Can Smart Import create Smart Learning content?
Dr Jue Wang Szilas & Dr Patrizia Birchler Emery
But what will teachers do with all the free time they will have thanks to AI?
Sitting in front of a computer, giving a text or video to an AI tool to read, and a few minutes later having a complete online course, including a learning path with interactive activities: is this a scenario for the future?
The answer to this rhetorical question is obviously "no", because a number of developers are already making such tools available, assuring teachers that they will be able to reduce the time they spend preparing their lessons.
At least that's what we heard during a webinar extolling the educational potential of H5P Smart Import, a new tool for creating text-based interactive learning content. These claims piqued our curiosity and prompted us to try Smart Import to see how true they were.
However, as H5P Smart Import was only available in English, we decided to find a similar tool in French to complete the test and we chose Nolej.
We therefore tested each of these two tools to gain a better understanding of their capabilities and limitations: the first, H5P Smart Import, using a 30-minute YouTube video of a presentation intended for a general audience, and the second, Nolej, using a 3-minute video on digital imaging techniques as part of a university-level digital humanities course. The results are more or less the same.
Presenting the tools...
 |
 |
| H5P Smart Import: AI engine generated Interactive content documentation | Nolej AI: OpenAi-Based Instructional Design Generator |
H5P Smart Import and Nolej are innovative solutions for AI-assisted instructional design, particularly for the creation of text-based interactive learning content. Both tools use AI to analyse and transform existing documents - such as text, images and video - into engaging, interactive formats.
Key features of H5P Smart Import and Nolej include the conversion of content into interactive activities such as quizzes, flashcards, drag-and-drop activities and multiple choice questions. Both solutions can also be used to create simulations and mind maps. The videos generate automatic transcripts that users are invited to check and correct. What's more, users can make changes to the interactive activities they create, ensuring that the content is of the highest quality.
In terms of customisation, H5P Smart Import and Nolej offer basic adjustments, while Nolej also offers advanced options. Both tools integrate easily with the main LMS platforms (Moodle, Canvas, Blackboard), making them easy to use in a variety of teaching environments. H5P Smart Import is particularly effective for structured content, while Nolej excels with more varied media and encourages collaboration between teachers.
There are differences between the two tools, in terms of access on the one hand and use of the content produced on the other. H5P Smart Import offers a 30-day trial period, whereas Nolej allows a 10-day trial to create up to three learning modules. With H5P Smart Import, the content generated is reusable, but requires a subscription for permanent access after the trial, whereas with Nolej it remains visible even without a subscription.
Last but not least, H5P Smart Import is limited to English, whereas Nolej is available in several languages.
These tools are suitable for a wide range of teachers, from primary to higher education, and are adapted to different types of teaching, including vocational training. Basic digital skills are recommended to integrate them effectively into educational practice.
In conclusion, H5P Smart Import and Nolej offer complementary solutions for creating interactive activities, meeting the varied needs of teachers depending on the nature and complexity of the educational content.
...and tests
Here we present the testing process, the results obtained as well as our analysis and comments at each stage. The results were very similar for both tools. However, as H5P Smart Import requires a subscription to share the results, we had to use Nolej to make them available.
 |
 |
| Click on the image above to see the generated content | Click on the image above to see the generated content |
Test with H5P Smart Import
1. Context and resources used
After a convincing demonstration by the H5P team, the authors were granted temporary access to test Smart Import's features with colleagues. For this evaluation, a video extract of a 30-minute conference in English was selected: it is a presentation of a MOOC aimed at preserving an endangered script, entitled Teaching an Endangered Script through MOOC. This resource was chosen to assess Smart Import's ability to generate interactive learning activities from long and complex content.
In this video, the author explores the Dongba script, a unique UNESCO heritage specific to the Naxi people in China. She outlines the goals of the MOOC, which are to raise awareness of this endangered script and to promote its preservation. Learning the basic 300 characters is at the heart of the course, with interactive activities to engage participants.
2. Process and results
The authors used Smart Import to convert the video into a text transcript. Any errors were corrected by human proofreading, thus guaranteeing the quality of the basic text. From this corrected content, Smart Import generated eight types of learning activities, including an interactive video with keywords and questions, a quiz, a glossary, a concept map, a drag-and-drop activity, flashcards and a summary. These activities were then shared with colleagues, who were invited to complete them and give their feedback.
Here is the link to the content created (link to the Nolej site for the reasons given above): https://h5p-live.nolej.app/documents/deccc566-2605-49fc-9a06-e564120e838f/previews/ibook/index.html.
3. Analysis of results
In evaluating this video and the associated teaching activities, a number of positive points and areas for improvement emerged (see table below).
| Categories | Positif points | Areas for Improvement |
|---|---|---|
| Transcription | Overall accuracy, with human proofreading necessary | Some errors related to voice recognition |
| Keywords | Relevant and well-explained (Dongba script, Endangered languages, etc.) | Frequent repetitions |
| Exercises | Effective interactive activities (quizzes, flashcards, drag-and-drop). |
|
| Conception | Interactive approach suitable for simple factual learning. | Insufficient variety of activities, resulting in a certain degree of monotony. |
While automatic transcription is largely accurate, human proofreading is required to ensure accuracy, particularly for specific technical or cultural terms. For example, certain terms relating to the Dongba script or Naxi culture, which are essential elements of the content, could be misinterpreted without proper proofreading. For example, a term such as 'Dongba script' could be transcribed incorrectly if it is not checked in its cultural context, highlighting the importance of double-checking to avoid errors.
The selected keywords are highly relevant. Each term, such as Dongba script or Endangered Languages, is accompanied by explanations that enrich the content and contextualise the concepts covered. For example, the term "endangered languages" is not only defined, but also linked to concrete examples of endangered languages around the world. However, this richness is sometimes undermined by unnecessary repetition in the glossaries, flashcards and other materials. For example, certain terms appear in several exercises with the same explanations, which can make learning redundant and less engaging.
The structure of the exercises shows a clear desire to make learning interactive and engaging. Activities such as quizzes and drag-and-drop games allow users to associate Dongba symbols with their meanings, making the experience more fun and memorable. However, these exercises sometimes suffer from unnecessary repetition. For example, the question about the objectives of the MOOC is repeated in several formats in different exercises, without adding any new information or perspective. This can make the learning experience monotonous and less stimulating. A variation in the presentation of questions or pedagogical approaches could help to enliven this part of the course.
In terms of consistency, discrepancies in expected responses can be problematic. A key question about the goals of the MOOC, "What is the goal of the MOOC?" is answered differently depending on whether it is asked in a quiz or in a concept map. For instance, in the quiz, the answer focuses on the preservation of the Dongba script, whereas in the concept map, the goal is broadened to explore the role of plurilingualism in language preservation. These inconsistencies can confuse learners, who are faced with several versions of what they perceive to be the 'correct answer'. Clarification of the expected answer and greater harmonisation of teaching materials would be beneficial.
Another aspect that deserves attention is the quality of feedback provided in the exercises. It sometimes happens that correct answers are reported as wrong because of incorrect settings or errors in the validation system. For example, a correct answer such as 'To preserve and revitalise the Dongba script' in a quiz might be incorrectly marked as wrong, which can be frustrating and detract from the learning experience. Furthermore, the lack of open-ended questions limits the opportunity for learners to think critically and deeply about issues such as the preservation of endangered languages or the role of multilingualism in script preservation. Open-ended questions would allow more nuanced viewpoints to be explored and encourage personal reflection by learners.
To sum up, although this video and its educational resources have some solid assets, such as an interactive structure and relevant keywords, a few adjustments are needed. Greater consistency in responses and feedback, as well as greater diversity in the pedagogical exercises, would help to improve interactivity and the level of learning on offer.
4. Conclusion
The results indicate that Smart Import is effective in generating text-based learning activities, particularly for the acquisition of declarative knowledge. The tool reinforces mastery of key concepts by repeatedly integrating these elements into the proposed activities. However, certain limitations remain. H5P Smart Import is not yet capable of generating open-ended questions that stimulate critical thinking, designing complex distractors in multiple-choice questions, or facilitating collaborative learning. In addition, the tool currently only supports English-language content, which highlights the need to add other languages to increase the tool's accessibility and usefulness.
Test with Nolej
1. Context and resources used
The video used as a basis for the test with Nolej was presented as part of a course on digital imaging techniques (university level) aimed at students of the digital humanities working in any field where objects are involved (archaeology, art history, manuscripts, etc.). Different types of imaging techniques are examined and tested in order to learn about their specific characteristics and practical uses, with the aim of being able to apply the most appropriate technique in the research context.
This video introduces one of these techniques, RTI (Reflectance Transformation Imaging), and its application to archaeology. It is an advertisement for a commercial tool and students are encouraged to be critical of it.
It is therefore not a course video, but a demonstration video (which has no bearing on the test results). The video is shown after a theoretical introduction to RTI and how it works for the following reasons:
- It gives a visual presentation of a specific RTI tool: components, handling of the equipment, set-up, etc.
- It shows the tool in use.
- It presents the results obtained.
- It presents the usefulness and interest of photographing objects with this tool and technique (without mentioning its limitations).
2. Process and results
The first step was to use Nolej to convert the video into a text transcript. As with Smart Import, human proofreading was required to remove the few transcription errors. From this corrected content, Nolej generated learning activities, interactive videos, concept maps, quizzes, drag-and-drops, flashcards, as well as a glossary, key points and a summary. Nolej also provided additional content: crosswords, hidden words, project-based learning ideas.
Here is the link to the content created: https://h5p-live.nolej.app/documents/c1e9e42c-1714-4b96-9184-2ab3ba37eab5/previews/ibook/index.html.
3. Analysis of the result
| Categories | Positive points | Areas for Improvement |
|---|---|---|
| Transcription | Overall accuracy, but human proofreading necessary | Some errors due to background noises other than human voices (e.g., engine sounds, equipment movement) |
| Keywords | Basic and general (other terms mentioned in the video would have been more appropriate) | Not all activities created from the keywords were relevant (interactive video, glossary, concept maps, drag-and-drop, crossword puzzles, hidden words). Redundancy here presents liabilities. |
| Exercises | Satisfactory interactive activities (quizzes, flashcards, project-based learning ideas) |
|
| Conception | Interactive approach suitable for simple factual learning | Insufficient variety of activities, leading to a certain degree of monotony; moreover, based solely on speech or text, with no image recognition. |
As the text is very short, the transcription did not require any major intervention, apart from eliminating clusters of letters corresponding to transcription errors due to the various sounds (for example, an engine) and music heard in the video.
The number of activities proposed is significant and would, in principle, make it possible to develop a learning path with a variety of activities: interactive video, glossary, key words, concept maps, then quiz, drag and drop, flashcards, key points and summary, and finally, as complementary activities, crosswords, secret word and ideas for project-based learning.
The whole group of activities based on the glossary (keywords, interactive video, concept maps, crosswords, secret word) was unfortunately not very useful because the words chosen were too general (reconstruction, animated) and reflected basic concepts that did not need to be tested (excavation, photography, web environment, battery-powered solution, state-of-the-art equipment, infrastructure, know-how). Other terms that were more specific to RTI technology and used in the text were not retained even though they would have been more relevant (research, turning the object over, light and shadow, engraving, inscription, iconography).
The quiz questions were more interesting, as they were based on sentences in the text rather than isolated words. For example: ‘What does the methodology improve through the amalgamation of photographic materials from disparate institutions around the world?’ fits in with the aims of the viewing, as it serves to demonstrate one of the uses of the results. In general, the quiz questions were satisfactory, even if limited.
The flashcards were of mixed quality: those based on the quiz questions were relevant, while those based on the glossary were useless.
The key points and the summary were satisfactory, as were the first three suggestions for project-based learning (the last two were either unrealistic or too vague). As concerns the additional content: the crosswords and hidden words did not contribute anything, as they were not very relevant to the teaching objectives and did not encourage in-depth thinking.
Last but not least, as AI is text-based, the proposed interactive course lacks a set of questions to explore that which is shown in the video and which is just as important as the discourse in terms of viewing objectives. A few examples: size and components of the equipment, environment in which the RTI will be carried out, common characteristics of the objects undergoing the process (size, appearance, mobile/immobile)?
There is also a lack of open questions that require critical thinking and reflection, on the possibility, for example, of applying this technology to all archaeological objects, or on the possible existence of other techniques of the same type, and so on.
4. Conclusion
Nolej is a very effective tool, but it is preferable to generate the basic text yourself, incorporating the keywords and concepts that the students need to grasp, otherwise you have to invest a fair amount of time correcting the transcription and skimming the activities to keep only what is relevant (even if it means adding questions). This AI's learning model is still basic, corresponding only to the first level of Bloom's taxonomy (knowledge acquisition).
A major flaw is that the AI does not analyse images and therefore does not generate keywords or quizzes on what is shown in a video, whereas many fields of study involve learning linked to a set of gestures to be practised or items to be observed.
Finally, there are no reflection, analysis or open-ended questions.
Conclusion and reflection
The results revealed that these two tools effectively generated numerous text-based teaching activities, focusing mainly on the learning of declarative knowledge. In particular, they improved mastery and understanding of key concepts by increasing their frequency in activities. However, limitations have been identified, such as the need for human intervention in the transcription phase, the inability to generate open-ended questions requiring critical reflection, to incorporate in-depth distractors in multiple-choice questions, or to facilitate collaborative learning. In addition, these tools do not allow the development of activities on visual objects, which limits their scope in many areas. In addition, the current version of H5P Smart Import only supports English-language resources, with other languages still to be developed.
Through this trial, the benefits of an AI engine have been recognised, as well as its potential to significantly increase productivity in the creation of text-based interactive learning content and to improve accessibility. At the same time, the test provided a better understanding of the current limitations of these tools, which can only generate content based on text and, above all, content that is basic in pedagogical terms, focusing solely on the acquisition of knowledge.
However, it is clear that the use of these AI tools in educational design raises a number of questions. For example, to what extent will interactive learning content created by AI be credible in a university environment? Or how can a balance be struck between automation and creativity in the design of learning content? When integrating AI tools such as Smart Import into instructional design, in what phases of the process can human roles not be replaced by AI?
The test carried out and its results invite reflection on the evolution of the role of AI in the future development of intelligent learning content, but perhaps also on the notion of intelligence itself.
Appendix: presentation of the article with screenshots
References:
- Farrelly, T., & Baker, N. (2023). Generative artificial intelligence: Implications and considerations for higher education practice. Education Sciences, 13 (11), 1109.
- García-Peñalvo, F. J., Llorens Largo, F., & Vidal, J. (2023). The new reality of education in the face of advances in generative artificial intelligence.
- Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2023). The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493.
- ANNEMISSELBROOK. (2023, November 22). H5P software trial of the feature ‘Smart Import’ using AI technology [Blog post https://blogs.northampton.ac.uk/learntech/2023/11/22/h5p-software-trial-of-the-feature-smart-import-using-ai-technology/].
- BERSIN, J. (2023, April 21). EdTech Is Going Crazy For AI [Blog post https://joshbersin.com/2023/04/edtech-is-going-crazy-for-ai/].
- Lindsay, R. (2023, November 10). Embracing Interactive Learning - H5P at the University of Liverpool [Blog post https://www.liverpool.ac.uk/centre-for-innovation-in-education/blog/general/embracing-interactive-learning-h5p-uol/].
- O'Donnell, F. (2023). Let AI automatically create learning activities [Blog post https://teachnet.ie/let-ai-automatically-create-learning-activities/].
- Zhgenti, S., & Holmes, W. (2023, July 24). Generative AI and Education: Adopting a Critical Approach [Blog post https://botpopuli.net/generative-ai-and-education-adopting-a-critical-approach/].
- Wladawsky-Berger, I. (2023, September 28). The Critical Value of Human-Created Knowledge to the Future of Generative AI [Blog post https://blog.irvingwb.com/blog/2023/09/the-critical-value-of-human-created-knowledge-to-generative-ai.html].
- Packer, H. (2023, July 4). UK universities set out plans to use AI in teaching [Newspaper article https://thepienews.com/news/uk-universities-ai-teaching/]
