Five Levels of Intelligent Textbooks
A Proposed Scale for AI-Powered Textbooks
I have been working on developing tools and standards for learning graphs for several months. When ChatGPT o1-preview came out, the increase in quality of creating concept graphs blew me away. I gained massive speedups using the o1-preview model (really a workflow) to generate learning graphs. I started generating more learning graphs for new topics that friends were interested in. But it got me thinking about how to help my readers see how mature these courses were and prepare them to load into a commercial learning management system (LMS).
Inspired by the five levels of autonomous driving, I created my own “five levels of intelligent textbooks.” This blog will review these five levels and how they can be used to guide any learning organization's intelligent textbook strategy. I will conclude with a look at what is coming next.
Background on Learning Graphs
Representation is the hardest part of AI. — Jeff Hawkins
Learning graphs are concept dependency graphs that help guide intelligent agents in building customized learning paths for each student. I reviewed them in my prior blog, but here is a quick recap.
Learning graphs are often about a specific subject, such as circuits, signal processing, Systems Thinking, MicroPython, Graph Algorithms, STEM robots, or Data Science. I usually ask a generative AI tool to help me write a course description or outline for a course. Much of this is organized by first focusing on defining core concepts with precise definitions and examples and proceeding to more complex concepts that build on these foundational concepts.
One of the critical steps to building websites for these courses is to anchor our understanding of intelligent content generation with MicroSims and interactive exercises where students use buttons and sliders to control interactive elements.
We then review this outline, do some cleanup, and create a complete learning graph for the course that intelligent agents can use. Today, there are only a few hundred MicroSims on these websites, but in the future, there will be tens of thousands or millions of them that AI agents can recommend and customize.
I have learned that many individuals and organizations are discussing “intelligent textbooks” without a precise scale to rank these textbooks. Moving a textbook from print to the web is an excellent first step. But that is just the beginning. Here is my attempt to create a five-level model of intelligent textbooks similar to the five-level scale for autonomous cars. Each level requires increasing investment and has proportional risks and rewards.
Level 1 — Static Textbooks
These textbooks include any traditional printed or digital textbooks without any interactive elements. They consist of purely text and static images. No dynamic content is generated, and no internal or external links to external resources such as MicroSims exist.
Static textbooks are suited for simple, short, and straightforward content delivery where no interaction is required. Static textbooks can still provide many illustrations, figures, tables, charts, and a detailed concept index at the back of the textbook.
Over 90% of textbooks used by college students today are required to use Level 1 static textbooks. However, this is starting to change.
Level 2 — Interactive Content Textbooks
A Level 2 textbook incorporates essential interactive elements such as keyword search, clickable links to internal content, and external resources such as Wikipedia Pages. Level 2 textbooks can use embedded videos or simple quizzes. Users can engage with multimedia or complete basic assessments directly in the textbook.
I use the excellent free, open-source mkdocs-material system with Visual Studio Code to create my online textbook examples. It is easy to do the following:
- Add a search function by adding a single element to a configuration file
- Provide deep links to each page section so you can put a link into a chat that moves the user direction to the correct chapter section.
- Configuring social media previews so your links to sample pages on social media will look great
- Monitoring usage with Google Analytics so you can see how often each page is being used to check student progress
- Every page footer can contain a link to a course feedback form, such as a Google form hosted on another server to keep student feedback confidential.
My example textbooks use powerful AI-generated MicroSims to demonstrate how tools like ChatGPT o1-preview can generate fully interactive simulations that run directly in your browser. I believe that MicroSims are one of the easiest and most cost-effective ways to add interactivity to any interactive textbook.
My examples also have a detailed glossary of terms so that the first time a concept is introduced on a page, the author can use a markdown editor to link to that concept in the glossary. Markdown editors built into VS Code do auto-completion to these resources.
I have also used the mkdocs-material md_in_html extension to embed review questions in the text so that students can click on a user interface element to reveal the answer to a question.
Good interactive sites have features that are designed to help facilitate distributed learning. For example, instructors can copy a deep link to a specific page, example, exercise, or quiz directly into the chat of a Zoom call, and the chat will display a descriptive and attractive preview of a chart or simulation using the social media metadata standard Open Graph Protocol. Open Graph metadata also has limited support for the free mkdocs-material tools. Paid users of mkdocs-material get full support for customizing the previews. The paid option is a good choice if you are serious about having custom images in your link previews.
Level 2 textbooks can also provide precise feedback tools, with every page having a link to a feedback form where content developers and instructors can see where students are visiting most often and how long they remain on a content page or animation.
Level 2.9 — Getting Ready for the Leap to Level 3
You will note that Level 2 textbooks don’t change their content based on the background or needs of the student. They are a one-size-fits-all solution. Students can’t quickly put in their personal background and personal learning goals and have content change based on their prior understanding of concepts. Everyone gets the same content regardless of their background. If you are taking a class on machine learning, there is just a single path that needs to be worked on, even if your students have little or no background in linear algebra.
Advancement to a Level 3 textbook requires getting ready to integrate with students’ needs, which means that free public hosting may not be sufficient. However, becoming “Level 3 Ready” is still a difficult and time-consuming process that requires continual feedback and testing.
Level 3 — Adaptive Textbooks
Level 3 intelligent textbooks adjust content based on analysis of user input or performance (e.g., adapting difficulty levels, providing additional resources). They incorporate simple deterministic rules like concept graph traversal algorithms to personalize learning pathways.
Level 3 intelligent textbooks select from fixed levels of curated content based on simple assessment scores and delay-time-based rules. If a user didn’t do well on last week’s review quiz, the content can be reviewed before generating new material. They can skip content if users show high proficiency.
Level 3 intelligent textbooks continuously record the concepts each student has mastered on the learning graph. When we add personalized concept proficiency data to a learning graph, we call it a personal learning graph. Each student needs their own personal learning graph. Because this is information tied to an individual, it needs to be carefully protected to be consistent with regulatory requirements such as GDPR in the EU and FERPA. These regulations even impact using simple web cookies stored in a student’s web browser.
Hosting Personal Learning Graphs
Regulatory compliance significantly negatively impacts the cost of hosting intelligent textbooks. It is not just a question of using AI to generate high-quality MicroSims quickly. A team of outside auditors must be engaged to certify compliance with the appropriate regulations. The need for auditing is a significant opportunity for new startups that can provide the infrastructure to support the hosting and auditing of personal learning graphs for a course.
However, these level-3 systems may still lack real-time responsiveness and the ability to leverage LLM to generate custom lesson plans and MicroSims. No ml is used.
Level-3 adaptive textbooks bridge the gap between mature online textbooks and fully dynamically student-responsive textbooks. Although they allow us to leverage generative AI to generate fantastic MicroSims at course design time, Level-3 textbooks can use simple graph algorithms to personalize content. They can use a vector store and graph embeddings to find similar content with positive approval by similar students. These are the same techniques that real-time product recommendation engines have used for several years. LLMs and open-source vector stores make building these solutions more accessible and cost-effective on an academic budget.
However, to truly integrate generative AI engines at course runtime, we must move to level 4: generative AI-powered textbooks with chatbot interfaces.
Level 4 — Textbooks with Chatbots
So, let’s assume you have a fantastic online course with hundreds of GenAI-created interactive MicroSims. You use simple graph algorithms to suggest learning content based on the needs of each student driven by their mastery of the concepts on the shelf. You are already using off-the-shelf graph algorithms such as similarity of students and recommendation engines.
But where do you go next? The answer is to combine the power of concept graph algorithms with LLMs to build a chatbot that can quickly answer questions from students. This combination uses the GraphRAG architecture in the context of an intelligent textbook. GraphRAG combines NLP pipelines, LLMs, embeddings, and vector stores to convert documents into graphs. These graphs can then be the basis for a curated Learning Graph. The Learning Graph becomes the “ground truth” upon which intelligent agents ride.
Level 4 includes using many GenAI and classic machine-learning-powered features like real-time feedback from a chatbot to adjust content recommendations. It requires careful logging of each student’s progress tracking and responsive question-answering systems. We will see these types of textbooks as early as next year in learning institutions that are forward-thinking thought leaders. Commercial organizations like Kahn Academy are already working on these features.
Level 4 Intelligent Textbooks know user context and provide tailored responses and recommendations based on student progress. They must be designed carefully to protect sensitive data and minimize the cost of excessively using larger LLMs when smaller LLMs, such as Ollama running on local GPUs, may be sufficient.
In summary, Level 4 intelligent textbooks enable a personalized learning experience that adapts to the learner’s needs in real-time. However, they still have limitations, and because of this, teachers and instructors will still be around for many years.
Finally, let’s look at the features of the full Level-5 AI Textbooks that are imagined in science fiction.
Level 5 — Autonomous AI Textbooks
Fully AI-driven adaptive intelligent textbooks have been imagined since Neal Stevenson wrote The Diamond Age: Or, A Young Lady’s Illustrated Primer in 1995. This cyberpunk novel imagined an AI-powered tablet capable of generating animated stories and lessons in real-time adapted to the context of each student’s goals. Level 5 textbooks deeply understand individual students’ current knowledge status and learning goals, answering complex questions through natural language processing and generating customized lessons.
Level 5 textbooks are just like cars that drive themselves under all conditions: during a heavy snowstorm at night on roads under heavy construction—in other words, not happening in the near term. But an exciting vision of the future.
Level 5 textbooks are something we can all dream about, but making them happen with today’s CPU hardware and unreliable LLMs will take work. We need hardware vendors to focus on cost-effective chips customized for graph traversal to build truly smart textbooks successfully.
But it will happen. My grandchildren will probably have comprehensive teaching assistants, mimicking a one-on-one tutor experience. These integrated AI tutors will go way beyond a single textbook. They will always support your learning objectives and protect your learning graph privacy. We will eventually see fully autonomous and interactive learning agents with advanced analytics to dynamically predict and respond to student needs.
Breaking Through The Layers
All models are wrong, but some are useful - George Box
One of the reasons that I created these layers is to expose them as applicable models, but I hope you recognize them as just a simple model that is easy to change. No regulation says you can’t build a simple website that includes complex forms where teachers or students respond to a quiz or survey, and the results of this process generate text that can be copied directly as a prompt for a tool like ChatGPT. Many of my websites feature sample prompts that use templates you can easily customize.
Skipping layers means we can go from a static Level 2 directly to take advantage of LLMs used in Level 4, skipping all the burden of regulatory compliance. The prompts still may contain student-specific data, but this data never needs to be stored on your website. The layers are just a model to help us create objective ratings for our work and guide us on the next steps.
Conclusion
Never mistake a clear view for a short distance. — Paul Saffo
I hope this scale provides a clear framework for evaluating and improving textbooks along the spectrum of interactivity and adaptability, guiding educators, software developers, and publishers in their development journey. Unfortunately, there are many unknowns. Although I am confident that the quality of LLMs will increase, I can’t predict when there will be standards for exchanging intelligent textbooks between authoring, hosting, and LMS systems. So much of this depends on standard graph databases to execute agents.
I can tell you with a high degree of certainty that my students prefer hands-on interaction to static textbooks and lectures. I encourage everyone to try to increase the levels of their textbooks to meet the needs of a diverse population of students. I hope all children on planet Earth have equal learning opportunities, regardless of country, race, gender, and socioeconomic background.
