This blog post was originally published via the Community for Advancing Discovery Research in Education (CADRE) on November 25, 2024. The original blog can be accessed here.
Authors: Jeremy Roschelle, Executive Director of Learning Sciences Research, Digital Promise; Gautam Biswas, Cornelius Vanderbilt Professor of Engineering and Professor of Computer Science and Computer Engineering, Vanderbilt University; Cindy Hmelo-Silver, Distinguished Professor and Associate Dean for Research and Development, Indiana University; James Lester, Goodnight Distinguished University Professor in Artificial Intelligence and Machine Learning, North Carolina State University
Artificial intelligence can strengthen the role of narratives in games for learning. In the NSF-funded AI Institute for Engaged Learning (EngageAI), we’re working across disciplines and with practitioners to enable more engaging experiences with story-driven learning in games for students while also strengthening the opportunities for students to learn science more deeply.
A starting place for exploring the connections between gamed-based and narrative-based learning is the role of scenes in structuring the overall student experience. In a traditional game, a “level” often introduces a change in scene, with possible changes in objective, difficulty, and opportunities to earn rewards. In narrative-centered learning, a “story beat” is a shift in the storyline, potentially with new characters, new plot aspects, or a new spatial setting, along with new opportunities for interacting with the characters or setting. Within this apparent contrast, many features can be similar, including ways of navigating the world, mechanics of interacting in the world, and more. Practitioners and researchers seek to introduce games into education for similar reasons; synthetic worlds motivate students, and more importantly, entering into an immersive world can strengthen students’ opportunities to take on scientific roles, engage in problem-solving, and seek understanding of concepts while continuing to engage in the game.
As one of five NSF-funded institutes that connect AI and education, our team sees powerful roles for AI in the process of building narrative-centered learning environments:
- One of our tools, SceneCraft, leverages large language models to ease the process of authoring student experiences with an interactive story that includes multiple story beats, each of which adds to a coherent plot. SceneCraft provides a practitioner-friendly workflow for writing and editing interactive stories about science. In one sense, working with SceneCraft is like working with a chatbot—the technology makes a proposed narrative-centered learning experience from a series of teacher prompts. In another sense, it enables tuning the elements of the story—the context, the characters, the plot, and what the characters know and can say—so the teacher can shape an engaging, contextually relevant pedagogical experience.
- Our featured exploration of narrative-centered curricular experiences is Food Justice. Food Justice is enabling our team to explore how collaborative inquiry can be fostered in an AI-enabled science unit. It also explores student embodied activity and screen interactions. AI enhances the game through the use of conversational agents—non-player characters that can support students in finding information, making decisions, and providing feedback on their arguments. We see AI as multimodal, not just as textual prompts and responses. Our conversational agents can take on multiple roles, including mentor agents that support student learning and help them overcome their difficulties or peer agents that can provide sociocognitive support to the students.
- On the research side, our Multimodal Analytics team is creating a pipeline that can integrate multiple types of data from our game-based learning environments to enable online analysis and empower learning sciences research. As they do so, they are also integrating new, foundational AI technologies that analyze raw data, such as vision and speech data, and transforming it into more useful insights for research, such as helping educators and researchers identify highlights or teachable moments in a long stream of information. Online multimodal analytics and AI algorithms also hold the potential to support adaptation, allowing the agents in the environment to interpret students’ cognitive, metacognitive, and affective states and tailor their conversations to meet their needs.
- Across all activities, a strong Ethics Team and Practitioner Advisory Board are working to ensure that uses of AI are safe, responsible and aligned with the goals of science education.
Stepping back on all this detailed activity, EngageAI sees three major ways in which AI can strengthen the role of games and narratives in STEM education (as long as appropriate guardrails are in place).
First, AI opens novel possibilities for student interaction in these environments. Rather than using “gamification” as an extrinsic motivational device, AI can help developers transport students into immersive worlds and thereby engage them in the collaborative inquiry, problem-solving, and sense-making activities at the heart of science education.
Second, AI can make authoring more accessible and make it easier to co-design with practitioners. The process of creating either a game or a narrative-centered learning environment has been laborious and out of research for most educators. AI can reduce costs and ensure that educators can be involved in ensuring the quality of the science content, appropriate pedagogy, and responsiveness to their students’ strengths and needs.
Third, AI can help researchers who study these environments by making it easier to work with complex streams of temporal data that combine video, speech, and students’ problem-solving behaviors in the narrative-centered learning environment.
EngageAI invites researchers, practitioners, innovators and others to connect with us via our website and X/Twitter.