Authors: Sarah Burriss, Kip Glazer, Bill Bass and Judi Fusco
Are you searching for a way through all the new AI systems in the education landscape? Would you like a compass to guide your path to positive impacts on members of your school ecosystem (including teachers, principals, district staff, students, and families)? It seems educators everywhere are navigating the complexities of evaluating new AI systems’ effects on pedagogy, learning, and relationships of members in the school ecosystem. At the recent AI K12 Deeper Learning Summit, hosted in Anaheim, California, researchers and practitioners representing EngageAI’s Practitioner Advisory Board spoke to the audience about this exact topic. This convening served as an opportunity for us to connect with a broader audience of people interested in AI in education and we had the opportunity to facilitate one of the sessions.
Our session, Finding Your Compass: Tools and Frameworks to Navigate AI, presented tools and frameworks to help education leaders chart a more informed course. This session was led by Bill Bass, Parkway School District (MO), Kip Glazer, Mountain View High School (CA), Sarah Burriss, Vanderbilt University, and Judi Fusco, Digital Promise; the speakers represent a group of school and district leaders and researchers who are seeking solutions for the issues that educators everywhere are facing.
We focused the session on the cyclical, iterative evaluation of AI for education, by a new framework and tool called An AI Compass Evaluation Tool for Education. The cycle, shown in the image below, can help educators consider specific AI systems for adoption.
The AI Compass Evaluation Tool, while still in development, systematizes earlier work. It grew out of the development of An AI Bill of Rights for Educators1. While a compass and a Bill of Rights may seem disconnected, they are both important facets of work towards including AI systems in an educational system. Following the Bill of Rights work, several “conversation starters” were developed to facilitate further discussions among the members of the school ecosystem. When complete, the AI Compass evaluation tool will include an adaptable survey form that can be sent to all members of the school ecosystem to gather a wide range of perspectives to help those leading implementations understand challenges and impacts. Importantly, the AI Compass will also include a dashboard for visualizing results from the evaluation responses.
The audience members at the AI K12 Deeper Learning Summit session were mostly district administrators or school-based practitioners (teachers or principals). They were able to explore an initial version of the AI Compass that focused on teachers as respondents, and then give feedback to the session leaders. In the session, we heard:
- A positive attitude towards the idea of the “bill of rights” for educators and students
- An appreciation of the compass evaluation cycle — there was a strong acknowledgement that it’s not possible to just do an initial evaluation and stop there.
- Further appreciation of how the new compass tool delineated various roles, and how it attempted to address the role-specific needs of the different groups.
The session participants’ feedback underscored the necessity of this work. Their comments illuminated the AI Compass tool’s potential to facilitate discussions around specific tools by bringing in varied perspectives as it is completed. A participant made suggestions for expanding its question set to understand specific needs, such as those of special education teachers working with students who have varying cognitive abilities. However, we also heard some uncertainty around its usefulness for teachers in some districts where they have limited input around new technology adoption. Participants noted that the AI Compass tool could help teachers, administrators, students, or parents initiate conversations that are currently not taking place around new AI systems. The participants expressed a need for more conversations like this one to think about what is needed and from different perspectives.
The Practitioner Advisory Board thanks the National Science Foundation’s AI Institute for Engaged Learning (EngageAI) for the opportunity to do this work. The board works to facilitate connections between practitioners and researchers at EngageAI. Session participants also appreciated hearing from practitioners who have a vision for AI. For example, Mountain View Principal Kip Glazer shared how her students have been engaging with the AI Bill of Rights, including creating an educational game and developing their own Student Bill of Rights. This kind of work enriches and deepens the impact of the new systems and helps others understand them.
We were pleased to hear enthusiasm for the AI Compass tool and framework, the desire to see the completed version of the evaluation tool, and the positive response towards the development of materials and processes to encourage and support these types of important conversations.
Acknowledgements
The Practitioner Advisory Board members are: Pam Amendola, Bill Bass, Krystal Chatman, Gina Fugate, Kip Glazer, Shawn Higgins, Paula Johnson, Nneka McGee, Julio Vazquez, & Julie York. Sarah Burriss organized and led the work of the board with Judi Fusco, Pati Ruiz, and Nicole Hutchins participating, facilitating, and co-leading in different stages of the work.
- The Bill of Rights was created in 2024 as the development and deployment of AI, including generative AI, was accelerating. It supports the safe implementation of these new technologies and centers educators in the very human endeavor of education. Please see the AI Bill of Rights for Educators for more information. ↩︎