From Spoken Words to Prompt Triggers: Technical Iterations of a Semi-Intelligent Conversational Agent to Promote Early Literacy

AI technology is rapidly evolving and has vast potential for educational applications. This paper focuses on the technical iterations that took place as our project team developed a semi-intelligent conversational agent (CA) that uses speech recognition to fire spoken prompts to promote caregiver-child interaction as they read books aloud together. Situating this work in a design-based research methodology, the technical iterations reported here are part of the iterative "build" phase. (Easterday et al., 2018; Hoadley & Campos, 2022). The CA app promotes conversations between caregivers and children by listening to the human dyads as they read, matching their spoken words to marker words that would pinpoint the page of the storybook the dyads are reading, and playing a prompt corresponding to the page. The dynamic system that supports the app involves multiple components: web accessible services, data processing services, and human outputs, and it has gone through a combined seven iterations in three prototypes. Though a very small part of the DBR cycle, the technical iterations presented here have the potential to inform others interested in incorporating text-to-speech and other AI technologies into educational applications. We close with considerations for future directions.