Spoken Dialogs with Children for Science Learning and Literacy

Abstract

Advances in human language and character animation technologies have enabled a new generation of intelligent tutoring systems that support conversational interaction between young learners and a lifelike computer character that was designed to behave like a sensitive and effective human tutor My Science Tutor is a spoken dialog system in which children learn to construct science explanations through conversations with Marni, the virtual science tutor, in multimedia environments. MyST displays illustrations, silent animations or interactive simulations to the student, while Marni asks open-ended questions like “Whats going here?”. Based on MySTs analysis of the students spoken response, the system decides what the student understands about the science and what the student has not yet explained (or doesnt know), and generates a follow-on question a new prompt, and possibly a new animation, that is designed to scaffold learning and challenge the student to reason about the science. Two large scale evaluations were conducted in which third, fourth and fifth grade students received over 5 hours of tutoring during sixteen 20-minute sessions in four different areas of science. The results revealed that, relative to students who did not receive tutoring, students who used My Science Tutor achieved significant learning gains in standardized tests of science achievement, equivalent to gains achieved by students who received tutoring by expert human tutors. In recent research, we have extended the technologies used in MyST to a develop a new generation of interactive books that use text, speech and dialog technologies to help children learn to read science texts fluently, expressively, and with good comprehension. We will demonstrate these MindStars Books and present initial results of classroom testing.

Invited talk.