Gaze Awareness in Agent-Based Early-Childhood Learning Application

  • Deepak AkkilEmail author
  • Prasenjit Dey
  • Deepshika Salian
  • Nitendra Rajput
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10514)


Use of technological devices for early childhood learning is increasing. Now, kindergarten and primary school children use interactive applications on mobile phones and tablet computers to support and complement classroom learning. With increase in cognitive technologies, there is further potential to make such applications more engaging by understanding the user context. In this paper, we present the Little Bear, a gaze aware pedagogical agent, that tailors its verbal and non-verbal behavior based on the visual attention of the child and employs means to reorient the attention of the child, when distracted from the learning activity. We used the Little Bear agent in a learning application to enable teaching the vocabulary of everyday fruits and vegetables. Our user-study (n = 12) with preschoolers shows that children interacted longer and showed improved short-term retention of the vocabulary using the gaze aware agent compared to a baseline touch-based application. Our results demonstrate the potential of gaze aware application design for early childhood learning.


Gaze Touch Pedagogical agent Early-childhood learning Vocabulary building Games Mobile devices Engagement Comparative study 



We thank Nathalie Henry Riche for her valuable inputs while shepherding this paper. We also thank the staff members at Kidzee Kindergarten, Domlur, Bangalore for their support to conduct the study.


  1. 1.
    Aladé, F., Lauricella, A.R., Beaudoin-Ryan, L., Wartella, E.: Measuring with Murray: touchscreen technology and preschoolers’ STEM learning. Comput. Hum. Behav. 62, 433–441 (2016)CrossRefGoogle Scholar
  2. 2.
    Atkinson, R.K.: Optimizing learning from examples using animated pedagogical agents. J. Educ. Psychol. 94(2), 416 (2002)CrossRefGoogle Scholar
  3. 3.
    Bee, N., Wagner, J., André, E., Charles, F., Pizzi, D., Cavazza, M.: Interacting with a gaze-aware virtual character. In: Proceedings of the 2010 workshop on Eye Gaze in Intelligent Human Machine Interaction, pp. 71–77. ACM (2010)Google Scholar
  4. 4.
    Black, M., Chang, J., Chang, J., Narayanan, S.: Comparison of child-human and child-computer interactions based on manual annotations. In: Proceedings of the 2nd Workshop on Child, Computer and Interaction, p. 2. ACM (2009)Google Scholar
  5. 5.
    Borgestig, M., Sandqvist, J., Parsons, R., Falkmer, T., Hemmingsson, H.: Eye gaze performance for children with severe physical impairments using gaze-based assistive technology: a longitudinal study. Assist. Technol. 28(2), 93–102 (2016)CrossRefGoogle Scholar
  6. 6.
    Chipman, G., Druin, A., Beer, D., Fails, J.A., Guha, M.L., Simms, S.: A case study of tangible flags: a collaborative technology to enhance field trips. In: Proceedings of Conference on Interaction Design and Children, pp. 1–8. ACM (2006)Google Scholar
  7. 7.
    D’Mello, S., Olney, A., Williams, C., Hays, P.: Gaze tutor: a gaze-reactive intelligent tutoring system. Int. J. Hum.-Comput. Stud. 70(5), 377–398 (2012)CrossRefGoogle Scholar
  8. 8.
    Dugard, P.: Randomization tests: a new gold standard? J. Context. Behav. Sci. 3(1), 65–68 (2014)CrossRefGoogle Scholar
  9. 9.
    Eichner, T., Prendinger, H., André, E., Ishizuka, M.: Attentive presentation agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS, vol. 4722, pp. 283–295. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-74997-4_26 CrossRefGoogle Scholar
  10. 10.
    Evans, M.A., Drechsel, E., Woods, E., Cui, G.: Multi-touch tabletop computing for early childhood mathematics: 3D interaction with tangible user interfaces. In: Proceedings of the 9th International Conference of the Learning Sciences, pp. 274–275 (2010)Google Scholar
  11. 11.
    Feng, G.: Eye tracking: a brief guide for developmental researchers. J. Cogn. Dev. 12(1), 1–11 (2011)CrossRefGoogle Scholar
  12. 12.
    Fisher, A.V., Godwin, K.E., Seltman, H.: Visual environment, attention allocation, and learning in young children when too much of a good thing may be bad. Psychol. Sci. 25(7), 1362–1370 (2014)CrossRefGoogle Scholar
  13. 13.
    McKnight, L., Fitton, D.: Touch-screen technology for children: giving the right instructions and getting the right responses. In: Proceedings of the 9th International Conference on Interaction Design and Children, pp. 238–241. ACM (2010)Google Scholar
  14. 14.
    Heidig, S., Clarebout, G.: Do pedagogical agents make a difference to student motivation and learning? Educ. Res. Rev. 6(1), 27–54 (2011)CrossRefGoogle Scholar
  15. 15.
    Helo, A., Pannasch, S., Sirri, L., Rämä, P.: The maturation of eye movement behavior: scene viewing characteristics in children and adults. Vis. Res. 103, 83–91 (2014)CrossRefGoogle Scholar
  16. 16.
    Hiniker, A., Sobel, K., Hong, S.R., Suh, H., Kim, D., Kientz, J.A.: Touchscreen prompts for preschoolers: designing developmentally appropriate techniques for teaching young children to perform gestures. In: Proceedings of International Conference on Interaction Design and Children, pp. 109–118. ACM (2015)Google Scholar
  17. 17.
    Homer, B.D., Kinzer, C.K., Plass, J.L., Letourneau, S.M., Hoffman, D., Bromley, M., Hayward, E.O., Turkay, S., Kornak, Y.: Moved to learn: the effects of interactivity in a Kinect-based literacy game for beginning readers. Comput. Educ. 74, 37–49 (2014)CrossRefGoogle Scholar
  18. 18.
    Hornof, A.J., Cavender, A.: EyeDraw: enabling children with severe motor impairments to draw with their eyes. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 161–170. ACM (2005)Google Scholar
  19. 19.
    Hourcade, J.P.: Interaction design and children. Found. Trends Hum.-Comput. Interact. 1(4), 277–392 (2008)CrossRefGoogle Scholar
  20. 20.
    Gottlieb, J., Oudeyer, P.Y., Lopes, M., Baranes, A.: Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends Cogn. Sci. 17(11), 585–593 (2013)CrossRefGoogle Scholar
  21. 21.
    Kam, M., Rudraraju, V., Tewari, A., Canny, J.: Mobile gaming with children in rural India: contextual factors in the use of game design patterns. In: Proceedings of 3rd Digital Games Research Association International Conference (2007)Google Scholar
  22. 22.
    Kangas, J., Akkil, D., Rantala, J., Isokoski, P., Majaranta, P., Raisamo, R.: Gaze gestures and haptic feedback in mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 435–438. ACM (2014)Google Scholar
  23. 23.
    Krämer, N.C., Bente, G.: Personalizing e-learning. the social effects of pedagogical agents. Educ. Psychol. Rev. 22(1), 71–87 (2010)CrossRefGoogle Scholar
  24. 24.
    LaFrance, M., Mayo, C.: Cultural aspects of nonverbal communication. Int. J. Interc. Relat. 2(1), 71–89 (1978)CrossRefGoogle Scholar
  25. 25.
    Lahiri, U., Warren, Z., Sarkar, N.: Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 4, 443–452 (2014)Google Scholar
  26. 26.
    Lu, C., Frye, D.: Mastering the machine: a comparison of the mouse and touch screen for children’s use of computers. In: Tomek, I. (ed.) ICCAL 1992. LNCS, vol. 602, pp. 417–427. Springer, Heidelberg (1992). doi: 10.1007/3-540-55578-1_88 CrossRefGoogle Scholar
  27. 27.
    Luna, B.: Developmental changes in cognitive control through adolescence. Adv. Child Dev. Behav. 37, 233–278 (2009)CrossRefGoogle Scholar
  28. 28.
    Nacher, V., Jaen, J., Navarro, E., Catala, A., González, P.: Multi-touch gestures for pre-kindergarten children. Int. J. Hum.-Comput. Stud. 73, 37–51 (2015)CrossRefGoogle Scholar
  29. 29.
    Ovaska, S., Hietala, P., Kangassalo, M.: Electronic whiteboard in kindergarten: opportunities and requirements. In: Proceedings of the 2003 Conference on Interaction Design and Children, pp. 15–22. ACM (2003)Google Scholar
  30. 30.
    Pfeuffer, K., Alexander, J., Chong, M.K., Gellersen, H.: Gaze-touch: combining gaze with multi-touch for interaction on the same surface. In: Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, pp. 509–518. ACM (2014)Google Scholar
  31. 31.
    Plowman, L., McPake, J.: Seven myths about young children and technology. Child. Educ. 89(1), 27–33 (2013)CrossRefGoogle Scholar
  32. 32.
    Ramloll, R., Trepagnier, C., Sebrechts, M., Finkelmeyer, A.: A gaze contingent environment for fostering social attention in autistic children. In: Proceedings of the 2004 Symposium on Eye Tracking Research & Applications, pp. 19–26. ACM (2004)Google Scholar
  33. 33.
    Robertson, J., Macvean, A., Howland, K.: Robust evaluation for a maturing field: the train the teacher method. Int. J. Child-Comput. Interact. 1(2), 50–60 (2013)CrossRefGoogle Scholar
  34. 34.
    Rogers, Y., Price, S., Randell, C., Fraser, D.S., Weal, M., Fitzpatrick, G.: Ubi-learning integrates indoor and outdoor experiences. Commun. ACM 48(1), 55–59 (2005)CrossRefGoogle Scholar
  35. 35.
    Romeo, G., Edwards, S., McNamara, S., Walker, I., Ziguras, C.: Touching the screen: issues related to the use of touchscreen technology in early childhood education. Br. J. Educ. Technol. 34(3), 329–339 (2003)CrossRefGoogle Scholar
  36. 36.
    Schroeder, E.L., Kirkorian, H.L.: When seeing is better than doing: preschoolers’ transfer of STEM skills using touchscreen games. Front. Psychol. 7, 1–10 (2016)CrossRefGoogle Scholar
  37. 37.
    Sibert, J.L., Gokturk, M., Lavine, R.A.: The reading assistant: eye gaze triggered auditory prompting for reading remediation. In: Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology, pp. 101–107. ACM (2000)Google Scholar
  38. 38.
    Vatavu, R.D., Cramariuc, G., Schipor, D.M.: Touch interaction for children aged 3 to 6 years: experimental findings and relationship to motor skills. Int. J. Hum.-Comput. Stud. 74, 54–76 (2015)CrossRefGoogle Scholar
  39. 39.
    Vidal, M., Turner, J., Bulling, A., Gellersen, H.: Wearable eye tracking for mental health monitoring. Comput. Commun. 35(11), 1306–1311 (2011)CrossRefGoogle Scholar
  40. 40.
    Ygge, J., Aring, E., Han, Y., Bolzani, R., Hellström, A.: Fixation stability in normal children. Ann. N. Y. Acad. Sci. 1039(1), 480–483 (2005)CrossRefGoogle Scholar
  41. 41.
    Zaman, B., Abeele, V.V., De Grooff, D.: Measuring product liking in preschool children: an evaluation of the Smileyometer and this or that methods. Int. J. Child-Comput. Interact. 1(2), 61–70 (2013)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Deepak Akkil
    • 1
    Email author
  • Prasenjit Dey
    • 2
  • Deepshika Salian
    • 3
  • Nitendra Rajput
    • 4
  1. 1.University of TampereTampereFinland
  2. 2.IBM ResearchBangaloreIndia
  3. 3.Kidzee Kindergarten DomlurBangaloreIndia
  4. 4.Infoedge India LimitedNew DelhiIndia

Personalised recommendations