Using a Serious Game to Assess Spatial Memory in Children and Adults

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10714)


Short-term spatial memory has traditionally been assessed using visual stimuli, but not auditory stimuli. In this paper, we design and test a serious game with auditory stimuli for assessing short-term spatial memory. The interaction is achieved by gestures (by raising your arms). The auditory stimuli are emitted by smart devices placed at different locations. A total of 70 participants (32 children and 38 adults) took part in the study. The outcomes obtained with our game were compared with traditional methods. The results indicated that the outcomes in the game for the adults were significantly greater than those obtained by the children. This result is consistent with the assumption that the ability of humans increases continuously during maturation. Correlations were found between our game and traditional methods, suggesting its validity for assessing spatial memory. The results indicate that both groups easily learn how to perform the task and are good at recalling the locations of sounds emitted from different positions. With regard to satisfaction with our game, the mean scores of the children were higher for nearly all of the questions. The mean scores for all of the questions, except one, were greater than 4 on a scale from 1 to 5. These results show the satisfaction of the participants with our game. The results suggest that our game promotes engagement and allows the assessment of spatial memory in an ecological way.


Gamification Serious game Auditory Short-term memory  Karotz Color-depth sensor Microsoft KinectTM Natural User Interface 



This work was mainly funded by the Spanish Ministry of Economy and Competitiveness (MINECO) through the CHILDMNEMOS project (TIN2012-37381-C02-01) and cofinanced by the European Regional Development Fund (FEDER).

Other financial support was received from the Government of the Republic of Ecuador through the Scholarship Program of the Secretary of Higher Education, Science, Technology and Innovation (SENESCYT), the Conselleria d’Educació, Investigació, Cultura i Esport through the grant for consolidable research groups in favour of the Computer Graphics and Multimedia group of the ai2 (PI. Prof. M.-Carmen Juan; Ref. AICO/2017/041) (2017–2018), the Government of Aragon (Department of Industry and Innovation), and the European Social Fund for Aragon.

We would like to thank the following for their contributions: Jimena Bonilla; the users who participated in the study; and the reviewers for their valuable comments.


  1. 1.
    Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining “Gamification”. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (MindTrek 2011), pp. 9–15. ACM, New York (2011)Google Scholar
  2. 2.
    Wiemeyer, J., Kliem, A.: Serious games in prevention and rehabilitation - a new panacea for elderly people? Eur. Rev. Aging Phys. Act. 9, 41–50 (2012)CrossRefGoogle Scholar
  3. 3.
    Catalano, C.E., Luccini, A.M., Mortara, M.: Best practices for effective design and evaluation of serious games. Int. J. Serious Games 1, e1–e13 (2014)CrossRefGoogle Scholar
  4. 4.
    Whyte, E.M., Smyth, J.M., Scherf, K.S.: Designing serious game interventions for individuals with Autism. J. Autism Dev. Disord. 45, 3820–3831 (2015)CrossRefGoogle Scholar
  5. 5.
    Kim, K.-W., Choi, Y., You, H., Na, D.L., Yoh, M.-S., Park, J.-K., Seo, J.-H., Ko, M.-H.: Effects of a serious game training on cognitive functions in older adults. J. Am. Geriatr. Soc. 63, 603–605 (2015)CrossRefGoogle Scholar
  6. 6.
    Lezak, M.D.: Neuropsychological Assessment. Oxford University Press, New York (1995)Google Scholar
  7. 7.
    Rourke, B.P.: Arithmetic disabilities, specific and otherwise: a neuropsychological perspective. J. Learn. Disabil. 26, 214–226 (1993)CrossRefGoogle Scholar
  8. 8.
    Arai, S., Okamoto, Y., Fujioka, T., Inohara, K., Ishitobi, M., Matsumura, Y., Jung, M., Kawamura, K., Takiguchi, S., Tomoda, A., Wada, Y., Hiratani, M., Matsuura, N., Kosaka, H.: Altered frontal pole development affects self-generated spatial working memory in ADHD. Brain Dev. 38, 471–480 (2016)CrossRefGoogle Scholar
  9. 9.
    Dehn, M.J.: Cognitive processing deficits. In: Morris, R.J., Mather, N. (eds.) Evidence-Based Interventions for Students with Learning and Behavioral Challenges, pp. 258–287. Routledge, New York and London (2008)Google Scholar
  10. 10.
    Graham, J.A., Heywood, S.: The effects of elimination of hand gestures and of verbal codability on speech performance. Eur. J. Soc. Psychol. 5, 189–195 (1975)CrossRefGoogle Scholar
  11. 11.
    Rauscher, F.H., Krauss, R.M., Chen, Y.: Gesture, speech, and lexical access: the role of lexical movements in speech production. Psychol. Sci. 7, 226–231 (1996)CrossRefGoogle Scholar
  12. 12.
    Khan, R.Z., Ibraheem, N.A.: Hand gesture recognition: a literature review. Int. J. Artif. Intell. Appl. 3(4), 161–174 (2012)CrossRefGoogle Scholar
  13. 13.
    Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141, 152–165 (2015)CrossRefGoogle Scholar
  14. 14.
    Agrawal, S., Constandache, I., Gaonkar, S., Choudhury, R.R., Caves, K., DeRuyter, F.: Using mobile phones to write in air. In: Proceedings of the 7th ACM International Conference on Mobile Systems, Applications, and Services, Washington, DC, USA, pp. 15–28 (2011)Google Scholar
  15. 15.
    Park, T., Lee, J., Hwang, I., Yoo, C., Nachman, L., Song, J.: Egesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices. In: Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems, pp. 260–273. ACM, Seattle (2011)Google Scholar
  16. 16.
    Beh, J., Han, D.K., Durasiwami, R., Ko, H.: Hidden Markov model on a unit hyper-sphere space for gesture trajectory recognition. Pattern Recognit. Lett. 36, 144–153 (2014)CrossRefGoogle Scholar
  17. 17.
    Suk, H.I., Sin, B.K., Lee, S.W.: Hand gesture recognition based on dynamic Bayesian network framework. Pattern Recognit. 43(9), 3059–3072 (2010)CrossRefzbMATHGoogle Scholar
  18. 18.
    Yang, M.H., Ahuja, N., Tabb, M.: Extraction of 2D motion trajectories and its application to hand gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 24(8), 1061–1074 (2002)CrossRefGoogle Scholar
  19. 19.
    Shen, X.H., Hua, G., Williams, L., Wu, Y.: Dynamic hand gesture recognition: an exemplar-based approach from motion divergence fields. Image Vis. Comput. 30(3), 227–235 (2012)CrossRefGoogle Scholar
  20. 20.
    Patwardhan, K.S., Roy, S.D.: Hand gesture modelling and recognition involving changing shapes and trajectories, using a predictive eigentracker. Pattern Recognit. Lett. 28, 329–334 (2007)CrossRefGoogle Scholar
  21. 21.
    Shin, M.C., Tsap, L.V., Goldgof, D.B.: Gesture recognition using Bezier curves for visualization navigation from registered 3-D data. Pattern Recognit. 37(5), 1011–1024 (2004)CrossRefGoogle Scholar
  22. 22.
    Kuremoto, T., Kinoshita, Y., Feng, L., Watanabe, S., Kobayashi, K., Obayashi, M.: A gesture recognition system with retina-v1 model and one-pass dynamic programming. Neurocomputing 116, 291–300 (2012)CrossRefGoogle Scholar
  23. 23.
    Corradini, A.: Dynamic time warping for off-line recognition of a small gesture vocabulary. In: Proceedings of the IEEE International Workshop on Computer Vision (ICCVW 2001), pp. 82–89. IEEE (2001)Google Scholar
  24. 24.
    Breuer, P., Eckes, C., Müller, S.: Hand gesture recognition with a novel IR time-of-flight range camera–a pilot study. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2007. LNCS, vol. 4418, pp. 247–260. Springer, Heidelberg (2007). CrossRefGoogle Scholar
  25. 25.
    Zhang, Z.: Microsoft Kinect sensor and its effect. IEEE Multi Med. 19(2), 4–10 (2012)CrossRefGoogle Scholar
  26. 26.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: The IEEE Computer Vision and Pattern Recognition, pp. 116–124. ACM, New York (2011)Google Scholar
  27. 27.
    Martín-SanJosé, J.F., Juan, M.C., Gil-Gómez, J.A., Rando, N.: Flexible learning itinerary vs. linear learning itinerary. Sci. Comput. Program. 88, 3–21 (2014)CrossRefGoogle Scholar
  28. 28.
    Martín-SanJosé, J.F., Juan, M.C., Torres, E., Vicent, M.J.: Playful interaction for learning collaboratively and individually. J. Ambient Intell. Smart Environ. 6, 295–311 (2014)Google Scholar
  29. 29.
    Martín-SanJosé, J.F., Juan, M.C., Mollá, R., Vivó, R.: Advanced displays and natural user interfaces to support learning. Interact. Learn. Environ. 25(1), 17–34 (2017)CrossRefGoogle Scholar
  30. 30.
    Rodríguez-Andrés, D., Juan, M. C., Mollá, R., Méndez-López, M.: A 3D serious game for dental learning in higher education. In: Proceedings of the 17th IEEE International Conference on Advanced Learning Technologies (ICALT2017), pp. 111–115. IEEE (2017)Google Scholar
  31. 31.
    Homer, B., Kinzer, C., Plass, J., Letourneau, S., Hoffman, D., Bromley, M., Hayward, E., Turkay, S., Kornak, Y.: Moved to learn: the effects of interactivity in a Kinect-based literacy game for beginning readers. Comput. Educ. 74, 37–49 (2014)CrossRefGoogle Scholar
  32. 32.
    Lin, J., Sun, Q., Li, G., He, Y.: SnapBlocks: a snapping interface for assembling toy blocks with XBOX kinect. Multimed. Tools Appl. 73, 2009–2032 (2014)CrossRefGoogle Scholar
  33. 33.
    Sun, C., Zhang, T., Bao, B.K., Xu, C., Mei, T.: Discriminative exemplar coding for sign language recognition with Kinect. IEEE Trans. Cybern. 43, 1418–1428 (2013)CrossRefGoogle Scholar
  34. 34.
    Lee, G.C., Yeh, F.-H., Hsiao, Y.-H.: Kinect-based Taiwanese sign-language recognition system. Multimed. Tools Appl. 75, 261–279 (2016)CrossRefGoogle Scholar
  35. 35.
    Armin, K., Mehrana, Z., Fatemeh, D.: Using kinect in teaching children with hearing and visual impairment. In: Proceedings of the 4th International Conference on e-Learning and e-Teaching (ICELET 2013), pp. 86–90. IEEE (2013)Google Scholar
  36. 36.
    Retalis, S., Boloudakis, M., Altanis, G., Nikou, N.: Children with motor impairments play a kinect learning game: first findings from a pilot case in an authentic classroom environment. Interact. Des. Archit. 19, 91–104 (2014)Google Scholar
  37. 37.
    Luna-Oliva, L., Ortiz-Gutiérrez, R.M., Cano-de la Cuerda, R., Piédrola, R.M., Alguacil-Diego, I.M., Sánchez-Camarero, C., Martínez Culebras, M.D.C.: Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: a preliminary study. NeuroRehabilitation 33, 513–521 (2013)Google Scholar
  38. 38.
    Jordan, K., King, M., Hellersteth, S., Wirén, A., Mulligan, H.: Feasibility of using a humanoid robot for enhancing attention and social skills in adolescents with autism spectrum disorder. Int. J. Rehabil. Res. 36, 221–227 (2013)CrossRefGoogle Scholar
  39. 39.
    Moriguchi, Y., Kanda, T., Ishiguro, H., Shimada, Y., Itakura, S.: Can young children learn words from a robot? Interact. Stud. 12, 107–118 (2011)CrossRefGoogle Scholar
  40. 40.
    Movellan, J.R., Eckhardt, M., Virnes, M., Rodriguez, A.: Sociable robot improves toddler vocabulary skills. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, pp. 307–308. ACM, New York (2009)Google Scholar
  41. 41.
    Keren, G., Fridin, M.: Kindergarten social assistive robot (KindSAR) for children’s geometric thinking and metacognitive development in preschool education: a pilot study. Comput. Hum. Behav. 35, 400–412 (2014)CrossRefGoogle Scholar
  42. 42.
    Keren, G., Ben-David, A., Fridin, M.: Kindergarten assistive robotics (KAR) as a tool for spatial cognition development in pre-school education. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1084–1089. IEEE (2012)Google Scholar
  43. 43.
    Timms, M.J.: Letting artificial intelligence in education out of the box: educational cobots and smart classrooms. Int. J. Artif. Intell. Educ. 26, 701–712 (2016)CrossRefGoogle Scholar
  44. 44.
    Randell, C., Price, S., Rogers, Y., Harris, E., Fitzpatrick, G.: The ambient horn: designing a novel audio-based learning experience. Pers. Ubiquitous Comput. 8, 177–183 (2004)CrossRefGoogle Scholar
  45. 45.
    de Graaf, M.M.A., Allouch, S.B., Klamer, T.: Sharing a life with Harvey: exploring the acceptance of and relationship-building with a social robot. Comput. Hum. Behav. 43, 1–14 (2015)CrossRefGoogle Scholar
  46. 46.
    Juan, M.-C., Mendez-Lopez, M., Perez-Hernandez, E., Albiol-Perez, S.: Augmented reality for the assessment of children’s spatial memory in real settings. PLoS ONE 9, e113751 (2014)CrossRefGoogle Scholar
  47. 47.
    Mendez-Lopez, M., Perez-Hernandez, E., Juan, M.C.: Learning in the navigational space: age differences in a short-term memory for objects task. Learn. Individ. Differences 60, 11–22 (2016)CrossRefGoogle Scholar
  48. 48.
    Rodríguez-Andrés, D., Juan, M.-C., Méndez-López, M., Pérez-Hernández, E., Lluch, J.: MnemoCity task: assessment of children’s spatial memory using stereoscopy and virtual environments. PLoS ONE 11, e0161858 (2016)CrossRefGoogle Scholar
  49. 49.
    Cárdenas-Delgado, S., Méndez-López, M., Juan, M.-C., Pérez-Hernández, E., Lluch, J., Vivó, R.: Using a virtual maze task to assess spatial short-term memory in adults. In: Proceedings of the International Conference on Computer Graphics Theory and Applications, pp. 46–57. Scitepress (2017)Google Scholar
  50. 50.
    Lumsden, J., Edwards, E.A., Lawrence, N.S., Coyle, D., Munafò, M.R.: Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 4, e11 (2016)CrossRefGoogle Scholar
  51. 51.
    Reynolds, C.R., Bigler, E.D.: TOMAL test of memory and learning: examiner’s manual. Austin, TX: Pro-Ed [TOMAL Test de memoria y aprendizaje. Manual de interpretación (E. Goikoetxea, & Departamento I + D de TEA Ediciones, Adapters), (TEA Ediciones, Madrid)] (2001)Google Scholar
  52. 52.
    Brancal, M.F., Alcantud, F., Ferrer, A.M., Quiroga, M.E.: EDAF: Evaluación de la discriminación auditiva y fonológica. TEA Ediciones, Lebón, Madrid (2009)Google Scholar
  53. 53.
    Kamphaus, K.W., Perez-Hernandez, E., Sanchez-Sanchez, F.: Cuestionario de Evaluación Clínica de la Memoria. TEA Ediciones, Madrid (in press)Google Scholar
  54. 54.
    Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995)CrossRefGoogle Scholar
  55. 55.
    Lund, A.M.: Measuring usability with the USE questionnaire. Usability User Exp. Newslett. STC Usability SIG. 8(2), 1–4 (2001)Google Scholar
  56. 56.
    Vallejo, V., Wyss, P., Chesham, A., Mitache, A.V., Müri, R.M., Mosimann, U.P., Nef, T.: Evaluation of a new serious game based multitasking assessment tool for cognition and activities of daily living: comparison with a real cooking task. Comput. Hum. Behav. 70, 500–506 (2017)CrossRefGoogle Scholar
  57. 57.
    Tarnanas, I., Tsolaki, M., Nef, T.M., Müri, R., Mosimann, U.P.: Can a novel computerized cognitive screening test provide additional information for early detection of Alzheimer’s disease? Alzheimer’s Dementia 10, 790–798 (2014)CrossRefGoogle Scholar
  58. 58.
    Spooner, D., Pachana, N.: Ecological validity in neuropsychological assessment: a case for greater consideration in research with neurologically intact populations. Arch. Clin. Neuropsychol. 21, 327–337 (2006)CrossRefGoogle Scholar
  59. 59.
    Tarnanas, I., Schlee, W., Tsolaki, M., Müri, R., Mosimann, U., Nef, T.: Ecological validity of virtual reality daily living activities screening for early dementia: longitudinal study. JMIR Serious Games 1, e1 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Instituto Universitario de Automática e Informática IndustrialUniversitat Politècnica de ValènciaValenciaSpain
  2. 2.Departamento de Psicología y SociologíaUniversidad de ZaragozaZaragozaSpain
  3. 3.Departamento de Psicología Evolutiva y de la EducaciónUniversidad Autónoma de MadridMadridSpain

Personalised recommendations