The Smartkuber Case Study: Lessons Learned from the Development of an Augmented Reality Serious Game for Cognitive Screening

  • Costas BoletsisEmail author
  • Simon McCallum
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10324)


In this work, we present a case study, examining the design, development, and evaluation of an Augmented Reality serious game for cognitive screening (namely Smartkuber), which aims to provide reliable and motivating cognitive screening for the elderly. This case study can be of interest for the game designers and researchers, allowing them to build on previous experiences and lessons learned. Smartkuber’s development process took place in four stages: (1) analysing the state of the art and defining characteristics, (2) setting up and examining the interaction method, (3) adding and evaluating the game content, and (4) evaluating cognitive screening performance and future direction. The “lessons learned” around the design and development of serious games for cognitive screening are discussed, with focus on Augmented Reality, interaction, test validity, and game motivation aspects.


Augmented Reality Cognitive screening Elderly Serious games 


  1. 1.
    Barakova, E., van Wanrooij, G., van Limpt, R., Menting, M.: Using an emergent system concept in designing interactive games for autistic children. In: Proceedings of the 6th International Conference on Interaction Design and Children, IDC 2007, pp. 73–76 (2007)Google Scholar
  2. 2.
    Billinghurst, M., Kato, H., Poupyrev, I.: Tangible augmented reality. In: Proceedings of ACM SIGGRAPH Asia, pp. 1–10 (2008)Google Scholar
  3. 3.
    Boletsis, C., McCallum, S.: Augmented reality cube game for cognitive training: an interaction study. Stud. Health Technol. Inf. 200, 81–87 (2014)Google Scholar
  4. 4.
    Boletsis, C., McCallum, S.: Connecting the player to the doctor: utilising serious games for cognitive training & screening. DAIMI PB 597, 5–8 (2015)Google Scholar
  5. 5.
    Boletsis, C., McCallum, S.: Augmented reality cubes for cognitive gaming: preliminary usability and game experience testing. Int. J. Serious Games 3(1), 3–18 (2016)CrossRefGoogle Scholar
  6. 6.
    Boletsis, C., McCallum, S.: Evaluating a gaming system for cognitive screening and sleep duration assessment of elderly players: a pilot study. In: Bottino, R., Jeuring, J., Veltkamp, R.C. (eds.) GALA 2016. LNCS, vol. 10056, pp. 107–119. Springer, Cham (2016). doi: 10.1007/978-3-319-50182-6_10 CrossRefGoogle Scholar
  7. 7.
    Boletsis, C., McCallum, S.: Smartkuber: a cognitive training game for cognitive health screening of elderly players. Games Health J. 5(4), 241–251 (2016)CrossRefGoogle Scholar
  8. 8.
    Brooke, J.: SUS-A quick and dirty usability scale. In: Usability Evaluation in Industry, pp. 189–194. Taylor & Francis (1996)Google Scholar
  9. 9.
    Cockrell, J.R., Folstein, M.F.: Mini-mental state examination (MMSE). Psychopharmacol. Bull. 24(4), 689–692 (1988)Google Scholar
  10. 10.
    Connolly, A., Gaehl, E., Martin, H., Morris, J., Purandare, N.: Underdiagnosis of dementia in primary care: variations in the observed prevalence and comparisons to the expected prevalence. Aging Mental Health 15(8), 978–984 (2011)CrossRefGoogle Scholar
  11. 11.
    Correa, A., de Assis, G.A., Nascimento, M.d., Ficheman, I., de Deus Lopes, R.: GenVirtual: an augmented reality musical game for cognitive and motor rehabilitation. In: Virtual Rehabilitation, pp. 1–6 (2007)Google Scholar
  12. 12.
    Frazer, A., Argles, D., Wills, G.: Assessing the usefulness of mini-games as educational resources. In: ALT-C 2007: Beyond Control (2007)Google Scholar
  13. 13.
    Gamberini, L., Martino, F., Seraglia, B., Spagnolli, A., Fabregat, M., Ibanez, F., Alcaniz, M., Andres, J.M.: Eldergames project: an innovative mixed reality table-top solution to preserve cognitive functions in elderly people. In: Proceedings of the 2nd Conference on Human System Interactions, pp. 164–169 (2009)Google Scholar
  14. 14.
    Gerling, K., Masuch, M.: When gaming is not suitable for everyone: playtesting Wii games with frail elderly. In: Proceeding of the 1st Workshop on Game Accessibility: Xtreme Interaction Design (FDG 2011), Bordeaux, France (2011)Google Scholar
  15. 15.
    Gregor, P., Newell, A.F.: Designing for dynamic diversity: making accessible interfaces for older people. In: Proceedings of the 2001 EC/NSF Workshop on Universal Accessibility of Ubiquitous Computing: Providing for the Elderly, WUAUC 2001, pp. 90–92 (2001)Google Scholar
  16. 16.
    Gregor, P., Newell, A.F., Zajicek, M.: Designing for dynamic diversity: interfaces for older people. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, ASSETS 2002, pp. 151–156 (2002)Google Scholar
  17. 17.
    Heller, R., Jorge, J., Guedj, R.: EC/NSF workshop on universal accessibility of ubiquitous computing: providing for the elderly event report. In: Proceedings of the 2001 EC/NSF Workshop on Universal Accessibility of Ubiquitous Computing: Providing for the Elderly, WUAUC 2001, pp. 1–10. ACM (2001)Google Scholar
  18. 18.
    IJsselsteijn, W., De Kort, Y., Poels, K.: The game experience questionnaire: development of a self-report measure to assess the psychological impact of digital games. Manuscript in preparation. FUGA technical report Deliverable 3.3 (2013)Google Scholar
  19. 19.
    IJsselsteijn, W., De Kort, Y., Poels, K., Jurgelionis, A., Bellotti, F.: Characterising and measuring user experiences in digital games. In: International Conference on Advances in Computer Entertainment Technology, vol. 2, p. 27 (2007)Google Scholar
  20. 20.
    IJsselsteijn, W., van den Hoogen, W., Klimmt, C., de Kort, Y., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., Vorderer, P.: Measuring the experience of digital game enjoyment. In: Proceedings of Measuring Behavior, pp. 88–89 (2008)Google Scholar
  21. 21.
    Ijsselsteijn, W., Nap, H.H., de Kort, Y., Poels, K.: Digital game design for elderly users. In: Proceedings of the 2007 Conference on Future Play 2007, pp. 17–22 (2007)Google Scholar
  22. 22.
    Kato, P.: What do you mean when you say your serious game has been validated? Experimental vs. Test Validity (2013). Accessed 8 Apr 2017
  23. 23.
    Manera, V., Petit, P.D., Derreumaux, A., Orvieto, I., Romagnoli, M., Lyttle, G., David, R., Robert, P.: ‘Kitchen and cooking’, a serious game for mild cognitive impairment and Alzheimer’s disease: a pilot study. Front. Aging Neurosci. 7(24), 1–10 (2015)Google Scholar
  24. 24.
    McCallum, S., Boletsis, C.: Augmented reality & gesture-based architecture in games for the elderly. In: pHealth, Studies in Health Technology and Informatics, vol. 189, pp. 139–144. IOS Press (2013)Google Scholar
  25. 25.
    McCallum, S., Boletsis, C.: Dementia games: a literature review of dementia-related serious games. In: Ma, M., Oliveira, M.F., Petersen, S., Hauge, J.B. (eds.) SGDA 2013. LNCS, vol. 8101, pp. 15–27. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40790-1_2 CrossRefGoogle Scholar
  26. 26.
    McCallum, S., Boletsis, C.: A taxonomy of serious games for dementia. In: Schouten, B., Fedtke, S., Bekker, T., Schijven, M., Gekker, A. (eds.) Games for Health, pp. 219–232. Springer, Wiesbaden (2013). doi: 10.1007/978-3-658-02897-8_17 CrossRefGoogle Scholar
  27. 27.
    Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE Trans. Inf. Syst. 77(12), 1321–1329 (1994)Google Scholar
  28. 28.
    Nasreddine, Z.S., Phillips, N.A., Bedirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J.L., Chertkow, H.: The montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 53(4), 695–699 (2005)CrossRefGoogle Scholar
  29. 29.
    Neistadt, M.: A critical analysis of occupational therapy approaches for perceptual deficits in adults with brain injury. Am. J. Occup. Ther. 44(4), 299–304 (1990)CrossRefGoogle Scholar
  30. 30.
    Scanlon, L., O’Shea, E., O’Caoimh, R., Timmons, S.: Usability and validity of a battery of computerised cognitive screening tests for detecting cognitive impairment. Gerontology 62(2), 247–252 (2016)CrossRefGoogle Scholar
  31. 31.
    Sharlin, E., Itoh, Y., Watson, B., Kitamura, Y., Sutphen, S., Liu, L.: Cognitive cubes: a tangible user interface for cognitive assessment. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2002, pp. 347–354 (2002)Google Scholar
  32. 32.
    Sottas, P.E., Robinson, N., Rabin, O., Saugy, M.: The athlete biological passport. Clin. Chem. 57(7), 969–976 (2011)CrossRefGoogle Scholar
  33. 33.
    Tong, T., Guana, V., Jovanovic, A., Tran, F., Mozafari, G., Chignell, M., Stroulia, E.: Rapid deployment and evaluation of mobile serious games: a cognitive assessment case study. Procedia Comput. Sci. 69, 96–103 (2015)CrossRefGoogle Scholar
  34. 34.
    Waldemar, G., Phung, K., Burns, A., Georges, J., Hansen, F.R., Iliffe, S., Marking, C., Rikkert, M.O., Selmes, J., Stoppe, G., Sartorius, N.: Access to diagnostic evaluation and treatment for dementia in europe. Int. J. Geriatr. Psychiatry 22(1), 47–54 (2007)CrossRefGoogle Scholar
  35. 35.
    Zhou, Z., Cheok, A., Pan, J.: 3D story cube: an interactive tangible user interface for storytelling with 3D graphics and audio. Pers. Ubiquit. Comput. 8(5), 374–376 (2004)CrossRefGoogle Scholar
  36. 36.
    Zucchella, C., Sinforiani, E., Tassorelli, C., Cavallini, E., Tost-Pardell, D., Grau, S., Pazzi, S., Puricelli, S., Bernini, S., Bottiroli, S., et al.: Serious games for screening pre-dementia conditions: from virtuality to reality? A pilot project. Funct. Neurol. 29(3), 153–158 (2014)Google Scholar
  37. 37.
    Zygouris, S., Giakoumis, D., Votis, K., Doumpoulakis, S., Ntovas, K., Segkouli, S., Karagiannidis, C., Tzovaras, D., Tsolaki, M.: Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J. Alzheimers Dis. 44(4), 1333–1347 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.SINTEF DigitalOsloNorway
  2. 2.Norwegian University of Science and TechnologyGjøvikNorway

Personalised recommendations