Technological advances are increasing the range of applications for artificial intelligence, especially through its embodiment within humanoid robotics platforms. This promotes the development of novel systems for automated screening of neurological conditions to assist the clinical practitioners in the detection of early signs of mild cognitive impairments. This article presents the implementation and the experimental validation of the first robotic system for cognitive assessment, based on one of the most popular platforms for social robotics, Softbank “Pepper”, which administers and records a set of multi-modal interactive tasks to engage the user cognitive abilities. The robot intelligence is programmed using the state-of-the-art IBM Watson AI Cloud services, which provide the necessary capabilities for improving the social interaction and scoring the tests. The system has been tested by healthy adults (N = 35) and we found a significant correlation between the automated scoring and one of the most widely used Paper-and-Pencil tests. We conclude that the system can be considered as a screening instrument for cognitive assessment.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Feil-Seifer D, Mataric M J. Defining socially assistive robotics. Proceedigns of the 9th International Conference on Rehabilitation Robotics, Chicago, IL, USA, 2005, 465–468.
Tapus A, Mataric M J, Scasselati B. Socially assistive robotics [Grand Challenges of Robotics]. IEEE Robotics & Automation Magazine, 2007, 14, 35–42.
Mataric M J, Scassellati B. Socially assistive robotics. In Siciliano B, Khatib O eds., Springer Handbook of Robotics, Cham, 2016, 1973–1994.
Furht B, Escalante A. Handbook of Cloud Computing, Springer, New York, NY, USA, 2010.
Hu G, Tay W P, Wen Y. Cloud robotics: Architecture, challenges and applications. IEEE Network, 2012, 26, 21–28.
Kehoe B, Patil S, Abbeel P, Goldberg K. A survey of research on cloud robotics and automation. IEEE Transactions on Automation Science and Engineering, 2015, 12, 398–409.
Novoa J, Wuth J, Escudero J P, Fredes J, Mahu R, Yoma N B. DNN-HMM based automatic speech recognition for HRI scenarios. Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 150–159.
Di Nuovo A, Broz F, Wang N, Belpaeme T, Cangelosi A, Jones R, Esposito R, Cavallo F, Dario P. The multi-modal interface of Robot-Era multi-robot services tailored for the elderly. Intelligent Service Robotics, 2018, 11, 109–126.
Wang N, Di Nuovo A, Cangelosi A, Jones R. Temporal patterns in multi-modal social interaction between elderly users and service robot. Interaction Studies, 2019, 20, 1–9.
Di Nuovo A, Broz F, Cavallo F, Dario P. New frontiers of service robotics for active and healthy ageing. International Journal of Social Robotics, 2016, 8, 353–354.
Cavallo F, Esposito R, Limosani R, Manzi A, Bevilacqua R, Felici E, Di Nuovo A, Cangelosi A, Lattanzio F, Dario P. Robotic services acceptance in smart environments with older adults: User satisfaction and acceptability study. Journal of Medical Internet Research, 2018, 20, 264.
Iroju O, Ojerinde O, Ikono R. State of the art: A study of human-robot interaction in healthcare. International Journal of Information Engineering and Electronic Business, 2017, 3, 43–55.
Conti D, Di Nuovo S, Trabia G, Buono S, Di Nuovo A. Use of robotics to stimulate imitation in children with autism spectrum disorder: A pilot study in a clinical setting. Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN, Kobe, Japan, 2015, 1–6.
Conti D, Di Nuovo S, Buono S, Di Nuovo A. Robots in education and care of children with developmental disabilities: A study on acceptance by experienced and future professionals. International Journal of Social Robotics, 2017, 9, 51–62.
Conti D, Cirasa C, Di Nuovo S, Di Nuovo A. “Robot, tell me a tale!”: A social robot as tool for teachers in kindergarten. Interaction Studies, 2019, 20, 1–16.
Rabbitt S M, Kazdin A E, Scassellati B. Integrating socially assistive robotics into mental healthcare interventions: Applications and recommendations for expanded use. Clinical Psychology Review, 2015, 35, 35–16.
Boumans R, van Meulen F, Hindriks K, Neerincx M, Olde Rikkert M. Proof of concept of a cocial robot for patient reported outcome measurements in elderly persons. Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 73–74.
Conti D, Trubia G, Buono S, Di Nuovo S, Di Nuovo A. Evaluation of a robot-assisted therapy for children with autism and intellectual disability. Proceedings of 19th Annual Conference on Towards Autonomous Robotic Systems, Bristol, UK, 2018, 405–415.
Di Nuovo A, Conti D, Trubia G, Buono S, Di Nuovo S. Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability. Robotics, 2018, 7, https://doi.org/10.3390/robotics7020025.
Petric F, Miklic D, Kovacic Z. Robot-assisted autism spectrum disorder diagnostics using POMDPs. Proceedings of Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, 2017, 369–370.
Wijayasinghe I B, Ranatunga I, Balakrishnan N, Bugnariu, N, Popa D O. Human-robot gesture analysis for objective assessment of autism spectrum disorder. International Journal of Social Robotics, 2016, 8, 695–707.
Kojima H, Takaeda K, Nihel M, Sadohara K, Ohnaka S, Inoue T. Acquisition and evaluation of a human-robot elderly spoken dialog corpus for developing computerized cognitive assessment systems. Journal of the Acoustical Society of America, 2016, 140, 2963–2963.
Varrasi S, Di Nuovo S, Conti D, Di Nuovo A. A social robot for cognitive assessment. Proceedings of Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 269–270.
Feingold Polak R, Elishay A, Shahar Y, Stein M, Edan Y, Levy-Tzedek S. Differences between young and old users when interacting with a humanoid robot: A qualitative usability study. Proceedings of Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 2018, 107–108.
Scassellati B, Admoni H, Mataric M. Robots for use in autism research. Annual Review of Biomedical Engineering, 2012, 14, 275–294.
Desideri L, Ottaviani C, Malavasi M, di Marzio R, Bonifacci, P. Emotional processes in human-robot interaction during brief cognitive testing. Computers in Human Behavior, 2019, 90, 331–342.
Rossi S, Santangelo G, Staffa M, Varrasi S, Conti D, Di Nuovo A. Psychometric evaluation supported by a social robot: Personality factors and technology acceptance. Proceedings of the 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China, 2018, 802–807.
Petersen R C. Mild cognitive impairment as a diagnostic entity. Journal of Internal Medicine, 2004, 256, 183–194.
Luis C, Loewenstein D, Acevedo A, Barker W W, Duara R. Mild cognitive impairment: Directions for future research. Neurology, 2003, 61, 438–444.
Landau S M, Harvey D, Madison C M, Reiman E M, Foster N L, Aisen P S, Petersen R C, Shaw L M, Trojanowski J Q, Jack C R, Weiner M W, Jagust W J. Comparing predictors of conversion and decline in mild cognitive impairment. Neurology, 2010, 75, 230–238.
Di Nuovo S, De La Cruz V M, Conti D, Buono S, Di Nuovo A. Mental imagery: Rehabilitation through simulation. Life Span and Disability, 2014, 17, 89–118.
Caraci F, Castellano S, Salomone S, Drago F, Bosco P, Di Nuovo S. Searching for disease-modifying drugs in AD: Can we combine neuropsychological tools with biological markers? CNS & Neurological Disorders — Drug Targets, 2014, 13, 173–186.
Fisekovic S, Memic A, Pasalic A. Correlation between MoCA and MMSE for the assessment of cognition in schizophrenia. Acta Informatica Medica, 2012, 20, 186–189.
Moirand R, Galvao F, Lecompte M, Poulet E, Haesebaert F, Brunelin J. Usefulness of the Montreal Cognitive Assessment (MoCA) to monitor cognitive impairments in depressed patients receiving electroconvulsive therapy. Psychiatry Research, 2018, 259, 476–481.
Pandey A K, Gelin R A mass-produced sociable humanoid robot: Pepper: The first machine of its kind. IEEE Robotics & Automation Magazine, 2018, 25, 40–48.
Varrasi S, Di Nuovo S, Conti D, Di Nuovo A. Social robots as psychometric tools for cognitive assessment: A pilot test. Proceedings of 10th International Workshop on Human Friendly Robotics, Naples, Italy, 2019, 99–112.
Varrasi S, Lucas A, Soranzo A, McNamara J, Di Nuovo A. IBM cloud services enhance automatic cognitive assessment via human-robot interaction. In Carbone G, Ceccarelli M, Pisla D eds., New Trends in Medical and Service Robotics, Cassino, Italy, 2019, 169–176.
Nasreddine Z S, Phillips N A, Bédirian V, Charbonneau S, Whitehead V, Collin I, Cummings J L, Chertkow H. The montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 2005, 53, 695–699.
Nasreddine Z. MoCA Montreal Cognitive Assessment, [2018-10-18], https://doi.org/www.mocatest.org
Olson R A, Chhanabhai T, McKenzie M. Feasibility study of the Montreal Cognitive Assessment (MoCA) in patients with brain metastases. Supportive Care in Cancer, 2008, 16, 1273–1278.
Wong A, Xiong Y Y, Kwan P W L, Chan A Y Y, Lam W W M, Wang K, Chu W C W, Nyenhuis D L, Nasreddine Z, Wong L K S, Mok V C T. The validity, reliability and clinical utility of the Hong Kong Montreal Cognitive Assessment (HK-MoCA) in patients with cerebral small vessel disease. Dementia and Geriatric Cognitive Disorders, 2009, 28, 81–87.
Dalrymple-Alford J C, MacAskill M R, Nakas C T, Livingston L, Graham C, Crucian G P, Melzer T R, Kirwan, J, Keenan R, Wells S, Porter R J, Watts R, Anderson T J. The MoCA Well-suited screen for cognitive impairment in Parkinson disease. Neurology, 2010, 75, 1717–1725.
Videnovic A, Bernard B, Fan W, Jaglin J, Leurgans S, Shannon K M. The Montreal Cognitive Assessment as a screening tool for cognitive dysfunction in Huntington’s disease. Movement Disorders, 2010, 25, 401–404.
Bertrand J-A, Génier Marchand D, Postuma R B, Gagnon J-F. Cognitive dysfunction in rapid eye movement sleep behavior disorder. Sleep and Biological Rhythms, 2013, 11, 21–26.
Carson N, Leach L, Murphy KJ. A re-examination of Montreal Cognitive Assessment (MoCA) cutoff scores. International Journal of Geriatric Psychiatry, 2017, 33, 379–388.
Chertkow H, Nasreddine Z S, Johns E, Phillips N A, McHenry C. The Montreal Cognitive Assessment (MoCA): Validation of alternate forms and new recommendations for education corrections. Alzheimer’s and Dementia, 2011, 7, S157.
Xu T (Linger), Zhang H, Yu C. See you see me: The role of eye contact in multimodal human-robot Interaction. ACM Transactions on Interactive Intelligent Systems, 2016, 6, 1–22.
Sciutti A, Rea F, Sandini G. When you are young, (robot&#c2019;s) looks matter. Developmental changes in the desired properties of a robot friend. Proceedigns of the 23rd IEEE International Symposium on Robot and Human Interactive Communication (IEEE RO-MAN), Edinburgh, Scotland, UK, 2014, 567–573.
Streiner D L. Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 2003, 80, 99–103.
Cohen J. Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates, Hillsdale, New Jersey, USA, 1988.
Landau S, Everitt B S. A Handbook of Statistical Analyses Using SPSS, CRC Press, Boca Raton, FL, USA, 2004.
Field A. Discovering Statistics Using IBM SPSS Statistics, SAGE Publications, Los Angeles, CA, USA, 2013.
Lakens D, Scheel A M, Isager P M. Equivalence testing for psychological research: A tutorial. Advances in Methods and Practices in Psychological Science, 2018, 1, 259–269.
Purpura J E. An analysis of the relationships between Ttest takers’ cognitive and metacognitive strategy use and second language test performance. Language Learning, 1997, 47, 289–325.
The authors gratefully thank all university staff and students who participated in this study. This work won the IBM Shared University Research (SUR) Award. The work of Daniela Conti and Alessandro Di Nuovo was supported by the European Union’s H2020 research and innovation program under the MSCA-Individual Fellowship grant agreement no. 703489 (CARER-AID). Alessandro Di Nuovo was also partially supported by the EPSRC through project grant EP/P030033/1 (NUMBERS).
Rights and permissions
About this article
Cite this article
Di Nuovo, A., Varrasi, S., Lucas, A. et al. Assessment of Cognitive skills via Human-robot Interaction and Cloud Computing. J Bionic Eng 16, 526–539 (2019). https://doi.org/10.1007/s42235-019-0043-2
- socially assistive robotics
- brief cognitive testing
- human-robot interaction
- neurological screening
- cloud computing