Skip to main content
SpringerLink
Log in

Nadine Robot in Elderly Care Simulation Recreational Activity: Using Computer Vision and Observations for Analysis

  • Conference paper
  • First Online:
Human Aspects of IT for the Aged Population. Technology in Everyday Living (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13331))

Included in the following conference series:

Abstract

Many elderly residing in nursing homes have age-prevalent cognitive impairment and would benefit from regular social and cognitive engagements to maintain their well-being. However, manpower and resources to cater for both healthy elderly and their cognitively impaired co-inhabitants effectively are often inadequate in elderly care settings; therefore, other measures must be considered. Here, we subjectively and objectively evaluated the feasibility and impact of Nadine, our humanoid social robot, conducting Bingo games for a group of cognitively impaired elderly at a nursing home, compared to Staff-led sessions. The elderly engaged more and sought less attention from staff during Nadine-led Bingo games than during Staff-championed sessions. They were also calmer, happier, more at ease, and more communicative during Nadine-led games. These findings can guide future designs of humanoid robots that could be extended to become mobile robots to engage with the elderly of various functional and cognitive abilities and provide more manpower.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://alz.org.sg/dementia/singapore/.

  2. 2.

    http://dlib.net/.

  3. 3.

    https://github.com/open-mmlab/mmaction2.

References

  1. Algar, K., Woods, R.T., Windle, G.: Measuring the quality of life and well-being of people with dementia: a review of observational measures. Dementia 15(4), 832–857 (2016)

    Article  Google Scholar 

  2. Baka, E., Vishwanath, A., Mishra, N., Vleioras, G., Thalmann, N.M.: “Am i talking to a human or a robot?’’: a preliminary study of human’s perception in human-humanoid interaction and its effects in cognitive and emotional states. In: Gavrilova, M., Chang, J., Thalmann, N.M., Hitzer, E., Ishikawa, H. (eds.) CGI 2019. LNCS, vol. 11542, pp. 240–252. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22514-8_20

    Chapter  Google Scholar 

  3. Barrett, E., et al.: Evaluation of a companion robot for individuals with dementia: quantitative findings of the MARIO project in an Irish residential care setting. J. Gerontol. Nurs. 45(7), 36–45 (2019)

    Article  Google Scholar 

  4. Beck, A., Zhijun, Z., Magnenat-Thalmann, N.: Motion control for social behaviors. In: Magnenat-Thalmann, N., Yuan, J., Thalmann, D., You, B.-J. (eds.) Context Aware Human-Robot and Human-Agent Interaction. HIS, pp. 237–256. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-19947-4_11

    Chapter  Google Scholar 

  5. Beerens, H.C., et al.: The association between aspects of daily life and quality of life of people with dementia living in long-term care facilities: a momentary assessment study. Int. Psychogeriatr. 28(8), 1323–1331 (2016)

    Article  Google Scholar 

  6. Beerens, H.C., et al.: The relation between mood, activity, and interaction in long-term dementia care. Aging Mental Health 22(1), 26–32 (2018)

    Article  Google Scholar 

  7. Blond, L.: Studying robots outside the lab: HRI as ethnography. Paladyn J. Behav. Robot. 10(1), 117–127 (2019). https://doi.org/10.1515/pjbr-2019-0007

  8. Bridges, J., Gould, L., Hope, J., Schoonhoven, L., Griffiths, P.: The quality of interactions schedule (QuIS) and person-centred care: concurrent validity in acute hospital settings. Int. J. Nurs. Stud. Adv. 1, 100001 (2019)

    Article  Google Scholar 

  9. Casey, D., et al.: What people with dementia want: designing MARIO an acceptable robot companion. In: Miesenberger, K., Bühler, C., Penaz, P. (eds.) ICCHP 2016. LNCS, vol. 9758, pp. 318–325. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41264-1_44

    Chapter  Google Scholar 

  10. Chang, W., Šabanovic, S., Huber, L.: Use of seal-like robot PARO in sensory group therapy for older adults with dementia. In: 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 101–102 (2013)

    Google Scholar 

  11. Chang, W.L., Šabanović, S.: Exploring Taiwanese nursing homes as product ecologies for assistive robots. In: 2014 IEEE International Workshop on Advanced Robotics and its Social Impacts, pp. 32–37. IEEE (2014)

    Google Scholar 

  12. Cohen-Mansfield, J., Dakheel-Ali, M., Marx, M.S.: Engagement in persons with dementia: the concept and its measurement. Am. J. Geriatr. Psychiatry 17(4), 299–307 (2009)

    Article  Google Scholar 

  13. Cohen-Mansfield, J., Marx, M.S., Dakheel-Ali, M., Regier, N.G., Thein, K., Freedman, L.: Can agitated behavior of nursing home residents with dementia be prevented with the use of standardized stimuli? J. Am. Geriatr. Soc. 58(8), 1459–1464 (2010)

    Article  Google Scholar 

  14. Gaugler, J.E., Hobday, J.V., Savik, K.: The cares® observational tool: a valid and reliable instrument to assess person-centered dementia care. Geriatr. Nurs. 34(3), 194–198 (2013)

    Article  Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of European Conference on Computer Vision, pp. 770–778 (2016)

    Google Scholar 

  16. Huisman, C., Kort, H.: Two-year use of care robot Zora in Dutch nursing homes: an evaluation study. In: Healthcare, vol. 7, p. 31. Multidisciplinary Digital Publishing Institute (2019)

    Google Scholar 

  17. Khosla, R., Chu, M.T.: Embodying care in matilda: an affective communication robot for emotional wellbeing of older people in Australian residential care facilities. ACM Trans. Manag. Inf. Syst. (TMIS) 4(4), 1–33 (2013)

    Article  Google Scholar 

  18. Kinney, J.M., Rentz, C.A.: Observed well-being among individuals with dementia: memories in the making, an art program, versus other structured activity. Am. J. Alzheimer’s Disease Other Dementias® 20(4), 220–227 (2005)

    Google Scholar 

  19. Law, M., et al.: Developing assistive robots for people with mild cognitive impairment and mild dementia: a qualitative study with older adults and experts in aged care. BMJ Open 9(9), e031937 (2019)

    Article  Google Scholar 

  20. Lee, H.R., Tan, H., Šabanović, S.: That robot is not for me: addressing stereotypes of aging in assistive robot design. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 312–317. IEEE (2016)

    Google Scholar 

  21. Li, J., Louie, W.G., Mohamed, S., Despond, F., Nejat, G.: A user-study with tangy the bingo facilitating robot and long-term care residents. In: 2016 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pp. 109–115 (2016)

    Google Scholar 

  22. Li, J., Louie, W.Y.G., Mohamed, S., Despond, F., Nejat, G.: A user-study with tangy the bingo facilitating robot and long-term care residents. In: 2016 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pp. 109–115. IEEE (2016)

    Google Scholar 

  23. Liu, Z., Luo, P., Wang, X., Tang, X.: Large-scale CelebFaces attributes (CelebA) dataset (2018). Accessed 15 Aug 2018

    Google Scholar 

  24. Louie, W.Y.G., Li, J., Mohamed, C., Despond, F., Lee, V., Nejat, G.: Tangy the robot bingo facilitator: a performance review. J. Med. Dev. 9(2) (2015)

    Google Scholar 

  25. Louie, W.Y.G., Nejat, G.: A social robot learning to facilitate an assistive group-based activity from non-expert caregivers. Int. J. Soc. Robot. 12, 1–18 (2020)

    Article  Google Scholar 

  26. Magnenat Thalmann, N., Zhang, Z.: Social robots and virtual humans as assistive tools for improving our quality of life. In: 2014 5th International Conference on Digital Home, pp. 1–7. IEEE (2014)

    Google Scholar 

  27. Mannion, A., et al.: Introducing the social robot MARIO to people living with dementia in long term residential care: reflections. Int. J. Soc. Robot. 12, 1–13 (2019)

    Google Scholar 

  28. Martinez-Martin, E., Escalona, F., Cazorla, M.: Socially assistive robots for older adults and people with autism: an overview. Electronics 9(2), 367 (2020)

    Article  Google Scholar 

  29. Melkas, H., Hennala, L., Pekkarinen, S., Kyrki, V.: Impacts of robot implementation on care personnel and clients in elderly-care institutions. Int. J. Med. Inform. 134, 104041 (2020)

    Article  Google Scholar 

  30. Mishra, N., Ramanathan, M., Satapathy, R., Cambria, E., Magnenat Thalmann, N.: Can a humanoid robot be part of the organizational workforce? A user study leveraging sentiment analysis. In: 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1–7. IEEE (2019)

    Google Scholar 

  31. Molinari, V., et al.: Impact of serious mental illness online training for certified nursing assistants in long term care. Gerontol. Geriatr. Educ. 38(4), 359–374 (2017)

    Article  Google Scholar 

  32. Neven, L.: ‘But obviously not for me’: robots, laboratories and the defiant identity of elder test users. Sociol. Health Illn. 32(2), 335–347 (2010)

    Article  Google Scholar 

  33. Norman, R.: Observations of the experiences of people with dementia on general hospital wards. J. Res. Nurs. 11(5), 453–465 (2006)

    Article  Google Scholar 

  34. Paudel, A., Resnick, B., Galik, E.: The quality of interactions between staff and residents with cognitive impairment in nursing homes. Am. J. Alzheimer’s Disease Other Dementias® 35, 1533317519863259 (2020)

    Google Scholar 

  35. Ramanathan, M., Mishra, N., Thalmann, N.M.: Nadine humanoid social robotics platform. In: Gavrilova, M., Chang, J., Thalmann, N.M., Hitzer, E., Ishikawa, H. (eds.) CGI 2019. LNCS, vol. 11542, pp. 490–496. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22514-8_49

    Chapter  Google Scholar 

  36. Šabanović, S., Bennett, C.C., Chang, W.L., Huber, L.: Paro robot affects diverse interaction modalities in group sensory therapy for older adults with dementia. In: 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), pp. 1–6. IEEE (2013)

    Google Scholar 

  37. Šabanović, S., Chang, W.L.: Socializing robots: constructing robotic sociality in the design and use of the assistive robot PARO. AI Soc. 31(4), 537–551 (2016)

    Article  Google Scholar 

  38. Alzheimer’s Society: Observation as an evaluation tool for people with advanced dementia. https://www.alzheimers.org.uk/dementia-professionals/dementia-experience-toolkit/real-life-examples/strategy-evaluation/observation-evaluation-tool-people-advanced-dementia

  39. Business Case Studies: Meet stevie the social robot that holds bingo lessons in a care home (2020). https://businesscasestudies.co.uk/meet-stevie-the-social-robot-that-holds-bingo-lessons-in-a-care-home/

  40. Teed, Z., Deng, J.: RAFT: recurrent all-pairs field transforms for optical flow. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 402–419. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_24

    Chapter  Google Scholar 

  41. Thompson, C., Mohamed, S., Louie, W.Y.G., He, J.C., Li, J., Nejat, G.: The robot tangy facilitating trivia games: a team-based user-study with long-term care residents. In: 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pp. 173–178. IEEE (2017)

    Google Scholar 

  42. Tuisku, O., Pekkarinen, S., Hennala, L., Melkas, H.: Robots do not replace a nurse with a beating heart. Inf. Technol. People (2019)

    Google Scholar 

  43. Wang, L., et al.: Temporal segment networks for action recognition in videos. IEEE Trans. Pattern Anal. Mach. Intell. 41(11), 2740–2755 (2018)

    Article  Google Scholar 

  44. Xiao, Y., Zhang, Z., Beck, A., Yuan, J., Thalmann, D.: Human-robot interaction by understanding upper body gestures. Presence: Teleoperators Virtual Environ. 23(2), 133–154 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Nanyang Technological University, Singapore, Singapore

    Nidhi Mishra, Gauri Tulsulkar & Nadia Magnenat Thalmann

  2. Miralab, University of Geneva, Geneva, Switzerland

    Nadia Magnenat Thalmann

Authors
  1. Nidhi Mishra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gauri Tulsulkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadia Magnenat Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nidhi Mishra .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Qin Gao

  2. Chongqing University, Chongqing, China

    Jia Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mishra, N., Tulsulkar, G., Magnenat Thalmann, N. (2022). Nadine Robot in Elderly Care Simulation Recreational Activity: Using Computer Vision and Observations for Analysis. In: Gao, Q., Zhou, J. (eds) Human Aspects of IT for the Aged Population. Technology in Everyday Living. HCII 2022. Lecture Notes in Computer Science, vol 13331. Springer, Cham. https://doi.org/10.1007/978-3-031-05654-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05654-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05653-6

  • Online ISBN: 978-3-031-05654-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation