Advertisement

MemHolo: mixed reality experiences for subjects with Alzheimer’s disease

  • Beatrice AruannoEmail author
  • Franca Garzotto
Article
  • 20 Downloads

Abstract

HoloLens is the most recent and advanced forms of wearable Mixed Reality (MR) technology. It enables the user wearing a head-mounted device to experience 3D holographic objects “inside” the visualization of the real environment where he or she is located. Existing HoloLens applications have been developed in domains such as data visualization, entertainment, industrial training, education, and tourism, but the use of this technology in the arena of mental health is largely unexplored. The paper presents a HoloLens-based system called MemHolo that addresses persons with mild Alzheimer’s Disease (AD). AD is associated to a chronic progressive neurodegenerative process that severely affects cognitive functioning (especially memory) and some motor functions. MemHolo is intended to be used as a cognitive training tool to practice short-term and spatial memory in a safe and controlled virtual environment, and to mitigate the effects of mental decline. The paper discusses the design process of MemHolo, and describes three evaluation studies on progressive prototypes. To our knowledge, MemHolo is the first HoloLens application designed natively for persons with AD. Our empirical work sheds a light on how these people experience HoloLens applications, highlights some challenges and potential benefits of using MR technology in the AD arena, and may pave the ground towards new forms of treatment.

Keywords

Mixed reality Augmented reality HoloLens Elderly Alzheimer disease (AD) Cognitive training 

Notes

Acknowledgments

We thank Dr. Valoriani and Dr. Oliveto from Sixth Element for their collaboration in the initial phase of our research. We are grateful Dr. Olivieri and his team at the Gerontological Division at the Hospital in Palermo for their contribution to MemHolo design process. Special thanks go to the staff and guests of the Filo di Arianna Center for their help during the evaluation process.

References

  1. 1.
    Abe M, Yoshimura T, Koizumi S, Hasegawa N, Osaki T, Yasukawa N, Koba K, Moriya K, Sakai T (2005) Virtual forest: design and evaluation of a walk-through system for forest education. J For Res 10:189–197.  https://doi.org/10.1007/s10310-004-0131-x CrossRefGoogle Scholar
  2. 2.
    Alzheimer Association (2016) 2016 Alzheimer’s disease facts and figures. Alzheimers Dement (12):1–80.  https://doi.org/10.1016/j.jalz.2016.03.001
  3. 3.
    Alzheimer’s Disease Fact Sheet. https://www.nia.nih.gov/health/alzheimers-disease-fact-sheet. Accessed 3 May 2018
  4. 4.
    American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, ArlingtonCrossRefGoogle Scholar
  5. 5.
    Aruanno B, Garzotto F, Torelli E, Vona F (2018) HoloLearn - wearable mixed reality for people with neurodevelopmental disorders (NDD). In: ASSETS 2018: The 20th International ACM SIGACCESS Conference of Computers and AccessibilityGoogle Scholar
  6. 6.
    Astur RS, Taylor LB, Mamelak AN, Philpott L, Sutherland RJ (2002) Humans with hippocampus damage display severe spatial memory impairments in a virtual Morris water task. Behav Brain Res 132:77–84.  https://doi.org/10.1016/S0166-4328(01)00399-0 CrossRefGoogle Scholar
  7. 7.
    Blaschke CM, Freddolino PP, Mullen EE (2009) Ageing and technology: a review of the research literature. Br J Soc Work 39:641–656.  https://doi.org/10.1093/bjsw/bcp025 CrossRefGoogle Scholar
  8. 8.
    Brooke J (1996) SUS - a quick and dirty usability scale. Usability Eval Ind 189:4–7.  https://doi.org/10.1002/hbm.20701 CrossRefGoogle Scholar
  9. 9.
    Burgess N (2006) Spatial memory: how egocentric and allocentric combine. Trends Cogn Sci.  https://doi.org/10.1016/j.tics.2006.10.005
  10. 10.
    Byagowi A, Mohaddes D, Moussavi Z (2014) Design and application of a novel virtual reality navigational technology (VRNchair). J Exp Neurosci 8:7–14.  https://doi.org/10.4137/JEn.s13448 CrossRefGoogle Scholar
  11. 11.
    Cushman LA, Stein K, Duffy CJ (2008) Detecting navigational deficits in cognitive aging and Alzheimer disease using virtual reality. Neurology 71:888–895.  https://doi.org/10.1212/01.wnl.0000326262.67613.fe CrossRefGoogle Scholar
  12. 12.
    Design - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/design. Accessed 3 May 2018
  13. 13.
    Folstein MF, Folstein SE, McHugh PR (1975) Mini-mental state. J Psychiatr Res 12:189–198.  https://doi.org/10.1016/0022-3956(75)90026-6 CrossRefGoogle Scholar
  14. 14.
    Garzotto F, Gelsomini M, Clasadonte F, Montesano D, Occhiuto D (2016) Wearable Immersive Storytelling for Disabled Children. In: Proceedings of the International Working Conference on Advanced Visual Interfaces - AVI ‘16. p 196–203Google Scholar
  15. 15.
    Garzotto F, Gelsomini M, Occhiuto D, Matarazzo V, Messina N (2017) Wearable immersive virtual reality for children with disability: a case study. In: Proceedings of the 2017 Conference on Interaction Design and Children - IDC ‘17Google Scholar
  16. 16.
    Gaze - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/gaze. Accessed 3 May 2018
  17. 17.
    Gestures - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/gestures. Accessed 3 May 2018
  18. 18.
    GitHub - Microsoft/MixedRealityToolkit-Unity. https://github.com/Microsoft/MixedRealityToolkit-Unity. Accessed 7 May 2018
  19. 19.
    HoloLens hardware details - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/hololens-hardware-details. Accessed 3 May 2018
  20. 20.
    Iancu I, Olmer A (2006) The minimental state examination - an up-to-date review. Harefuah 145:687–690Google Scholar
  21. 21.
    Klinger E, Chemin I, Lebreton S, Marié RM (2004) A virtual supermarket to assess cognitive planning. Cyberpsychol Behav 7:292–293Google Scholar
  22. 22.
    Lee JY, Kho S, Bin YH, Park S, Choi JS, Kwon JS, Cha KR, Jung HY (2014) Spatial memory impairments in amnestic mild cognitive impairment in a virtual radial arm maze. Neuropsychiatr Dis Treat 10:653–660.  https://doi.org/10.2147/ndt.s58185 CrossRefGoogle Scholar
  23. 23.
    Milgram P, Kishino F (1994) Taxonomy of mixed reality visual displays. IEICE Trans Inf Syst E77–D:1321–1329Google Scholar
  24. 24.
    Mitchell AJ (2009) A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res 43:411–431.  https://doi.org/10.1016/j.jpsychires.2008.04.014 CrossRefGoogle Scholar
  25. 25.
    Mungas D (1991) In-office mental status testing: a practical guide. Geriatrics 46:54–58 63, 66Google Scholar
  26. 26.
  27. 27.
    Ohyama S, Nishiike S, Watanabe H, Matsuoka K, Akizuki H, Takeda N, Harada T (2007) Autonomic responses during motion sickness induced by virtual reality. Auris Nasus Larynx 34:303–306.  https://doi.org/10.1016/j.anl.2007.01.002 CrossRefGoogle Scholar
  28. 28.
    Prince M, Comas-Herrera A, Knapp M, Guerchet M, Karagiannidou M (2016) World Alzheimer report 2016 improving healthcare for people living with dementia. Coverage, quality and costs now and in the future. Alzheimer’s Dis Int:1–140Google Scholar
  29. 29.
    Sitzer DI, Twamley EW, Jeste DV (2006) Cognitive training in Alzheimer’s disease: a meta-analysis of the literature. Acta Psychiatr Scand 114:75–90CrossRefGoogle Scholar
  30. 30.
    Spatial mapping - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/spatial-mapping. Accessed 3 May 2018
  31. 31.
    Tamura H, Yamamoto H, Katayama A (2001) Mixed reality: future dreams seen at the border between real and virtual worlds. IEEE Comput Graph Appl 21:64–70.  https://doi.org/10.1109/38.963462 CrossRefGoogle Scholar
  32. 32.
    Using the Windows Device Portal - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/using-the-windows-device-portal. Accessed 7 May 2018
  33. 33.
    Voice input - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/voice-input. Accessed 3 May 2018
  34. 34.
    Werner P, Rabinowitz S, Klinger E, Korczyn AD, Josman N (2009) Use of the virtual action planning supermarket for the diagnosis of mild cognitive impairment. Dement Geriatr Cogn Disord 27:301–309.  https://doi.org/10.1159/000204915 CrossRefGoogle Scholar
  35. 35.
    What is mixed reality? - Mixed Reality | Microsoft Docs. https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality. Accessed 3 May 2018

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Mechanical EngineeringPolitecnico di MilanoMilanItaly
  2. 2.Department of Electronics, Information and BioengineeringPolitecnico di MilanoMilanItaly

Personalised recommendations