Skip to main content

Health Data Management for Nursing Practice: An Intelligent, Holographic Mixed-Reality System

Part of the Communications in Computer and Information Science book series (CCIS,volume 1420)


The elderly population in need of long-term care is rising, adding to the workload of healthcare workers in nursing homes. Throughout a work day health workers often must perform the time consuming process of collecting various types of health data from the residents. A recent study has revealed that frontline nurses and caregivers spend over half of their time accessing and updating patient records instead of providing care. New technology could enhance these data-oriented operations, and free up time so that staff can care for patients in other ways. Intelligent human-computer interaction paradigms can enhance data retrieval, visualization, and communication processes efficiently in a way to improve the overall productivity during work. In this study, we present an intelligent holographic mixed reality (MR) system that leverages Microsoft Hololens for work practices of healthcare work in nursing homes. Supported by computer vision, this system provides automatic data retrieval, in addition to interactive data visualization and mid-air data entry mechanism. Using face recognition through the Hololens’ visual sensors, the system recognizes individual patients, and automatically retrieves relevant health data and supports data entry.


  • Mixed reality
  • Intelligent user interfaces
  • Eldercare
  • Nursing homes
  • Health Informatics

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.


  1. Organization, W. H. Ageing And Health. news-room/fact-sheets/detail/ageing-and-health%20%5C#:\(\sim \):text=Today% 5C%2C%5C%20125%5C%20million%5C%20people%5C%20are%20%5C% 20aged%5C%2080%5C%20years%5C%20or%5C%20older

    Google Scholar 

  2. SOTI: Critical Technology for Critical Care: The State of Mobility in Health- care 2020/21 Report.

  3. Microsoft: Mixed Reality.

  4. Lindlbauer, D., Feit, A.M., Hilliges, O.: Context-aware online adaptation of mixed reality interfaces. In: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 147–160 (2019)

    Google Scholar 

  5. Daling, L., Abdelrazeq, A., Sauerborn, C., Hees, F.: A comparative study of augmented reality assistant tools in assembly. In: Ahram, T., Falcão, C. (eds.) AHFE 2019. AISC, vol. 972, pp. 755–767. Springer, Cham (2020). ISBN: 978-3-030-19135-1

    CrossRef  Google Scholar 

  6. Hanna, M.G., Ahmed, I., Nine, J., Prajapati, S., Pantanowitz, L.: Augmented reality technology using Microsoft HoloLens in anatomic pathology. Arch. Pathol. Laborat. Medi. 142, 638–644 (2018). ISSN: 0003–9985

    Google Scholar 

  7. Technologies, S. Skype: Communication Tool.

  8. Andersson, H.B., BØrresen, T., Prasolova-FØrland, E., McCallum, S., Estrada, J.G.: Developing an AR application for neurosurgical training: lessons learned for medical specialist education. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 407–412 (2020).

  9. Hauze, S., Hoyt, H., Marshall, J., Frazee, J., Greiner, P.: An evaluation of nursing student motivation to learn through holographic mixed reality simulation. In: 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), pp. 1058–1063 (2018).

  10. Teng, C.-C., et al.: Mixed reality patients monitoring application for critical care nurses. In: Proceedings of the Third International Conference on Medical and Health Informatics 2019, Xiamen, China, pp. 49–53. Association for Computing Machinery (2019). ISBN: 9781450371995. 3340037.3340050.

  11. Bakshi, A.M., Simson, J., de Castro, C., Yu, C.C., Dias, A.: Bright: an augmented reality assistive platform for visual impairment. In: 2019 IEEE Games, Entertainment, Media Conference (GEM), pp. 1–4 (2019). GEM.2019.8811556.

  12. Gupta, A., Samad, M., Kin, K., Kristensson, P.O., Benko, H.: Investigating remote tactile feedback for mid-air text-entry in virtual reality. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 350–360 (2020)

    Google Scholar 

Download references


This work was supported by the Norwegian University of Science and Technology (NTNU).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Syed Hammad Hussain Shah .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shah, S.H.H., Longva, B., Hameed, I.A., Solberg, M., Susanne T. Karlsen, A. (2021). Health Data Management for Nursing Practice: An Intelligent, Holographic Mixed-Reality System. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2021 - Posters. HCII 2021. Communications in Computer and Information Science, vol 1420. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78641-0

  • Online ISBN: 978-3-030-78642-7

  • eBook Packages: Computer ScienceComputer Science (R0)