Skip to main content
Log in

A Spatiotemporal and Multisensory Approach to Designing Wearable Clinical ICU Alarms

  • Original Paper
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

In health care, auditory alarms are an important aspect of an informatics system that monitors patients and alerts clinicians attending to multiple concurrent tasks. However, the volume, design, and pervasiveness of existing Intensive Care Unit (ICU) alarms can make it difficult to quickly distinguish their meaning and importance. In this study, we evaluated the effectiveness of two design approaches not yet explored in a smartwatch-based alarm system designed for ICU use: (1) using audiovisual spatial colocalization and (2) adding haptic (i.e., touch) information. We compared the performance of 30 study participants using ICU smartwatch alarms containing auditory icons in two implementations of the audio modality: colocalized with the visual cue on the smartwatch’s low-quality speaker versus delivered from a higher quality speaker located two feet away from participants (like a stationary alarm bay situated near patients in the ICU). Additionally, we compared participant performance using alarms with two sensory modalities (visual and audio) against alarms with three sensory modalities (adding haptic cues). Participants were 10.1% (0.24s) faster at responding to alarms when auditory information was delivered from the smartwatch instead of the higher quality external speaker. Meanwhile, adding haptic information to alarms improved response times to alarms by 12.2% (0.23s) and response times on their primary task by 10.3% (0.08s). Participants rated learnability and ease of use higher for alarms with haptic information. These small but statistically significant improvements demonstrate that audiovisual colocalization and multisensory alarm design can improve user response times.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

By request to the corresponding author.

References

  1. Berglund, B., Lindvall, T., Schwela, Dietrich H, & World Health Organization. Occupational and Environmental Health Team. (1999). Guidelines for community noise. Who.int. https://apps.who.int/iris/handle/10665/66217

  2. Johansson, L., Bergbom, I., Waye, K. P., Ryherd, E., & Lindahl, B. (2012). The sound environment in an ICU patient room—a content analysis of sound levels and patient experiences. Intensive and Critical Care Nursing, 28(5), 269–279. https://doi.org/10.1016/j.iccn.2012.03.004

    Article  PubMed  Google Scholar 

  3. Christensen, M., Dodds, A., Sauer, J., & Watts, N. (2014). Alarm setting for the critically ill patient: a descriptive pilot survey of nurses' perceptions of current practice in an Australian Regional Critical Care Unit. Intensive & critical care nursing, 30(4), 204–210. https://doi.org/10.1016/j.iccn.2014.02.003

    Article  Google Scholar 

  4. Pal, J., Taywade, M., Pal, R., & Sethi, D. (2022). Noise Pollution in Intensive Care Unit: A Hidden Enemy affecting the Physical and Mental Health of Patients and Caregivers. Noise & health, 24(114), 130–136. https://doi.org/10.4103/nah.nah_79_21

    Article  Google Scholar 

  5. Maidl-Putz, C., McAndrew, N. S., & Leske, J. S. (2014). Noise in the ICU. Nursing Critical Care, 9(5), 29–35. https://doi.org/10.1097/01.ccn.0000453470.88327.2f

    Article  Google Scholar 

  6. Darbyshire, J. L., Müller‐Trapet, M., Cheer, J., Fazi, F. M., & Young, J. D. (2019). Mapping sources of noise in an intensive care unit. Anaesthesia, 74(8), 1018–1025. https://doi.org/10.1111/anae.14690

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Sangari, A., Emhardt, E. A., Salas, B., Avery, A., Freundlich, R. E., Fabbri, D., Shotwell, M. S., & Schlesinger, J. J. (2021). Delirium variability is influenced by the sound environment (devise study): How changes in the intensive care unit soundscape affect delirium incidence. Journal of Medical Systems, 45(8). https://doi.org/10.1007/s10916-021-01752-5

  8. Simpao, A. F., & Rehman, M. A. (2018). Anesthesia Information Management Systems. Anesthesia and analgesia, 127(1), 90–94. https://doi.org/10.1213/ANE.0000000000002545

    Article  PubMed  Google Scholar 

  9. Simpao, A. F., Nelson, O., & Ahumada, L. M. (2021). Automated anesthesia artifact analysis: can machines be trained to take out the garbage?. Journal of clinical monitoring and computing, 35(2), 225–227. https://doi.org/10.1007/s10877-020-00589-6

    Article  PubMed  Google Scholar 

  10. Miles, M., Mueller, D., Gay-Betton, D., Baum Miller, S. H., Massa, S., Shi, Y., Shotwell, M. S., Lane-Fall, M., & Schlesinger, J. J. (2020). Observational Study of Clinician Attentional Reserves (OSCAR): Acuity-Based Rounds Help Preserve Clinicians' Attention. Critical care medicine, 48(4), 507–514. https://doi.org/10.1097/CCM.0000000000004205

    Article  PubMed  Google Scholar 

  11. Haas, E. C., & Edworthy, J. (1996). Measuring Perceived Urgency to Create Safe Auditory Warnings. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 40(16), 845–849. https://doi.org/10.1177/154193129604001611

    Article  Google Scholar 

  12. Giang, W., Santhakumaran, S., Masnavi, E., Glussich, D., Kline, J., Chui, F., ... & Zelek, J. (2010). Multimodal interfaces: Literature review of ecological interface design, multimodal perception and attention, and intelligent adaptive multimodal interfaces.

  13. Bruder, A. L., Edworthy, J., Schlesinger, J., & Rothwell, C. (2019). The multisensory benefit of informative sound in visual task performance. The Journal of the Acoustical Society of America, 146(4), 2834–2834. https://doi.org/10.1121/1.5136825

    Article  Google Scholar 

  14. Burdick, K. J., Bell, A. S., McCoy, M. C., Samuels, J. L., Jolly, A. S., Patel, S. S., Balas, J. B., Patten, K. J., & Schlesinger, J. J. (2019). Using multisensory haptic integration to improve monitoring in the Intensive Care Unit. Auditory Perception & Cognition, 2(4), 188–206. https://doi.org/10.1080/25742442.2020.1773194

    Article  Google Scholar 

  15. Foley, L., Anderson, C. J., & Schutz, M. (2020). Re-sounding alarms: Designing ergonomic auditory interfaces by embracing musical insights. Healthcare, 8(4), 389. https://doi.org/10.3390/healthcare8040389

    Article  PubMed  PubMed Central  Google Scholar 

  16. Salas, B., & Schlesinger, J. J. (2019). Enhancing the sounds of urgency. Anesthesia & Analgesia, 129(4), 915–917. https://doi.org/10.1213/ane.0000000000004310

    Article  Google Scholar 

  17. Edworthy, J. R., Parker, C. J., & Martin, E. V. (2022). Discriminating between simultaneous audible alarms is easier with auditory icons. Applied Ergonomics, 99, 103609. https://doi.org/10.1016/j.apergo.2021.103609

    Article  PubMed  Google Scholar 

  18. Edworthy, J., Page, R., Hibbard, A., Kyle, S., Ratnage, P., & Claydon, S. (2014). Learning three sets of alarms for the same medical functions: A perspective on the difficulty of learning alarms specified in an international standard. Applied Ergonomics, 45(5), 1291–1296. https://doi.org/10.1016/j.apergo.2013.10.003

    Article  PubMed  Google Scholar 

  19. Edworthy, J., Reid, S., McDougall, S., Edworthy, J., Hall, S., Bennett, D., Khan, J., & Pye, E. (2017). The recognizability and localizability of auditory alarms: Setting Global Medical Device Standards. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(7), 1108–1127. https://doi.org/10.1177/0018720817712004

    Article  Google Scholar 

  20. McNeer, R. R., Horn, D. B., Bennett, C. L., Edworthy, J. R., & Dudaryk, R. (2018). Auditory icon alarms are more accurately and quickly identified than current standard melodic alarms in a simulated clinical setting. Anesthesiology, 129(1), 58–66. https://doi.org/10.1097/aln.0000000000002234

    Article  PubMed  Google Scholar 

  21. Bennett, C., Dudaryk, R., Crenshaw, N., Edworthy, J., & McNeer, R. (2019). Recommendation of New Medical Alarms Based on Audibility, Identifiability, and Detectability in a Randomized, Simulation-Based Study. Critical Care Medicine, 47(8), 1050–1057. https://doi.org/10.1097/ccm.0000000000003802

    Article  PubMed  Google Scholar 

  22. Bruder, A. L., Rothwell, C. D., Fuhr, L. I., Shotwell, M. S., Edworthy, J. R., & Schlesinger, J. J. (2021). The Influence of Audible Alarm Loudness and Type on Clinical Multitasking. Journal of Medical Systems, 46(1). https://doi.org/10.1007/s10916-021-01794-9

  23. Beranek, L. L. & Mellow, T. J. (2012). Chapter 6 - Electrodynamic loudspeakers. In L. L. Beranek & T. J. Mellow (Eds.), Acoustics: Sound Fields and Transducers, (pp. 241-288). Academic Press. https://doi.org/10.1016/B978-0-12-391421-7.00006-3.

  24. Bolton, M. L., Zheng, X., Li, M., Edworthy, J. R., & Boyd, A. D. (2020). An Experimental Validation of Masking in IEC 60601-1-8:2006-Compliant Alarm Sounds. Human factors, 62(6), 954–972. https://doi.org/10.1177/0018720819862911

    Article  PubMed  Google Scholar 

  25. Foley, L., Schlesinger, J., & Schutz, M. (2022). More detectable, less annoying: Temporal variation in amplitude envelope and spectral content improves auditory interface efficacy. The Journal of the Acoustical Society of America, 151(5), 3189. https://doi.org/10.1121/10.0010447

    Article  PubMed  Google Scholar 

  26. Klueber, S., Wolf, E., Grundgeiger, T., Brecknell, B., Mohamed, I., & Sanderson, P. (2019). Supporting multiple patient monitoring with head-worn displays and Spearcons. Applied Ergonomics, 78, 86–96. https://doi.org/10.1016/j.apergo.2019.01.009

    Article  PubMed  Google Scholar 

  27. Short, K., & Chung, Y. “J. (2019). Solving alarm fatigue with smartphone technology. Nursing, 49(1), 52–57. https://doi.org/10.1097/01.nurse.0000549728.37810.d9

    Article  PubMed  Google Scholar 

  28. Ng, J. Y. C., Man, J. C. F., Fels, S., Dumont, G., & Ansermino, J. M. (2005). An evaluation of a vibro-tactile display prototype for physiological monitoring. Anesthesia and analgesia, 101(6), 1719–1724. https://doi.org/10.1213/01.ANE.0000184121.03150.62

    Article  PubMed  Google Scholar 

  29. White, T. L., & Hancock, P. A. (2020). Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information. Applied ergonomics, 88, 103146. https://doi.org/10.1016/j.apergo.2020.103146

    Article  PubMed  Google Scholar 

  30. Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical issues in ergonomics science, 3(2), 159-177.

    Article  Google Scholar 

  31. Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule. Annals of the New York Academy of Sciences, 1296, 31–49. https://doi.org/10.1111/nyas.12121

    Article  PubMed  Google Scholar 

  32. Frens, M. A., Van Opstal, A. J., & Van der Willigen, R. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception & psychophysics, 57(6), 802–816. https://doi.org/10.3758/bf03206796

    Article  CAS  Google Scholar 

  33. Meredith, M. A., & Stein, B. E. (1986). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain research, 365(2), 350–354. https://doi.org/10.1016/0006-8993(86)91648-3

    Article  CAS  PubMed  Google Scholar 

  34. Royal, D. W., Carriere, B. N., & Wallace, M. T. (2009). Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions. Experimental brain research, 198(2-3), 127–136. https://doi.org/10.1007/s00221-009-1772-y

    Article  PubMed  PubMed Central  Google Scholar 

  35. Fleming, J. T. (2021). Multisensory Integration of Spatio-Temporal Information in Attention and Working Memory. Dash.harvard.edu. https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37369454

  36. Marian, V., Hayakawa, S., & Schroeder, S. R. (2021). Cross-Modal Interaction Between Auditory and Visual Input Impacts Memory Retrieval. Frontiers in Neuroscience, 15, 661477. https://doi.org/10.3389/fnins.2021.661477

    Article  PubMed  PubMed Central  Google Scholar 

  37. Iordanescu, L., Grabowecky, M., Franconeri, S., Theeuwes, J., & Suzuki, S. (2010). Characteristic sounds make you look at target objects more quickly. Attention, perception & psychophysics, 72(7), 1736–1741. https://doi.org/10.3758/APP.72.7.1736

    Article  Google Scholar 

  38. Bolognini, N., Senna, I., Maravita, A., Pascual-Leone, A., & Merabet, L. B. (2010). Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neuroscience Letters, 477(3), 109–114. https://doi.org/10.1016/j.neulet.2010.04.044

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Hameed, S., Ferris, T., Jayaraman, S., & Sarter, N. (2009). Using Informative Peripheral Visual and Tactile Cues to Support Task and Interruption Management. Human Factors, 51(2), 126–135. https://doi.org/10.1177/0018720809336434

    Article  PubMed  Google Scholar 

  40. Ferris, T. K., & Sarter, N. (2011). Continuously informing vibrotactile displays in support of attention management and multitasking in anesthesiology. Human factors, 53(6), 600–611. https://doi.org/10.1177/0018720811425043

    Article  PubMed  Google Scholar 

  41. Burdick, K. J., Gupta, M., Sangari, A., & Schlesinger, J. J. (2022). Improved Patient Monitoring with a Novel Multisensory Smartwatch Application. Journal of medical systems, 46(12), 83. https://doi.org/10.1007/s10916-022-01869-1

    Article  PubMed  PubMed Central  Google Scholar 

  42. Ng, G., Barralon, P., Dumont, G., Schwarz, S. K., & Ansermino, J. M. (2007). Optimizing the tactile display of physiological information: vibro-tactile vs. electro-tactile stimulation, and forearm or wrist location. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference, 2007, 4202–4205. https://doi.org/10.1109/IEMBS.2007.4353263

    Article  CAS  Google Scholar 

  43. Korres, G., Birgitte Falk Jensen, C., Park, W., Bartsch, C., & Eid, M. (2018). A Vibrotactile Alarm System for Pleasant Awakening. IEEE transactions on haptics, https://doi.org/10.1109/TOH.2018.2804952. Advance online publication. https://doi.org/10.1109/TOH.2018.2804952

    Article  PubMed  Google Scholar 

  44. Watson, N., Shah, A., Patel, P., & Soni, S. (2023). Alarming tones: the unexplored phenomenon of auditory roughness. British journal of anaesthesia, S0007-0912(23)00246-5. Advance online publication. https://doi.org/10.1016/j.bja.2023.05.010

  45. Burdick, K., Yang, S., Elizondo Lopez, A.E., Wessel, C., Schutz, M., Schlesinger, J.J. (2023). Auditory roughness: a delicate balance. British journal of anaesthesia. Journal Article - in press.

    Google Scholar 

  46. Photinos, P. (2021). The Physics of Sound Waves: Music, instruments, and sound equipment. IOP Publishing.

    Book  Google Scholar 

  47. Kane SP. Sample Size Calculator. ClinCalc: https://clincalc.com/stats/samplesize.aspx. Updated July 24, 2019. Accessed July 20, 2023.

  48. Stoet, G. (2010). PsyToolkit: A software package for programming psychological experiments using linux. Behavior Research Methods, 42(4), 1096–1104. https://doi.org/10.3758/brm.42.4.1096

    Article  PubMed  Google Scholar 

  49. Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.

    Article  Google Scholar 

  50. Schlesinger, J. J., Baum Miller, S. H., Nash, K., Bruce, M., Ashmead, D., Shotwell, M. S., Edworthy, J. R., Wallace, M. T., & Weinger, M. B. (2018). Acoustic features of auditory medical alarms - An experimental study of alarm volume. The Journal of the Acoustical Society of America, 143(6), 3688. https://doi.org/10.1121/1.5043396

    Article  PubMed  PubMed Central  Google Scholar 

  51. Pallant, J. (2016). SPSS Survival Manual: A Step By Step Guide to Data Analysis Using SPSS Program (6th ed.). London, UK: McGraw-Hill Education.

    Google Scholar 

  52. Sedgwick, P. (2012). Multiple significance tests: the Bonferroni correction. BMJ, 2012;344:e509. https://doi.org/10.1136/bmj.e509

    Article  Google Scholar 

  53. Schlesinger, J. J., Stevenson, R. A., Shotwell, M. S., & Wallace, M. T. (2014). Improving pulse oximetry pitch perception with multisensory perceptual training. Anesthesia and analgesia, 118(6), 1249–1253. https://doi.org/10.1213/ANE.0000000000000222

    Article  PubMed  Google Scholar 

  54. Li, Q., Yang, H., Sun, F., & Wu, J. (2015). Spatiotemporal Relationships among Audiovisual Stimuli Modulate Auditory Facilitation of Visual Target Discrimination. Perception, 44(3), 232–242. https://doi.org/10.1068/p7846

    Article  PubMed  Google Scholar 

  55. Mahoney, J. R., Molholm, S., Butler, J. S., Sehatpour, P., Gomez-Ramirez, M., Ritter, W., & Foxe, J. J. (2015). Keeping in touch with the visual system: spatial alignment and multisensory integration of visual-somatosensory inputs. Frontiers in psychology, 6, 1068. https://doi.org/10.3389/fpsyg.2015.01068

    Article  PubMed  PubMed Central  Google Scholar 

  56. Fleming, J. T., Noyce, A. L., & Shinn-Cunningham, B. G. (2020). Audio-visual spatial alignment improves integration in the presence of a competing audio-visual stimulus. Neuropsychologia, 146, 107530. https://doi.org/10.1016/j.neuropsychologia.2020.107530

    Article  PubMed  PubMed Central  Google Scholar 

  57. Bartel, L., & Mosabbir, A. (2021). Possible Mechanisms for the Effects of Sound Vibration on Human Health. Healthcare, 9(5), 597. https://doi.org/10.3390/healthcare9050597

    Article  PubMed  PubMed Central  Google Scholar 

  58. Yau, J. M., Olenczak, J. B., Dammann, J. F., & Bensmaia, S. J. (2009). Temporal Frequency Channels Are Linked across Audition and Touch. Current Biology, 19(7), 561–566. https://doi.org/10.1016/j.cub.2009.02.013

    Article  CAS  PubMed  Google Scholar 

  59. Yau, J. M., Weber, A. I., & Bensmaia, S. J. (2010). Separate mechanisms for audio-tactile pitch and loudness interactions. Frontiers in Psychology, 1. https://doi.org/10.3389/fpsyg.2010.00160

  60. Tivadar, R. I., Arnold, R. C., Turoman, N., Knebel, J. F., & Murray, M. M. (2022). Digital haptics improve speed of visual search performance in a dual-task setting. Scientific reports, 12(1), 9728. https://doi.org/10.1038/s41598-022-13827-5

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Shanmugham, M., Strawderman, L., Babski-Reeves, K., & Bian, L. (2018). Alarm-Related Workload in Default and Modified Alarm Settings and the Relationship Between Alarm Workload, Alarm Response Rate, and Care Provider Experience: Quantification and Comparison Study. JMIR human factors, 5(4), e11704. https://doi.org/10.2196/11704

    Article  PubMed  PubMed Central  Google Scholar 

  62. Linzer, M., Jin, J. O., Shah, P., Stillman, M., Brown, R., Poplau, S., Nankivil, N., Cappelucci, K., & Sinsky, C. A. (2022). Trends in Clinician Burnout With Associated Mitigating and Aggravating Factors During the COVID-19 Pandemic. JAMA health forum, 3(11), e224163. https://doi.org/10.1001/jamahealthforum.2022.4163

    Article  PubMed  PubMed Central  Google Scholar 

  63. Sendelbach, S., & Funk, M. (2013). Alarm fatigue: a patient safety concern. AACN advanced critical care, 24(4), 378–388. https://doi.org/10.1097/NCI.0b013e3182a903f9

    Article  PubMed  Google Scholar 

  64. Rouch, J., Parizet, E. (2020). Relation between perceived urgency of alarm signals and psychoacoustic parameters. Forum Acusticum, 2435-2439

  65. Katzman, N., Gellert, M., Schlesinger, J. J., Oron-Gilad, T., Cooperstock, J. R., & Bitan, Y. (2019). Evaluation of tactile cues for simulated patients’ status under high and low workload. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 658–662. https://doi.org/10.1177/1071181319631285

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. Judy Edworthy and Dr. Michael R. King for their input and assistance. The authors would also like to thank Russ Beebe, interactions designer within the Department of Anesthesiology at Vanderbilt University Medical Center, for his assistance in formatting figures.

Funding

This work was supported by the Office of Naval Research Grant N00014-22-1-2184.

Author information

Authors and Affiliations

Authors

Contributions

A.S., M.B., M.C., A.T., and J.S. contributed to the study design and conception. M.B., M.C., A.T., and J.S. assisted with data collection. All authors assisted with data analysis. All authors contributed to writing, revising, and finalizing the manuscript.

Corresponding author

Correspondence to Ayush Sangari.

Ethics declarations

Ethics approval

This study was performed in accordance with the principles of the Declaration of Helsinki and approved by the Vanderbilt University Institutional Review Board.

Consent to participate

Informed consent was obtained from all participants prior to participation.

Competing interests

The authors have no conflicts of interest or competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (WAV 6848 KB)

Supplementary file2 (DOCX 21.8 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sangari, A., Bingham, M.A., Cummins, M. et al. A Spatiotemporal and Multisensory Approach to Designing Wearable Clinical ICU Alarms. J Med Syst 47, 105 (2023). https://doi.org/10.1007/s10916-023-01997-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-023-01997-2

Keywords

Navigation