Abstract
In health care, auditory alarms are an important aspect of an informatics system that monitors patients and alerts clinicians attending to multiple concurrent tasks. However, the volume, design, and pervasiveness of existing Intensive Care Unit (ICU) alarms can make it difficult to quickly distinguish their meaning and importance. In this study, we evaluated the effectiveness of two design approaches not yet explored in a smartwatch-based alarm system designed for ICU use: (1) using audiovisual spatial colocalization and (2) adding haptic (i.e., touch) information. We compared the performance of 30 study participants using ICU smartwatch alarms containing auditory icons in two implementations of the audio modality: colocalized with the visual cue on the smartwatch’s low-quality speaker versus delivered from a higher quality speaker located two feet away from participants (like a stationary alarm bay situated near patients in the ICU). Additionally, we compared participant performance using alarms with two sensory modalities (visual and audio) against alarms with three sensory modalities (adding haptic cues). Participants were 10.1% (0.24s) faster at responding to alarms when auditory information was delivered from the smartwatch instead of the higher quality external speaker. Meanwhile, adding haptic information to alarms improved response times to alarms by 12.2% (0.23s) and response times on their primary task by 10.3% (0.08s). Participants rated learnability and ease of use higher for alarms with haptic information. These small but statistically significant improvements demonstrate that audiovisual colocalization and multisensory alarm design can improve user response times.
Similar content being viewed by others
Data availability
By request to the corresponding author.
References
Berglund, B., Lindvall, T., Schwela, Dietrich H, & World Health Organization. Occupational and Environmental Health Team. (1999). Guidelines for community noise. Who.int. https://apps.who.int/iris/handle/10665/66217
Johansson, L., Bergbom, I., Waye, K. P., Ryherd, E., & Lindahl, B. (2012). The sound environment in an ICU patient room—a content analysis of sound levels and patient experiences. Intensive and Critical Care Nursing, 28(5), 269–279. https://doi.org/10.1016/j.iccn.2012.03.004
Christensen, M., Dodds, A., Sauer, J., & Watts, N. (2014). Alarm setting for the critically ill patient: a descriptive pilot survey of nurses' perceptions of current practice in an Australian Regional Critical Care Unit. Intensive & critical care nursing, 30(4), 204–210. https://doi.org/10.1016/j.iccn.2014.02.003
Pal, J., Taywade, M., Pal, R., & Sethi, D. (2022). Noise Pollution in Intensive Care Unit: A Hidden Enemy affecting the Physical and Mental Health of Patients and Caregivers. Noise & health, 24(114), 130–136. https://doi.org/10.4103/nah.nah_79_21
Maidl-Putz, C., McAndrew, N. S., & Leske, J. S. (2014). Noise in the ICU. Nursing Critical Care, 9(5), 29–35. https://doi.org/10.1097/01.ccn.0000453470.88327.2f
Darbyshire, J. L., Müller‐Trapet, M., Cheer, J., Fazi, F. M., & Young, J. D. (2019). Mapping sources of noise in an intensive care unit. Anaesthesia, 74(8), 1018–1025. https://doi.org/10.1111/anae.14690
Sangari, A., Emhardt, E. A., Salas, B., Avery, A., Freundlich, R. E., Fabbri, D., Shotwell, M. S., & Schlesinger, J. J. (2021). Delirium variability is influenced by the sound environment (devise study): How changes in the intensive care unit soundscape affect delirium incidence. Journal of Medical Systems, 45(8). https://doi.org/10.1007/s10916-021-01752-5
Simpao, A. F., & Rehman, M. A. (2018). Anesthesia Information Management Systems. Anesthesia and analgesia, 127(1), 90–94. https://doi.org/10.1213/ANE.0000000000002545
Simpao, A. F., Nelson, O., & Ahumada, L. M. (2021). Automated anesthesia artifact analysis: can machines be trained to take out the garbage?. Journal of clinical monitoring and computing, 35(2), 225–227. https://doi.org/10.1007/s10877-020-00589-6
Miles, M., Mueller, D., Gay-Betton, D., Baum Miller, S. H., Massa, S., Shi, Y., Shotwell, M. S., Lane-Fall, M., & Schlesinger, J. J. (2020). Observational Study of Clinician Attentional Reserves (OSCAR): Acuity-Based Rounds Help Preserve Clinicians' Attention. Critical care medicine, 48(4), 507–514. https://doi.org/10.1097/CCM.0000000000004205
Haas, E. C., & Edworthy, J. (1996). Measuring Perceived Urgency to Create Safe Auditory Warnings. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 40(16), 845–849. https://doi.org/10.1177/154193129604001611
Giang, W., Santhakumaran, S., Masnavi, E., Glussich, D., Kline, J., Chui, F., ... & Zelek, J. (2010). Multimodal interfaces: Literature review of ecological interface design, multimodal perception and attention, and intelligent adaptive multimodal interfaces.
Bruder, A. L., Edworthy, J., Schlesinger, J., & Rothwell, C. (2019). The multisensory benefit of informative sound in visual task performance. The Journal of the Acoustical Society of America, 146(4), 2834–2834. https://doi.org/10.1121/1.5136825
Burdick, K. J., Bell, A. S., McCoy, M. C., Samuels, J. L., Jolly, A. S., Patel, S. S., Balas, J. B., Patten, K. J., & Schlesinger, J. J. (2019). Using multisensory haptic integration to improve monitoring in the Intensive Care Unit. Auditory Perception & Cognition, 2(4), 188–206. https://doi.org/10.1080/25742442.2020.1773194
Foley, L., Anderson, C. J., & Schutz, M. (2020). Re-sounding alarms: Designing ergonomic auditory interfaces by embracing musical insights. Healthcare, 8(4), 389. https://doi.org/10.3390/healthcare8040389
Salas, B., & Schlesinger, J. J. (2019). Enhancing the sounds of urgency. Anesthesia & Analgesia, 129(4), 915–917. https://doi.org/10.1213/ane.0000000000004310
Edworthy, J. R., Parker, C. J., & Martin, E. V. (2022). Discriminating between simultaneous audible alarms is easier with auditory icons. Applied Ergonomics, 99, 103609. https://doi.org/10.1016/j.apergo.2021.103609
Edworthy, J., Page, R., Hibbard, A., Kyle, S., Ratnage, P., & Claydon, S. (2014). Learning three sets of alarms for the same medical functions: A perspective on the difficulty of learning alarms specified in an international standard. Applied Ergonomics, 45(5), 1291–1296. https://doi.org/10.1016/j.apergo.2013.10.003
Edworthy, J., Reid, S., McDougall, S., Edworthy, J., Hall, S., Bennett, D., Khan, J., & Pye, E. (2017). The recognizability and localizability of auditory alarms: Setting Global Medical Device Standards. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(7), 1108–1127. https://doi.org/10.1177/0018720817712004
McNeer, R. R., Horn, D. B., Bennett, C. L., Edworthy, J. R., & Dudaryk, R. (2018). Auditory icon alarms are more accurately and quickly identified than current standard melodic alarms in a simulated clinical setting. Anesthesiology, 129(1), 58–66. https://doi.org/10.1097/aln.0000000000002234
Bennett, C., Dudaryk, R., Crenshaw, N., Edworthy, J., & McNeer, R. (2019). Recommendation of New Medical Alarms Based on Audibility, Identifiability, and Detectability in a Randomized, Simulation-Based Study. Critical Care Medicine, 47(8), 1050–1057. https://doi.org/10.1097/ccm.0000000000003802
Bruder, A. L., Rothwell, C. D., Fuhr, L. I., Shotwell, M. S., Edworthy, J. R., & Schlesinger, J. J. (2021). The Influence of Audible Alarm Loudness and Type on Clinical Multitasking. Journal of Medical Systems, 46(1). https://doi.org/10.1007/s10916-021-01794-9
Beranek, L. L. & Mellow, T. J. (2012). Chapter 6 - Electrodynamic loudspeakers. In L. L. Beranek & T. J. Mellow (Eds.), Acoustics: Sound Fields and Transducers, (pp. 241-288). Academic Press. https://doi.org/10.1016/B978-0-12-391421-7.00006-3.
Bolton, M. L., Zheng, X., Li, M., Edworthy, J. R., & Boyd, A. D. (2020). An Experimental Validation of Masking in IEC 60601-1-8:2006-Compliant Alarm Sounds. Human factors, 62(6), 954–972. https://doi.org/10.1177/0018720819862911
Foley, L., Schlesinger, J., & Schutz, M. (2022). More detectable, less annoying: Temporal variation in amplitude envelope and spectral content improves auditory interface efficacy. The Journal of the Acoustical Society of America, 151(5), 3189. https://doi.org/10.1121/10.0010447
Klueber, S., Wolf, E., Grundgeiger, T., Brecknell, B., Mohamed, I., & Sanderson, P. (2019). Supporting multiple patient monitoring with head-worn displays and Spearcons. Applied Ergonomics, 78, 86–96. https://doi.org/10.1016/j.apergo.2019.01.009
Short, K., & Chung, Y. “J. (2019). Solving alarm fatigue with smartphone technology. Nursing, 49(1), 52–57. https://doi.org/10.1097/01.nurse.0000549728.37810.d9
Ng, J. Y. C., Man, J. C. F., Fels, S., Dumont, G., & Ansermino, J. M. (2005). An evaluation of a vibro-tactile display prototype for physiological monitoring. Anesthesia and analgesia, 101(6), 1719–1724. https://doi.org/10.1213/01.ANE.0000184121.03150.62
White, T. L., & Hancock, P. A. (2020). Specifying advantages of multi-modal cueing: Quantifying improvements with augmented tactile information. Applied ergonomics, 88, 103146. https://doi.org/10.1016/j.apergo.2020.103146
Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical issues in ergonomics science, 3(2), 159-177.
Spence, C. (2013). Just how important is spatial coincidence to multisensory integration? Evaluating the spatial rule. Annals of the New York Academy of Sciences, 1296, 31–49. https://doi.org/10.1111/nyas.12121
Frens, M. A., Van Opstal, A. J., & Van der Willigen, R. F. (1995). Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Perception & psychophysics, 57(6), 802–816. https://doi.org/10.3758/bf03206796
Meredith, M. A., & Stein, B. E. (1986). Spatial factors determine the activity of multisensory neurons in cat superior colliculus. Brain research, 365(2), 350–354. https://doi.org/10.1016/0006-8993(86)91648-3
Royal, D. W., Carriere, B. N., & Wallace, M. T. (2009). Spatiotemporal architecture of cortical receptive fields and its impact on multisensory interactions. Experimental brain research, 198(2-3), 127–136. https://doi.org/10.1007/s00221-009-1772-y
Fleming, J. T. (2021). Multisensory Integration of Spatio-Temporal Information in Attention and Working Memory. Dash.harvard.edu. https://nrs.harvard.edu/URN-3:HUL.INSTREPOS:37369454
Marian, V., Hayakawa, S., & Schroeder, S. R. (2021). Cross-Modal Interaction Between Auditory and Visual Input Impacts Memory Retrieval. Frontiers in Neuroscience, 15, 661477. https://doi.org/10.3389/fnins.2021.661477
Iordanescu, L., Grabowecky, M., Franconeri, S., Theeuwes, J., & Suzuki, S. (2010). Characteristic sounds make you look at target objects more quickly. Attention, perception & psychophysics, 72(7), 1736–1741. https://doi.org/10.3758/APP.72.7.1736
Bolognini, N., Senna, I., Maravita, A., Pascual-Leone, A., & Merabet, L. B. (2010). Auditory enhancement of visual phosphene perception: The effect of temporal and spatial factors and of stimulus intensity. Neuroscience Letters, 477(3), 109–114. https://doi.org/10.1016/j.neulet.2010.04.044
Hameed, S., Ferris, T., Jayaraman, S., & Sarter, N. (2009). Using Informative Peripheral Visual and Tactile Cues to Support Task and Interruption Management. Human Factors, 51(2), 126–135. https://doi.org/10.1177/0018720809336434
Ferris, T. K., & Sarter, N. (2011). Continuously informing vibrotactile displays in support of attention management and multitasking in anesthesiology. Human factors, 53(6), 600–611. https://doi.org/10.1177/0018720811425043
Burdick, K. J., Gupta, M., Sangari, A., & Schlesinger, J. J. (2022). Improved Patient Monitoring with a Novel Multisensory Smartwatch Application. Journal of medical systems, 46(12), 83. https://doi.org/10.1007/s10916-022-01869-1
Ng, G., Barralon, P., Dumont, G., Schwarz, S. K., & Ansermino, J. M. (2007). Optimizing the tactile display of physiological information: vibro-tactile vs. electro-tactile stimulation, and forearm or wrist location. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference, 2007, 4202–4205. https://doi.org/10.1109/IEMBS.2007.4353263
Korres, G., Birgitte Falk Jensen, C., Park, W., Bartsch, C., & Eid, M. (2018). A Vibrotactile Alarm System for Pleasant Awakening. IEEE transactions on haptics, https://doi.org/10.1109/TOH.2018.2804952. Advance online publication. https://doi.org/10.1109/TOH.2018.2804952
Watson, N., Shah, A., Patel, P., & Soni, S. (2023). Alarming tones: the unexplored phenomenon of auditory roughness. British journal of anaesthesia, S0007-0912(23)00246-5. Advance online publication. https://doi.org/10.1016/j.bja.2023.05.010
Burdick, K., Yang, S., Elizondo Lopez, A.E., Wessel, C., Schutz, M., Schlesinger, J.J. (2023). Auditory roughness: a delicate balance. British journal of anaesthesia. Journal Article - in press.
Photinos, P. (2021). The Physics of Sound Waves: Music, instruments, and sound equipment. IOP Publishing.
Kane SP. Sample Size Calculator. ClinCalc: https://clincalc.com/stats/samplesize.aspx. Updated July 24, 2019. Accessed July 20, 2023.
Stoet, G. (2010). PsyToolkit: A software package for programming psychological experiments using linux. Behavior Research Methods, 42(4), 1096–1104. https://doi.org/10.3758/brm.42.4.1096
Stoet, G. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24-31.
Schlesinger, J. J., Baum Miller, S. H., Nash, K., Bruce, M., Ashmead, D., Shotwell, M. S., Edworthy, J. R., Wallace, M. T., & Weinger, M. B. (2018). Acoustic features of auditory medical alarms - An experimental study of alarm volume. The Journal of the Acoustical Society of America, 143(6), 3688. https://doi.org/10.1121/1.5043396
Pallant, J. (2016). SPSS Survival Manual: A Step By Step Guide to Data Analysis Using SPSS Program (6th ed.). London, UK: McGraw-Hill Education.
Sedgwick, P. (2012). Multiple significance tests: the Bonferroni correction. BMJ, 2012;344:e509. https://doi.org/10.1136/bmj.e509
Schlesinger, J. J., Stevenson, R. A., Shotwell, M. S., & Wallace, M. T. (2014). Improving pulse oximetry pitch perception with multisensory perceptual training. Anesthesia and analgesia, 118(6), 1249–1253. https://doi.org/10.1213/ANE.0000000000000222
Li, Q., Yang, H., Sun, F., & Wu, J. (2015). Spatiotemporal Relationships among Audiovisual Stimuli Modulate Auditory Facilitation of Visual Target Discrimination. Perception, 44(3), 232–242. https://doi.org/10.1068/p7846
Mahoney, J. R., Molholm, S., Butler, J. S., Sehatpour, P., Gomez-Ramirez, M., Ritter, W., & Foxe, J. J. (2015). Keeping in touch with the visual system: spatial alignment and multisensory integration of visual-somatosensory inputs. Frontiers in psychology, 6, 1068. https://doi.org/10.3389/fpsyg.2015.01068
Fleming, J. T., Noyce, A. L., & Shinn-Cunningham, B. G. (2020). Audio-visual spatial alignment improves integration in the presence of a competing audio-visual stimulus. Neuropsychologia, 146, 107530. https://doi.org/10.1016/j.neuropsychologia.2020.107530
Bartel, L., & Mosabbir, A. (2021). Possible Mechanisms for the Effects of Sound Vibration on Human Health. Healthcare, 9(5), 597. https://doi.org/10.3390/healthcare9050597
Yau, J. M., Olenczak, J. B., Dammann, J. F., & Bensmaia, S. J. (2009). Temporal Frequency Channels Are Linked across Audition and Touch. Current Biology, 19(7), 561–566. https://doi.org/10.1016/j.cub.2009.02.013
Yau, J. M., Weber, A. I., & Bensmaia, S. J. (2010). Separate mechanisms for audio-tactile pitch and loudness interactions. Frontiers in Psychology, 1. https://doi.org/10.3389/fpsyg.2010.00160
Tivadar, R. I., Arnold, R. C., Turoman, N., Knebel, J. F., & Murray, M. M. (2022). Digital haptics improve speed of visual search performance in a dual-task setting. Scientific reports, 12(1), 9728. https://doi.org/10.1038/s41598-022-13827-5
Shanmugham, M., Strawderman, L., Babski-Reeves, K., & Bian, L. (2018). Alarm-Related Workload in Default and Modified Alarm Settings and the Relationship Between Alarm Workload, Alarm Response Rate, and Care Provider Experience: Quantification and Comparison Study. JMIR human factors, 5(4), e11704. https://doi.org/10.2196/11704
Linzer, M., Jin, J. O., Shah, P., Stillman, M., Brown, R., Poplau, S., Nankivil, N., Cappelucci, K., & Sinsky, C. A. (2022). Trends in Clinician Burnout With Associated Mitigating and Aggravating Factors During the COVID-19 Pandemic. JAMA health forum, 3(11), e224163. https://doi.org/10.1001/jamahealthforum.2022.4163
Sendelbach, S., & Funk, M. (2013). Alarm fatigue: a patient safety concern. AACN advanced critical care, 24(4), 378–388. https://doi.org/10.1097/NCI.0b013e3182a903f9
Rouch, J., Parizet, E. (2020). Relation between perceived urgency of alarm signals and psychoacoustic parameters. Forum Acusticum, 2435-2439
Katzman, N., Gellert, M., Schlesinger, J. J., Oron-Gilad, T., Cooperstock, J. R., & Bitan, Y. (2019). Evaluation of tactile cues for simulated patients’ status under high and low workload. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1), 658–662. https://doi.org/10.1177/1071181319631285
Acknowledgements
The authors would like to thank Dr. Judy Edworthy and Dr. Michael R. King for their input and assistance. The authors would also like to thank Russ Beebe, interactions designer within the Department of Anesthesiology at Vanderbilt University Medical Center, for his assistance in formatting figures.
Funding
This work was supported by the Office of Naval Research Grant N00014-22-1-2184.
Author information
Authors and Affiliations
Contributions
A.S., M.B., M.C., A.T., and J.S. contributed to the study design and conception. M.B., M.C., A.T., and J.S. assisted with data collection. All authors assisted with data analysis. All authors contributed to writing, revising, and finalizing the manuscript.
Corresponding author
Ethics declarations
Ethics approval
This study was performed in accordance with the principles of the Declaration of Helsinki and approved by the Vanderbilt University Institutional Review Board.
Consent to participate
Informed consent was obtained from all participants prior to participation.
Competing interests
The authors have no conflicts of interest or competing interests to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Sangari, A., Bingham, M.A., Cummins, M. et al. A Spatiotemporal and Multisensory Approach to Designing Wearable Clinical ICU Alarms. J Med Syst 47, 105 (2023). https://doi.org/10.1007/s10916-023-01997-2
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-023-01997-2