Multi-modal Interruptions on Primary Task Performance

  • Pooja P. BovardEmail author
  • Kelly A. Sprehn
  • Meredith G. Cunha
  • Jaemin Chun
  • SeungJun Kim
  • Jana L. Schwartz
  • Sara K. Garver
  • Anind K. Dey
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10916)


In this paper we have investigated a range of multi-modal displays (visual, auditory, haptic) to understand the effects of interruptions across various modalities on response times. Understanding these effects is particularly relevant in complex tasks that require perceptual attention, where pertinent information needs to be delivered to a user, e.g., driving. Multi-modal signal presentation, based on the Multiple Resource Theory framework, is a potential solution. To explore this solution, we conducted a study in which participants perceived and responded to a secondary task while conducting a visual, auditory, and haptic vigilance task during a driving scenario. We analyzed response times, errors, misses, and subjective responses and our results indicated that haptic interruptions of a primarily haptic task can be responded to the fastest, and visual interruptions are not the preferred modality in a driving scenario. With the results of this study, we can define logic for a context-based framework to better determine how to deliver incoming information in a driving scenario.


Augmented reality Interruptibility Multi-modal signaling 


  1. Abowd, G.D., Dey, A.K., Brown, P.J., Davies, N., Smith, M., Steggles, P.: Towards a better understanding of context and context-awareness. In: Gellersen, H.W. (ed.) Handheld and Ubiquitous Computing. LNCS, pp. 304–307. Springer, Heidelberg (1999). Scholar
  2. Adamcyzk, P.D., Bailey, B.P.: If not now, when? The effects of interruption at different moments within task execution. In: Proceedings of CHI 2004 Human Factors in Computing Systems, New York, pp. 271–278. Association for Computing Machinery Press (2004)Google Scholar
  3. Alghamdi, W., Shakshuki, E., Sheltami, T.R.: Context-aware driver assistance system. Procedia Comput. Sci. 10, 785–794 (2012)CrossRefGoogle Scholar
  4. Biondi, F., Rossi, R., Gastaldi, M., Mulatti, C.: Beeping ADAS: reflexive effect on drivers’ behavior. Transp. Res. Part F Traffic Psychol. Behav. 25, 27–33 (2014)CrossRefGoogle Scholar
  5. Biondi, F., Strayer, D.L., Rossi, R., Gastaldi, M., Mulatti, C.: Advanced driver assistance systems: using multimodal redundant warnings to enhance road safety. Appl. Ergon. 58, 238–244 (2017)CrossRefGoogle Scholar
  6. Bridgeman, G., Hendry, D., Stark, L.: Failure to detect displacement of visual world during saccadic eye movements. Vis. Res. 15, 719–722 (1975)CrossRefGoogle Scholar
  7. Burr, D.C., Ross, J.: Contrast sensitivity at high velocities. Vis. Res. 22(4), 479–484 (1982)CrossRefGoogle Scholar
  8. Cooper, R., Franks, B.: Interruptibility as a constraint on hybrid systems. Minds Mach. 3(1), 73–96 (1993)CrossRefGoogle Scholar
  9. Cutrell, E., Czerwinski, M., Horvitz, E.: Notification, disruption, and memory: effects of messaging interruptions on memory and performance. In: Interact 2001: IFIP Conference on Human-Computer Interaction, pp. 263–269. IFIP, Tokyo (2001)Google Scholar
  10. Drews, F.A., Yazdani, H., Godfrey, C.N., Cooper, J.M., Strayer, D.L.: Text messaging during simulated driving. Hum. Factors: J. Hum. Factors Ergon. Soc. 51, 762–770 (2009)CrossRefGoogle Scholar
  11. Egeth, H.E., Yantis, S.: Visual attention: control, representation, and time course. Ann. Rev. Psychol. 48(1), 269–297 (1997)CrossRefGoogle Scholar
  12. Evans, L.: Traffic Safety. Science Serving Society, Bloomfield Hills (2004)Google Scholar
  13. Fagerlonn, J.: Distracting effects of auditory warnings on experienced drivers. In: 16th International Conference on Auditory Display (ICAD-2010), pp. 127–132 (2010)Google Scholar
  14. Grane, C., Bengtsson, P.: Haptic addition to a visual menu selection interface controlled by an in-vehicle rotary device. Adv. Hum.-Comput. Interact. 2012, 1–12 (2011)CrossRefGoogle Scholar
  15. Hoffman, J., Lee, J., McGehee, D., Macias, M., Gellatly, A.: Visual sampling of in-vehicle text messages: effects of number of lines, page presentation, and message control. Transp. Res. Rec. J Transp. Res. Board (1937), 22–30 (2005)Google Scholar
  16. Jacobson, P.D., Gostin, L.O.: Reduced distracted driving: regulation and education to avert traffic injuries and fatalities. J. Am. Med. Assoc. 303, 1419–1420 (2010)CrossRefGoogle Scholar
  17. Maples, W.C., DeRosier, W., Hoenes, R., Bendure, R., Moore, S.: The effects of cell phone use on peripheral vision. Optom.-J. Am. Optom. Assoc. 79(1), 36–42 (2008)CrossRefGoogle Scholar
  18. McFarlane, D.C.: Comparison of four primary methods for coordinating the interruption of people in human-computer interaction. Appl. Cogn. Psychol. 18, 533–547 (2002)Google Scholar
  19. Moray, N.: Mental workload since 1979. In: Oborne, D.J. (ed.) International Review of Ergonomics, vol. 2, pp. 123–150. Taylor & Francis, London (1988)Google Scholar
  20. Petersen, H.E., Dugas, D.J.: The relative importance of contrast and motion in visual detection. J. Hum. Factors Ergon. 14(3), 207–216 (1972)CrossRefGoogle Scholar
  21. Scerra, V.E., Brill, J.C.: Effect of task modality on dual-task performance, response time, and ratings of operator workload. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 56, no 1, pp. 1456–1460. Sage Publications, Los Angeles, September 2012CrossRefGoogle Scholar
  22. Strayer, D.L.: Presentation at Cell Phones and Driver Distraction. Traffic Safety Coalition, Washington, DC (2007)Google Scholar
  23. Van Erp, J.B., Van Veen, H.A.: Vibrotactile in-vehicle navigation system. Transp. Res. Part F Traffic Psychol. Behav. 7(4), 247–256 (2004)CrossRefGoogle Scholar
  24. Wickens, C.D.: Processing resources in attention. In: Parasuraman, R., Davies, D.R. (eds.) Varieties of Attention, pp. 63–98. Academic Press Inc., London (1984)Google Scholar
  25. Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3(2), 159–177 (2002)CrossRefGoogle Scholar
  26. Wiese, E.E., Lee, J.D.: Auditory alerts for in-vehicle information systems: the effects of temporal conflict and sound parameters on driver attitudes and performance. Ergonomics 47(9), 965–986 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Pooja P. Bovard
    • 1
    Email author
  • Kelly A. Sprehn
    • 1
  • Meredith G. Cunha
    • 1
  • Jaemin Chun
    • 2
  • SeungJun Kim
    • 3
  • Jana L. Schwartz
    • 1
  • Sara K. Garver
    • 1
  • Anind K. Dey
    • 4
  1. 1.DraperCambridgeUSA
  2. 2.UX Innovation Lab, Samsung ResearchSeoulKorea
  3. 3.Gwangju Institute of Science and Technology (GIST)Buk-gu, GwangjuKorea
  4. 4.Carnegie Mellon UniversityPittsburghUSA

Personalised recommendations