Abstract
In the last decade, the number and variety of secondary tasks in modern vehicles has grown exponentially. To address this variety, drivers can choose between alternative input modalities to complete each task in the most adequate way. However, the process of switching between different modalities might cause increased cognitive effort and finally result in a loss of efficiency. Therefore, the effects of switching between input modalities have to be examined in detail. We present a user study with 18 participants that investigates these effects when switching between touch and speech input on task efficiency and driver distraction in a dual-task setup. Our results show that the sequential combination of adequate modalities for subtasks did not affect task completion times and thus reduced the duration of the entire interaction. We argue to promote modality switches and discuss the implications on application areas beyond the automotive context.
Similar content being viewed by others
References
Bradford JH, James H (1995) The human factors of speech-based interfaces. ACM SIGCHI Bull 27(2):61–67. https://doi.org/10.1145/202511.202527
Card SK, Moran TP, Newell A (1980) The keystroke-level model for user performance time with interactive systems. Commun ACM 23(7):396–410. https://doi.org/10.1145/358886.358895
Gondan M, Lange K, Rösler F, Röder B (2004) The redundant target effect is affected by modality switch costs. Psychon Bull Rev 11(2):307–313
Harbluk JL, Noy YI, Trbovich PL, Eizenman M (2007) An on-road assessment of cognitive distraction: impacts on drivers visual behavior and braking performance. Accid Anal Prev 39(2):372–379. https://doi.org/10.1016/j.aap.2006.08.013
Koons DB, Sparrell CJ, Thorisson KR (1993) Integrating simultaneous input from speech, gaze, and hand gestures. In: Maybury MT (ed) Intelligent multimedia interfaces. AAAI Press, Menlo Park, pp 257–276
Mitrevska M, Moniri MM, Nesselrath R, Schwartz T, Feld M, Korber Y, Deru M, Muller C (2015) SiAM—situation-adaptive multimodal interaction for innovative mobility concepts of the future. In: Proceedings of the international conference on intelligent environments—IE ’15. IEEE, pp 180–183. https://doi.org/10.1109/IE.2015.39
Monsell S (2003) Task switching. Trends Cogn Sci 7(3):134–140. https://doi.org/10.1016/S1364-6613(03)00028-7
Müller C, Weinberg G, Vetro A (2011) Multimodal input in the car, today and tomorrow. IEEE Multimed 18(1):98–103. https://doi.org/10.1109/MMUL.2011.14
Ohn-Bar E, Trivedi MM (2014) Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans Intell Transp Syst 15(6):2368–2377. https://doi.org/10.1109/TITS.2014.2337331
Oviatt S (2012) Multimodal interfaces. In: Jacko JA (ed) The human–computer interaction handbook: fundamentals, evolving technologies and emerging applications, 14th edn. CRC Press, Boca Raton, pp 286–304
Oviatt S, Coulston R, Lunsford R (2004) When do we interact multimodally? Cognitive load and multimodal communication patterns. In: Proceedings of the 2004 international conference on multimodal interfaces—ICMI ’04, pp 129–136. http://doi.acm.org/10.1145/1027933.1027957
Petzoldt T, Bellem H, Krems JF (2014) The critical tracking task: a potentially useful method to assess driver distraction? Hum Fact J Hum Fact Ergonom Soc 56(4):789–808. https://doi.org/10.1177/0018720813501864
Pfleging B, Schneegass S, Schmidt A (2012) Multimodal interaction in the car—combining speech and gestures on the steering wheel. In: Proceedings of the 4th international conference on automotive user interfaces and interactive vehicular applications—AutomotiveUI ’12. ACM, New York, pp 0–7. https://doi.org/10.1145/2390256.2390282
Pickering CA, Burnham KJ, Richardson MJ (2007) A research study of hand gesture recognition technologies and applications for human vehicle interaction. In: Institution of engineering and technology conference on automotive electronics, pp 1–15
Pickering CA, Burnham KKJ, Richardson MJM (2007) A review of automotive human machine interface technologies and techniques to reduce driver distraction. In: 2nd IET international conference on system safety. IEEE, pp 223–228. https://doi.org/10.1049/cp:20070468
Roider F, Rümelin S, Gross T (2018) Using visual cues to leverage the use of speech input in the vehicle. In: International conference on persuasive technology. Springer, Cham, pp 120–131. https://doi.org/10.1007/978-3-319-78978-1_10
Schneegaß S, Pfleging B, Kern D, Schmidt A (2011) Support for modeling interaction with automotive user interfaces. In: Proceedings of the 3rd international conference on automotive user interfaces and interactive vehicular applications—AutomotiveUI ’11, p 71. https://doi.org/10.1145/2381416.2381428
Strayer DL, Watson JM, Drews FA (2011) Cognitive distraction while multitasking in the automobile. In: Federmeier KD (ed) Psychology of learning and motivation, vol 54. Elsevier, Amsterdam. https://doi.org/10.1016/B978-0-12-385527-5.00002-4
Wickens CD, Sandry DL, Vidulich M (1983) Compatibility and resource competition between modalities of input, central processing, and output. Hum Fact 25(2):227–248. https://doi.org/10.1177/001872088302500209
Wierwille WW (1993) Demands on driver resources associated with introducing advanced technology into the vehicle. Transp Res Part C Emerg Technol 1(2):133–142. https://doi.org/10.1016/0968-090X(93)90010-D
Zhang Y, Stellmach S, Sellen A (2015) The costs and benefits of combining gaze and hand gestures for remote. In: Abascal J, Barbosa S, Fetter M, Gross T, Palanque P, Winckler M (eds) Human–computer interaction—INTERACT 2015. INTERACT 2015. Lecture notes in computer science. Springer, Cham. https://doi.org/10.1007/978-3-319-22698-9
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Roider, F., Rümelin, S., Pfleging, B. et al. Investigating the effects of modality switches on driver distraction and interaction efficiency in the car. J Multimodal User Interfaces 13, 89–97 (2019). https://doi.org/10.1007/s12193-019-00297-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-019-00297-9