Abstract
A basic operations approach to designing multimodal user interfaces has been explored in two experimental studies. By basic operations is here meant the most fundamental operations that make up a(ny) interaction task. Study A investigated if certain basic operations have a preferred modal counterpart and execution and Study B what happens when several basic operations are combined into tasks. The results suggest interdependencies between basic operations and specific modalities but also indicate that users may prefer unimodal interaction or to choose modality as desired. Hence, further studies are needed to assess the feasibility of a basic operations approach to multimodal interaction design.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Althoff F, McGlaun G, Lang M, Rigoll G (2004) Evaluating multimodal interaction patterns in various application scenarios. In: Camurri A, Volpe G (eds) Gesture-based communication in human-computer interaction. Lecture notes in computer science, vol 2915. Springer, Heidelberg, pp 421–435
Carrino F et al (2012) In-vehicle natural interaction based on electromyography. In: Adjunct proceedings of the AutomotiveUI 2012. ACM, New York, pp 23–24
Chen X, Tremaine M (2005) Multimodal user input patterns in a non-visual context. In: Proceedings of the 7th international ACM SIGACCESS conference on computers and accessibility, pp 206–207
Hausen D, Richter H, Hemme A, Butz A (2013) Comparing input modalities for peripheral interaction: a case study on peripheral music control. In: Kotzé P, Marsden G, Lindgaard G, Wesson J, Winckler M (eds) Human-computer interaction – INTERACT 2013. Lecture notes in computer science, vol 8119. Springer, Heidelberg, pp 162–179
Janowski K, Kistler F, André E (2013) Gestures or Speech? Comparing modality selection for different interaction tasks in a virtual environment. In: Proceedings of the Tilburg gesture research meeting
Krahnstoever N, Kettebekov S, Yeasin M, Sharma R (2002) A real-time framework for natural multimodal interaction with large screen displays. In: Proceedings of the 4th IEEE international conference on multimodal interfaces, Washington D.C. IEEE Computer Society Press, pp 349–354
Muller C, Weinberg G (2011) Multimodal input in the car, today and tomorrow. IEEE Multimed 18(1):98–103
Naumann AB, Wechsung I, Möller S (2008) Factors influencing modality choice in multimodal applications. In: André E, Dybkjær L, Minker W, Neumann H, Pieraccini R, Weber M (eds) Perception in multimodal dialogue systems. PIT 2008. Lecture notes in computer science, vol 5078. Springer, Heidelberg, pp 37–43
Oviatt S (1999) Ten myths of multimodal interaction. Commun ACM 42(11):74–81
Oviatt S, Coulston R, Lunsford R (2004) When do we interact multimodally? Cognitive load and multimodal communication patterns. In: Proceedings of the 6th International Conference on Multimodal Interfaces. ACM, New York, pp 129–136
Oviatt S et al (2003) Toward a theory of organized multimodal integration patterns during human-computer interaction. In: Proceedings of the 5th international conference on multimodal interfaces. ACM, New York, pp 44–51
Shi Y, Taib R, Ruiz N, Choi E, Chen F (2007) Multimodal human-machine interface and user cognitive load measurement. IFAC Proc Vol 40(16):200–205
Tang L, Lundgren J (2016) Exploring multimodality and natural interaction in a driving context. Master Thesis, Chalmers University of Technology, Gothenburg
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
MariAnne Karlsson, I.C., Ekman, F., Johansson, M. (2019). Designing Multi-modal Interaction – A Basic Operations Approach. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds) Proceedings of the 20th Congress of the International Ergonomics Association (IEA 2018). IEA 2018. Advances in Intelligent Systems and Computing, vol 827. Springer, Cham. https://doi.org/10.1007/978-3-319-96059-3_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-96059-3_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-96058-6
Online ISBN: 978-3-319-96059-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)