Abstract
With variety of interaction technologies like speech, pen, touch, hand or body gestures, eye gaze, etc., being now available for users, it is a challenge to design optimal and effective multimodal combinations for specific tasks. For designing that, it is important to understand how these modalities can be combined and used in a coordinated manner. We performed an experimental evaluation of combinations of different multimodal inputs, such as keyboard, speech and touch with pen etc, in an attempt to investigate, which combinations are efficient for diverse needs of the users. In our study, multimodal combination of three modalities was found to be more effective in terms of performance, accuracy and user experience than that of two modalities. Further, we also inferred the roles that each of the modalities play in a multimodal combination to achieve the usability goals.
Chapter PDF
Similar content being viewed by others
References
Dybkaer, L., Bernsen, N.O., Minker, W.: New Challenges in Usability Evaluation-Beyond Task-Oriented Spoken Dialogue Systems. In: ICSLP, vol. 3, pp. 2261–2264 (2004)
Bretan, I., Karlgren, J.: Synergy Effects in Natural Language-Based Multimodal Interaction. SICS Research Report (1994)
Wechsung, I., Engelbrecht, K.P., Schaffer, S., Seebode, J., Metze, F., Möller, S.: Usability Evaluation of Multimodal Interfaces: Is the Whole the Sum of Its Parts? In: Jacko, J.A. (ed.) HCI International 2009, Part II. LNCS, vol. 5611, pp. 113–119. Springer, Heidelberg (2009)
Oviatt, S.: User-centered modeling and evaluation of multimodal interfaces. IEEE 91(9), 1457–1468 (2003)
Ren, X., Zhang, G., Dai, G.: An experimental study of input modes for multimodal human-computer interaction. In: Tan, T., Shi, Y., Gao, W. (eds.) ICMI 2000. LNCS, vol. 1948, pp. 49–56. Springer, Heidelberg (2000)
Metze, F., Wechsung, I., Schaffer, S., Seebode, J., Möller, S.: Reliable Evaluation of Multimodal Dialogue Systems. In: Jacko, J.A. (ed.) Human-Computer Interaction, Part II, HCII 2009. LNCS, vol. 5611, pp. 75–83. Springer, Heidelberg (2009)
Bernsen, N.O., Dybkjær, L.: Evaluation of spoken multimodal conversation. In: 6th ACM International Conference on Multimodal Interfaces, pp. 38–45 (2004)
Serrano, M., Nigay, L.: A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces. J. Multimodal User Interfaces 3(3), 215–225 (2010)
Bernhaupt, R., Palanque, P., Winckler, M., Navarre, D.: Usability Study of Multi-modal Interfaces Using Eye-Tracking. In: Baranauskas, C., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4663, pp. 412–424. Springer, Heidelberg (2007)
Bernhaupt, R., Navarre, D., Palanque, P., Winckler, M.: Model-Based Evaluation: A New Way to Support Usability Evaluation of Multimodal Interactive Applications. In: Maturing Usability: Quality in Software, Interaction and Quality, pp. 96–122 (2007)
Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., Clow, J.: QuickSet: Multimodal Interaction for Distributed Applications. In: 5th ACM International Conference on Multimedia, pp. 31–40 (1997)
Kaster, T., Pfeiffer, M., Bauckhage, C.: Combining Speech and Haptics for Intuitive and Efficient Navigation through Image Databases. In: 5th ACM International Conference on Multimodal Interfaces, pp. 180–187 (2003)
Beelders, T.R., Blignaut, P.J.: The Usability of Speech and Eye Gaze as a Multimodal Interface for a Word Processor. In: Speech Technologies, pp. 386–404 (2011)
Microsoft Voice Recognition System, http://www.microsoft.com/enable/products/windowsvista/speech.aspx (last retrieved on February 26, 2013)
Sauro, J., Kindlund, E.A.: Method to standardize usability metrics into a single score. In: ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 401–409 (2005)
Hornbæk, K., Law, E.L.C.: Meta-analysis of correlations among usability measures. In: ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 617–626 (2007)
Cohen, P.R., Johnston, M., McGee, D., Oviatt, S.L., Clow, J., Smith, I.: The efficiency of multimodal interaction: A case study. In: International Conference on Spoken Language Processing, vol. 2, pp. 249–252 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ghosh, S., Joshi, A., Tripathi, S. (2013). Empirical Evaluation of Multimodal Input Interactions. In: Yamamoto, S. (eds) Human Interface and the Management of Information. Information and Interaction Design. HIMI 2013. Lecture Notes in Computer Science, vol 8016. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39209-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-39209-2_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39208-5
Online ISBN: 978-3-642-39209-2
eBook Packages: Computer ScienceComputer Science (R0)