Journal on Multimodal User Interfaces

, Volume 7, Issue 4, pp 299–310

Influencing factors on multimodal interaction during selection tasks

Original Paper

DOI: 10.1007/s12193-012-0117-5

Cite this article as:
Schüssel, F., Honold, F. & Weber, M. J Multimodal User Interfaces (2013) 7: 299. doi:10.1007/s12193-012-0117-5


When developing multimodal interactive systems it is not clear which importance should be given to which modality. In order to study influencing factors on multimodal interaction, we conducted a Wizard of Oz study on a basic recurrent task: 53 subjects performed diverse selections of objects on a screen. The way and modality of interaction was not specified nor predefined by the system, and the users were free in how and what to select. Natural input modalities like speech, gestures, touch, and arbitrary multimodal combinations of these were recorded as dependent variables. As independent variables, subjects’ gender, personality traits, and affinity towards technical devices were surveyed, as well as the system’s varying presentation styles of the selection. Our statistical analyses reveal gender as a momentous influencing factor and point out the role of individuality for the way of interaction, while the influence of the system output seems to be quite limited. This knowledge about the prevalent task of selection will be useful for designing effective and efficient multimodal interactive systems across a wide range of applications and domains.


Natural human computer interaction  Multimodal interaction User study Gender differences Selection task Input Output 

Copyright information

© OpenInterface Association 2012

Authors and Affiliations

  1. 1.Institute of Media InformaticsUlm UniversityUlmGermany

Personalised recommendations