Toward Optimization of Multimodal User Interfaces for Tactical Audio Applications

Abstract

Tactical audio uses audio feedback to facilitate the precise and accurate positioning of an object with respect to some other object. Existing solutions in pointing and trajectory based Human Computer Interface (HCI) tasks have primarily explored visual feedback, sometimes in rather limited conditions. In this paper we have examined different sonification paradigms for tactical audio to improve the accuracy of pursuit tracking. We have developed a multimodal simulation system as an open environment for evaluation of different sonification and visualization techniques in tactical audio applications. The environment is implemented as an audio-visual scene using the Java3D package. The paper presents the quantitative results of of three pursuit tracking applications using a combination of acoustic and visual guidance and different background conditions. Experiments with 19 participants have shown that acoustic presentation improves the quality of human-computer interaction and reduces the error during pursuit tracking tasks for up to 19%. Moreover, experiments have shown that benefits do not exist in all conditions, indicating the existence of perceptual boundaries of multimodal HCI for different scene complexity and target speeds. Significant benefits of audio modes exist for medium complexity of interaction. User’s questionnaires indicate that users are not aware of quantitative benefits of applied sonification paradigms. We have shown that the most appealing paradigms are not the most effective ones, which necessitates a careful quantitative analysis of proposed multi-modal HCI paradigms.