Abstract
Users of mobile systems often simultaneously perform some other task, and multimodality tends to give them greater opportunities to do so. One goal in the design of mobile multimodal systems should therefore be the support of effective user multitasking. Previous research in several areas has made many contributions that are relevant to this goal, but some key issues require further work. Using the example of voice dialling with a mobile phone, we discuss task analyses of two voice dialling methods, showing how such analyses can help to identify possible obstacles to the simultaneous performance of voice dialling and other tasks. Detailed observations of users doing multitasking, supplemented with survey results, confirm that these analyses capture important aspects of the multitasking problem; but also that users' decisions and behaviour are strongly influenced by factors not covered by the task analyses, such as previous experience and beliefs about social acceptability. Conclusions are drawn concerning the implications of this research for design methods and for future research in support of user multitasking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baber, C. and Mellor, B. (2001). Using critical path analysis to model multimodal human-computer interaction. International Journal of Human-Computer Studies, 54:613–636.
Bernsen, N. O. (2001). Multimodality in language and speech systems—from theory to design support tool. In Granström, B., editor, Multimodality in Language and Speech Systems, pages 93–147. Kluwer Academic Publishers, Dordrecht, The Netherlands.
Bhavnani, S. K. and John, B. E. (2000). The strategic use of complex computer systems. Human-Computer Interaction, 1(2/3):107–137.
Brewster, S., Lumsden, J., Bell, M., Hall, M., and Tasker, S. (2003). Multimodal ‘eyes-free’ interaction techniques for wearable devices. In Terveen, L., Wixon, D., Comstock, E., and Sasse, A., editors, Human Factors in Computing Systems: CHI 2003 Conference Proceedings, pages 473–480. ACM, New York, New York, USA.
Card, S. K., Moran, T. P., and Newell, A. (1983). The Psychology of Human-Computer Interaction. Erlbaum, Hillsdale, New Jersey, USA.
Carroll, J. M. and Rosson, M. B. (1987). The paradox of the active user. In Carroll, J. M., editor, Interfacing Thought: Cognitive Aspects of Human-Computer Interaction, pages 80–111. MIT Press, Cambridge, Massachusetts, USA.
Carroll, J. M., Smith-Kerker, P. L., Ford, J. R., and Mazur-Rimetz, S. A. (1987). The minimal manual. Human-Computer Interaction, 3:123–153.
Cutrell, E., Czerwinski, M., and Horvitz, E. (2001). Notification, disruption and memory: Effects of messaging interruptions on memory and performance. In Proceedings of Interact, pages 263–269, Tokyo, Japan.
Damos, D. L., editor (1991). Multiple-Task Performance. Taylor and Francis, London, United Kingdom.
Gray, W. D., John, B. E., and Atwood, M. E. (1992). The precis of Project Ernestine or an overview of a validation of GOMS. In Bauersfeld, P., Bennett, J., and Lynch, G., editors, Human Factors in Computing Systems: CHI '92 Conference Proceedings, pages 307–312. ACM, New York, New York, USA.
Green, P. (2003). Motor vehicle driver interfaces. In Jacko, J. A. and Sears, A., editors, Human-Computer Interaction Handbook, pages 844–860. Lawrence Erlbaum Assoc., Mahwah, New Jersey, USA.
John, B. E. and Kieras, D. E. (1996). The GOMS family of user interface analysis techniques: Comparison and contrast. ACM Transactions on Computer-Human Interaction, 3:320–351.
Kieras, D. E. and Meyer, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part 2. Accounts of psychological refractory-period phenomena. Psychological Review, 104(4):749–791.
Lyons, K. and Starner, T. (2001). Mobile capture for wearable computer usability testing. In Martin, T. and Bahl, V., editors, Proceedings of the Fifth International Symposium on Wearable Computers, pages 69–76. IEEE Computer, Los Alamitos, California, USA.
Meyer, D. E. and Kieras, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. Psychological Review, 104(1):3–65.
Miyata, Y. and Norman, D. (1986). Psychological issues in support of multiple activities. In User Centered System Design: New Perspectives on Human-Computer Interaction, pages 265–284. Lawrence Erlbaum Assoc., Mahwah, New Jersey, USA, Hillsdale, New Jersey, USA.
Navon, D. (1984). Resources — a theoretical soup stone? Psychological Review, 91(2):216–234.
Oviatt, S. (2000). Multimodal system processing in mobile environments. In Proceedings of Annual ACM Symposium on User Interface Software and Technology, pages 21–30, San Diego, California, USA.
Pascoe, J., Ryan, N., and Morse, D. (2000). Using while moving: HCI issues in fieldwork environments. ACM Transactions on Computer-Human Interaction, 7(3):417–437.
Recarte, M. A. and Nunes, L. M. (2000). Effects of verbal and spatial-imagery tasks on eye fixations while driving. Journal of Experimental Psychology: Applied, 6(1):31–43.
Salvucci, D. D. (2001). Predicting the effects of in-car interface use on driver performance: An integrated model approach. International Journal of Human-Computer Studies, 55:85–107.
Sawhney, N. and Schmandt, C. (2000). Nomadic Radio: Speech and audio interaction for contextual messaging in nomadic environments. ACM Transactions on Computer-Human Interaction, 7:353–383.
Sodhi, M., Reimer, B., Cohen, J. L., Vastenburg, E., Kaars, R., and Kirschenbaum, S. (2002). On-road driver eye movement tracking using head-mounted devices. In Proceedings of ETRA 2002: Eye Tracking Research and Applications Symposium, pages 61–68. New Orleans, Louisiana, USA.
Wickens, C. D. (1984). Processing resources in attention. In Parasuraman, R. and Davies, D. R., editors, Varieties of Attention, pages 63–102. Academic Press, Orlando, Florida, USA.
Wickens, C. D. and Hollands, J. G. (2000). Engineering Psychology and Human Performance. Prentice Hall, Upper Saddle River, New Jersey, USA, 3rd edition.
Wierwille, W. W. (1993). Visual and manual demands of in-car controls and displays. In Peacock, B. and Karwowski, W., editors, Automotive Ergonomics, pages 299–320. Taylor and Francis, London, United Kingdom.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer
About this chapter
Cite this chapter
Jameson, A., Klöckner, K. (2005). User Multitasking with Mobile Multimodal Systems. In: Minker, W., Bühler, D., Dybkjær, L. (eds) Spoken Multimodal Human-Computer Dialogue in Mobile Environments. Text, Speech and Language Technology, vol 28. Springer, Dordrecht. https://doi.org/10.1007/1-4020-3075-4_19
Download citation
DOI: https://doi.org/10.1007/1-4020-3075-4_19
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-3073-4
Online ISBN: 978-1-4020-3075-8
eBook Packages: Computer ScienceComputer Science (R0)