Abstract
An important research question in multimodal information presentation is, which presentation modes, such as text, speech, images and animations, or combinations thereof, are most suitable for meeting different communicative goals (e.g., instruct, inform, and persuade)? This chapter describes three experiments carried out to find answers to this question. Experiment 1 investigated how people produced (multimodal) information presentations in the medical domain. In Experiment 2, people were asked to evaluate these information presentations on their informativeness and attractiveness. Finally, Experiment 3 evaluated two different methods (caption-based selection versus section-based selection) to automatically illustrate answers to medical questions, and compared the results to those of manually created presentations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arens Y, Hovy E, Vossers M (1993) On the knowledge underlying multimedia presentations. In: MayburyM(ed) Intelligent Multimedia Interfaces, AAAI Press
Berkow R, Beers MH, Fletcher AJ (eds) (2005) Merck manual medisch handboek, 2nd edn. Bohn Stafleu van Loghum, Houten, the Netherlands
Bernsen N (1994) Foundations of multimodal representations. a taxonomy of representational presentation mode. Interacting with Computers 6(4):347ā371
Bosma W (2005) Image retrieval supports multimedia authoring. In: Zudilova- Seinstra E, Adriaansen T (eds) Linguistic Engineering meets Cognitive Engineering in Multimodal Systems, ITC-irst, Trento, Italy, ICMI Workshop, pp 89ā94
Bosma W, Theune M, van Hooijdonk C, Krahmer E, Maes A (2008) Illustrating answers: an evaluation of automatically retrieved illustrations of answers to medical questions. In: Proceedings of the AISB Symposium on Multimodal Output Generation (MOG 2008), pp 34ā41
Bosma W, Marsi E, Krahmer E, Theune M (2010) Text-to-text generation for question answering. In: Bouma G, van den Bosch A (eds) Interactive Multi-modal Question Answering, Springer Verlag
Carney R, Levin J (2002) Pictorial illustrations still improve studentsā learning from text. Educational Psychology Review 14(1):5ā26
van Deemter K, Power R (2003) High-level authoring of illustrated documents. Natural Language Engineering 2(9):101ā126
Fiedeldij Dop P, Vermeent S (eds) (1974)Winkler Prins medische encyclopedie, 3rd edn. Spectrum
Heiser J, Phan D, Agrawala M, Tversky B, Hanrahan P (2004) Identification and validation of cognitive design principles for automated generation of assembly instructions. In: Proceedings of Advanced Visual Interfaces, pp 311ā319
van Hooijdonk C, Krahmer E (2008) Information modalities for procedural instructions: the influence of text, static and dynamic visuals on learning and executing rsi exercises. IEEE Transactions on Professional Communication 51(1):50ā62
van Hooijdonk C, Krahmer E, Maes A, Theune M, Bosma W (2007a) Towards automatic generation of multimodal answers to medical questions: a cognitive engineering approach. In: Proceedings of the Workshop on Multimodal Output Generation (MOG 2007), pp 93ā104
van Hooijdonk C, de Vos J, Krahmer E, Maes A, Theune M, Bosma W (2007b) On the role of visuals in multimodal answers to medical questions. In: Proceedings of the 2007 Conference of the IEEE Professional Communication Society, IEEE
de Jong FMG, Westerveld T, de Vries AP (2007) Multimedia search without visual analysis: the value of linguistic and contextual information. IEEE Transactions on Circuits and Systems for Video Technology 17(3):365ā371
Maybury MT, Merlino AE (1997) Multimedia summaries of broadcast news. In: 1997 IASTED International Conference on Intelligent Information Systems, IEEE
Mayer RE, Moreno R (2002) Aids to computer-based multimedia learning. Learning & Instruction 12(1):107ā119
Mayer RE (2005) The Cambridge handbook of multimedia learning. Cambridge University Press, Cambridge
Michas I, Berry D (2000) Learning a procedural task: effectiveness of multimedia presentations. Applied Cognitive Psychology 14(6):555ā575
Nagao K, Ohira S, YoneokaM(2002) Annotation-based multimedia summarization and translation. In: Proceedings of the 19th international conference on Computational linguistics, Association for Computational Linguistics, Morristown, NJ, USA, pp 1ā7
Petrushin VA (2007) Introduction into Multimedia Data Mining and Knowledge Discovery, Springer London, pp 3ā13
Porter M (1997) An algorithm for suffix stripping. In: Jones KS, Willet P (eds) Readings in Information Retrieval, Morgan Kaufmann, pp 313ā316
Tversky B, Morrison J, BĀ“etrancourt M (2002) Animation; can it facilitate? Int J Human-Computer Studies 57(4):247ā262
Veenker T (2005) WWStim: A CGI script for presenting webbased questionnaires and experiments. Website: http://www.let.uu.nl/ĖTheo.Veenker/personal/projects/wwstim/doc/en/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
van Hooijdonk, C., Bosma, W., Krahmer, E., Maes, A., Theune, M. (2011). Experiments in Multimodal Information Presentation. In: van den Bosch, A., Bouma, G. (eds) Interactive Multi-modal Question-Answering. Theory and Applications of Natural Language Processing. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17525-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-17525-1_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17524-4
Online ISBN: 978-3-642-17525-1
eBook Packages: Computer ScienceComputer Science (R0)