Abstract
Just like humans, conversational computer systems should not listen silently to their input and then respond. Instead, they should enforce the speaker-listener link by attending actively and giving feedback on an utterance while perceiving it. Most existing systems produce direct feedback responses to decisive (e.g. prosodic) cues. We present a framework that conceives of feedback as a more complex system, resulting from the interplay of conventionalized responses to eliciting speaker events and the multimodal behavior that signals how internal states of the listener evolve. A model for producing such incremental feedback, based on multi-layered processes for perceiving, understanding, and evaluating input, is described.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Allwood, J., Cerrato, L.: A study of gestural feedback expressions. In: Paggio, P., K. J.K., Jönsson, A. (eds.) First Nordic Symposium on Multimodal Communication, Copenhagen, 23-24 September, pp. 7–22 (2003)
Allwood, J., Nivre, J., Ahlsen, E.: On the semantics and pragmatics of linguistic feedback. Journal of semantics 9(1), 1–26 (1992)
Cassell, J., Bickmore, T.W., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H.H., Yan, H.: Embodiment in Conversational Interfaces: Rea. In: Proceedings of the CHI 1999 Conference, Pittsburgh, PA, pp. 520–527 (1999)
Cathcart, N., Carletta, J., Klein, E.: A shallow model of backchannel continuers in spoken dialogue. In: EACL 10. Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics, Budapest, April 2003, pp. 51–58 (2003)
Clark, H.H., Schaefer, E.F.: Contributing to discourse. Cognitive Science 13, 259–294 (1989)
Fujie, S., Fukushima, K., Kobayashi, T.: A conversation robot with back-channel feedback function based on linguistic and nonlinguistic information. In: Proc. Int. Conference on Autonomous Robots and Agents (2004)
Graesser, A.C., Lu, S., Jackson, G.T., Mitchell, H., Ventura, M., Olney, A., Louwerse, M.M.: A tutor with dialogue in natural language. Behavioral Research Methods, Instruments, and Computers 36, 180–193 (2004)
Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S., Morales, M., van der Werf, R., Morency, L.-P.: Virtual Rapport. In: Gratch, J., Young, M., Aylett, R., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)
Kopp, S., Allwood, J., Grammer, K., Ahlsen, E., Stocksmeier, T.: Modeling embodied feedback in virtual humans. In: Wachsmuth, I., Knoblich, G. (eds.) Modeling Communication With Robots and Virtual Humans, Springer, Heidelberg (to appear)
Kopp, S., Gesellensetter, L., Krämer, N.C., Wachsmuth, I.: A Conversational Agent as Museum Guide – Design and Evaluation of a Real-World Application. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 329–343. Springer, Heidelberg (2005)
Kopp, S., Wachsmuth, I.: Synthesizing multimodal utterances for conversational agents. Computer Animation & Virtual Worlds 15(1), 39–52 (2004)
Schmid, H.: Improvements in Part-of-Speech Tagging With an Application To German (1995), http://www.ims.uni-stuttgart.de/ftp/pub/corpora/tree-tagger1.pdf
Stocksmeier, T., Kopp, S., Gibbon, D.: Synthesis of prosodic attitudinal variants in german backchannel ’ja’. In: Proc. of Interspeech 2007 (2007)
Takeuchi, M., Kitaoka, N., Nakagawa, S.: Timing detection for realtime dialog systems using prosodic and linguistic information. In: SP 2004. Proc. of the International Conference Speech Prosody, pp. 529–532 (2004)
Thórisson, K.R.: Communicative Humanoids - A Computational Model of Psychosocial Dialogue Skills. PhD thesis, School of Architecture & Planning, Massachusetts Institute of Technology (September 1996)
Ward, N., Tsukahara, W.: Prosodic features which cue back-channel responses in English and Japanese (2000)
Yngve, V.H.: On getting a word in edgewise. In: Papers from the Sixth Regional Meeting of the Chicago Linguistics Society, April 16-18, pp. 567–578. University of Chicago, Department of Linguistics (1970)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kopp, S., Stocksmeier, T., Gibbon, D. (2007). Incremental Multimodal Feedback for Conversational Agents. In: Pelachaud, C., Martin, JC., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds) Intelligent Virtual Agents. IVA 2007. Lecture Notes in Computer Science(), vol 4722. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74997-4_13
Download citation
DOI: https://doi.org/10.1007/978-3-540-74997-4_13
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74996-7
Online ISBN: 978-3-540-74997-4
eBook Packages: Computer ScienceComputer Science (R0)