Language Resources and Evaluation

, Volume 41, Issue 3–4, pp 255–272 | Cite as

The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation

  • Jens Allwood
  • Stefan Kopp
  • Karl Grammer
  • Elisabeth Ahlsén
  • Elisabeth OberzaucherEmail author
  • Markus Koppensteiner


Communicative feedback refers to unobtrusive (usually short) vocal or bodily expressions whereby a recipient of information can inform a contributor of information about whether he/she is able and willing to communicate, perceive the information, and understand the information. This paper provides a theory for embodied communicative feedback, describing the different dimensions and features involved. It also provides a corpus analysis part, describing a first data coding and analysis method geared to find the features postulated by the theory. The corpus analysis part describes different methods and statistical procedures and discusses their applicability and the possible insights gained with these methods.


Communicative embodied feedback Contact Perception Understanding Emotions Multimodal Embodied communication 



We thank the Ludwig Boltzmann Institute for Urban Ethology in Vienna for help with data collection and transcription, the Department of Linguistics and SSKKII Center for Cognitive Science, Göteborg University, and the ZiF Center of Interdisciplinary Research in Bielefeld/Germany.


  1. Allwood, J. (1976). Linguistic communication as action and cooperation. Gothenburg monographs in linguistics 2. Göteborg University, Department of Linguistics.Google Scholar
  2. Allwood, J. (2000). Structure of dialog. In M. Taylor, D. Bouwhuis, & F. Neel (Eds.), The structure of multimodal dialogue II (pp. 3–24). Amsterdam: Benjamins.Google Scholar
  3. Allwood, J., Cerrato, L., Dybjær, L., Jokinen, K., Navaretta, C., & Paggio, P. (2005). The MUMIN Multimodal Coding Scheme. NorFA Yearbook 2005.Google Scholar
  4. Allwood, J., NivreJ., & Ahlsén, E. (1992). On the semantics and pragmatics of linguistic feedback. Journal of Semantics, 9(1), 1–26.CrossRefGoogle Scholar
  5. Bakeman, R., & Quera, V. (1995). Analyzing interaction: Sequential analysis with SDIS and GSEQ. New York: Cambridge University Press.Google Scholar
  6. Grammer K., Honda R., Schmitt A., & Jütte A. (1999). Fuzziness of nonverbal courtship communication. Unblurred by motion energy detection. Journal of Personality and Social Psychology, 77(3), 487–508.CrossRefGoogle Scholar
  7. Grammer K., Kruck K., Juette A., & Fink B. (2000): Non-verbal behavior as courtship signals: The role of control and choice in selecting partners. Evolution and Human Behavior, 21, 371–390.CrossRefGoogle Scholar
  8. Kopp, S., Gesellensetter, L., Krämer, N., & Wachsmuth, I. (2005): A conversational agents as museum guide: design and evaluation of a real-world application. In Panayiotopoulos et al. (Eds.) Intelligent Virtual Agents, LNAI 3661 (pp. 329–343). Berlin: Springer-Verlag.Google Scholar
  9. Kopp, S., Allwood, J., Grammer, K., Ahlsen, E., & Stocksmeier, T. (2008): Modeling Embodied Feedback With Virtual Humans. In I. Wachsmuth & G. Knoblich (Eds.), Modeling embodied communication in robots and virtual humans, LNAI 4930. Springer-Verlag.Google Scholar
  10. Magnusson, M. S. (1996). Hidden real-time patterns in intra- and inter-individual behavior: description and detection. European Journal of Psychology Assessment, 12, 112–123.CrossRefGoogle Scholar
  11. Magnusson, M. S. (2000). Discovering hidden time patterns in behavior: T-patterns and their detection. Behavior Research Methods and Instrumentation, 32, 93–110.Google Scholar
  12. Peirce, C. S. (1931). In C. Hartshorne, P. Weiss & A. Burks (Eds.), Collected Papers of Charles Sanders Peirce, 1931–1958, 8 vols. Cambridge, Mass.: Harvard University Press.Google Scholar
  13. Popper, K. (1959). The logic of scientific discovery (trans. Logik der Forschung). London: Hutchinson.Google Scholar
  14. Scherer, K. T. (1999). Appraisal theory. In T. Dalgleish & M. J. Power (Eds.), Handbook of emotion and cognition (pp. 637–663). Chichester: New York.CrossRefGoogle Scholar
  15. Ward, N., & Tsukahara, W. (2000). Prosodic features which cue back-channel responses in English and Japanese. Journal of Pragmatics, 32, 1177–1207.CrossRefGoogle Scholar
  16. Wiener, N. (1948). Cybernetics or control and communication in the animal and the machine. MIT Press.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Jens Allwood
    • 1
  • Stefan Kopp
    • 2
  • Karl Grammer
    • 3
  • Elisabeth Ahlsén
    • 1
  • Elisabeth Oberzaucher
    • 3
    Email author
  • Markus Koppensteiner
    • 3
  1. 1.Department of LinguisticsGöteborg UniversityGöteborgSweden
  2. 2.Artificial Intelligence GroupBielefeld UniversityBielefeldGermany
  3. 3.Ludwig Boltzmann Institute for Urban Ethology, Department of AnthropologyUniversity of ViennaViennaAustria

Personalised recommendations