Non-verbal communication is important in order to maintain fluency of communication. Gestures, facial expressions and eye-gazing function as non-verbal means to convey feedback and provide subtle cues to control and organise conversations. In this paper, verbal and non-verbal feedback are discussed from the point of view of how they contribute to the communicative activity in conversations, especially the type of strategies that the speakers deploy when they aim to construct shared understanding of the tasks and duties in interaction in general. The study concerns conversational data, collected for the purposes of designing and developing more natural interactive systems.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., Paggio, P.: The MUMIN coding scheme for the annotation of feedback, turn management and sequencing phenomena. In: Martin, J.C., Paggio, P., Kuenlein, P., Stiefelhagen, R., Pianesi, F. (eds.) Multimodal corpora for modelling human multimodal behaviour. Special issue of the International Journal of Language Resources and Evaluation, vol. 41(3–4), pp. 273–287 (2007)Google Scholar
  2. 2.
    André, E., Pelachaud, C.: Interacting with Embodied Conversational Agents. In: Jokinen, K., Cheng, F. (eds.) Speech-based Interactive Systems: Theory and Applications. Springer, Heidelberg (2009)Google Scholar
  3. 3.
    Bennewitz, M., Faber, F., Joho, D., Behnke, S.: Fritz - A Humanoid Communication Robot. In: Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (2007)Google Scholar
  4. 4.
    Campbell, N.: On the Use of NonVerbal Speech Sounds in Human Communication. In: Esposito, A., Faundez-Zanuy, M., Keller, E., Marinaro, M. (eds.) COST Action 2102. LNCS (LNAI), vol. 4775, pp. 117–128. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  5. 5.
    Campbell, N.: Speech and expression; the value of a longitudinal corpus. In: The 4th International Conference on Language Resources and Evaluation (LREC), pp. 183–186 (2004)Google Scholar
  6. 6.
    Campbell, N., Ohara, R.: How far can non-verbal information help us follow a conversation? Preliminary experiments with speech-style and gesture tracking. In: Proceedings of ATR Symposium on the Cross-Modal Processing of Faces & Voices. No laughing matter (2005)Google Scholar
  7. 7.
    Carletta, J.: Announcing the AMI Meeting Corpus. The ELRA Newsletter 11(1), 3–5 (2006)Google Scholar
  8. 8.
    Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.): Embodied Conversational Agents. MIT Press, Cambridge (2003)Google Scholar
  9. 9.
    Clark, H.H., Schaefer, E.F.: Contributing to Discourse. Cognitive Science 13, 259–294 (1989)CrossRefGoogle Scholar
  10. 10.
    Douglas, C.E., Campbell, N., Cowie, R., Roach, P.: Emotional speech: Towards a new generation of databases. Speech Communication 40, 33–60 (2003)CrossRefMATHGoogle Scholar
  11. 11.
    Ekman, P.: Universal and cultural differences in facial expression of emotion. In: Cole, J.R. (ed.) Nebraska Symposium on Motivation, pp. 207–283. Nebraska University Press, Lincoln (1972)Google Scholar
  12. 12.
    Feldman, R.S., Rim, B.: Fundamentals of Nonverbal Behavior. Cambridge University Press, Cambridge (1991)Google Scholar
  13. 13.
    Jokinen, K.: Constructive Dialogue Management: Speech Interaction and Rational Agents. John Wiley and Sons, Chichester (2009a)CrossRefGoogle Scholar
  14. 14.
    Jokinen, K.: Natural Language and Dialogue Interfaces. In: Stephanidis, C. (ed.) The Universal Access Handbook, Cenveo, ch. 31, pp. 495–506 (2009b)Google Scholar
  15. 15.
    Jokinen, K.: Non-verbal Feedback in Interactions. In: Tao, J.H., Tan, T.N. (eds.) Affective Information Processing, pp. 227–240. Science+Business Media LLC. Springer, London (2008)Google Scholar
  16. 16.
    Jokinen, K., Campbell, N.: Non-verbal Information Sources for Constructive Dialogue Management. In: LREC 2008, Marrakech, Marocco (2008)Google Scholar
  17. 17.
    Jokinen, K., Vanhasalo, M.: Stand-up gestures – Annotation for Communication Management. In: Proceedings of the Multimodal Workshop at Nodalida Conference, Denmark (2009)Google Scholar
  18. 18.
    Kendon, A.: Gesture: Visible action as utterance. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar
  19. 19.
    Koiso, H., Horiuchi, Y., Tutiya, S., Ichikawa, A., Den, Y.: An analysis of turn taking and backchannels based on prosodic and syntactic features in Japanese Map Task dialogs. Language and Speech 41(3-4), 295–321 (1998)Google Scholar
  20. 20.
    McNeill, D.: Gesture and Thought. University of Chicago Press, Chicago (2005)CrossRefGoogle Scholar
  21. 21.
    Norros, L., Kaasinen, E., Plomp, J., Rämä, P.: Human-Technology Interaction Research and Design. VTT Roadmap. VTT Industrial Systems, VTT Research Notes 2220. Espoo (2003), http://www.vtt.fi/inf/pdf/tiedotteet/2003/T2220.pdf
  22. 22.
    Traum, D.: Computational models of grounding in collaborative systems. In: Working Papers of the AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems, pp. 124–131. AAAI, Menlo Park (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Kristiina Jokinen
    • 1
  1. 1.University of HelsinkiFinland

Personalised recommendations