KI - Künstliche Intelligenz

, Volume 27, Issue 2, pp 137–143 | Cite as

Co-constructing Grounded Symbols—Feedback and Incremental Adaptation in Human–Agent Dialogue

  • Hendrik Buschmeier
  • Stefan Kopp
Technical Contribution


Grounding in dialogue concerns the question of how the gap between the individual symbol systems of interlocutors can be bridged so that mutual understanding is possible. This problem is highly relevant to human–agent interaction where mis- or non-understanding is common. We argue that humans minimise this gap by collaboratively and iteratively creating a shared conceptualisation that serves as a basis for negotiating symbol meaning. We then present a computational model that enables an artificial conversational agent to estimate the user’s mental state (in terms of contact, perception, understanding, acceptance, agreement and based upon his or her feedback signals) and use this information to incrementally adapt its ongoing communicative actions to the user’s needs. These basic abilities are important to reduce friction in the iterative coordination process of co-constructing grounded symbols in dialogue.


Symbol grounding Dialogue Feedback Adaptation Human-agent Interaction 



This research is supported by the Deutsche Forschungsgemeinschaft (DFG) in the Center of Excellence EXC 277 in ‘Cognitive Interaction Technology’ (CITEC).


  1. 1.
    Allwood J, Nivre J, Ahlsén E (1992) On the semantics and pragmatics of linguistic feedback. J Semant 9:1–26 CrossRefGoogle Scholar
  2. 2.
    Bavelas JB, Coates L, Johnson T (2000) Listeners as co-narrators. J Pers Soc Psychol 79:941–952 CrossRefGoogle Scholar
  3. 3.
    Bavelas JB, Coates L, Johnson T (2002) Listener responses as a collaborative process: the role of gaze. J Commun 52:566–580 CrossRefGoogle Scholar
  4. 4.
    Brennan SE, Clark HH (1996) Conceptual pacts and lexical choice in conversation. J Exp Psychol Learn Mem Cogn 22:1482–1493 CrossRefGoogle Scholar
  5. 5.
    Buschmeier H, Baumann T, Dosch B, Kopp S, Schlangen D (2012) Combining incremental language generation and incremental speech synthesis for adaptive information presentation. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Seoul, South Korea, pp 295–303 Google Scholar
  6. 6.
    Buschmeier H, Kopp S (2011) Towards conversational agents that attend to and adapt to communicative user feedback. In: Proceedings of the 11th International Conference on Intelligent Virtual Agents, Reykjavík, Iceland, pp 169–182 CrossRefGoogle Scholar
  7. 7.
    Buschmeier H, Kopp S (2012) Using a Bayesian model of the listener to unveil the dialogue information state. In: SemDial 2012: Proceedings of the 16th Workshop on the Semantics and Pragmatics of Dialogue, Paris, France, pp 12–20 Google Scholar
  8. 8.
    Chao YR (1968) Language and symbolic systems. Cambridge University Press, Cambridge Google Scholar
  9. 9.
    Clark HH (1996) Using language. Cambridge University Press, Cambridge CrossRefGoogle Scholar
  10. 10.
    Clark HH, Krych MA (2004) Speaking while monitoring addressees for understanding. J Mem Lang 50:62–81 CrossRefGoogle Scholar
  11. 11.
    Clark HH, Schaefer EF (1989) Contributing to discourse. Cogn Sci 13:259–294 CrossRefGoogle Scholar
  12. 12.
    Clark HH, Wilkes-Gibbs D (1986) Referring as a collaborative process. Cognition 22:1–39 CrossRefGoogle Scholar
  13. 13.
    Gravano A, Hirschberg J (2011) Turn-taking cues in task-oriented dialogue. Comput Speech Lang 25:601–634 CrossRefGoogle Scholar
  14. 14.
    Grice HP (1975) Logic and conversation. In: Cole P, Morgan JL (eds) Syntax and semantics 3: Speech acts. Academic Press, New York, pp 41–58 Google Scholar
  15. 15.
    Harnad S (1990) The symbol grounding problem. Physica D 42:335–346 CrossRefGoogle Scholar
  16. 16.
    Heldner M, Edlund J, Hirschberg J (2010) Pitch similarity in the vicinity of backchannels. In: Proceedings of INTERSPEECH 2010. Makuhari, Japan, pp 3054–3057 Google Scholar
  17. 17.
    de Kok I, Heylen D (2011) The MultiLis corpus – Dealing with individual differences in nonverbal listening behavior. In: Proceedings of the 3rd COST 2102 International Training School, Caserta, Italy, pp 362–375 Google Scholar
  18. 18.
    de Kok I, Ozkan D, Heylen D, Morency LP (2010) Learning and evaluating response prediction models using parallel listener consensus. In: Proceedings of the 12th International Conference on Multimodal Interfaces, Beijing, China Google Scholar
  19. 19.
    Kopp S, Allwood J, Grammar K, Ahlsén E, Stocksmeier T (2008) Modeling embodied feedback with virtual humans. In: Wachsmuth I, Knoblich G (eds) Modeling communication with robots and virtual humans. Springer, Berlin, pp 18–37 CrossRefGoogle Scholar
  20. 20.
    Morency LP, de Kok I, Gratch J (2010) A probabilistic multimodal approach for predicting listener backchannels. Auton Agents Multiagent Syst 20:70–84 CrossRefGoogle Scholar
  21. 21.
    Reidsma D, de Kok I, Neiberg D, Pammi S, van Straalen B, Truong K, van Welbergen H (2011) Continuous interaction with a virtual human. J Multimodal User Interfaces 4:97–118 CrossRefGoogle Scholar
  22. 22.
    Sacks H, Schegloff EA, Jefferson G (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50:696–735 CrossRefGoogle Scholar
  23. 23.
    Schröder M, Bevacqua E, Cowie R, Eyben F, Gunes H, Heylen D, ter Maat M, McKeown G, Pammi S, Pantic M, Pelachaud C, Schuller B, de Sevin E, Valstar M, Wollmer M (2012) Building autonomous sensitive artificial listeners. IEEE Trans Affect Comput 3:165–183 CrossRefGoogle Scholar
  24. 24.
    Wang Z, Lee J, Marsella S (2011) Towards more comprehensive listening behavior: beyond the bobble head. In: Proceedings of the 11th International Conference on Intelligent Virtual Agents, Reykjavík, Iceland, pp 216–227 CrossRefGoogle Scholar
  25. 25.
    Ward N (2006) Non-lexical conversational sounds in American English. Pragmat Cogn 14:129–182 CrossRefGoogle Scholar
  26. 26.
    Ward N, Tsukahara W (2000) Prosodic features which cue back-channel responses in English and Japanese. J Pragmat 38:1177–1207 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Sociable Agents Group, CITECBielefeld UniversityBielefeldGermany

Personalised recommendations