An Agent-Based Dialog Simulation Technique to Develop and Evaluate Conversational Agents

Conference paper
Part of the Advances in Intelligent and Soft Computing book series (AINSC, volume 88)


In this paper, we present an agent-based dialog simulation technique for learning new dialog strategies and evaluate conversational agents. Using this technique the effort necessary to acquire data required to train the dialog model and then explore new dialog strategies is considerably reduced. A set of measures has also been defined to evaluate the dialog strategy that is automatically learned and compare different dialog corpora. We have applied this technique to explore the space of possible dialog strategies and evaluate the dialogs acquired for a conversational agent that collects monitored data from patients suffering from diabetes.


Automatic Speech Recognition Conversational Agent Agent Simulator User Simulation System Dialog 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ai, H., Raux, A., Bohus, D., Eskenazi, M., Litman, D.: Comparing Spoken Dialog Corpora Collected with Recruited Subjects versus Real Users. In: Proc. of the 8th SIGdial Workshop on Discourse and Dialogue, Antwerp, Belgium, pp. 124–131 (2007)Google Scholar
  2. 2.
    Black, L., McTear, M.F., Black, N.D., Harper, R., Lemon, M.: Appraisal of a conversational artefact and its utility in remote patient monitoring. In: Proc. of the 18th IEEE Symposium CBMS 2005, Dublin, Ireland, pp. 506–508 (2005)Google Scholar
  3. 3.
    Griol, D., Sánchez-Pi, N., Carbó, J., Molina, J.: An Architecture to Provide Context-Aware Services by means of Conversational Agents. Advances in Intelligent and Soft Computing 79, 275–282 (2010)CrossRefGoogle Scholar
  4. 4.
    Paek, T., Horvitz, E.: Conversation as action under uncertainty. In: Proc. of the 16th Conference on Uncertainty in Artificial Intelligence, San Francisco (USA), pp. 455–464 (2000)Google Scholar
  5. 5.
    Schatzmann, J., Georgila, K., Young, S.: Quantitative Evaluation of User Simulation Techniques for Spoken Dialogue Systems. In: Proc. of the 6th SIGdial Workshop on Discourse and Dialogue, Lisbon, Portugal, pp. 45–54 (2005)Google Scholar
  6. 6.
    Schatzmann, J., Thomson, B., Weilhammer, K., Ye, H., Young, S.: Agenda-Based User Simulation for Bootstrapping a POMDP Dialogue System. In: Proc. of Human Language Technologies HLT/NAACL 2007 Conference, Rochester, USA, pp. 149–152 (2007)Google Scholar
  7. 7.
    Schatzmann, J., Weilhammer, K., Stuttle, M., Young, S.: A Survey of Statistical User Simulation Techniques for Reinforcement-Learning of Dialogue Management Strategies. Knowledge Engineering Review 21(2), 97–126 (2006)CrossRefGoogle Scholar
  8. 8.
    Young, S.: The Statistical Approach to the Design of Spoken Dialogue Systems. Tech. rep., CUED/F-INFENG/TR 433, Cambridge University Engineering Department, Cambridge, UK (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  1. 1.Group of Applied Artificial Intelligence (GIAA), Computer Science DepartmentCarlos III University of MadridSpain

Personalised recommendations