Machine Learning for Spoken Dialogue Management: An Experiment with Speech-Based Database Querying

  • Olivier Pietquin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4183)


Although speech and language processing techniques achieved a relative maturity during the last decade, designing a spoken dialogue system is still a tailoring task because of the great variability of factors to take into account. Rapid design and reusability across tasks of previous work is made very difficult. For these reasons, machine learning methods applied to dialogue strategy optimization has become a leading subject of researches since the mid 90’s. In this paper, we describe an experiment of reinforcement learning applied to the optimization of speech-based database querying. We will especially emphasize on the sensibility of the method relatively to the dialogue modeling parameters in the framework of the Markov decision processes, namely the state space and the reinforcement signal. The evolution of the design will be exposed as well as results obtained on a simple real application.


Spoken Dialogue Systems Reinforcement Learning Dialogue Management 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)Google Scholar
  2. 2.
    Levin, E., Pieraccini, R., Eckert, W.: Learning Dialogue Strategies within the Markov Decision Process Framework. In: Proceedings of ASRU 1997, Santa Barbara, California (1997)Google Scholar
  3. 3.
    Singh, S., Kearns, M., Litman, D., Walker, M.: Reinforcement Learning for Spoken Dialogue Systems. In: Proceedings of NIPS 1999, Denver, USA (1999)Google Scholar
  4. 4.
    Scheffler, K., Young, S.: Corpus-Based Dialogue Simulation for Automatic Strategy Learning and Evaluation. In: Proceedings of NAACL Workshop on Adaptation in Dialogue Systems (2001)Google Scholar
  5. 5.
    Pietquin, O., Dutoit, T.: A Probabilistic Framework for Dialog Simulation and Optimal Strategy Learning. IEEE Transactions on Audio, Speech and Language Processing 14(2), 589–599 (2006)CrossRefGoogle Scholar
  6. 6.
    Walker, M., Litman, D., Kamm, C., Abella, A.: PARADISE: A Framework for Evaluating Spoken Dialogue Agents. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, Madrid, Spain, pp. 271–280 (1997)Google Scholar
  7. 7.
    Pietquin, O., Beaufort, R.: Comparing ASR Modeling Methods for Spoken Dialogue Simulation and Optimal Strategy Learning. In: Proceedings of Interspeech/Eurospeech 2005, Lisbon, Portugal (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Olivier Pietquin
    • 1
  1. 1.SupélecMetzFrance

Personalised recommendations