Skip to main content
SpringerLink
Log in
Book cover

Data-Driven Methods for Adaptive Spoken Dialogue Systems pp 5–17Cite as

Developing Dialogue Managers from Limited Amounts of Data

  • Chapter
  • First Online:

Abstract

One of the central problems in developing a spoken dialogue system (SDS) is in how the system makes the decision of “what to say next” at any specific point in a conversation. This selection of an appropriate action is the core problem of dialogue management (DM), and it depends on having a representation of the conversational context at each decision point. This context information could consist of, for example, what information has already been conveyed in the dialogue, what the user has said in the preceding utterance (according to a speech recogniser), and the length of the dialogue so far. Making decisions regarding what to say next has been approached in a variety of ways.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    See www.voicexml.org.

  2. 2.

    Note that a common misunderstanding is that the Markov property constrains the state to exclude the dialogue history. However, we can employ variables in the current state which explicitly represent features of the history.

  3. 3.

    This similarity measure is known as a linear kernel, see the discussion in [16].

  4. 4.

    http://www.macs.hw.ac.uk/iLabArchive/CLASSiCProject/Data/myaccount.php.

  5. 5.

    http://www.talk-project.eurice.eu/.

References

  1. Ai, H., Tetreault, J., Litman, D.: Comparing user simulation models for dialog strategy learning. In: Proc. of the North American Meeting of the Association of Computational Linguistics (NAACL), pp. 1–4. Rochester, New York, USA, April 2007

    Google Scholar 

  2. Asher, N., Lascarides, A., Lemon, O., Guhe, M., Rieser, V., Muller, P., Afantenos, S., Benamara, F., Vieu, L., Denis, P., Paul, S., Keizer, S., Degremont, C.: Modelling Strategic Conversation: the STAC project. The 16th workshop on the semantics and Pragmatics of Dialogue (SeineDial’12). Paris, 2012

    Google Scholar 

  3. Bennett, C., Rudnicky, A.: The carnegie mellon communicator corpus. In: Proc. of the International Conference of Spoken Language Processing (ICSLP), 2002

    Google Scholar 

  4. Bohus, D., Langner, B., Raux, A., Black, A.W., Eskenazi, M., Rudnicky, A.L: Online supervised learning of non-understanding recovery policies. In: Proc. of the IEEE/ACL workshop on Spoken Language Technology (SLT), pp. 170–173. Aruba, December 2006

    Google Scholar 

  5. Bos, J., Klein, E., Lemon, O., Oka, T.: DIPPER: Description and formalisation of an Information-State Update dialogue system architecture. In: Proc. of the 4th SIGdial Workshop on Discourse and Dialogue, 2002

    Google Scholar 

  6. Dethlefs, N., Cuayáhuitl, H.: Hierarchical reinforcement learning and hidden markov models for task-oriented natural language generation. In: Proc. of 49th Annual Meeting of the Association for Computational Linguistics, 2011

    Google Scholar 

  7. Dietterrich, T.G.: Machine learning. Nature Encyclopedia of Cognitive Science, 2003

    Google Scholar 

  8. Doran, C., Aberdeen, J., Damianos, L., Hirschman, L.: Comparing several aspects of Human-Computer and Human-Human dialogues. In: Proc. of the 2nd SIGDIAL Workshop on Discourse and Dialogue, 2001

    Google Scholar 

  9. Eckert, W., Levin,  E.,  Pieraccini, R.: User modeling for spoken dialogue system evaluation. In: Proc. of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 80–87. Santa Barbara, CA , USA, December 1997

    Google Scholar 

  10. Forbes-Riley, K., Litman, D.: Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system. Computer Speech and Language 25(1), 105–126 (2011)

    Article  Google Scholar 

  11. Fraser, N.M., Gilbert, G.N.: Simulating speech systems. Computer Speech and Language 5, 81–99 (1991)

    Article  Google Scholar 

  12. Gargett, A., Garoufi, K., Koller, A., Striegnitz, K.: The GIVE-2 corpus of giving instructions in virtual environments. In: Proceedings of the 7th Conference on International Language Resources and Evaluation (LREC), 2010

    Google Scholar 

  13. Gasic, M., Jurcicek, F., Keizer, S., Mairesse, F., Thomson, B., Yu, K., Young, S.: Gaussian processes for fast policy optimisation of a pomdp dialogue manager for a real-world task. In: Proceedings of SIGDIAL, 2010

    Google Scholar 

  14. Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Young, S.: Training and Evaluation of the HIS POMDP Dialogue System in Noise. In: Proc. of SIGdial Workshop on Discourse and Dialogue, 2008

    Google Scholar 

  15. Gesmundo, A., Henderson, J., Merlo, P., Titov, I.: A latent variable model of synchronous syntactic-semantic parsing for multiple languages. In: CoNLL 2009 Shared Task, Conf. on Computational Natural Language Learning, 2009

    Google Scholar 

  16. Henderson, J., Lemon, O., Georgila, K.: Hybrid Reinforcement / Supervised Learning of Dialogue Policies from Fixed Datasets. Computational Linguistics 34(4), 487–513 (2008)

    Article  Google Scholar 

  17. Janarthanam, S., Lemon, O.: Learning to Adapt to Unknown Users: Referring Expression Generation in Spoken Dialogue Systems. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 69–78. Uppsala, Sweden, July 2010

    Google Scholar 

  18. Janarthanam, S., Hastie, H., Lemon, O., Liu, X.: ’The day after the day after tomorrow? ’ A machine learning approach to adaptive temporal expression generation: training and evaluation with real users. In: Proceedings of SIGDIAL, 2011

    Google Scholar 

  19. Jönsson, A., Dahlbäck, N.: Talking to a computer is not like talking to your best friend. In: Proc. of the Scandinavian Conference on Artificial Intelligence, 1988

    Google Scholar 

  20. Koehn, P.: Europarl: A parallel corpus for statistical machine translation. In: Proceedings of the MT Summit, 2005

    Google Scholar 

  21. Larsson, S., Traum, D.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering Special Issue on Best Practice in Spoken Language Dialogue Systems Engineering 6(3-4), 323–340 (2000)

    Google Scholar 

  22. Lemon, O., Pietquin, O.: Machine Learning for spoken dialogue systems. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), pp. 2685–2688. Antwerp, Belgium, September 2007

    Google Scholar 

  23. Mairesse, F., Gašić, M., Jurčíček, F., Keizer, S., Thomson, B., Yu, K., Young, S.: Phrase-based statistical language generation using graphical models and active learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, pp. 1552–1561. Stroudsburg, PA, USA (2010) Association for Computational Linguistics

    Google Scholar 

  24. Moore, R.K., Morris, A.: Experiences collecting genuine spoken enquiries using woz techniques. In: Proc. 5th DARPA workshop on Speech and Natural Language, 1992

    Google Scholar 

  25. Orkin, J., Roy, D.: The restaurant game: Learning social behavior and language from thousands of players online. Journal of Game Development 3(1), 39–60 (2007)

    Google Scholar 

  26. Parent, G., Eskenazi, M.: Toward better crowdsourced transcription: Transcription of a year of the let’s go bus information system data. In: Proc. of the IEEE/ACL Spoken Language Technology (SLT), 2010

    Google Scholar 

  27. Pieraccini, R., Suendermann, D., Dayanidhi, K., Liscombe, J.: Are we there yet? research in commercial spoken dialog systems. In: Proceedings of TSD’09, pp. 3–13, 2009

    Google Scholar 

  28. Pietquin, O., Geist, M., Chandramohan, S., Frezza-Buet, H.: Sample-Efficient Batch Reinforcement Learning for Dialogue Management Optimization. ACM Transactions on Speech and Language Processing 7(3), 21 (2011)

    Article  Google Scholar 

  29. Pietquin, O., Hastie, H., Janarthanam, S., Keizer, S., Putois, G., van der Plas, L.: D6.5 annotated data archive. Technical report, CLASSiC project deliverable, 2011

    Google Scholar 

  30. van der Plas, L., Merlo, P., Henderson, J.: Scaling up cross-lingual semantic annotation transfer. In: In Proceedings of ACL/HLT, 2011

    Google Scholar 

  31. Prommer, T., Holzapfel, H., Waibel, A.: Rapid simulation-driven Reinforcement Learning of multimodal dialog strategies in human-robot interaction. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), pp. 1918–1924. Pittsburgh, Pennsylvania, USA, September 2006

    Google Scholar 

  32. Raux, A., Bohus, D., Langner, B., Black, A.W., Eskenazi, M.: Doing research on a deployed spoken dialogue system: One year of let’s go! experience. In: Proc. of Interspeech, 2006

    Google Scholar 

  33. Rieser, V.: Bootstrapping Reinforcement Learning-based Dialogue Strategies from Wizard-of-Oz data. PhD thesis, Saarbrueken Dissertations in Computational Linguistics and Language Technology, vol. 28, 2008

    Google Scholar 

  34. Rieser, V., Kruijff-Korbayová, I., Lemon, O.: A corpus collection and annotation framework for learning multimodal clarification strategies. In: Proc. of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 97–106. Lisbon, Portugal, September 2005

    Google Scholar 

  35. Rieser, V., Lemon, O., Liu, X.: Optimising Information Presentation for Spoken Dialogue Systems. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pp. 1009–1018. Uppsala, Sweden, July 2010

    Google Scholar 

  36. Rieser, V., Lemon, O.: Learning effective multimodal dialogue strategies from Wizard-of-Oz data: Bootstrapping and evaluation. In: Proc. of the 21st International Conference on Computational Linguistics and 46th Annual Meeting of the Association for Computational Linguistics (ACL/HLT), pp. 638–646. Columbus, Ohio, USA, June 2008

    Google Scholar 

  37. Rieser, V., Lemon, O.: Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets. Computational Linguistics 37(1), 2011

    Google Scholar 

  38. Rieser, V., Lemon, O.: Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems. In: Proc. of the Conference of European Chapter of the Association for Computational Linguistics (EACL), 2009

    Google Scholar 

  39. Rieser, V., Lemon, O.: Reinforcement Learning for Adaptive Dialogue Systems: A Data-driven Methodology for Dialogue Management and Natural Language Generation. Theory and Applications of Natural Language Processing, Springer (2011)

    Google Scholar 

  40. Schatzmann, J., Thomson, B., Weilhammer, K., Ye, H., Young, S.: Agenda-based user simulation for bootstrapping a POMDP dialogue system. In: Proc. of the North American Meeting of the Association of Computational Linguistics (NAACL), pp. 149–152. Rochester, New York, USA, April 2007

    Google Scholar 

  41. Schatzmann, J., Weilhammer, K., Stuttle, M., Young, S.: A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies. Knowledge Engineering Review 21(2), 97–126 (2006)

    Article  Google Scholar 

  42. Stent, A., Walker, M., Whittaker, S., Maloor, P.: User-tailored generation for spoken dialogue: an experiment. In: Proc. of ICSLP, 2002

    Google Scholar 

  43. Stuttle, M.N., Williams, J.D., Young, S.: A framework for dialogue data collection with a simulated ASR channel. In: Proc. of the International Conference of Spoken Language Processing (Interspeech/ICSLP), Jeju, South Korea, October 2004

    Google Scholar 

  44. Sutton, R., Barto, A.: Reinforcement Learning. MIT Press (1998)

    Google Scholar 

  45. Thomson, B., Young, S.: Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems. Computer Speech and Language 24(4), 562–588 (2010)

    Article  Google Scholar 

  46. Tvarožek, J., Bieliková, M.: Wizard-of-oz-driven bootstrapping of a socially intelligent tutoring strategy. In: Bastiaens, T., Ebner, M. (eds.) Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2011, pp. 3635–3644. Lisbon, Portugal, June 2011, AACE

    Google Scholar 

  47. Walker, M., Kamm, C., Litman, D.: Towards developing general models of usability with PARADISE. Natural Language Engineering 6(3), 363–377 (2000)

    Article  Google Scholar 

  48. Whittaker, S., Walker, M., Maloor, P.: Should i tell all? an experiment on conciseness in spoken dialogue. In: Proc. European Conference on Speech Processing (EUROSPEECH), 2003

    Google Scholar 

  49. Williams, J., Young, S.: Using Wizard-of-Oz simulations to bootstrap Reinforcement-Learning-based dialog management systems. In: Proc. of the 4th SIGDIAL Workshop on Discourse and Dialogue, pp. 135–139. Sapporo, Japan, July 2004

    Google Scholar 

  50. Young, S., Gasic, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The Hidden Information State Model: a practical framework for POMDP-based spoken dialogue management. Computer Speech and Language 24(2), 150–174 (2009)

    Article  Google Scholar 

  51. Young, S.: Probabilistic methods in spoken dialogue systems. Philosophical Trans Royal Society (Series A) 358(1769), 1389–1402 (2000)

    Google Scholar 

Download references

Acknowledgements

The research leading to these results has received partial support from the European Community’s Seventh Framework Programme (FP7) under grant agreement no. 216594 (classic project), from the EPSRC, project no. EP/G069840/1, and from the European Community’s Seventh Framework Programme (FP7) under grant agreement no. 269427 (STAC project), under grant agreement no. 270019 (SpaceBook project), under grant agreement no. 270435 (james project), and under grant agreement no. 287615 (PARLANCE project).

Author information

Authors and Affiliations

  1. Heriot-Watt University, Edinburgh, EH14 4AS, UK

    Verena Rieser & Oliver Lemon

Authors
  1. Verena Rieser
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oliver Lemon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Verena Rieser .

Editor information

Editors and Affiliations

  1. , Mathematics and Computer Science, Heriot Watt University, Edinburgh, EH14 4AS, United Kingdom

    Oliver Lemon

  2. , Metz Campus - IMS Research Group, SUPELEC, rue Edouard Belin 2, Metz, 57070, France

    Olivier Pietquin

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media New York

About this chapter

Cite this chapter

Rieser, V., Lemon, O. (2012). Developing Dialogue Managers from Limited Amounts of Data. In: Lemon, O., Pietquin, O. (eds) Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4803-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-4803-7_2

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-4802-0

  • Online ISBN: 978-1-4614-4803-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation