Abstract
Spoken dialogue systems (SDS) allow users to interact with a wide variety of information systems using speech as the primary, and often the only, communication medium. The principal elements of an SDS are a speech understanding component which converts each spoken input into an abstract semantic representation called a user dialogue act (see Chap. 3), a dialogue manager which responds to the user’s input and generates a system act a t in response, and a message generator which converts each system act back into speech (see Chap. 6). At each turn t, the system updates its state s t , and based on a policy π, it determines the next system act a t = π(s t ). The state consists of the variables needed to track the progress of the dialogue and the attribute values (often called slots) that determine the user’s requirements. In conventional systems, as discussed in Chap. 8, the policy is usually defined by a flow chart with nodes representing states and actions and arcs representing user inputs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amari, S.: Natural gradient works efficiently in learning. Neural. Comput., 10(2), 251–276 (1998)
Bishop, C.: Pattern Recognition and Machine Learning, Springer (2006)
Cassandra, A.R.: POMDP solver [Computer Software] (2005). Available from http://www.cassandra.org/pomdp/code/
Cohn, D., Atlas, L., Ladner, R.: Improving Generalization with Active Learning. Mach. Learn. 15, 201–221 (1994)
Deisenroth, M.P., Rasmussen, C.E., Peters, J.: Gaussian Process Dynamic Programming. Neurocomputing 72(7-9), 1508–1524 (2009)
Engel, Y.: Algorithms and Representations for Reinforcement Learning. PhD thesis, Hebrew University (2005)
Engel, Y., Mannor, S., Meir, R.: Reinforcement learning with Gaussian processes. In: Proceedings of ICML (2005)
Engel, E., Mannor, S., Meir, R.: Reinforcement learning with Gaussian processes. In: ICML 2005, Bonn, Germany (2005)
Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K., Young, S.: Training and evaluation of the HIS-POMDP dialogue system in noise. In: Proceedings of SIGDIAL (2008)
Gašić, M., Jurčíček, F., Keizer, S., Mairesse, F., Thomson, B., Yu, K., Young, S.J.: Gaussian Processes for Fast Policy Optimisation of a POMDP Dialogue Manager for a Real-world Task. In: SigDial 2010, Tokyo, Japan (2010)
Gašić, M., Young, S.: Effective Handling of Dialogue State in the Hidden Information State POMDP Dialogue Manager. ACM Transactions on Speech and Language Processing 7(3), 2011
Gašić, M.: Statistical dialogue modelling. PhD thesis, University of Cambridge (2011)
Jurčíček, F., Thomson, B., Keizer, S., Mairesse, F., Gašić, M., Yu, K., Young, S.J.: Natural Belief-Critic: a reinforcement algorithm for parameter estimation in statistical spoken dialogue systems. In Takao Kobayashi, Keikichi Hirose, and Satoshi Nakamura, editors, Proc. Interspeech, 90–93, ISCA (2010)
Jurčíček, F., Thomson, B., Young, S.J.: Natural Actor and Belief Critic: Reinforcement algorithm for learning parameters of dialogue systems modelled as POMDPs. ACM Transactions on Speech and Language Processing 7(3), 2011
Keizer, S., Gašić, M., Mairesse, F., Thomson, B., Yu, K., Young, S.: Modelling user behaviour in the HIS-POMDP dialogue manager. In: Proceedings of SLT, pp. 121–124, 2008
MacKay, D.J.C.: Information-based objective functions for active data selection. Neural. Comput. 4(4), 590–604 (1992)
Minka, T.P.: Expectation Propagation for Approximate Bayesian Inference. In: Proc 17th Conf in Uncertainty in Artificial Intelligence, pp. 362–369. Seattle, Morgan-Kaufmann (2001)
Paquet, U.: Bayesian inference for latent variable models. PhD thesis, University of Cambridge (2007)
Peters, J., Schaal, S.: Natural Actor-Critic. m Neurocomputing 71(7-9), 1180–1190 (2008)
Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. MIT Press (2006)
Roy, N., Pineau, J., Thrun, S: Spoken Dialogue Management Using Probabilistic Reasoning. In: Proceedings of the ACL 2000, 2000
Schatzmann, J., Stuttle, M.N., Weilhammer, K., Young, S.: Effects of the user model on simulation-based learning of dialogue strategies. In: IEEE ASRU ’05: Proc. IEEE Workshop Automatic Speech Recognition and Understanding (2005)
Schatzmann, J.: Statistical User and Error Modelling for Spoken Dialogue Systems. PhD thesis, University of Cambridge (2008)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning. MIT Press, Cambridge, Massachusetts (1998)
Thomson, B.: Statistical methods for spoken dialogue management. PhD thesis, University of Cambridge (2009)
Thomson, B., Schatzmann, J., Young, S.J.: Bayesian Update of Dialogue State for Robust Dialogue Systems. In: Int Conf Acoustics Speech and Signal Processing ICASSP, Las Vegas (2008)
Thomson, B., Young, S.J.: Bayesian update of dialogue state: A POMDP framework for spoken dialogue systems. Computer Speech and Language 24(4), 562–588 (2010)
Thomson, B., Jurčíček, F., Gašić, M., Keizer, S., Mairesse, F., Yu, K., Young, S.J.: Parameter learning for POMDP spoken dialogue models. In: IEEE Workshop on Spoken Language Technology (SLT 2010), Berkeley, CA (2010)
Williams, J.D.: Partially Observable Markov Decision Processes for Spoken Dialogue Management. PhD thesis, University of Cambridge (2006)
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992)
Williams, J.D., Poupart, P., Young, S.: Factored partially observable Markov decision processes for dialogue management. In: Proceedings of the IJCAI Workshop on Knowledge and Reasoning in Practical Dialog Systems, 2005
Williams, J.D., Young, S.J.: Partially Observable Markov Decision Processes for Spoken Dialog Systems. Computer Speech and Language 21(2), 393–422 (2007)
Young, S.J.: Talking to Machines (Statistically Speaking). In: Int Conf Spoken Language Processing, Denver, Colorado (2002)
Young, S.J., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The Hidden Information State Model: a practical framework for POMDP-based spoken dialogue management. Computer Speech and Language 24(2), 150–174 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media New York
About this chapter
Cite this chapter
Gašić, M., Jurčíček, F., Thomson, B., Young, S. (2012). Optimisation for POMDP-Based Spoken Dialogue Systems. In: Lemon, O., Pietquin, O. (eds) Data-Driven Methods for Adaptive Spoken Dialogue Systems. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-4803-7_5
Download citation
DOI: https://doi.org/10.1007/978-1-4614-4803-7_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-4802-0
Online ISBN: 978-1-4614-4803-7
eBook Packages: Computer ScienceComputer Science (R0)