Abstract
We present a Deep Learning approach to dialogue management for multiple domains. Instead of training multiple models (e.g. one for each domain), we train a single domain-independent policy network that is applicable to virtually any information-seeking domain. We use the Deep Q-Network algorithm to train our dialogue policy, and evaluate against simulated and paid human users. The results show that our algorithm outperforms previous approaches while being more practical and scalable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Casanueva I, Hain T, Christensen H, Marxer R, Green P (2015) Knowledge transfer between speakers for personalised dialogue management. In: 16th SIGDial, p 12
Chandramohan S, Geist M, Lefevre F, Pietquin O (2014) Co-adaptation in spoken dialogue systems. In: Natural interaction with robots, knowbots and smartphones, pp 343–353
Cuayáhuitl, H, Yu S, Williamson A, Carse J (2016) Deep reinforcement learning for multi-domain dialogue systems. arXiv preprint arXiv:1611.08675
Gašić M, Kim D, Tsiakoulis P, Young S (2015) Distributed dialogue policies for multi-domain statistical dialogue management. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 5371–5375
Gašić M, Mrkšić N, Su P, Vandyke D, Wen T, Young S (2015) Policy committee for adaptation in multi-domain spoken dialogue systems. In: IEEE workshop on automatic speech recognition and understanding (ASRU). IEEE, pp 806–812
Lee S, Stent A (2016) Task lineages: dialog state tracking for flexible interaction. In: 17th annual meeting of the special interest group on discourse and dialogue, p 11
Lemon O, Georgila K, Henderson J (2006) Evaluating effectiveness and portability of reinforcement learned dialogue strategies with real users: the talk towninfo evaluation. In: IEEE spoken language technology workshop. IEEE, pp 178–181
Margolis A, Livescu K, Ostendorf M (2010) Domain adaptation with unlabeled data for dialog act tagging. In: Proceedings of the 2010 workshop on domain adaptation for natural language processing. ACL, pp 45–52
Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540): 529–533
Papangelis A, Stylianou Y (2016) Multi-domain spoken dialogue systems using domain-independent parameterisation. In: Domain adaptation for dialogue agents
Schaul T, Quan J, Antonoglou I, Silver D (2015) Prioritized experience replay. arXiv preprint arXiv:1511.05952
Sutton R, Barto A (1998) Reinforcement learning: an introduction. MIT Press
Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10: 1633–1685
Tur G, Guz U, Hakkani-Tur D (2006) Model adaptation for dialog act tagging. In: IEEE spoken language technology workshop. IEEE, pp 94–97
Ultes S, Rojas-Barahona L, Su P, Vandyke D, Kim D, Casanueva I, Budzianowski P, Mrkšić N, Wen T, Gašić M, Young S (2017) Pydial: a multi-domain statistical dialogue system toolkit. In: ACL 2017 Demo, Vancouver, ACL
Walker M, Stent A, Mairesse F, Prasad R (2007) Individual and domain adaptation in sentence planning for dialogue. J Artif Intell Res 30:413–456
Wang Z, Wen T, Su P, Stylianou Y (2015) Learning domain-independent dialogue policies via ontology parameterisation. In: 16th annual meeting of the SIGDial, p 412
Young S, Gašić M, Keizer S, Mairesse F, Schatzmann J, Thomson B, Yu K (2010) The hidden information state model: a practical framework for POMDP-based spoken dialogue management. Comput Speech Lang 24(2):150–174
Zhao T, Eskenazi M (2016) Towards end-to-end learning for dialog state tracking and management using deep reinforcement learning. arXiv preprint arXiv:1606.02560
Zhu S, Chen L, Sun K, Zheng D, Yu K (2014) Semantic parser enhancement for dialogue domain extension with little data. In: SLT workshop. IEEE, pp 336–341
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Papangelis, A., Stylianou, Y. (2019). Single-Model Multi-domain Dialogue Management with Deep Learning. In: Eskenazi, M., Devillers, L., Mariani, J. (eds) Advanced Social Interaction with Agents . Lecture Notes in Electrical Engineering, vol 510. Springer, Cham. https://doi.org/10.1007/978-3-319-92108-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-92108-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92107-5
Online ISBN: 978-3-319-92108-2
eBook Packages: EngineeringEngineering (R0)