Finite-to-Infinite N-Best POMDP for Spoken Dialogue Management

  • Guohua WuEmail author
  • Caixia Yuan
  • Bing Leng
  • Xiaojie Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9427)


Partially Observable Markov Decision Process (POMDP) has been widely used as dialogue management in slot-filling Spoken Dialogue System (SDS). But there are still lots of open problems. The contribution of this paper lies in two aspects. Firstly, the observation probability of POMDP is estimated from the N-Best list of Automatic Speech Recognition (ASR) rather than the top one. This modification gives SDS a chance to address the uncertainty of ASR. Secondly, a dynamic binding technique is proposed for slots with infinite values so as to deal with uncertainty of talking object. The proposed methods have been implemented on a teach-and-learn spoken dialogue system. Experimental results show that performance of system improves significantly by introducing the proposed methods.


Partially Observable Markov Decision Process (POMDP) Spoken Dialogue System (SDS) Dynamic binding N-best 



This work was partially supported by National Natural Science Foundation of China (No.61273365, No.61202248), discipline building plan in 111 base (No.B08004) and Engineering Research Center of Information Networks, Ministry of Education.


  1. 1.
    Barto, A.G.: Reinforcement Learning: An Introduction. MIT press, Cambridge (1998)Google Scholar
  2. 2.
    Hastie, H., Aufaure, M.a., Alexopoulos, P., Cuayáhuitl, H., Dethlefs, N., Gasic, M., Henderson, J., Lemon, O., Liu, X., Mika, P., Mustapha, N.B., Rieser, V., Thomson, B., Tsiakoulis, P., Vanrompay, Y., Villazon-terrazas, B., Young, S.: Demonstration of the Parlance system: a data-driven, incremental, spoken dialogue system for interactive search. In: Proceedings of the SIGDIAL 2013 Conference, pp. 154–156 (2013).
  3. 3.
    Jokinen, K., McTear, M.: Spoken dialogue systems. Synth. Lect. Hum. Lang. Technol. 2(1), 1–151 (2009)CrossRefGoogle Scholar
  4. 4.
    Kaelbling, L.P., Littman, M.L., Cassandra, A.R.: Planning and acting in partially observable stochastic domains. Artif. Intell. 101(1), 99–134 (1998)zbMATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    Kurniawati, H., Hsu, D., Lee, W.S.: SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces. In: Robotics: Science and Systems (2008)Google Scholar
  6. 6.
    Levin, E., Pieraccini, R., Eckert, W.: A stochastic model of human-machine interaction for learning dialog strategies. IEEE Trans. Speech Audio Process. 8(1), 11–23 (2000)CrossRefGoogle Scholar
  7. 7.
    McTear, M.: Spoken dialogue technology: enabling the conversational user interface. ACM Comput. Surv. (CSUR) 34(1), 90–169 (2002)CrossRefGoogle Scholar
  8. 8.
    Seneff, S., Polifroni, J.: Dialogue management in the Mercury flight reservation system. In: Proceedings of the 2000 ANLP/NAACL Workshop on Conversational Systems, vol. 3. pp. 11–16. Association for Computational Linguistics (2000)Google Scholar
  9. 9.
    Shani, G., Pineau, J., Kaplow, R.: A survey of point-based POMDP solvers. Auton. Agent. Multi-Agent Syst. 27(1), 1–51 (2012). CrossRefGoogle Scholar
  10. 10.
    Williams, J.D.: A case study of applying decision theory in the real world: POMDPs and spoken dialog systems. In: Decision Theory Models for Applications in Artificial Intelligence: Concepts and Solutions, pp. 315–342 (2010)Google Scholar
  11. 11.
    Williams, J.D., Young, S.: Partially observable Markov decision processes for spoken dialog systems. Comput. Speech Lang. 21(2), 393–422 (2007)CrossRefGoogle Scholar
  12. 12.
    Young, S., Gasic, M., Thomson, B., Williams, J.D.: Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE 101(5), 1160–1179 (2013)CrossRefGoogle Scholar
  13. 13.
    Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Comput. Speech Lang. 24(2), 150–174 (2010). CrossRefGoogle Scholar
  14. 14.
    Zue, V., Seneff, S., Glass, J.R., Polifroni, J., Pao, C., Hazen, T.J., Hetherington, L.: JUPITER: A telephone-based conversational interface for weather information. IEEE Trans. Speech Audio Process. 8(1), 85–96 (2000)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Guohua Wu
    • 1
    Email author
  • Caixia Yuan
    • 1
  • Bing Leng
    • 1
  • Xiaojie Wang
    • 1
  1. 1.School of ComputerBeijing University of Posts and TelecommunicationsBeijingChina

Personalised recommendations