Chat-Like Conversational System Based on Selection of Reply Generating Module with Reinforcement Learning

  • Tomohide Shibata
  • Yusuke Egashira
  • Sadao Kurohashi
Part of the Signals and Communication Technology book series (SCT)


This paper presents a chat-like conversational system, and that generates a reply by selecting an appropriate reply generating module. Such modules consist in selecting a sentence from an article of Web news, retrieving a definition sentence in Wikipedia, question-answering, and so on. A dialogue strategy corresponds to which reply generating module should be chosen according to a user input and the dialogue history, and is learned in the MDP framework. User evaluations showed that our system could learn an appropriate dialogue strategy, and perform natural dialogues.


  1. 1.
    Bellman R (1957) A markov decision process. J Math Mech 6(679–684):2Google Scholar
  2. 2.
    Kaelbling L, Littman M, Cassandra A (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1–2):99–134Google Scholar
  3. 3.
    Levin E, Pieraccini R, Eckert W (1998) Using markov decision process for learning dialogue strategies. In: Proceedings of the 1998 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 201–204Google Scholar
  4. 4.
    Meguro T, Higashinaka R, Minami Y, Dohsaka K (2010) Controlling listening-oriented dialogue using partially observable markov decision processes. In: Proceedings of the 23rd international conference on computational linguistics, pp 761–769Google Scholar
  5. 5.
    Nakano M, Funakoshi K, Hasegawa Y, Tsujino H (2008) A framework for building conversational agents based on a multi-expert model. In: Proceedings of the 9th SIGdial workshop on discourse and dialogue, pp 88–91Google Scholar
  6. 6.
    Weizenbaum J (1966) Eliza—a computer program for the study of natural language communication between man and machine. Commun ACM 9(1):36–45CrossRefGoogle Scholar
  7. 7.
    Yoshino K, Mori S, Kawahara T (2011) Spoken dialogue system based on information extraction using similarity of predicate argument structures. In: Proceedings of the SIGDIAL 2011 Conference, pp 59–66Google Scholar
  8. 8.
    Young S, Gasic M, Keizer S, Mairesse F, Schatzmann J, Thomson B, Yu K (2010) The hidden information state model: a practical framework for pomdp-based spoken dialogue management. Comput Speech Lang 24(2):150–174CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Tomohide Shibata
    • 1
  • Yusuke Egashira
    • 1
  • Sadao Kurohashi
    • 1
  1. 1.Kyoto UniversityKyotoJapan

Personalised recommendations