SimpleDS: A Simple Deep Reinforcement Learning Dialogue System

  • Heriberto CuayáhuitlEmail author
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 427)


This article presents SimpleDS, a simple and publicly available dialogue system trained with deep reinforcement learning. In contrast to previous reinforcement learning dialogue systems, this system avoids manual feature engineering by performing action selection directly from raw text of the last system and (noisy) user responses. Our initial results, in the restaurant domain, report that it is indeed possible to induce reasonable behaviours with such an approach that aims for higher levels of automation in dialogue control for intelligent interactive systems and robots.


Dialogue systems Reinforcement learning Deep learning 



Funding from the European Research Council (ERC) project “STAC: Strategic Conversation” no. 269427 is gratefully acknowledged.


  1. 1.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. In: Proceedings of the NIPS Deep Learning Workshop (2013)Google Scholar
  2. 2.
    Paek, T., Pieraccini, R.: Automating spoken dialogue management design using machine learning: an industry perspective. Speech Commun. 50(8–9) (2008)Google Scholar
  3. 3.
    Cuayáhuitl, H., Keizer, S., Lemon, O.: Strategic dialogue management via deep reinforcement learning. In: Proceedings of the NIPS Deep Reinforcement Learning Workshop (2015)Google Scholar
  4. 4.
    Szepesvári, C.: Algorithms for Reinforcement Learning. Morgan and Claypool Publishers (2010)Google Scholar
  5. 5.
    Karpathy, A.: ConvNetJS: Javascript library for deep learning (2015).
  6. 6.
    Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of ICML (2010)Google Scholar
  7. 7.
    Cuayáhuitl, H., Renals, S., Lemon, O., Shimodaira, H.: Evaluation of a hierarchical reinforcement learning spoken dialogue system. Comput. Speech Lang. 24(2) (2010)Google Scholar
  8. 8.
    Cuayáhuitl, H., Dethlefs, N.: Spatially-aware dialogue control using hierarchical reinforcement learning. TSLP 7(3) (2011)Google Scholar
  9. 9.
    Cuayáhuitl, H., Kruijff-Korbayová, I., Dethlefs, N.: Nonstrict hierarchical reinforcement learning for interactive systems and robots. TiiS 4(3) (2014)Google Scholar
  10. 10.
    Sainath, T.N., Vinyals, O., Senior, A.W., Sak, H.: Convolutional, long short-term memory, fully connected deep neural networks. In: Proceedings of the ICASSP (2015)Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  1. 1.School of Computer ScienceUniversity of LincolnLincolnUK

Personalised recommendations