Reinforced Memory Network for Question Answering

  • Anupiya Nugaliyadde
  • Kok Wai Wong
  • Ferdous Sohel
  • Hong Xie
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10635)


Deep learning techniques have shown to perform well in Question Answering (QA) tasks. We present a framework that combines Memory Network (MN) and Reinforcement Learning (Q-learning) to perform QA, termed Reinforced MN (R-MN). We investigate the proposed framework by the use of Long Short Term Memory Network (LSTM) and Dynamic Memory Network (DMN). We call them Reinforced LSTM (R-LSTM) and Reinforced DMN (R-DMN), respectively. The input text sequence and question are passed to both MN and Q-Learning. The output of the MN is then fed to Q-Learning as a second input for refinement. The R-MN is trained end-to-end. We evaluated R-MNs on the bAbI 1 K QA dataset for all of the 20 tasks. We achieve superior performance when compared to conventional method of RL, LSTM and the state of the art technique, DMN. Using only half of the training data, both R-LSTM and R-DMN achieved all of the bAbI tasks with high accuracies. The experimental results demonstrated that the proposed framework of combining MN and Q-learning enhances the QA tasks while using less training data.


Question Answering Long Short Term Memory Network Reinforcement Learning Dynamic Memory Network 



This work was partially supported by a Murdoch University internal grant.


  1. 1.
    Cambria, E., White, B.: Jumping NLP curves: a review of natural language processing research. IEEE Comput. Intell. Mag. 9(2), 48–57 (2014)CrossRefGoogle Scholar
  2. 2.
    Weston, J., Bordes, A., Chopra, S., Rush, A.M., van Merriënboer, B., Joulin, A., Mikolov, T.: Towards ai-complete question answering: a set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698 (2015)
  3. 3.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  4. 4.
    Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., Socher, R.: Ask me anything: dynamic memory networks for natural language processing. In: International Conference on Machine Learning (2016)Google Scholar
  5. 5.
    Sukhbaatar, S., Weston, J., Fergus, R.: End-to-end memory networks. In: Advances in Neural Information Processing Systems, pp. 2440–2448 (2015)Google Scholar
  6. 6.
    Bordes, A., Usunier, N., Chopra, S., Weston, J.: Large-scale simple question answering with memory networks. arXiv preprint arXiv:1506.02075 (2015)
  7. 7.
    Weston, J., Chopra, S., Bordes, A.: Memory networks. arXiv preprint arXiv:1410.3916 (2014)
  8. 8.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  9. 9.
    Tan, M., Santos, C.d., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108 (2015)
  10. 10.
    Peng, B., Lu, Z., Li, H., Wong, K.F.: Towards neural network-based reasoning. arXiv preprint arXiv:1508.05508 (2015)
  11. 11.
    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  12. 12.
    Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937 (2016)Google Scholar
  13. 13.
    Branavan, S.R., Chen, H., Zettlemoyer, L.S., Barzilay, R.: Reinforcement learning for mapping instructions to actions. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (2009)Google Scholar
  14. 14.
    Yu, L., Hermann, K.M., Blunsom, P., Pulman, S.: Deep learning for answer sentence selection. arXiv preprint arXiv:1412.1632 (2014)
  15. 15.
    Guo, X., Klinger, T., Rosenbaum, C., Bigus, J.P., Campbell, M., Kawas, B., Talamadupula, K., Tesauro, G., Singh, S.: Learning to query, reason, and answer questions on ambiguous texts. In: 5th International Conference on Learning Representations (2017)Google Scholar
  16. 16.
    Bakker, B.: Reinforcement learning with long short-term memory. In: Neural Information Processing Systems (2002)Google Scholar
  17. 17.
    Cho, K., Van Merriënboer, B., Bahdanau, D., Bengio, Y.: On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Anupiya Nugaliyadde
    • 1
  • Kok Wai Wong
    • 1
  • Ferdous Sohel
    • 1
  • Hong Xie
    • 1
  1. 1.Murdoch UniversityPerthAustralia

Personalised recommendations