Advertisement

Question Rewrite Based Dialogue Response Generation

  • Hengrui Liu
  • Wenge Rong
  • Libin Shi
  • Yuanxin Ouyang
  • Zhang Xiong
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11306)

Abstract

Dialogue response generation is a fundamental technique in natural language processing, which can be used in human-computer interaction. As the quick development in neural networks, the sequence to sequence (seq2seq) model which employed recurrent neural networks (RNN) encoder-decoder has archived great success in machine translation. Many researchers began to apply this model in dialogue response generation. However, the conventional seq2seq model counters several problems, e.g., grammatical mistake, safe response and etc. In this paper, motivated by the great success of generative adversarial networks (GANs) in generating images, we propose an improved seq2seq framework by employing GANs to rewrite questions in order to retrieve more information from the question. Afterwards we combine the original question and the rewritten question together to generate responses. The experiments on the public Yahoo! Answers dataset demonstrated the proposed framework’s potential in dialogue response generation.

Keywords

Dialogue generation Generative adversarial networks Question rewriting 

Notes

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (No. 61332018).

References

  1. 1.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473 (2014)Google Scholar
  2. 2.
    Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. SIGKDD Explorations 19(2), 25–35 (2017)CrossRefGoogle Scholar
  3. 3.
    Denton, E.L., Chintala, S., Fergus, R., et al.: Deep generative image models using a laplacian pyramid of adversarial networks. In: Proceedings of 2015 Annual Conference on Neural Information Processing Systems, pp. 1486–1494 (2015)Google Scholar
  4. 4.
    Goodfellow, I., et al.: Generative adversarial nets. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  5. 5.
    He, Y., Tang, J., Ouyang, H., Kang, C., Yin, D., Chang, Y.: Learning to rewrite queries. In: Proceedings of the 25th ACM International Conference on Information and Knowledge Management, pp. 1443–1452 (2016)Google Scholar
  6. 6.
    Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  8. 8.
    Huang, C., Zaïane, O.R., Trabelsi, A., Dziri, N.: Automatic dialogue generation with expressed emotions. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 49–54 (2018)Google Scholar
  9. 9.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pp. 1746–1751 (2014)Google Scholar
  10. 10.
    Kingma, D.P., Salimans, T., Józefowicz, R., Chen, X., Sutskever, I., Welling, M.: Improving variational autoencoders with inverse autoregressive flow. In: Proceedings of 2016 Annual Conference on Neural Information Processing Systems, pp. 4736–4744 (2016)Google Scholar
  11. 11.
    Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 110–119 (2016)Google Scholar
  12. 12.
    Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A., Jurafsky, D.: Adversarial learning for neural dialogue generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2157–2169 (2017)Google Scholar
  13. 13.
    Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421 (2015)Google Scholar
  14. 14.
    Meng, F., Lu, Z., Wang, M., Li, H., Jiang, W., Liu, Q.: Encoding source language with convolutional neural network for machine translation. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pp. 20–30 (2015)Google Scholar
  15. 15.
    Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., Khudanpur, S.: Recurrent neural network based language model. In: Proceedings of 11th Annual Conference of the International Speech Communication Association, pp. 1045–1048 (2010)Google Scholar
  16. 16.
    Mou, L., Song, Y., Yan, R., Li, G., Zhang, L., Jin, Z.: Sequence to backward and forward sequences: a content-introducing approach to generative short-text conversation. In: Proceedings of 26th International Conference on Computational Linguistics, pp. 3349–3358 (2016)Google Scholar
  17. 17.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311–318 (2002)Google Scholar
  18. 18.
    Riezler, S., Liu, Y.: Query rewriting using monolingual statistical machine translation. Comput. Linguist. 36(3), 569–582 (2010)CrossRefGoogle Scholar
  19. 19.
    Ritter, A., Cherry, C., Dolan, W.B.: Data-driven response generation in social media. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 583–593 (2011)Google Scholar
  20. 20.
    Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Sig. Process. 45(11), 2673–2681 (1997)CrossRefGoogle Scholar
  21. 21.
    Shao, L., Gouws, S., Britz, D., Goldie, A., Strope, B., Kurzweil, R.: Generating long and diverse responses with neural conversation models. CoRR abs/1701.03185 (2017)Google Scholar
  22. 22.
    Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Proceedings of 2014 Annual Conference on Neural Information Processing Systems, pp. 3104–3112 (2014)Google Scholar
  23. 23.
    Wang, H., Lu, Z., Li, H., Chen, E.: A dataset for research on short-text conversations. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 935–945 (2013)Google Scholar
  24. 24.
    Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 229–256 (1992)zbMATHGoogle Scholar
  25. 25.
    Xing, C., et al.: Topic aware neural response generation. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 3351–3357 (2017)Google Scholar
  26. 26.
    Xu, J., Sun, X., Ren, X., Lin, J., Wei, B., Li, W.: DP-GAN: diversity-promoting generative adversarial network for generating informative and diversified text. CoRR abs/1802.01345 (2018)Google Scholar
  27. 27.
    Yu, L., Zhang, W., Wang, J., Yu, Y.: SeqGAN: sequence generative adversarial nets with policy gradient. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence, pp. 2852–2858 (2017)Google Scholar
  28. 28.
    Zhu, J.-Y., Krähenbühl, P., Shechtman, E., Efros, A.A.: Generative visual manipulation on the natural image manifold. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 597–613. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_36CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Hengrui Liu
    • 1
  • Wenge Rong
    • 1
  • Libin Shi
    • 2
  • Yuanxin Ouyang
    • 1
  • Zhang Xiong
    • 1
  1. 1.School of Computer Science and EngineeringBeihang UniversityBeijingChina
  2. 2.Sino-French Engineer SchoolBeihang UniversityBeijingChina

Personalised recommendations