Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Neural personalized response generation as domain adaptation


One of the most crucial problem on training personalized response generation models for conversational robots is the lack of large scale personal conversation data. To address the problem, we propose a two-phase approach, namely initialization then adaptation, to first pre-train an optimized RNN encoder-decoder model (LTS model) in a large scale conversational data for general response generation and then fine-tune the model in a small scale personal conversation data to generate personalized responses. For evaluation, we propose a novel human aided method, which can be seen as a quasi-Turing test, to evaluate the performance of the personalized response generation models. Experimental results show that the proposed personalized response generation model outperforms the state-of-the-art approaches to language model personalization and persona-based neural conversation generation on the automatic evaluation, offline human judgment and the quasi-Turing test.

This is a preview of subscription content, log in to check access.

Figure 1
Figure 2
Figure 3
Figure 4


  1. 1.


  2. 2.


  3. 3.


  4. 4.


  5. 5.


  6. 6.

    Here, a tweet is a message sent using Twitter.

  7. 7.

    Here, one-to-one means one post is only corresponded to one response.

  8. 8.


  9. 9.


  10. 10.



  1. 1.

    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. Computer Science (2014)

  2. 2.

    Banchs, R.E.: Movie-dic: A movie dialogue corpus for research and development. In: ACL (2012)

  3. 3.

    Banchs, RE, Li, H: Iris: A chat-oriented dialogue system based on the vector space model. In: ACL, pp. 37–42 (2012)

  4. 4.

    Bang, J, Noh, H, Kim, Y, Lee, GG: Example-based chat-oriented dialogue system with personalized long-term memory. In: ICBDSC, pp. 238–243 (2015)

  5. 5.

    Bergstra, J., Bastien, F., Breuleux, O., Lamblin, P., Pascanu, R., Delalleau, O., Desjardins, G., Warde-Farley, D., Goodfellow, I.J., Bergeron, A., Bengio, Y.: Theano: Deep learning on gpus with python. In: NIPS (2011)

  6. 6.

    Berry, P.M., Gervasio, M., Peintner, B., Yorke-Smith, N.: Ptime: Personalized assistance for calendaring. ACM Trans. Intell. Syst. Technol. (TIST) 2(4), 40 (2011)

  7. 7.

    Bin, Y, Yang, Y, Shen, F, Xie, N, Shen, HT, Li, X: Describing video with attention-based bidirectional LSTM. IEEE Transactions on Cybernetics (2018)

  8. 8.

    Cao, J., Wu, Z., Mao, B., Zhang, Y.: Shilling attack detection utilizing semi-supervised learning method for collaborative recommender system. World Wide Web-internet Web Inf. Syst. 16(5–6), 729–748 (2013)

  9. 9.

    Casanueva, I, Hain, T, Christensen, H, Marxer, R, Green, P: Knowledge transfer between speakers for personalised dialogue management. In: SIGDD, pp. 12–21 (2015)

  10. 10.

    Cho, K., Merrienboer, B.V., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H, Bengio, Y: Learning phrase representations using rnn encoder-decoder for statistical machine translation (2014)

  11. 11.

    Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76(5), 378 (1971)

  12. 12.

    Genevay, A, Laroche, R: Transfer learning for user adaptation in spoken dialogue systems. In: ICAAMS, pp. 975–983 (2016)

  13. 13.

    Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks. In: ICML, pp. 1319–1327 (2013)

  14. 14.

    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

  15. 15.

    Kim, Y., Bang, J., Choi, J., Ryu, S., Koo, S., Lee, G.G.: Acquisition and use of long-term memory for personalized dialog systems. In: MAAAHMI, pp. 78–87 (2014)

  16. 16.

    Li, J., Galley, M., Brockett, C., Gao, J, Dolan, B: A diversity-promoting objective function for neural conversation models. NAACL (2015)

  17. 17.

    Li, J, Galley, M, Brockett, C, Spithourakis, G, Gao, J, Dolan, B: A persona-based neural conversation model. In: ACL, pp. 994–1003 (2016)

  18. 18.

    Li, J, Monroe, W, Ritter, A, Dan, J: Deep reinforcement learning for dialogue generation, pp. 1192–1202 (2016)

  19. 19.

    Luan, Y., Ji, Y, Ostendorf, M: Lstm based conversation models (2016)

  20. 20.

    Luan, Y., Brockett, C., Dolan, B., Gao, J., Galley, M.: Multi-task learning for speaker-role adaptation in neural conversation models. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1, Long Papers). Asian Federation of Natural Language Processing, pp. 605–614. Taipei (2017)

  21. 21.

    Mairesse, F., Young, S.: Stochastic language generation in dialogue using factored language models. Comput Linguis. 40(4), 763–799 (2014)

  22. 22.

    Mairesse, F, Jurcicek, M, Ek, F, Keizer, S, Thomson, B, Yu, K, Young, S: Phrase-based statistical language generation using graphical models and active learning. In: ACL, pp. 1552–1561 (2010)

  23. 23.

    Marjan, G., Chris, B., Ming-Wei, C., Bill, D., Jianfeng, G., Wen-tau, Y, Michel, G: A knowledge-grounded neural conversation model (2018)

  24. 24.

    Mei, H., Bansal, M, Walter, M: Coherent dialogue with attention-based language models (2017)

  25. 25.

    Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. NIPS 26, 3111–3119 (2013)

  26. 26.

    Mo, K., Li, S., Zhang, Y., Li, J, Yang, Q: Personalizing a dialogue system with transfer learning (2016)

  27. 27.

    Papineni, K, Roukos, S, Ward, T, Zhu, WJ: Bleu: A method for automatic evaluation of machine translation. In: ACL, pp. 311–318 (2002)

  28. 28.

    Peng, M., Zeng, G., Sun, Z., Huang, J., Wang, H., Tian, G.: Personalized app recommendation based on app permissions. World Wide Web-internet Web Inf. Syst., 1–16 (2017)

  29. 29.

    Rasch, K., Li, F., Sehic, S., Ayani, R., Dustdar, S.: Context-driven personalized service discovery in pervasive environments. World Wide Web-internet Web Inf. Syst. 14(4), 295–319 (2011)

  30. 30.

    Ritter, A, Cherry, C, Dolan, B: Unsupervised modeling of twitter conversations. In: NAACL, pp. 172–180 (2010)

  31. 31.

    Ritter, A, Cherry, C, Dolan, WB: Data-driven response generation in social media. In: EMNLP, pp. 583–593 (2011)

  32. 32.

    Serban, I.V., Sordoni, A., Bengio, Y., Courville, A, Pineau, J: Hierarchical neural network generative models for movie dialogues (2015)

  33. 33.

    Serban, I.V., Sordoni, A., Bengio, Y., Courville, A, Pineau, J: Building end-to-end dialogue systems using generative hierarchical neural network models. Computer Science (2016)

  34. 34.

    Shang, L, Lu, Z, Li, H: Neural responding machine for short-text conversation. In: ACL, pp. 1577–1586 (2015)

  35. 35.

    Sordoni, A., Bengio, Y., Vahabi, H., Lioma, C., Grue Simonsen, J., Nie, JY: A hierarchical recurrent encoder-decoder for generative context-aware query suggestion. In: CIKM (2015)

  36. 36.

    Sordoni, A, Galley, M, Auli, M, Brockett, C, Ji, Y, Mitchell, M, Nie, JY, Gao, J, Dolan, B: A neural network approach to context-sensitive generation of conversational responses. In: NAACL, pp. 196–205 (2015)

  37. 37.

    Sutskever, I., Vinyals, O., Le, Q.V., Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. NIPS 4, 3104–3112 (2014)

  38. 38.

    Vinyals, O, Le, Q: A neural conversational model. Computer Science (2015)

  39. 39.

    Wang, C, Zhang, M, Ma, S, Ru, L: Automatic online news issue construction in web environment. In: WWW, pp. 457–466 (2008)

  40. 40.

    Wang, D., Jojic, N., Brockett, C., Nyberg, E.: Steering output style and topic in neural response generation. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp. 2140-2150. Copenhagen (2017)

  41. 41.

    Wen, TH, Heidel, A, Lee, H.Y., Tsao, Y, Lee, LS: Recurrent neural network based language model personalization by social network crowdsourcing. In: INTERSPEECH, pp. 2703–2707 (2013)

  42. 42.

    Wen, T.H., Gasic, M., Kim, D., Mrksic, N., Su, P.H., Vandyke, D, Young, S: Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking. SIGDial (2015)

  43. 43.

    Wen, T.H., Gasic, M., Mrksic, N., Su, P.H., Vandyke, D, Young, S: Semantically conditioned lstm-based natural language generation for spoken dialogue systems. EMNLP (2015)

  44. 44.

    Wen, TH, Gašic, M, Mrkšic, N, Rojas-Barahona, LM, Su, PH, Vandyke, D, Young, S: Multi-domain neural network language generation for spoken dialogue systems. In: NAACL, pp. 120–129 (2016)

  45. 45.

    Xing, C, Wu, W, Wu, Y, Liu, J, Huang, Y, Zhou, M, Ma, WY: Topic augmented neural response generation with a joint attention mechanism. arXiv:1606.08340 (2016)

  46. 46.

    Yang, M, Zhao, Z, Zhao, W, Chen, X, Zhu, J, Zhou, L, Cao, Z: Personalized response generation via domain adaptation. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1021–1024. ACM (2017)

  47. 47.

    Yao, W., He, J., Huang, G., Cao, J., Zhang, Y.: Personalized recommendation on multi-layer context graph. Lect. Notes Comput. Sci. 8180, 135–148 (2013)

  48. 48.

    Young, S., Gasic, M., Thomson, B., Williams, J.D.: Pomdp-based statistical spoken dialog systems: A review. Proc. IEEE, 1160–1179 (2013)

  49. 49.

    Zhang, X, LeCun, Y: Text Understanding from Scratch. Computer Science (2015)

  50. 50.

    Zhou, X, He, J, Huang, G, Zhang, Y: A personalized recommendation algorithm based on approximating the singular value decomposition (approsvd). In: Ieee/wic/acm International Conferences on Web Intelligence and Intelligent Agent Technology, pp. 458–464 (2012)

Download references


This paper is supported by NSFC (No. 61502120, 61472105, 61772153) and Heilongjiang philosophy and social science research project (No. 16TQD03).

Author information

Correspondence to Wei-Nan Zhang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Zhu, Q., Wang, Y. et al. Neural personalized response generation as domain adaptation. World Wide Web 22, 1427–1446 (2019). https://doi.org/10.1007/s11280-018-0598-6

Download citation


  • Personalized response generation
  • Conversation generation
  • Sequence to sequence learning
  • Domain adaptation