Abstract
We study multi-turn response selection in open domain dialogue systems, where the best-matched response is selected according to a conversation context. The widely used sequential matching models match a response candidate with each utterance in the conversation context through a representation-interaction-aggregation framework, but do not pay enough attention to the inter-utterance dependencies at the representation stage and global information guidance at the interaction stage. They may lead to the result that the matching features of utterance-response pairs may be one-sided or even noisy. In this paper, we propose a hierarchical interactive matching network (HIMN) to model both aspects in a unified framework. In HIMN, we model the dependencies between adjacency utterances in the context with multi-level attention mechanism. Then a two-level hierarchical interactive matching is exploited to introduce the global context information to assist in distilling important matching features of each utterance-response pair at the interaction stage. Finally, the two-level matching features are merged through gate mechanism. Empirical results on both Douban Corpus and Ecommerce Corpus show that HIMN can significantly outperform the competitive baseline models for multi-turn response selection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ba, J., Kiros, J.R., Hinton, G.E.: Layer normalization. arXiv abs/1607.06450 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition pp. 770–778 (2015)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv abs/1502.03167 (2015)
Kadlec, R., Schmid, M., Kleindienst, J.: Improved deep learning baselines for ubuntu corpus dialogs. arXiv abs/1510.03753 (2015)
Kingma, D., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2014)
Kyle, S., Lili, Y., Christopher, F., WohlwendJeremy, Tao, L.: Building a production model for retrieval-based chatbots. In: Proceedings of the First Workshop on NLP for Conversational AI, pp. 32–41 (2019)
Lowe, R., Pow, N., Serban, I., Pineau, J.: The Ubuntu dialogue corpus: a large dataset for research in unstructured multi-turn dialogue systems. In: Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue. pp. 285–294 (2015)
Lu, Z., Li, H.: A deep architecture for matching short texts. In: International Conference on Neural Information Processing Systems, pp. 1367–1375 (2013)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space (2013)
Schegloff, E.A., Sacks, H.: Opening up closings. J. Int. Assoc. Semiotic Stud. 8(4), (1973)
Tao, C., Wu, W., Xu, C., Hu, W., Zhao, D., Yan, R.: One time of interaction may not be enough:go deep with an interaction-over-interaction network for response selection in dialogues. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pp. 1–11 (2019)
Vaswani, A., et al.: Attention is all you need. In: International Conference on Neural Information Processing Systems (2017)
Vinyals, O., Le, Q.V.: A neural conversational model. arXiv abs/1506.05869 (2015)
Wan, S., Lan, Y., Xu, J., Guo, J., Pang, L., Cheng, X.: Match-srnn: Modeling the recursive matching structure with spatial RNN. In: IJCAI (2016)
Wang, M., Lu, Z., Li, H., Liu, Q.: Syntax-based deep matching of short texts. In: IJCAI, pp. 1354–1361 (2015)
Wang, S., Jiang, J.: Learning natural language inference with LSTM. In: NAACL, pp. 1442–1451 (2016)
Wu, Y., Wu, W., Xing, C., Zhou, M., Li, Z.: Sequential matching network: a new architecture for multi-turn response selection in retrieval-based chatbots. In: ACL, pp. 496–505 (2017)
Yan, R., Song, Y., Wu, H.: Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In: SIGIR 2016 (2016)
Zhang, H., Lan, Y., Pang, L., Guo, J., Cheng, X.: ReCoSa: detecting the relevant contexts with self-attention for multi-turn dialogue generation. In: ACL, pp. 3721–3730 (2019)
Zhang, Z., Li, J., Zhu, P., Zhao, H., Liu, G.: Modeling multi-turn conversation with deep utterance aggregation. In: Proceedings of the 27th International Conference on Computational Linguistics. pp. 3740–3752 (2018)
Zhou, X., et al.: Multi-view response selection for human-computer conversation. In: EMNLP, pp. 372–381 (2016)
Zhou, X., et al.: Multi-turn response selection for chatbots with deep attention matching network. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 1118–1127 (2018)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61771333, the Tianjin Municipal Science and Technology Project under Grant 18ZXZNGX00330.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, T., He, R., Wang, L., Zhao, X., Dang, J. (2020). Hierarchical Interactive Matching Network for Multi-turn Response Selection in Retrieval-Based Chatbots. In: Yang, H., Pasupa, K., Leung, A.CS., Kwok, J.T., Chan, J.H., King, I. (eds) Neural Information Processing. ICONIP 2020. Lecture Notes in Computer Science(), vol 12532. Springer, Cham. https://doi.org/10.1007/978-3-030-63830-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-63830-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63829-0
Online ISBN: 978-3-030-63830-6
eBook Packages: Computer ScienceComputer Science (R0)