Skip to main content
SpringerLink
Log in

Knowledge-aware response selection with semantics underlying multi-turn open-domain conversations

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Response selection is a critical issue in the AI community, with important applications on the Web. The accuracy of the selected responses, however, tends to be insufficient due to the lack of contextual awareness, especially in open-domain conversations where words tend to have several meanings in different contexts. Our solution, SemSol, is a knowledge-aware response selection model that tackles this problem by utilizing the context-specific semantics behind words that are implicitly shared among users throughout the dialogue. SemSol simultaneously learns word sense disambiguations (WSD) for the words in the dialogue on the basis of an open-domain knowledge graph, i.e. WordNet, while learning the match between the context and the response candidates. Then, SemSol improves the accuracy of the response by exploiting the semantic information in a knowledge graph in accordance with the dialogue context. Our model learns the topics of utterances in the context of the whole training dataset. This topic-level knowledge can provide topic-specific information in the dialogue context. This improves the WSDs and the response selection accuracy. Experiments with two open-domain conversational datasets, Douban (Chinese) and Reddit (English), demonstrated that the SemSol model outperformed state-of-the-art baselines. SemSol is ranked #1 on the Douban leaderboard.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Availability of data and materials

The dataset and code are available from github.

Notes

  1. https://openai.com/blog/chatgpt/

  2. https://openai.com

  3. https://developers.google.com/freebase

  4. https://www.reddit.com

  5. The results of this paper will be reflected in “https://paperswithcode.com/sota/conversational-response-selection-on-douban-1” after publication.

  6. We refer to the article: https://incora.software/insights/chatgpt-limitations

  7. https://github.com/google-research/bert

  8. Our code and datasets will be shared after publication. For the review phase, our code and learned results for Douban are available as anonymized materials on github “https://github.com/losmes/SemSol”.

  9. https://github.com/MarkWuNLP/MultiTurnResponseSelection

  10. https://bigquery.cloud.google.com/dataset/fh-bigquery

References

  1. Hardalov, M., Koychev, I., Nakov, P.: Enriched pre-trained transformers for joint slot filling and intent detection. CoRRarXiv:2004.14848 (2020)

  2. Rastogi, A., Zang, X., Sunkara, S., Gupta, R., Khaitan, P.: Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset. In: Proc. AAAI’20, pp. 8689–8696 (2020)

  3. Wang, J., Liu, J., Bi, W., Liu, X., He, K., Xu, R., Yang, M.: Improving knowledge-aware dialogue generation via knowledge base question answering. In: Proc. AAAI’20, pp. 9169–9176 (2020)

  4. Henderson, M., Casanueva, I., Mrkšić, N., Su, P.-H., Wen, T.-H., Vulić, I.: ConveRT: Efficient and accurate conversational representations from transformers. In: Proc. EMNLP’20, pp. 2161–2174 (2020)

  5. Henderson, M., Vulić, I., Gerz, D., Casanueva, I., Budzianowski, P., Coope, S., Spithourakis, G., Wen, T.-H., Mrkšić, N., Su, P.-H.: Training neural response selection for task-oriented dialogue systems. In: Proc. ACL’19, pp. 5392–5404 (2019)

  6. Whang, T., Lee, D., Lee, C. Yang, K., Oh, D., Lim, H.: An effective domain adaptive post-training method for BERT in response selection. In: Proc. Interspeech’20, pp. 1585–1589 (2020)

  7. Abd-alrazaq, A.A., Alajlani, M., Alalwan, A.A., Bewick, B.M., Gardner, P., Househ, M.: An overview of the features of chatbots in mental health: A scoping review. Int. J. Med. Inform. 132, 103978 (2019)

    Article  Google Scholar 

  8. Zhang, Z., Li, J., Zhu, P, Zhao, H., Liu, G.: Modeling multi-turn conversation with deep utterance aggregation. CoRRarXiv:1806.09102 (2018)

  9. Lowe, R., Pow, N., Serban, I., Pineau, J.: The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. CoRRarXiv:1506.08909 (2015)

  10. Chen, L., Zhao, Y., Lyu, B., Jin, L., Chen, Z., Zhu, S., Yu, K.: Neural graph matching networks for chinese short text matching. In: Proc. ACL’20, pp. 6152–6158 (2020)

  11. Tao, C., Wu, W., Xu, C., Hu, W., Zhao, D., Yan, R.: One time of interaction may not be enough: Go deep with an interaction-over-interaction network for response selection in dialogues. In: Proc. ACL’19, pp. 1–11. Association for Computational Linguistics (2019)

  12. Wu, Y., Wu, W., Xing, C., Xu, C., Li, Z., Zhou, M.: A sequential matching framework for multi-turn response selection in retrieval-based chatbots. Comput. Linguistics 45(1), 163–197 (2019)

    Article  MathSciNet  Google Scholar 

  13. Xu, Y., Zhao, H., Zhang, Z.: Topic-aware multi-turn dialogue modeling. In: Proc. AAAI’21, pp 14176–14184 (2021)

  14. Han, J., Hong, T., Kim, B., Ko, Y., Seo, J.: Fine-grained post-training for improving retrieval-based dialogue systems. In: Proc. NAACL-HLT’21, pp. 1549–1558 (2021)

  15. Zhang, Z., Zhao, H.: Structural pre-training for dialogue comprehension. In: Proc. ACL’21, pp. 5134–5145 (2021)

  16. Whang, T., Lee, D., Oh, D., Lee, C., Han, K., Lee, D.-h., Lee, S.: Do response selection models really know what’s next? utterance manipulation strategies for multi-turn response selection. In: Proc. AAAI’21, pp. 14041–14049 (2021)

  17. Su, Y., Cai, D., Zhou, Q., Lin, Z., Baker, S., Cao, Y., Shi, S., Collier, N., Wang, Y.: Dialogue response selection with hierarchical curriculum learning. In: Proc. ACL’21, pp. 1740–1751 (2021)

  18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Proc. NIPS’17, vol. 30 (2017)

  19. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. NAACL-HLT’19, pp. 4171–4186 (2019)

  20. Miller, G.A.: Wordnet: A lexical database for english. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  21. Wang, S., Bond, F.: Building the Chinese open Wordnet (COW): Starting from core synsets. In: Proceedings of the 11th Workshop on Asian Language Resources, pp. 10–18 (2013)

  22. Wu, Y., Wu, W., Xing, C., Zhou, M., Li, Z.: Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots. In: Proc. ACL’17, pp. 496–505 (2017)

  23. Zhang, H., Lan, Y., Pang, L., Chen, H., Ding, Z., Yin, D.: Modeling topical relevance for multi-turn dialogue generation. In: Proc. IJCAI’20, pp. 3737–3743 (2020)

  24. Nakatsuji, M., Okui, S.: Conclusion-supplement answer generation for non-factoid questions. In: Proc. AAAI’20, pp. 8520–8527 (2020)

  25. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., Liu, Q.: ERNIE: Enhanced language representation with informative entities. In: Proc. ACL’19, pp. 1441–1451 (2019)

  26. Bordes, A., Usunier, N., García-Durán, A., Weston, J., Yakhnenko, O.: Translating embeddings for modeling multi-relational data. In: Proc. NIPS’13, pp. 2787–2795 (2013)

  27. Liu, Z., Patwary, M., Prenger, R., Prabhumoye, S., Ping, W., Shoeybi, M., Catanzaro, B.: Multi-stage prompting for knowledgeable dialogue generation. In: Proc. ACL’22, pp. 1317–1337 (2022)

  28. Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Le Bras, R., Choi, Y., Hajishirzi, H.: Generated knowledge prompting for commonsense reasoning. In: Proc. ACL’22, pp. 3154–3169 (2022)

  29. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S, Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., Lowe, R.: Training language models to follow instructions with human feedback (2022)

  30. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. In: Proc. NIPS’20, vol. 33, pp. 1877–1901 (2020)

  31. Nakatsuji, M., Fujiwara, Y.: Linked taxonomies to capture users’ subjective assessments of items to facilitate accurate collaborative filtering. Artif. Intell. 207, 52–68 (2014)

    Article  MathSciNet  Google Scholar 

  32. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized BERT pretraining approach. CoRR, arXiv:1907.11692 (2019)

  33. Nakatsuji, M., Toda, H., Sawada, H., Zheng, J., Hendler, J.A.: Semantic sensitive tensor factorization. Artif. Intell. 230, 224–245 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ethayarajh, K.: How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In: EMNLP-IJCNLP’19, pp. 55–65 (2019)

Download references

Funding

not applicable

Author information

Authors and Affiliations

  1. NTT Resonant Inc., Otemachi First Square, East Tower, 1-5-1, 100-0004, Otemachi, Chiyoda-ku, Tokyo, Japan

    Makoto Nakatsuji, Yuka Ozeki, Shuhei Tateishi & Yoshihisa Kano

  2. The HKU Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong, China

    QingPeng Zhang

  3. The Department of Pharmacology and Pharmacy, LKS Faculty of Medicine, The University of Hong Kong, Hong Kong, China

    QingPeng Zhang

Authors
  1. Makoto Nakatsuji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuka Ozeki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhei Tateishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yoshihisa Kano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. QingPeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Makoto Nakatsuji is the corresponding author. Other authors gave advice on our method and the readability of the paper.

Corresponding author

Correspondence to Makoto Nakatsuji.

Ethics declarations

Ethical Approval

not applicable

Competing interests

not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Knowledge-Graph-Enabled Methods and Applications for the Future Web

Guest Editors: Xin Wang, Jeff Pan, Qingpeng Zhang, Yuan-Fang Li.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakatsuji, M., Ozeki, Y., Tateishi, S. et al. Knowledge-aware response selection with semantics underlying multi-turn open-domain conversations. World Wide Web 26, 3373–3388 (2023). https://doi.org/10.1007/s11280-023-01164-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-023-01164-0

Search

Navigation