Skip to main content
SpringerLink
Log in

A BERT-Based Question Representation for Improved Question Retrieval in Community Question Answering Systems

  • Conference paper
  • First Online:
Advances in Machine Learning and Computational Intelligence

Part of the book series: Algorithms for Intelligent Systems ((AIS))

Abstract

Community question answering (CQA) services have become a prominent place for seeking answers from experts and sharing knowledge in any domain. Retrieving semantically related historical questions for a new query is a critical task in CQA to eliminate duplicate questions and to avoid indefinite waiting time to get responses. One major challenge in question retrieval is the lexical gap between the new question and the question in the archive that is already answered. This problem can be eliminated to a great extent by giving proper embedding for questions. Traditional bag-of-words (BOW) representation for text is being replaced by semantic embedding models like word2vec, Glove, and FastText and the newly released Bidirectional Encoder Representations from Transformers (BERT) model. In this paper, we propose a modified BERT embedding using the topic information obtained by LDA on top of questions preprocessed using the RAKE keyword extraction algorithm. We perform question retrieval based on cosine similarity of question vectors and evaluated the accuracy on the Quora question pair dataset. Results show that the proposed question embedding improves the performance of question retrieval compared with competing models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. D.M. Blei, A.Y. Ng, M.I. Jordan, Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)

    MATH  Google Scholar 

  2. P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, Enriching word vectors with subword information. Trans. Assoc. Comput. Linguistics 5, 135–46 (2017)

    Article  Google Scholar 

  3. S.R. Bowman, G. Angeli, C. Potts, C.D. Manning, A Large Annotated Corpus for Learning Natural Language Inference. arXiv preprint arXiv:1508.05326 (2015)

    Google Scholar 

  4. X. Cao, G. Cong, B. Cui, C.S. Jensen, A Generalized Framework of Exploring Category Information for Question Retrieval in Community Question Answer Archives, in Proceedings of the 19th international conference on World Wide Web. ACM (2010), pp. 201–210

    Google Scholar 

  5. J. Devlin, M.W. Chang, K. Lee, T.K. Bert, Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (2018)

    Google Scholar 

  6. H. Duan, Y. Cao, C.Y. Lin, Y. Yu, Searching Questions by Identifying Question Topic and Question Focus, in Proceedings of ACL-08, HLT (2008), pp. 156–164

    Google Scholar 

  7. Z. Ji, F. Xu, B. Wang, B. He, Question-Answer Topic Model for Question Retrieval in Community Question Answering, in Proceedings of the 21st ACM International Conference on Information and Knowledge Management. ACM (2012), pp. 2471–2474

    Google Scholar 

  8. J. Jeon, W.B. Croft, J.H. Lee, Finding Similar Questions in Large Question and Answer Archives, in Proceedings of the 14th ACM International Conference on Information and Knowledge Management. ACM (2005), pp. 84–90

    Google Scholar 

  9. J.T. Lee, S.B. Kim, Y.I. Song, H.C. Rim, Bridging Lexical Gaps Between Queries and Questions on Large Online Q&A Collections with Compact Translation Models, in Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2008), pp. 410–418

    Google Scholar 

  10. T. Mikolov, K. Chen, G. Corrado, J. Dean, Efficient estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781 (2013)

    Google Scholar 

  11. N. Othman, R. Faiz, K. Smaïli, Enhancing Question Retrieval in Community Question Answering Using Word Embeddings, in Proceedings of the 23rd International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (2019)

    Google Scholar 

  12. J. Pennington, R. Socher, C.D. Manning, in Glove: Global Vectors for Word Representation (2014). URL: https://nlpstanford.edu/projects/glove/. Accessed 11 Jan 2018

  13. M.E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep Contextualized Word Representations. arXiv preprint arXiv:1802.05365 (2018)

    Google Scholar 

  14. N. Reimers, I. Gurevych, Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. arXiv preprint arXiv:1908.10084 (2019)

    Google Scholar 

  15. S. Rose, D. Engel, N. Cramer, W. Cowley, Automatic keyword extraction from individual documents, in Text Mining: Applications and Theory (2010)

    Google Scholar 

  16. X. Xue, J. Jeon, W.B. Croft, Retrieval Models for Question and Answer Archives, in Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM (2008), pp. 475–482

    Google Scholar 

  17. G. Zhou, Y. Zhou, T. He, W. Wu, Learning semantic representation with neural networks for community question answering retrieval. Knowl.-Based Syst. 93, 75–83 (2016)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Calicut, Calicut, Kerala, 673601, India

    C. M. Suneera & Jay Prakash

Authors
  1. C. M. Suneera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jay Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to C. M. Suneera .

Editor information

Editors and Affiliations

  1. School of Computer Science and Engineering, SOA University, Bhubaneswar, Odisha, India

    Srikanta Patnaik

  2. Simulation and Modelling, School of Science and Technology, Middlesex University, London, UK

    Xin-She Yang

  3. Department of Computer Science and Engineering, Oakland University, Rochester, MI, USA

    Ishwar K. Sethi

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Suneera, C.M., Prakash, J. (2021). A BERT-Based Question Representation for Improved Question Retrieval in Community Question Answering Systems. In: Patnaik, S., Yang, XS., Sethi, I. (eds) Advances in Machine Learning and Computational Intelligence. Algorithms for Intelligent Systems. Springer, Singapore. https://doi.org/10.1007/978-981-15-5243-4_31

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-5243-4_31

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-5242-7

  • Online ISBN: 978-981-15-5243-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation