Skip to main content
SpringerLink
Log in

Research on question retrieval method for community question answering

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

When the neural network model is applied to solve the question retrieval task of community question and answer, it needs a large corpus and long retrieval time. To address these problems, this paper proposes a two-stage question retrieval algorithm. In the second stage, the multi-feature fusion method is adopted to comprehensively judge the retrieved results according to the similarity of the query sentence to the candidate question sentence in lexical features and semantic features, as well as the answer quality features in the candidate answers. Experimental results ranked second with 78.3 on SemEval-2016 Task3 test set and ranked first with 48.20 on SemEval-2017 Task3 test set and and only took 500 ms to get the results from 1000 pieces of data. These results show that this algorithm can significantly improve the question retrieval effect while ensuring the retrieval efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

The data used to support the findings of this study are available from the corresponding author upon request.

Notes

  1. https://alt.qcri.org/semeval2016/task3/

  2. https://alt.qcri.org/semeval2017/task3/

References

  1. AlZu’bi S, Alsmadiv A, AlQatawneh S, et al. (2019) A brief analysis of amazon online reviews[C]//2019 sixth international conference on social networks analysis, management and security (SNAMS). IEEE, 555–560

  2. Alzubi S, Hawashin B, Mughaid A, et al. (2020) Whats trending? An efficient trending research topics extractor and recommender[C]//2020 11th international conference on information and communication systems (ICICS). IEEE, 191–196

  3. Charlet D, Damnati G (2017) Simbow at semeval-2017 task 3: Soft-cosine semantic similarity between questions for community question answering[C]//Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 315–319

  4. Chen Q, Zhu X, Ling ZH, et al. (2017) Enhanced LSTM for Natural Language Inference[A]. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)[C], 1657–1668

  5. Employing Deep Learning Methods for Predicting Helpful Reviews (n.d.)

  6. Franco-Salvador M, Kar S, Solorio T, et al. (2018) Uh-prhlt at semeval-2016 task 3: Combining lexical and semantic-based features for community question answering[J]. arXiv preprint arXiv:1807.11584

  7. Goyal N (2017) Learningtoquestion at semeval 2017 task 3: Ranking similar questions by learning to rank using rich features[C]//Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), 310–314

  8. Guo J, Fan Y, Ai Q, et al. (2016) A deep relevance matching model for ad-hoc retrieval[C]//Proceedings of the 25th ACM international on conference on information and knowledge management, 55–64

  9. Guo T, Lin T, Lu Y (2018) An interpretable LSTM neural network for autoregressive exogenous model[J]. arXiv preprint arXiv:1804.05251

  10. Guo J, Fan Y, Ji X, et al. (2019) Matchzoo: A learning, practicing, and developing system for neural text matching[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, 1297–1300

  11. Joachims T (2002) Optimizing search engines using clickthrough data[C]//Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, 133–142

  12. Joachims T (2006) Training linear SVMs in linear time[C]//Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, 217–226

  13. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980

  14. Li H, Xu J (2014) Semantic matching in search[J]. Found Trends Inf Retr 7(5):343–469

    Article  Google Scholar 

  15. Mitra B, Diaz F, Craswell N (2017) Learning to match using local and distributed representations of text for web search[C]//Proceedings of the 26th International Conference on World Wide Web, 1291–1299

  16. Nakov P, Màrquez L, Moschitti A, et al. (2016) SemEval-2016 Task 3: Community Question Answering[A]. Proceedings of the 10th International Workshop on Semantic Evaluation[C], 525–545

  17. Nakov P, Hoogeveen D, Màrquez L, et al. (2017) SemEval-2017 Task 3: Community Question Answering[A]. Proceedings of the 11th International Workshop on Semantic Evaluation, 27–48

  18. Palangi H, Deng L, Shen Y, et al. (2014) Semantic modelling with long-short-term memory for information retrieval[J]. arXiv preprint arXiv:1412.6629

  19. Peters M E, Neumann M, Iyyer M, et al. (2018) Deep contextualized word representations[J]. arXiv preprint arXiv:1802.05365

  20. Robertson S, Zaragoza H (2009) The probabilistic relevance framework: BM25 and beyond[M]. Now Publishers Inc

    Google Scholar 

  21. Shen Y, He X, Gao J, et al. (2014) Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd international conference on world wide web, 373–374

  22. Wang Z, Hamza W, Florian R et al. (2017) Bilateral multi-perspective matching for natural language sentences[a]. Twenty-Sixth International Joint Conference on Artificial Intelligence 4144–4150

  23. Wu Q, Burges CJC, Svore KM, Gao J (2010) Adapting boosting for information retrieval measures[J]. Inf Retr 13(3):254–270

    Article  Google Scholar 

  24. Xiong C, Dai Z, Callan J, et al. (2017) End-to-end neural ad-hoc ranking with kernel pooling[C]//Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval, 55–64

  25. Yu BG, Xu QT (2018) Research on answer selection algorithm based on collaboration attention mechanism[A]. Chinese Academy of Management, Fudan Premium Fund of Management. Proceedings of the 13th (2018) China management conference[C]. Chinese Academy of Management, Fudan Premium Fund of Management: Chinese Academy of Management, 7

  26. Zhang K, Wu W, Wu H, et al. (2014) Question retrieval with high quality answers in community question answering[C]//Proceedings of the 23rd ACM international conference on conference on information and knowledge management, 371–380

  27. Zhou G, Cai L, Zhao J, et al. (2011) Phrase-based translation model for question retrieval in community question answer archives[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 653–662

Download references

Funding

Supported by the National Natural Science Funds (No.62041305 and No.62072053) and Xizang Natural Science Foundation (No.XZ202001ZR0065G).

Author information

Authors and Affiliations

  1. Information and Network Management Department, Chang’an University, Xi’an, Shaanxi, 710064, China

    Yong Sun

  2. School of Information Engineering, Xizang Minzu University, Xian yang, Shaanxi, 712082, China

    Junfang Song

  3. School of IT, Faculty of Science, Engineering and Built Environment, Deakin University, Geelong, VIC, 3220, Australia

    Xiangyu Song

  4. Research and Development Department of China CITIC Bank Co., Ltd., Beijing, 100000, China

    Jiazheng Hou

Authors
  1. Yong Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junfang Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangyu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiazheng Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junfang Song.

Ethics declarations

Disclosures

The authors declare no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Y., Song, J., Song, X. et al. Research on question retrieval method for community question answering. Multimed Tools Appl 82, 24309–24325 (2023). https://doi.org/10.1007/s11042-023-14458-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14458-2

Keywords

Search

Navigation