Skip to main content
SpringerLink
Log in

Multi-grained unsupervised evidence retrieval for question answering

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Evidence retrieval is a crucial step in question answering (QA) tasks, which can filter original context to provide supporting evidence for reading comprehension and reduce the time of answer inference. Most research interests in evidence retrieval have been paid to supervised methods, while unsupervised evidence retrieval received limited attention. However, most existing works for unsupervised evidence retrieval regard sentence-level and passage-level evidence retrieval as two independent processes. In order to fuse these two levels of information, we propose an efficient and unsupervised multi-grained evidence retrieval method for QA, which considers multiple interactions, including query–sentence, query–passage, and passage–passage. Specifically, we use sentence-level retrieval to obtain an evidence framework. Then we propose a score fusion mechanism to model the unilateral guiding relationship between sentence-level and passage-level retrieval. On the basis of score fusion, we propose a gated selection mechanism to retrieve evidence passages, which can supplement reasoning information for the evidence framework. Extensive experiments on open-domain and multi-hop QA datasets show that our method has fast retrieval speed while achieving better retrieval and QA performance than baselines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. In the preliminary experiment phase, we found that the recall of sentence partitioning using NLTK is about 1 point lower than that of the regular partitioning method (based on the rule of “.”).

  2. https://www.nltk.org/.

  3. https://huggingface.co/mrm8488/spanbert-large-finetuned-squadv2.

  4. https://radimrehurek.com/gensim/.

References

  1. Dhingra B, Mazaitis K, Cohen WW (2017) Quasar: datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904

  2. Joshi M, Choi E, Weld DS, Zettlemoyer L (2017) Triviaqa: a large scale distantly supervised challenge dataset for reading comprehension. In: Proceedings of the 55th annual meeting of the association for computational linguistics: long papers, vol 1, pp 1601–1611

  3. Yang Z, Qi P, Zhang S, Bengio Y, Cohen W, Salakhutdinov R, Manning CD: Hotpotqa (2018) A dataset for diverse, explainable multi-hop question answering. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2369–2380

  4. Welbl J, Stenetorp P, Riedel S (2018) Constructing datasets for multi-hop reading comprehension across documents. Trans Assoc Comput Linguist 6:287–302

    Article  Google Scholar 

  5. Chen D, Fisch A, Weston J, Bordes A (2017) Reading wikipedia to answer open-domain questions. In: Proceedings of the 55th annual meeting of the association for computational linguistics: Long Papers, vol 1, pp 1870–1879

  6. Qiu L, Xiao Y, Qu Y, Zhou H, Li L, Zhang W, Yu Y (2019) Dynamically fused graph network for multi-hop reasoning. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 6140–6150

  7. Li S, Li X, Shang L, Jiang X, Liu Q, Sun C, Ji Z, Liu B (2021) Hopretriever: retrieve hops over Wikipedia to answer complex questions. Proceedings of the AAAI conference on artificial intelligence 35:13279–13287

    Article  Google Scholar 

  8. Cheng H, Shen Y, Liu X, He P, Chen W, Gao J: Unitedqa: a hybrid approach for open domain question answering. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing: long papers, vol 1, pp 3080–3090 (2021)

  9. Sachan D, Patwary M, Shoeybi M, Kant N, Ping W, Hamilton WL, Catanzaro B (2021) End-to-end training of neural retrievers for open-domain question answering. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, long papers, vol 1, pp 6648–6662

  10. Banerjee P, Baral C (2020) Self-supervised knowledge triplet learning for zero-shot question answering. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 151–162

  11. Min S, Seo M, Hajishirzi H (2017) Question answering through transfer learning from large fine-grained supervision data. In: Proceedings of the 55th annual meeting of the association for computational linguistics: short papers, vol 2, pp 510–517

  12. Hsu T-Y, Liu C-L, Lee H-Y (2019) Zero-shot reading comprehension by cross-lingual transfer learning with multi-lingual language representation model. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 5933–5940

  13. Khashabi D, Min S, Khot T, Sabharwal A, Tafjord O, Clark P, Hajishirzi H (2020) Unifiedqa: crossing format boundaries with a single qa system. Findings of the association for computational linguistics: EMNLP 2020, pp 1896–1907

    Google Scholar 

  14. Chen J, Lin S-t., Durrett G (2019) Multi-hop question answering via reasoning chains. arXiv preprint arXiv:1910.02610

  15. Yadav V, Bethard S, Surdeanu M (2019) Quick and (not so) Dirty: unsupervised selection of justification sentences for multi-hop question answering. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 2578–2589

  16. Ma J, Korotkov I, Yang Y, Hall K, McDonald R (2021) Zero-shot neural passage retrieval via domain-targeted synthetic question generation. In: Proceedings of the 16th conference of the European chapter of the association for computational linguistics: main, volume, pp 1075–1088

  17. Min S, Chen D, Zettlemoyer L, Hajishirzi H (2019) Knowledge guided text retrieval and reading for open domain question answering. arXiv preprint arXiv:1911.03868

  18. Yadav V, Bethard S, Surdeanu M (2020) Unsupervised alignment-based iterative evidence retrieval for multi-hop question answering. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 4514–4525

  19. Yadav V, Bethard S, Surdeanu M (2019) Alignment over heterogeneous embeddings for question answering. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies. Long and short papers, vol 1, pp 2681–2691

  20. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  21. Robertson S, Zaragoza H (2009) The probabilistic relevance framework: Bm25 and beyond. Found Trends Inf Retr 3(4):333–389

    Article  Google Scholar 

  22. Salton G (1991) Developments in automatic text retrieval. Science 253:974–980

    Article  MathSciNet  Google Scholar 

  23. Yang W, Xie Y, Lin A, Li X, Tan L, Xiong K, Li M, Lin J (2019) End-to-end open-domain question answering with bertserini. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (Demonstrations), pp 72–77

  24. Tu M, Huang K, Wang G, Huang J, He X, Zhou B (2020) Select, answer and explain: interpretable multi-hop reading comprehension over multiple documents. Proceedings of the AAAI conference on artificial intelligence 34:9073–9080

    Article  Google Scholar 

  25. Qi P, Lin X, Mehr L, Wang Z, Manning CD (2019) Answering complex open-domain questions through iterative query generation. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 2590–2602

  26. Yadav V, Bethard S, Surdeanu M (2021) If you want to go far go together: unsupervised joint candidate evidence retrieval for multi-hop question answering. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 4571–4581

  27. Berger A, Caruana R, Cohn D, Freitag D, Mittal V: Bridging the lexical chasm: statistical approaches to answer-finding. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, pp 192–199 (2000)

  28. Reimers N, Gurevych I (2021) The curse of dense low-dimensional information retrieval for large index sizes. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing. Short Papers, vol 2, pp 605–611

  29. Luan Y, Eisenstein J, Toutanova K, Collins M (2021) Sparse, dense, and attentional representations for text retrieval. Trans Assoc Comput Linguist 9:329–345

    Article  Google Scholar 

  30. Gao L, Dai Z, Chen T, Fan Z, Van Durme B, Callan J (2021) Complement lexical retrieval model with semantic residual embeddings. In: Advances in information retrieval: 43rd European conference on IR research, ECIR 2021, virtual event, March 28–April 1, 2021, proceedings, Part I, pp 146–160

  31. Devlin J, Chang M-W., Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American Chapter of the association for computational linguistics: human language technologies. Long and short papers, vol 1, pp 4171–4186

  32. Rajpurkar P, Jia R, Liang P: Know what you don’t know: Unanswerable questions for squad. In: Proceedings of the 56th annual meeting of the association for computational linguistics. Short papers, vol 2, pp 784–789 (2018)

  33. Joshi M, Chen D, Liu Y, Weld DS, Zettlemoyer L, Levy O (2020) Spanbert: improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64–77

    Article  Google Scholar 

  34. Izacard G, Grave E (2020) Distilling knowledge from reader to retriever for question answering. arXiv preprint arXiv:2012.04584

  35. Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100,000+ questions for machine comprehension of text. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 2383–2392

Download references

Author information

Authors and Affiliations

  1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing, 100093, China

    Hao You

  2. School of Cyber Security, University of Chinese Academy of Sciences, Beijing, 100049, China

    Hao You

Authors
  1. Hao You
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao You.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Availability of data

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, H. Multi-grained unsupervised evidence retrieval for question answering. Neural Comput & Applic 35, 21247–21257 (2023). https://doi.org/10.1007/s00521-023-08892-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08892-4

Keywords

Search

Navigation