Abstract
Passage re-ranking in question answering (QA) systems is a method to reorder a set of retrieved passages, related to a given question so that answer-containing passages are ranked higher than non-answer-containing passages. With recent advances in language models, passage ranking has become more effective due to improved natural language understanding of the relationship between questions and answer passages. With neural network models, question-passage pairs are used to train a cross-encoder that predicts the semantic relevance score of the pairs and is subsequently used to rank retrieved passages. This paper reports on the use of open information extraction (OpenIE) triples in the form \({<subject, verb, object>}\) for questions and passages to enhance answer passage ranking in neural network models. Coverage and overlap scores of question-passage triples are studied and a novel loss function is developed using the proposed triple-based features to better learn a cross-encoder model to rerank passages. Experiments on three benchmark datasets are compared to the baseline BERT and ERNIE models using the proposed loss function demonstrating improved passage re-ranking performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Callan, J., Hoy, M., Yoo, C., Zhao, L.: Clueweb09 data set (2009), https://lemurproject.org/clueweb09/ Accessed 28 Apr 2023
Craswell, N., Mitra, B., Yilmaz, E., Campos, D., Voorhees, E.M.: Overview of the TREC 2019 Deep Learning Track, https://arxiv.org/abs/2003.07820 Accessed 28 Apr 2023
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol 1. pp. 4171–4186 (2019)
Dhingra, B., Mazaitis, K., Cohen, W.W.: Quasar: Datasets for question answering by search and reading (2017), https://arxiv.org/abs/1707.03904
Dong, Q., et al.: Incorporating explicit knowledge in pre-trained language models for passage re-ranking. In: Proceedings of the 45th International ACM SIGIR Conference, pp. 1490–1501. SIGIR ’22, ACM, New York, NY, USA (2022). https://doi.org/10.1145/3477495.3531997
Gao, L., Dai, Z., Callan, J.: Understanding BERT rankers under distillation. In: Proc of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval. pp. 149–152 (2020)
Izacard, G., Grave, E.: Leveraging passage retrieval with generative models for open domain question answering. In: Proceeding of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. pp. 874–880. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.eacl-main.74
Karpukhin, V., et al.: Dense passage retrieval for open-domain question answering. In: Proceeding of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 6769–6781. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.emnlp-main.550
Khattab, O., Zaharia, M.: ColBERT: Efficient and effective passage search via contextualized late interaction over bert. In: Proc of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 39–48. SIGIR ’20, ACM, New York, NY, USA (2020). https://doi.org/10.1145/3397271.3401075
Kolluru, K., Adlakha, V., Aggarwal, S., Mausam, Chakrabarti, S.: OpenIE6: Iterative Grid Labeling and Coordination Analysis for Open Information Extraction. In: Proceeding of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pp. 3748–3761. Association for Computational Linguistics, Online (Nov 2020). https://doi.org/10.18653/v1/2020.emnlp-main.306
Mao, Y., He, P., Liu, X., Shen, Y., Gao, J., Han, J., Chen, W.: Reader-guided passage reranking for open-domain question answering. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. pp. 344–350 (2021)
Nguyen, T., et al.: MS MARCO: A human generated machine reading comprehension dataset. In: CoCo@ NIPs (2016)
Nogueira, R., Cho, K.: Passage re-ranking with BERT (2019), https://arxiv.org/abs/1901.04085
Nogueira, R., Yang, W., Cho, K., Lin, J.: Multi-stage document ranking with BERT (2019), https://arxiv.org/abs/1910.14424
Ofoghi, B.: Linguistic characterization of answer passages for fact-seeking question answering. In: Proceedings of the 37th ACM/SIGAPP Symposium on Applied Computing. p. 821–828. SAC ’22, ACM, New York, NY, USA (2022). https://doi.org/10.1145/3477314.3506999
Ofoghi, B., Mahdiloo, M., Yearwood, J.: Data envelopment analysis of linguistic features and passage relevance for open-domain question answering. Knowl.-Based Syst. 244, 108574 (2022). https://doi.org/10.1016/j.knosys.2022.108574
Ofoghi, B., Zarnegar, A.: Answer Passage Ranking Enhancement Using Shallow Linguistic Features. In: Torra, V., Narukawa, Y. (eds.) MDAI 2021. LNCS (LNAI), vol. 12898, pp. 286–298. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85529-1_23
Oguz, B., et al.: UniK-QA: Unified representations of structured and unstructured knowledge for open-domain question answering. In: Findings of the Association for Computational Linguistics: NAACL 2022. pp. 1535–1546. Association for Computational Linguistics, Seattle, United States (2022). https://doi.org/10.18653/v1/2022.findings-naacl.115
Pradeep, R., Liu, Y., Zhang, X., Li, Y., Yates, A., Lin, J.: Squeezing Water from a Stone: A Bag of Tricks for Further Improving Cross-Encoder Effectiveness for Reranking. In: Hagen, M., Verberne, S., Macdonald, C., Seifert, C., Balog, K., Nørvåg, K., Setty, V. (eds.) ECIR 2022. LNCS, vol. 13185, pp. 655–670. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99736-6_44
Qu, Y., et al.: RocketQA: an optimized training approach to dense passage retrieval for open-domain question answering. In: In Proceedings of NAACL (2021)
Reimers, N., Gurevych, I.: Making monolingual sentence embeddings multilingual using knowledge distillation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (2020), https://arxiv.org/abs/2004.09813
Vaswani, A., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 30 (2017)
Wang, S., et al.: Ernie 3.0 titan: exploring larger-scale knowledge enhanced pre-training for language understanding and generation (2021), https://arxiv.org/abs/2112.12731
Yan, M., Li, C., Bi, B., Wang, W., Huang, S.: A unified pretraining framework for passage ranking and expansion. In: Proceeding of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 4555–4563 (2021)
Yang, W., Xie, Y., Lin, A., Li, X., Tan, L., Xiong, K., Li, M., Lin, J.: End-to-end open-domain question answering with BERTserini. In: Proceeding of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations). pp. 72–77. Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-4013,https://aclanthology.org/N19-4013
Yu, D., et al.: KG-FiD: infusing knowledge graph in fusion-in-decoder for open-domain question answering. In: Proc of the 60th Annual Meeting of the Association for Computational Linguistics, Vol. 1, pp. 4961–4974. Dublin, Ireland (2022). https://doi.org/10.18653/v1/2022.acl-long.340
Acknowledgements
This research is partly funded by the Minerals Council of Australia.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nagumothu, D., Ofoghi, B., Eklund, P.W. (2023). Semantic Triple-Assisted Learning for Question Answering Passage Re-ranking. In: Fink, G.A., Jain, R., Kise, K., Zanibbi, R. (eds) Document Analysis and Recognition - ICDAR 2023. ICDAR 2023. Lecture Notes in Computer Science, vol 14189. Springer, Cham. https://doi.org/10.1007/978-3-031-41682-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-41682-8_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-41681-1
Online ISBN: 978-3-031-41682-8
eBook Packages: Computer ScienceComputer Science (R0)
