Advertisement

Enhancing Relevance Models with Adaptive Passage Retrieval

  • Xiaoyan Li
  • Zhigang Zhu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4956)

Abstract

Passage retrieval and pseudo relevance feedback/query expansion have been reported as two effective means for improving document retrieval in literature. Relevance models, while improving retrieval in most cases, hurts performance on some heterogeneous collections. Previous research has shown that combining passage-level evidence with pseudo relevance feedback brings added benefits. In this paper, we study passage retrieval with relevance models in the language-modeling framework for document retrieval. An adaptive passage retrieval approach is proposed to document ranking based on the best passage of a document given a query. The proposed passage ranking method is applied to two relevance-based language models: the Lavrenko-Croft relevance model and our robust relevance model. Experiments are carried out with three query sets on three different collections from TREC. Our experimental results show that combining adaptive passage retrieval with relevance models (particularly the robust relevance model) consistently outperforms solely applying relevance models on full-length document retrieval.

Keywords

Relevance models passage retrieval language modeling 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Abdul-Jaleel, N., et al.: UMASS at TREC2004. In: Thirteen Text Retrieval Conference Notebook (2004)Google Scholar
  2. 2.
    Clarke, C., Craswell, N., Soboroff, I.: Overview of the TREC 2004 terabyte track. In: Thirteen Text Retrieval Conference Notebook (2004)Google Scholar
  3. 3.
    Cronen-Townsend, S., Zhou, Y., Croft, W.B.: A framework for selective query expansion. In: Proc. 13th Int. Conf. on Information and Knowledge Management, pp. 236–237 (2004)Google Scholar
  4. 4.
    Hiemstra, D.: Using language models for information retrieval. PhD thesis, University of Twente (2001)Google Scholar
  5. 5.
    Jiang, J., Zhai, C.: UIUC in HARD 2004 – Passage Retrieval using HMMs. In: Thirteen Text Retrieval Conference Notebook (2004)Google Scholar
  6. 6.
    Kaszkiel, M., Zobel, J.: Passage retrieval revisited. In: Proc. 20th ACM-SIGIR Conf. on Research and Development in Information Retrieval, pp. 178–185 (1997)Google Scholar
  7. 7.
    Kaszkiel, M., Zobel, J.: Effective ranking with arbitrary passages. Journal of the American Society for Information Science and Technology 52(4), 344–364 (2001)CrossRefGoogle Scholar
  8. 8.
    Kraaij, W., Westerveld, T., Hiemstra, D.: The importance of prior probabilities for entry page search. In: 25th ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 27–34 (2002)Google Scholar
  9. 9.
    Krovetz, R.: Viewing morphology as an inference process. In: Proc. 16th ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 191–202 (1993)Google Scholar
  10. 10.
    Lafferty, J., Zhai, C.: Document language models, query models, and risk minimization for information retrieval. In: 24th ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 111–119 (2001)Google Scholar
  11. 11.
    Lavrenko, V., Croft, W.B.: Relevance-based language models. In: Proc. 24th ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 120–127 (2001)Google Scholar
  12. 12.
    Lavrenko, V., Croft, W.B.: Relevance models in information retrieval. In: Croft, W.B., Lafferty, J. (eds.) Language modeling for information retrieval, pp. 11–56. Kluwer Academic Publishers, Dordrecht (2003)Google Scholar
  13. 13.
  14. 14.
    Li, X., Croft, W.B.: Time-based language models. In: Proc. 12th Int. Conf. on Information and Knowledge Management, pp. 469–475 (2003)Google Scholar
  15. 15.
    Li, X.: A new robust relevance model in the language model framework, Information Processing and Management (2007), doi:10.1016/j.ipm.2007.07.005Google Scholar
  16. 16.
    Liu, X., Croft, W.B.: Passage retrieval based on language models. In: Proc. 11th Int. Conf. on Information and Knowledge Management, pp. 375–382 (2002)Google Scholar
  17. 17.
    Miller, D.H., Leek, T., Schwartz, R.: A hidden Markov model information retrieval system. In: Proc. 22nd ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 214–221 (1999)Google Scholar
  18. 18.
    Ponte, J.: A Language modeling approach to information retrieval. PhD thesis, UMass-Amherst (1998)Google Scholar
  19. 19.
    Ponte, J., Croft, W.B.: A language modeling approach to information retrieval. In: Proc. 21st ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 275–281 (1998)Google Scholar
  20. 20.
    Song, F., Croft, W.B.: A general language model for information retrieval. In: Proc. 22nd ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 279–280 (1999)Google Scholar
  21. 21.
    Tao, T., Zhai, C.: A two-stage mixture model for pseudo feedback. In: Proc.27th ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 486–487 (2004)Google Scholar
  22. 22.
    Tao, T., Zhai, C.: Regularized estimation of mixture models for robust pseudo-relevance feedback. In: Proc. 29th ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 162–169 (2006)Google Scholar
  23. 23.
    Zhai, C., Lafferty, J.: Model-based feedback in the language modeling approach to information retrieval. In: Proc. 10th Int. Conf. on Information and Knowledge Management, pp. 403–410 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Xiaoyan Li
    • 1
  • Zhigang Zhu
    • 2
  1. 1.Department of Computer ScienceMount Holyoke CollegeSouth HadleyUSA
  2. 2.Department of Computer ScienceCUNY City CollegeNew YorkUSA

Personalised recommendations