Selecting the N-Top Retrieval Result Lists for an Effective Data Fusion

  • Antonio Juárez-González
  • Manuel Montes-y-Gómez
  • Luis Villaseñor-Pineda
  • David Pinto-Avendaño
  • Manuel Pérez-Coutiño
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6008)

Abstract

Although the application of data fusion in information retrieval has yielded good results in the majority of the cases, it has been noticed that its achievement is dependent on the quality of the input result lists. In order to tackle this problem, in this paper we explore the combination of only the n-top result lists as an alternative to the fusion of all available data. In particular, we describe a heuristic measure based on redundancy and ranking information to evaluate the quality of each result list, and, consequently, to select the presumably n-best lists per query. Preliminary results in four IR test collections, containing a total of 266 queries, and employing three different DF methods are encouraging. They indicate that the proposed approach could significantly outperform the results achieved by fusion all available lists, showing improvements in mean average precision of 10.7%, 3.7% and 18.8% when it was used along with Maximum RSV, CombMNZ and Fuzzy Borda methods.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Agirre, E., Di Nunzio, G.M., Ferro, N., Mandl, T., Peters, C.: CLEF 2008: Ad Hoc Track Overview. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  2. 2.
    Arni, T., Clough, P., Sanderson, M., Grubinger, M.: Overview of the ImageCLEFphoto 2008 Photographic Retrieval Task. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  3. 3.
    Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley, Reading (1999)Google Scholar
  4. 4.
    Bartell, B.T., Cottrell, G.W., Belew, R.K.: Automatic Combination of Multiple Ranked Retrieval Systems. In: Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Dublin, Ireland (1994)Google Scholar
  5. 5.
    Diamond, T., Liddy, E.D.: Dynamic data fusion. In: Proceedings of the TIPSTER Text Program: Phase III. Annual Meeting of the Association for Computational Linguistics (ACL), Baltimore, Maryland, USA (1998)Google Scholar
  6. 6.
    Di Nunzio, G.M., Ferro, N., Jones, G.J.F., Peters, C.: CLEF 2005: Ad Hoc Track Overview. In: Working Notes for the CLEF 2005 Workshop, Vienna, Austria (2005)Google Scholar
  7. 7.
    Escalante, H.J., González, J.A., Hernández, C.A., López, A., Montes, M., Morales, E., Sucar, L.E., Villaseñor, L.: TIA-INAOE’s Participation at Image CLEF 2008. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  8. 8.
    Fox, E.A., Shaw, J.A.: Combination of Multiple Searches. In: Proceedings of The Second Text REtrieval Conference (TREC-2), Gaithersburg, Maryland, USA (1994)Google Scholar
  9. 9.
    Gopalan, N.P., Batri, K.: Adaptive Selection of Top-m Retrieval Strategies for Data Fusion in Information Retrieval. International Journal of Soft Computing 2(1), 11–16 (2007)Google Scholar
  10. 10.
    Hsu, D.F., Taksa, I.: Comparing Rank and Score Combination Methods for Data Fusion in Information Retrieval. Information Retrieval 8(3), 449–480 (2005)CrossRefGoogle Scholar
  11. 11.
    Kantor, P.B.: Decision level data fusión for routing of documents in the TREC3 context: A best case analysis of worst case results. In: Proceedings of The Third Text REtrieval Conference (TREC-3), Gaithersburg, Maryland, USA (1995)Google Scholar
  12. 12.
    Lebanon, G., Lafferty, J.: Cranking: Combining rankings using conditional probability models on permutations. In: Proceedings of the Nineteenth International Conference on Machine Learning, Sydney, Australia (2002)Google Scholar
  13. 13.
    Lee, J.H.: Analyses of Multiple Evidence Combination. In: Proceedings of the 20th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Philadelphia, PA, USA (1997)Google Scholar
  14. 14.
    Lillis, D., Toolan, F., Collier, R., Dunnion, J.: A probabilistic approach to data fusion. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, Washington, USA (2006)Google Scholar
  15. 15.
    Mandl, T., Carvalho, P., Gey, F., Larson, R., Santos, D., Womser-Hacker, C.: GeoCLEF 2008: the CLEF 2008 Cross-Language Geographic Information Retrieval Track Overview. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  16. 16.
    Montague, M., Aslam, J.A.: Condorcet fusion for improved retrieval. In: Proceedings of the 11th International Conference on Information Knowledge and Management (CIKM- ACM), McLean, VA, USA (2002)Google Scholar
  17. 17.
    Ng, K.B., Kantor, P.B.: Predicting the effectiveness of naive data fusion on the basis of system characteristics. Journal of American Society for Information Science 51, 1177–1189 (2000)CrossRefGoogle Scholar
  18. 18.
    Perea, J.M., Ureña, L.A., Buscaldi, D., Rosso, P.: TextMESS at GeoCLEF 2008: Result Merging with Fuzzy Borda Ranking. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  19. 19.
    Smucker, M.D., Allan, J., Carterette, B.: Agreement Among Statistical Significance Tests for Information Retrieval Evaluation at Varying Sample Sizes. Poster session for The 32nd Annual ACM SIGIR Conference (SIGIR 2009), Boston, MA, USA (2009)Google Scholar
  20. 20.
    Villatoro-Tello, E., Montes-y-Gómez, M., Villaseñor-Pineda, L.: INAOE at GeoCLEF 2008: A Ranking Approach based on Sample Documents. In: Working Notes for the CLEF 2008 Workshop, Aarhus, Denmark (2008)Google Scholar
  21. 21.
    Vogt, C., Cottrell, G., Belew, R., Bartell, B.: Using relevance to train a linear mixture of experts. In: Proceedings of The Fifth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland (1997)Google Scholar
  22. 22.
    Vogt, C.C., Cottrell, G.W.: Predicting the performance of linearly combined IR systems. In: Proceedings of the 21st ACM-SIGIR International Conference on Research and Development in Information Retrieval, Melbourne, Australia (1998)Google Scholar
  23. 23.
    Vorhees, E.M.: Overview of TREC 2007. In: Proceedings of the sixteenth Text Retrieval Conference (TREC 2007), Gaithersburg, Maryland, USA (2007)Google Scholar
  24. 24.
    Wu, S., McClean, S.: Performance prediction of data fusion for information retrieval. Information Processing and Management 42(4), 899–915 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Antonio Juárez-González
    • 1
  • Manuel Montes-y-Gómez
    • 1
  • Luis Villaseñor-Pineda
    • 1
  • David Pinto-Avendaño
    • 2
  • Manuel Pérez-Coutiño
    • 3
  1. 1.Laboratory of Language TechnologiesNational Institute of Astrophysics, Optics and Electronics (INAOE)Mexico
  2. 2.Faculty of Computer ScienceAutonomous University of Puebla (BUAP)Mexico
  3. 3.Vanguard Engineering Puebla (VEng)Mexico

Personalised recommendations