Abstract
We discuss the idea of modelling the statistical distributions of scores of documents, classified as relevant or non-relevant. Various specific combinations of standard statistical distributions have been used for this purpose. Some theoretical considerations indicate problems with some of the choices of pairs of distributions. Specifically, we revisit a generalisation of the well-known inverse relationship between recall and precision: some choices of pairs of distributions violate this generalised relationship. We identify the choices and the violations, and explore some of the consequences of this theoretical view.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Swets, J.A.: Information retrieval systems. Science 141(3577), 245–250 (1963)
Baumgarten, C.: A probabilistic solution to collection fusion problem in distributed information retrieval. In: Hearst, M., Gey, F., Tong, R. (eds.) SIGIR’99: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 246–253. ACM Press, New York (1999)
Arampatzis, A., van Hameren, A.: The score-distributional threshold optimization for adaptive binary classification tasks. In: Croft, W.B., et al. (eds.) SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 285–293. ACM Press, New York (2001)
Manmatha, R., Rath, T., Feng, F.: Modelling score distributions for combining the outputs of search engines. In: Croft, W.B., et al. (eds.) SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 267–275. ACM Press, New York (2001)
Swets, J.A.: Effectiveness of information retrieval methods. American Documentation 20, 72–89 (1969)
Bookstein, A.: When the most ‘pertinent’ document should not be retrieved – an analysis of the Swets model. Information Processing and Management 13, 377–383 (1977)
Collins-Thompson, K., et al.: Information filtering, novelty detection and named page finding. In: Voorhees, E.M., Harman, D.K. (eds.) The Eleventh Text REtrieval Conference, TREC 2002. NIST Special Publication, pp. 107–118. NIST (2003)
Robertson, S.E.: Explicit and implicit variables in information retrieval systems. Journal of the American Society for Information Science 26(4), 214–222 (1975)
van Rijsbergen, C.J.: Retrieval effectiveness. In: Voigt, M.J., Hanneman, G.J. (eds.) Progress in communication sciences, vol. 1, pp. 91–118. Ablex Publishing, Norwood (1979)
Hawking, D., Robertson, S.: On collection size and retrieval effectiveness. Information Retrieval 6, 99–150 (2003)
Robertson, S.E.: The probability ranking principle in information retrieval. Journal of Documentation 33, 294–304 (1977)
Robertson, S.E.: The parametric description of retrieval tests. part 1: The basic parameters. Journal of Documentation 25(1), 1–27 (1969)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Robertson, S. (2007). On Score Distributions and Relevance. In: Amati, G., Carpineto, C., Romano, G. (eds) Advances in Information Retrieval. ECIR 2007. Lecture Notes in Computer Science, vol 4425. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71496-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-71496-5_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71494-1
Online ISBN: 978-3-540-71496-5
eBook Packages: Computer ScienceComputer Science (R0)