On Theoretically Valid Score Distributions in Information Retrieval

  • Ronan Cummins
  • Colm O’Riordan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7224)


In this paper, we aim to investigate the practical usefulness of the Recall-Fallout Convexity Hypothesis (RFCH) for a number of document score distribution (SD) models. We compare SD models that do not automatically adhere to the RFCH to modified versions of the same SD models that do adhere to the RFCH. We compare these models using the inference of average precision as a measure of utility. For the three models studied in this paper, we conclude that adhering to the RFCH is practically useful for the two-normal model, makes no difference for the two-gamma model, and degrades the performance of the two-lognormal model.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arampatzis, A., Robertson, S.: Modeling score distributions in information retrieval. Inf. Retr. 14(1), 26–46 (2011)CrossRefGoogle Scholar
  2. 2.
    Cummins, R.: Measuring the Ability of Score Distributions to Model Relevance. In: Salem, M.V.M., Shaalan, K., Oroumchian, F., Shakery, A., Khelalfa, H. (eds.) AIRS 2011. LNCS, vol. 7097, pp. 25–36. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Robertson, S.: On Score Distributions and Relevance. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 40–51. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Robertson, S.E., Kanoulas, E., Yilmaz, E.: Extending average precision to graded relevance judgments. In: SIGIR 2010, pp. 603–610. ACM, New York (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ronan Cummins
    • 1
  • Colm O’Riordan
    • 1
  1. 1.Dept. of Information TechnologyNational University of IrelandGalwayIreland

Personalised recommendations