Probabilistic Score Normalization for Rank Aggregation
Rank aggregation is a pervading operation in IR technology. We hypothesize that the performance of score-based aggregation may be affected by artificial, usually meaningless deviations consistently occurring in the input score distributions, which distort the combined result when the individual biases differ from each other. We propose a score-based rank aggregation model where the source scores are normalized to a common distribution before being combined. Early experiments on available data from several TREC collections are shown to support our proposal.
KeywordsInformation Retrieval Score Distribution Rank List Rank Aggregation Individual Bias
Unable to display preview. Download preview PDF.
- 2.Croft, W.B.: Combining approaches to information retrieval. In: Advances in Information Retrieval: Recent Research from the Center for Intelligent Information Retrieval, pp. 1–36. Kluwer Academic Publishers, Dordrecht (2000)Google Scholar
- 3.Lee, J.H.: Analysis of multiple evidence combination. In: 20th ACM SIGIR Conf. on Research and Development in Information Retrieval (SIGIR 1997), New York, pp. 267–276 (1997)Google Scholar
- 4.Manmatha, R., Rath, R., Feng, F.: Modelling score distributions for combining the outputs of search engines. In: 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR, New Orleans, LA, pp. 267–275 (2001)Google Scholar
- 6.Montague, M., Aslam, J.A.: Relevance score normalization for metasearch. In: 10th Conf. on Information and Knowledge Management (CIKM 2001), Atlanta, GA, pp. 427–433 (2001)Google Scholar
- 7.Renda, M.E., Straccia, U.: Web metasearch: rank vs. score based rank aggregation methods. In: ACM symposium on Applied Computing, Melbourne, Florida, pp. 841–846 (2003)Google Scholar