Skip to main content
SpringerLink
Log in
SpringerLink
Log in

The Pairwise Approach

  • Chapter
Learning to Rank for Information Retrieval

  • 4798 Accesses

Abstract

In this chapter we will introduce the pairwise approach to learning to rank. Specifically we first introduce several example algorithms, whose major differences are in the loss functions. Then we discuss the limitations of these algorithms and present some improvements that enable better ranking performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    As far as we know, Microsoft Bing Search (http://www.bing.com/) is using the model trained with a variation of RankNet.

  2. 2.

    Note that Ranking SVM was originally proposed in [22] to solve the problem of ordinal regression. However, according to its formulation, it solves the problem of pairwise classification in an even more natural way.

  3. 3.

    See http://olivier.chapelle.cc/primal/ and http://www.cs.cornell.edu/People/tj/svm_light/svm_rank.html.

  4. 4.

    Note that there are many algorithms for rank aggregation proposed in the literature, such as BordaCount [2, 5, 16], median rank aggregation [17], genetic algorithm [4], fuzzy logic-based rank aggregation [1], and Markov chain-based rank aggregation [16]. Although BordaCount is used in [26] as an example, it by no means dictates that other methods cannot be used for the same purpose.

References

  1. Ahmad, N., Beg, M.: Fuzzy logic based rank aggregation methods for the World Wide Web. In: Proceedings of the International Conference on Artificial Intelligence in Engineering and Technology (ICAIET 2002), pp. 363–368 (2002)

    Google Scholar 

  2. Aslam, J.A., Montague, M.: Models for metasearch. In: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2001), pp. 276–284 (2001)

    Chapter  Google Scholar 

  3. Bax, E.: Nearly uniform validation improves compression-based error bounds. Journal of Machine Learning Research 9, 1741–1755 (2008)

    MathSciNet  MATH  Google Scholar 

  4. Beg, M.M.S.: Parallel rank aggregation for the World Wide Web. World Wide Web Journal 6(1), 5–22 (2004)

    Article  MathSciNet  Google Scholar 

  5. Borda, J.: Mémoire sur les élections au scrutin. In: Histoire de l’Académie royale des sciences, pp. 42–51 (1781)

    Google Scholar 

  6. Bradley, R., Terry, M.: The rank analysis of incomplete block designs: the method of paired comparisons. Biometrika 39, 324–345 (1952)

    MathSciNet  MATH  Google Scholar 

  7. Burges, C.J., Ragno, R., Le, Q.V.: Learning to rank with nonsmooth cost functions. In: Advances in Neural Information Processing Systems 19 (NIPS 2006), pp. 395–402 (2007)

    Google Scholar 

  8. Burges, C.J., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 89–96 (2005)

    Chapter  Google Scholar 

  9. Cao, Y., Xu, J., Liu, T.-Y., Li, H., Huang, Y., Hon, H.-W.: Adapting ranking SVM to document retrieval. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2006), pp. 186–193 (2006)

    Chapter  Google Scholar 

  10. Carvalho, V.R., Elsas, J.L., Cohen, W.W., Carbonell, J.G.: A meta-learning approach for robust rank learning. In: SIGIR 2008 Workshop on Learning to Rank for Information Retrieval (LR4IR 2008) (2008)

    Google Scholar 

  11. Chapelle, O., Keerthi, S.S.: Efficient algorithms for ranking with SVMs. Information Retrieval Journal. Special Issue on Learning to Rank 13(3), doi:10.1007/s10791-009-9109-9 (2010)

  12. Cohen, W.W., Schapire, R.E., Singer, Y.: Learning to order things. In: Advances in Neural Information Processing Systems (NIPS 1997), vol. 10, pp. 243–270 (1998)

    Google Scholar 

  13. Collobert, R., Sinz, F., Weston, J., Bottou, L.: Trading convexity for scalability. In: Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), pp. 201–208. ACM, New York (2006). http://doi.acm.org/10.1145/1143844.1143870

    Chapter  Google Scholar 

  14. Cortes, C., Mohri, M., et al.: Magnitude-preserving ranking algorithms. In: Proceedings of the 24th International Conference on Machine Learning (ICML 2007), pp. 169–176 (2007)

    Chapter  Google Scholar 

  15. Donmez, P., Svore, K.M., Burges, C.J.C.: On the local optimality of lambdarank. In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2009), pp. 460–467 (2009)

    Google Scholar 

  16. Dwork, C., Kumar, R., Naor, M., Sivakumar, D.: Rank aggregation methods for the web. In: Proceedings of the 10th International Conference on World Wide Web (WWW 2001), pp. 613–622. ACM, New York (2001)

    Google Scholar 

  17. Fagin, R., Kumar, R., Sivakumar, D.: Efficient similarity search and classification via rank aggregation. In: Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD 2003), pp. 301–312. ACM, New York (2003). http://doi.acm.org/10.1145/872757.872795

    Chapter  Google Scholar 

  18. Freund, Y., Iyer, R., Schapire, R., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research 4, 933–969 (2003)

    MathSciNet  Google Scholar 

  19. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and System Sciences 55(1), 119–139 (1995)

    Article  MathSciNet  Google Scholar 

  20. Friedman, J.: Greedy function approximation: a gradient boosting machine. Annual Statistics 29, 1189–1232 (2001)

    Article  MATH  Google Scholar 

  21. Gao, J., Qi, H., Xia, X., Nie, J.: Linear discriminant model for information retrieval. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2005), pp. 290–297 (2005)

    Chapter  Google Scholar 

  22. Herbrich, R., Obermayer, K., Graepel, T.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers, pp. 115–132 (2000)

    Google Scholar 

  23. Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2002), pp. 133–142 (2002)

    Google Scholar 

  24. Joachims, T.: Training linear svms in linear time. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2006), pp. 217–226 (2006)

    Chapter  Google Scholar 

  25. Matveeva, I., Burges, C., Burkard, T., Laucius, A., Wong, L.: High accuracy retrieval with multiple nested ranker. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2006), pp. 437–444 (2006)

    Chapter  Google Scholar 

  26. Qin, T., Liu, T.-Y., Lai, W., Zhang, X.-D., Wang, D.-S., Li, H.: Ranking with multiple hyperplanes. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pp. 279–286 (2007)

    Chapter  Google Scholar 

  27. Qin, T., Zhang, X.-D., Tsai, M.-F., Wang, D.-S., Liu, T.-Y., Li, H.: Query-level loss functions for information retrieval. Information Processing and Management 44(2), 838–855 (2008)

    Article  Google Scholar 

  28. Rigutini, L., Papini, T., Maggini, M., Scarselli, F.: Learning to rank by a neural-based sorting algorithm. In: SIGIR 2008 Workshop on Learning to Rank for Information Retrieval (LR4IR 2008) (2008)

    Google Scholar 

  29. Rudin, C.: Ranking with a p-norm push. In: Proceedings of the 19th Annual Conference on Learning Theory (COLT 2006), pp. 589–604 (2006)

    Google Scholar 

  30. Shen, L., Joshi, A.K.: Ranking and reranking with perceptron. Journal of Machine Learning 60(1–3), 73–96 (2005)

    Article  Google Scholar 

  31. Sun, Z., Qin, T., Tao, Q., Wang, J.: Robust sparse rank learning for non-smooth ranking measures. In: Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2009), pp. 259–266 (2009)

    Google Scholar 

  32. Thurstone, L.: A law of comparative judgement. Psychological Review 34, 34 (1927)

    Article  Google Scholar 

  33. Tsai, M.-F., Liu, T.-Y., Qin, T., Chen, H.-H., Ma, W.-Y.: Frank: a ranking method with fidelity loss. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pp. 383–390 (2007)

    Chapter  Google Scholar 

  34. Usunier, N., Buffoni, D., Gallinari, P.: Ranking with ordered weighted pairwise classification. In: Proceedings of the 26th International Conference on Machine Learning (ICML 2009), pp. 1057–1064 (2009)

    Google Scholar 

  35. Yager, R.R.: On ordered weighted averaging aggregation operators in multi-criteria decision making. IEEE Transactions on Systems, Man, and Cybernetics 18(1), 183–190 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  36. Zheng, Z., Chen, K., Sun, G., Zha, H.: A regression framework for learning ranking functions using relative relevance judgments. In: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pp. 287–294 (2007)

    Chapter  Google Scholar 

  37. Zheng, Z., Zha, H., Sun, G.: Query-level learning to rank using isotonic regression. In: SIGIR 2008 Workshop on Learning to Rank for Information Retrieval (LR4IR 2008) (2008)

    Google Scholar 

  38. Zheng, Z., Zha, H., Zhang, T., Chapelle, O., Chen, K., Sun, G.: A general boosting method and its application to learning ranking functions for web search. In: Advances in Neural Information Processing Systems 20 (NIPS 2007), pp. 1697–1704 (2008)

    Google Scholar 

  39. Zhou, K., Xue, G.-R., Zha, H., Yu, Y.: Learning to rank with ties. In: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2008), pp. 275–282 (2008)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research Asia, Bldg #2, No. 5, Dan Ling Street, Haidian District, Beijing, 100080, People’s Republic of China

    Tie-Yan Liu

Authors
  1. Tie-Yan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tie-Yan Liu .

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Liu, TY. (2011). The Pairwise Approach. In: Learning to Rank for Information Retrieval. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14267-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-14267-3_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-14266-6

  • Online ISBN: 978-3-642-14267-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation