Abstract
In this chapter, we introduce the generalization analysis on learning-to-rank methods. In particular, we first introduce the uniform generalization bounds and then the algorithm-dependent generalization bounds. The uniform bounds hold for any ranking function in a given function class. The algorithm-dependent bounds instead consider the specific ranking function learned by the given algorithm, thus can usually be tighter. The bounds introduced in this chapter are derived under different ranking frameworks, and can explain behaviors of different learning-to-rank algorithms. We also show the limitations of existing analyses and discuss how to improve them in future work.
This is a preview of subscription content, log in via an institution to check access.
Notes
- 1.
The three transformation functions are
-
⋄ Linear Functions: ϕ L (x)=ax+b,x∈[−BM,BM].
-
⋄ Exponential Functions: ϕ E (x)=e ax,x∈[−BM,BM].
-
⋄ Sigmoid Functions: \(\varphi_{S}(x)=\frac{1}{1+e^{-ax}}, x\in[-\mathit{BM},\mathit{BM}]\).
-
- 2.
Note that the disadvantage of algorithm-dependent bounds lies in that they can only be used for specific algorithms, and may not be derived for every algorithm.
References
Agarwal, S.: Generalization bounds for some ordinal regression algorithms. In: Proceedings of the 19th International Conference on Algorithmic Learning Theory (ALT 2008), pp. 7–21 (2008)
Agarwal, S., Graepel, T., Herbrich, R., Har-Peled, S., Roth, D.: Generalization bounds for the area under the roc curve. Journal of Machine Learning 6, 393–425 (2005)
Agarwal, S., Niyogi, P.: Stability and generalization of bipartite ranking algorithms. In: Proceedings of the 18th Annual Conference on Learning Theory (COLT 2005), pp. 32–47 (2005)
Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian complexities risk bounds and structural results. Journal of Machine Learning 3, 463–482 (2003)
Bousquet, O., Boucheron, S., Lugosi, G.: Introduction to statistical learning theory. In: Advanced Lectures on Machine Learning, pp. 169–207. Springer, Berlin (2004)
Bousquet, O., Elisseeff, A.: Stability and generalization. The Journal of Machine Learning Research 2, 449–526 (2002)
Cao, Y., Xu, J., Liu, T.Y., Li, H., Huang, Y., Hon, H.W.: Adapting ranking SVM to document retrieval. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2006), pp. 186–193 (2006)
Chen, W., Liu, T.Y., Ma, Z.M.: Two-layer generalization analysis for ranking using rademacher average. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Advances in Neural Information Processing Systems 23 (NIPS 2010), pp. 370–378 (2011)
Clemencon, S., Lugosi, G., Vayatis, N.: Ranking and empirical minimization of u-statistics. The Annals of Statistics 36(2), 844–874 (2008)
Freund, Y., Iyer, R., Schapire, R., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research 4, 933–969 (2003)
Herbrich, R., Obermayer, K., Graepel, T.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers, pp. 115–132 (2000)
Joachims, T.: Optimizing search engines using clickthrough data. In: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2002), pp. 133–142 (2002)
Lan, Y., Liu, T.Y.: Generalization analysis of listwise learning-to-rank algorithms. In: Proceedings of the 26th International Conference on Machine Learning (ICML 2009), pp. 577–584 (2009)
Lan, Y., Liu, T.Y., Qin, T., Ma, Z., Li, H.: Query-level stability and generalization in learning to rank. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), pp. 512–519 (2008)
Rajaram, S., Agarwal, S.: Generalization bounds for k-partite ranking. In: NIPS 2005 Workshop on Learning to Rank (2005)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Berlin (1995)
Vapnik, V.N.: Statistical Learning Theory. Wiley-Interscience, New York (1998)
Yilmaz, E., Robertson, S.: Deep versus shallow judgments in learning to rank. In: Proceedings of the 32st Annual International Conference on Research and Development in Information Retrieval (SIGIR 2009), pp. 662–663 (2009)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, TY. (2011). Generalization Analysis for Ranking. In: Learning to Rank for Information Retrieval. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14267-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-14267-3_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14266-6
Online ISBN: 978-3-642-14267-3
eBook Packages: Computer ScienceComputer Science (R0)