Learning to Rank with Nonlinear Monotonic Ensemble

  • Nikita Spirin
  • Konstantin Vorontsov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6713)


Over the last decade learning to rank (L2R) has gained a lot of attention and many algorithms have been proposed. One of the most successful approach is to build an algorithm following the ensemble principle. Boosting is the key representative of this approach. However, even boosting isn’t effective when used to increase the performance of individually strong algorithms, scenario when we want to blend already successful L2R algorithms in order to gain an additional benefit. To address this problem we propose a novel algorithm, based on a theory of nonlinear monotonic ensembles, which is able to blend strong base rankers effectively. Specifically, we provide the concept of defect of a set of algorithms that allows to deduce a popular pairwise approach in strict mathematical terms. Using the concept of defect, we formulate an optimization problem and propose a sound method of its solution. Finally, we conduct experiments with real data which shows the effectiveness of the proposed approach.


Monotonic Function Base Algorithm Correctness Condition Generalization Ability Isotonic Regression 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barlow, R., Bartholomew, D., Bremner, J., Brunk, H.: Statistical inference under order restrictions; the theory and application of isotonic regression (1972)Google Scholar
  2. 2.
    Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: ICML 2005 (2005)Google Scholar
  3. 3.
    Cao, Z., Qin, T., Liu, T.Y., Tsai, M.F., Li, H.: Learning to rank: From pairwise approach to listwise approach. In: ICML 2007, pp. 129–136 (2007)Google Scholar
  4. 4.
    Cooper, W.S., Gey, F.C., Dabney, D.P.: Probabilistic retrieval based on staged logistic regression. In: SIGIR 1992 (1992)Google Scholar
  5. 5.
    Freund, Y., Iyer, R.D., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research (2003)Google Scholar
  6. 6.
    Guz, I.S.: Nonlinear monotonic compositions of classifiers. In: MMRO 13 (2007)Google Scholar
  7. 7.
    Joachims, T.: Optimizing search engines using click through data. In: SIGIR 2002 (2002)Google Scholar
  8. 8.
    Li, P., Burges, C.J., Wu, Q.: Learning to rank with nonsmooth cost functions. In: Advances in NIPS 19, pp. 193–200 (2006)Google Scholar
  9. 9.
    Qin, T., Liu, T.Y., Lai, W., Zhang, X.D., Wang, D., Li, H.: Ranking with multiple hyperplanes. In: SIGIR 2007, pp. 279–286 (2007)Google Scholar
  10. 10.
    Schapire, R.E.: Theoretical views of boosting and applications. In: Watanabe, O., Yokomori, T. (eds.) ALT 1999. LNCS (LNAI), vol. 1720, p. 13. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    Sill, J., Abu-Mostafa, Y.: Monotonicity hints. In: Advances in NIPS 9 (1997)Google Scholar
  12. 12.
    Taylor, M., Guiver, J., Robertson, S., Minka, T.: Softrank: Optimising non-smooth rank metrics. In: WSDM 2008 (2008)Google Scholar
  13. 13.
    Tsai, M.F., Liu, T.Y., Qin, T., Chen, H.H., Ma, W.Y.: Frank: A ranking method with fidelity loss. In: SIGIR 2007 (2007)Google Scholar
  14. 14.
    Vorontsov, K.: Optimization methods for linear and monotone correction in the algebraic approach to the recognition problem. Comp. Math and Mat. Phys. (2000)Google Scholar
  15. 15.
    Vorontsov, K.: Combinatorial bounds for learning performance. Doklady Mathematics 69(1), 145 (2004)zbMATHGoogle Scholar
  16. 16.
    Weimer, M., Karatzoglou, A., Le, Q., Smola, A.: Cofirank — maximum margin matrix factorization for collaborative ranking. In: Advances in NIPS 19 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Nikita Spirin
    • 1
  • Konstantin Vorontsov
    • 2
  1. 1.University of IllinoisUrbana-ChampaignUSA
  2. 2.Dorodnicyn Computing Center of the Russian Academy of SciencesRussia

Personalised recommendations