Knowledge and Information Systems

, Volume 17, Issue 3, pp 381–395 | Cite as

Ranking with decision tree

  • Fen Xia
  • Wensheng Zhang
  • Fuxin Li
  • Yanwu Yang
Regular Paper


Ranking problems have recently become an important research topic in the joint field of machine learning and information retrieval. This paper presented a new splitting rule that introduces a metric, i.e., an impurity measure, to construct decision trees for ranking tasks. We provided a theoretical basis and some intuitive explanations for the splitting rule. Our approach is also meaningful to collaborative filtering in the sense of dealing with categorical data and selecting relative features. Some experiments were made to illustrate our ranking approach, whose results showed that our algorithm outperforms both perceptron-based ranking and the classification tree algorithms in term of accuracy as well as speed.


Machine learning Ranking Decision tree Splitting rule 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Buntine W, Niblett T (1992) A further comparison of splitting rules for decision-tree induction. Mach Learn 8: 75–85Google Scholar
  2. 2.
    Burges C, Shaked T, Renshaw E et al. (2005) Learning to ranking using gradient descent. In: Proceedings of the 22nd international conference on maching learning (ICML-2005), Bonn, Germany, pp 89–96Google Scholar
  3. 3.
    Breiman L, Friedman J, Olshen RA et al (1984) Classification and regression trees. Wadsworth, BelmontzbMATHGoogle Scholar
  4. 4.
    Chu W, Ghahramani Z (2005) Gaussian processes for ordinal regression. J Mach Learn Res 6: 1019–1041MathSciNetGoogle Scholar
  5. 5.
    Crammer K, Singer Y (2002) Pranking with ranking. Advances in Neural Information Processing Systems 14. MIT Press, Cambridge, pp 641–647Google Scholar
  6. 6.
    Cohen WW, Schapire RE, Singer Y (1999) Learning to order things. J Artif Intell Res 10: 243–270zbMATHMathSciNetGoogle Scholar
  7. 7.
    Freund Y, Iyer R, Schapire RE et al (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4: 933–969CrossRefMathSciNetGoogle Scholar
  8. 8.
    Genest D, Chein M (2004) A content-search information retrieval process based on conceptual graphs. Knowl Inform Syst 8: 292–309CrossRefGoogle Scholar
  9. 9.
    Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning: data mining, inference, and prediction. Springer, New YorkzbMATHGoogle Scholar
  10. 10.
    Herbrich R, Graepel T, Obermayer K (2000) Large margin rank boundaries for ordinal regression. Advance in large margin classifiers. MIT Press, Cambridge, MA. pp 115–132Google Scholar
  11. 11.
    Harrington EF (2003) Online ranking/collaborative filtering using the perceptron algorithm. In: Proceedings of the twentieth international conference on machine learning (ICML-2003), Washington, DCGoogle Scholar
  12. 12.
    Li T, Zhu S-H, Ogihara M (2006) Using discriminat analysis for multi-class classification: an experimental investigation. Knowl Inform Syst 10: 453–472CrossRefGoogle Scholar
  13. 13.
    Mitchell TM (1997) Machine Learning. The McGraw-Hill New York, pp 52–78Google Scholar
  14. 14.
    Quinlan JR (1986) Induction of decision trees. Machine Learning 1: 81–106Google Scholar
  15. 15.
    Quinlan JR (1993) C4.5: programs for Machine learning. Kaufmann, MorganGoogle Scholar
  16. 16.
    Rosset S, Perlich C, Zadrozny B (2007) Ranking-based evaluation of regression models. Knowl Inform Syst 12: 331–353CrossRefGoogle Scholar
  17. 17.
    Spink A, Greisdorf H, Bateman J (1998) From highly relevant to not relevant: examining different regions of relevance. Inform Process Manag Int J 34: 599–621CrossRefGoogle Scholar
  18. 18.
    Shen L, Joshi AK (2005) Ranking and reranking with perceptron. Mach Learn 60: 73–96CrossRefGoogle Scholar
  19. 19.
    Shaw WM, Wood JB, Wood RE et al (1991) The cystic fibrosis database: content and research opportunities. LISR 13: 347–366Google Scholar
  20. 20.
    Shashua A, Levin A (2003) Ranking with large margin principle: two approaches. In: Proceedings of the conference on Neural information processing systems, (NIPS) 14Google Scholar
  21. 21.
    Thabtah F-A, Cowling P, Peng Y-H (2005) Multiple lables associative classification. Knowl Inform Syst 9: 109–129CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2007

Authors and Affiliations

  1. 1.The Key Laboratory of Complex Systems and Intelligence Science, Institute of AutomationChinese Academy of SciencesBeijingPeople’s Republic of China

Personalised recommendations