Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

Joint European Conference on Machine Learning and Knowledge Discovery in Databases

ECML PKDD 2012: Machine Learning and Knowledge Discovery in Databases pp 83–98Cite as

  1. Home
  2. Machine Learning and Knowledge Discovery in Databases
  3. Conference paper
Probability Estimation for Multi-class Classification Based on Label Ranking

Probability Estimation for Multi-class Classification Based on Label Ranking

  • Weiwei Cheng21 &
  • Eyke Hüllermeier21 
  • Conference paper
  • 4901 Accesses

  • 2 Citations

Part of the Lecture Notes in Computer Science book series (LNAI,volume 7524)

Abstract

We consider the problem of probability estimation in the setting of multi-class classification. While this problem has already been addressed in the literature, we tackle it from a novel perspective. Exploiting the close connection between probability estimation and ranking, our idea is to solve the former on the basis of the latter, taking advantage of recently developed methods for label ranking. More specifically, we argue that the Plackett-Luce ranking model is a very natural choice in this context, especially as it can be seen as a multinomial extension of the Bradley-Terry model. The latter provides the basis of pairwise coupling techniques, which arguably constitute the state-of-the-art in multi-class probability estimation. We explore the relationship between the pairwise and the ranking-based approach to probability estimation, both formally and empirically. Using synthetic and real-world data, we show that our method does not only enjoy nice theoretical properties, but is also competitive in terms of accuracy and efficiency.

Keywords

  • Class Label
  • Reconstruction Error
  • Ranking Model
  • Brier Score
  • Machine Learn Research

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Download conference paper PDF

References

  1. Bradley, R., Terry, M.: Rank analysis of incomplete block designs I. the method of paired comparisons. Biometrika 39, 324–345 (1952)

    MathSciNet  MATH  Google Scholar 

  2. Buja, A., Stuetzle, W., Shen, Y.: Loss functions for binary class probability estimation: Structure and applications. Technical report, University of Pennsylvania (2005)

    Google Scholar 

  3. Cheng, W., Dembczyński, K., Hüllermeier, E.: Label ranking methods based on the Plackett-Luce model. In: Proc. ICML 2010, pp. 215–222 (2010)

    Google Scholar 

  4. Cheng, W., Hühn, J., Hüllermeier, E.: Decision tree and instance-based learning for label ranking. In: Proc. ICML 2009, pp. 161–168 (2009)

    Google Scholar 

  5. Clemencon, S., Lugosi, G., Vayatis, N.: Ranking and empirical minimization of U-statistics. The Annals of Statistics 36(2), 844–874 (2008)

    CrossRef  MathSciNet  MATH  Google Scholar 

  6. Cour, T., Sapp, B., Taskar, B.: Learning from partial labels. Journal of Machine Learning Research 12, 1225–1261 (2011)

    MathSciNet  Google Scholar 

  7. Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29, 103–137 (1997)

    CrossRef  MATH  Google Scholar 

  8. Flach, P.A.: Putting Things in Order: On the Fundamental Role of Ranking in Classification and Probability Estimation. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 2–3. Springer, Heidelberg (2007)

    CrossRef  Google Scholar 

  9. Frank, A., Asuncion, A.: UCI machine learning repository (2010)

    Google Scholar 

  10. Fürnkranz, J.: Round robin classification. Journal of Machine Learning Research 2, 721–747 (2003)

    Google Scholar 

  11. Hastie, T., Tibshirani, R.: Classification by pairwise coupling. The Annals of Statistics 26(1), 451–471 (1998)

    MathSciNet  MATH  Google Scholar 

  12. Herbei, R., Wegkamp, M.: Classification with reject option. Canadian Journal of Statistics 34(4), 709–721 (2006)

    CrossRef  MathSciNet  MATH  Google Scholar 

  13. Luce, R.: Individual Choice Behavior: A Theoretical Analysis. Wiley (1959)

    Google Scholar 

  14. Mallows, C.: Non-null ranking models. Biometrika 44(1), 114–130 (1957)

    MathSciNet  MATH  Google Scholar 

  15. Marden, J.: Analyzing and Modeling Rank Data. CRC Press (1995)

    Google Scholar 

  16. Niculescu-Mizil, A., Caruana, R.: Predicting good probabilities with supervised learning. In: Proc. ICML, pp. 625–632 (2005)

    Google Scholar 

  17. Plackett, R.: The analysis of permutations. Applied Statistics 24(2), 193–202 (1975)

    CrossRef  MathSciNet  Google Scholar 

  18. Platt, J.C.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in Large Margin Classifiers, pp. 61–74. MIT Press (1999)

    Google Scholar 

  19. Rifkin, R., Klautau, A.: In defense of one-vs-all classification. The Journal of Machine Learning Research 5, 101–141 (2004)

    MathSciNet  MATH  Google Scholar 

  20. Wellman, M.P.: Some varieties of qualitative probability. In: Proc. IPMU 1994, Paris, pp. 437–442 (1994)

    Google Scholar 

  21. Wu, T., Lin, C., Weng, R.: Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 5, 975–1005 (2004)

    MathSciNet  MATH  Google Scholar 

  22. Zadrozny, B., Elkan, C.: Learning and making decisions when costs and probabilities are both unknown. In: Proc. KDD, pp. 204–213 (2001)

    Google Scholar 

  23. Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Proc. KDD, pp. 694–699 (2002)

    Google Scholar 

  24. Zhang, T.: Statistical behavior and consistency of classification methods based on convex risk minimization. Annals of Statistics 32(1), 5–85 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics and Computer Science Department, University of Marburg, Germany

    Weiwei Cheng & Eyke Hüllermeier

Authors
  1. Weiwei Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eyke Hüllermeier
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Intelligent Systems Laboratory, University of Bristol, Merchant Venturers Building, Woodland Road, BS8 1UB, Bristol, UK

    Peter A. Flach

  2. Intelligent Systems Laboratory, University of Bristol, Merchant Venturers Building, Woodland Road,, BS8 1UB, Bristol, UK

    Tijl De Bie & Nello Cristianini & 

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Cheng, W., Hüllermeier, E. (2012). Probability Estimation for Multi-class Classification Based on Label Ranking. In: Flach, P.A., De Bie, T., Cristianini, N. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2012. Lecture Notes in Computer Science(), vol 7524. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33486-3_6

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-33486-3_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-33485-6

  • Online ISBN: 978-3-642-33486-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature