Encyclopedia of Machine Learning

2010 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Generative and Discriminative Learning

  • Bin Liu
  • Geoffrey I. Webb
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-30164-8_332

Definition

Generative learning refers alternatively to any classification learning process that classifies by using an estimate of the joint probability P(y, x) or to any classification learning process that classifies by using estimates of the prior probability P(y) and the conditional probability P(x | y) (Bishop, 2007 Jaakkola & Haussler, 1999; Jaakkola, Meila & Jebara, 1999; Lasserre, Bishop & Minka, 2006; Ng & Jordan, 2002), where y is a class and x is a description of an object to be classified. Generative learning contrasts with discriminative learning in which a model or estimate of P(y | x) is formed without reference to an explicit estimate of any of P(y, x), P(x) or P(x | y).

It is also common to categorize as discriminative approaches based on a decision function that directly maps from input x onto the output y (such as support vector machines, neural networks, and decision trees), where the decision risk is minimized without estimation of P(y, x), P(x | y) or P(y | x) (Ja...

This is a preview of subscription content, log in to check access.

References

  1. Bishop, C. M. (2007). Pattern recognition and machine learning. Springer.Google Scholar
  2. Blum, A., & Mitchell, T. (1998). Combining labeled and unlabeled data with co-training. Proceedings of the eleventh annual conference on Computational learning theory, Madison, Wisconsin, USA. New York: ACM.Google Scholar
  3. Chapelle, O., Schölkopf, B., & Zien, A. (2006). Semi-supervised learning. Cambridge: The MIT Press.Google Scholar
  4. Efron, B. (1975). The efficiency of logistic regression compared to normal discriminant analysis. Journal of the American Statistical Association, 70(352), 892–898.MathSciNetMATHCrossRefGoogle Scholar
  5. Jaakkola, T. S., & Haussler, D. (1999). Exploiting generative models in discriminative classifiers. Advances in neural information processing systems, 11.Google Scholar
  6. Jaakkola, T., Meila, M., & Jebara, T. (1999). Maximum entropy discrimination. Advances in neural information processing systems, 12.Google Scholar
  7. Lasserre, J. A., Bishop, C. M., & Minka, T. P. (2006). Principled hybrids of generative and discriminative models. IEEE Conference on Computer Vision and Pattern Recognition, New York.Google Scholar
  8. Ng, A. Y., & Jordan, M. I. (2002). On discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes. Advances in neural information processing systems, 14.Google Scholar
  9. Taskar, B., Guestrin, C., & Koller, D. (2004). Max-margin Markov networks. Advances in neural information processing systems, 16.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Bin Liu
    • 1
  • Geoffrey I. Webb
    • 1
  1. 1.Monash UniversityVictoriaAustralia