Generative and Discriminative Learning
Generative learning refers alternatively to any classification learning process that classifies by using an estimate of the joint probability P(y, x) or to any classification learning process that classifies by using estimates of the prior probability P(y) and the conditional probability P(x | y) (Bishop, 2007 Jaakkola & Haussler, 1999; Jaakkola, Meila & Jebara, 1999; Lasserre, Bishop & Minka, 2006; Ng & Jordan, 2002), where y is a class and x is a description of an object to be classified. Generative learning contrasts with discriminative learning in which a model or estimate of P(y | x) is formed without reference to an explicit estimate of any of P(y, x), P(x) or P(x | y).
It is also common to categorize as discriminative approaches based on a decision function that directly maps from input x onto the output y (such as support vector machines, neural networks, and decision trees), where the decision risk is minimized without estimation of P(y, x), P(x | y) or P(y | x) (Ja...
- Bishop, C. M. (2007). Pattern recognition and machine learning. Springer.Google Scholar
- Blum, A., & Mitchell, T. (1998). Combining labeled and unlabeled data with co-training. Proceedings of the eleventh annual conference on Computational learning theory, Madison, Wisconsin, USA. New York: ACM.Google Scholar
- Chapelle, O., Schölkopf, B., & Zien, A. (2006). Semi-supervised learning. Cambridge: The MIT Press.Google Scholar
- Jaakkola, T. S., & Haussler, D. (1999). Exploiting generative models in discriminative classifiers. Advances in neural information processing systems, 11.Google Scholar
- Jaakkola, T., Meila, M., & Jebara, T. (1999). Maximum entropy discrimination. Advances in neural information processing systems, 12.Google Scholar
- Lasserre, J. A., Bishop, C. M., & Minka, T. P. (2006). Principled hybrids of generative and discriminative models. IEEE Conference on Computer Vision and Pattern Recognition, New York.Google Scholar
- Ng, A. Y., & Jordan, M. I. (2002). On discriminative vs. Generative classifiers: A comparison of logistic regression and naive Bayes. Advances in neural information processing systems, 14.Google Scholar
- Taskar, B., Guestrin, C., & Koller, D. (2004). Max-margin Markov networks. Advances in neural information processing systems, 16.Google Scholar