Introduction

Abstract

The objective of this monograph is to unite two powerful yet different paradigms in machine learning: generative and discriminative. Generative learning approaches such as Bayesian networks are at the heart of many pattern recognition, artificial intelligence and perception systems. These provide a rich framework for imposing structure and prior knowledge to learn a useful and detailed model of a phenomenon. Yet recent progress in discriminative learning, which includes the currently popular support vector machine approaches, has demonstrated that superior performance can be obtained by avoiding generative modeling and focusing only on the particular task the machine has to solve. The dividing gap between these two prevailing methods begs the question: is there a powerful connection between generative and discriminative learning that combines the complementary strengths of the two approaches? In this text, we undertake the challenge of building such a bridge and explicate a common formalism that spans both schools of thought.

lt is not knowledge, but the act 01 learning,...which grants the qreatest enjoyment. Karl Friedrich Gauss, 1808.