Maximum Entropy Model and Conditional Random Field

In this chapter, we will describe a statistical model that conforms to the maximum entropy principle (we will call it the maximum entropy model, or ME model in short) [68, 69]. Through mathematical derivations, we will show that the maximum entropy model is a kind of exponential model, and is a close sibling of the Gibbs distribution described in Chap. 6. An essential difference between the two models is that the former is a discriminative model, while the latter is a generative model. Through a model complexity analysis, we will show why discriminative models are generally superior to generative models in terms of data modeling power. We will also describe the Conditional Random Field (CRF), one of the latest discriminative models in the literature, and prove that CRF is equivalent to the maximum entropy model. At the end of this chapter, we will provide a case study where the ME model is applied to baseball highlight detections, and is compared with the HMM model described in Chap. 7.


Hide Markov Model Maximum Entropy Conditional Random Field Maximum Entropy Method Discriminative Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer Science+Business Media, LLC 2007

Personalised recommendations