Statistical Language Modeling Using Leaving-One-Out

Abstract

The need for a stochastic language model in speech recognition arises from Bayes’ decision rule for minimum error rate (Bahl et al., 1983). The word sequence w₁ ... w _N to be recognized from the sequence of acoustic observations x ₁ ... x _T is determined as that word sequence w ₁ ... w _N for which the posterior probability Pr(w ₁ ... w _N |x ₁ ... x _T ) attains its maximum. This rule can be rewritten in the form:

$$ \mathop {\arg \,\max }\limits_{{\omega _1} \cdot \cdot \cdot {\omega _N}} \{ \Pr ({\omega _1} \cdot \cdot \cdot {\omega _N}) \cdot \Pr ({x_1} \cdot \cdot \cdot {x_T}\left| {{\omega _1} \cdot \cdot \cdot {\omega _N}} \right.)\} , $$

, where Pr(x ₁ ... x _T |w ₁ ... w _N ) is the conditional probability of, given the word sequence w ₁ ... w _N , observing the sequence of acoustic measurements x ₁ ... x _T and where Pr(w ₁ ... w _N ) is the prior probability of producing the word sequence w ₁ ... w _N .