Statistical Inference in MLPs

Abstract

In Chapter 3, we showed that HMMs were stochastic models that dealt efficiently with the statistical and sequential character of the speech signal, but which also suffer from several limiting assumptions that are required for tractable solutions. In Chapter 4, we discussed ANNs and showed that they had their own attractive properties; in particular, they appear to rely on fewer basic assumptions. Chapter 5 briefly reviewed the most popular ANN approaches currently used for sequence processing in general and speech recognition in particular. We concluded that none of these were able to solve CSR properly using ANNs by themselves. Given these tradeoffs, we have been interested in using ANNs to overcome some HMM drawbacks while staying within the latter’s formalism. This kind of hybrid is frequently not straightforward, however; for instance, it is difficult to optimally incorporate rule-based speech knowledge in an HMM-based ASR system.¹

One never goes so far as when one doesn’t know where one is going. - Johann Wolfgang von Goethe -

Of course, systems do incorporate such knowledge, but it is difficult to do this in an integrated way; most commonly, all but the coarsest knowledge about the language or the world is built into a post-processor that is loosely-coupled with the acoustic recognizer.