Visionary Speech: Looking Ahead to Practical Speechreading Systems

Abstract

We present an overview of speechreading systems, paying particular attention to the various approaches to key design decisions and the benefits and drawbacks of each. Our own speechreading efforts have centered on the use of Hidden Markov Models, and, for feature extraction, de-formable templates. We have explored several sensory integration schemes — early, late, and a novel intermediate method made possible by a new learning method, called Boltzmann zippers. We describe several possible practical applications of speechreading systems, and conclude with a list of important outstanding problems.