Hard-Testing the Multi-stream Approach to Automatic Speech Recognition

Abstract

It has been experimentally demonstrated that optimized multi-stream based speech recognisers can perform substantially better than the corresponding conventional systems on some particular recognition tasks. Typically, those applications present critical robustness problems, with the speech signal affected by noise that is localised in the acoustic space. In general, substantial recognition improvement is obtained extracting multiple feature streams that encode highly complementary information related with the speech signal. The main goal of this experimental study is to assess the potential of the multi-stream statistical formalism on standard clean speech recognition tasks not particularly favourable to this approach and, adding to this, intentionally using highly correlated feature streams. Notwithstanding, it is here demonstrated that a careful design of the streams recombination model, adapting their local influence on the decoding process according to several information sources, can lead to significant performance gains comparing to the single-stream corresponding systems.