Recurrent neural networks for speech recognition
In this paper we present some results from a net-like structure for Hidden Markov Models, applied to speech recognition. Net topology is a Recurrent Neural Network in which each temporary step is identified as a layer. Backpropagation techniques are used to train the RNN-HMM. Two types of training estimations are used: Maximum Likelihood and Competitive Training. Maximum Likelihood estimation algorithm using backpropagation provides the same updating equations as Baum-Welch algorithm used in HMM. Competitive Training is based on the probability of correct labelling the sequences from the Maximum Likelihood measures. Our results have shown that the best procedure is to train first with Maximum Likelihood estimation and then with Competitive Training reestimation.
Unable to display preview. Download preview PDF.
- Bourlard, H; Wellekens, C.J. "Speech dynamics and Recurrent Neural Networks". Proc. ICASSP-89 pp. 33–36.Google Scholar
- Demichelis, P; et als. "On the use of Neural Networks for Speaker Independent Isolated Word Recognition". Proc. ICASSP-89 pp. 314–317.Google Scholar
- Sakoe, H.: et als. "Speaker Independent Word Recognition Using Dynamic Programming Neural Networks" in Proc. ICASSP-89 pp. 29–32.Google Scholar
- Hwang, J.N.; Vlontzos, J.; Kung, S. "A Systolic Neural Network Architecture for Hidden Markov Models", in IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 1967–1979, Dec. 1989.Google Scholar
- Kung, S.; Hwang, J. "A Unifying Algorithm/Architecture for Artificial Neural Networks" in Proc. ICASSP-89, pp. 2505–2508.Google Scholar
- Bahl,L.R; et als. "Maximum Mutual Information Estimation of Hidden Markov Model Parameters for Speech Recognition" in Proc. ICASSP-86, pp. 49–52. Tokio.Google Scholar
- Ephrain, Y.; Rabiner, L. "On the Relations Between Modelling Approaches for Speech Recognition", in IEEE Trans. Acoust., Speech, Signal Processing, vol. 36, pp. 372–379. March, 1990.Google Scholar
- Bridle, J.S. "Alpha-nets: A Recurrent Neural Network Architecture with a Hidden Markov Model Interpretation" in Speech Communication, vol. 9, 1990.Google Scholar
- Rabiner, L. "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition". in Proc. of the IEEE, vol. 77, n. 2, Feb. 1989.Google Scholar
- Sadaoki, Furui. "Speaker-Independent Isolated Word Recognition Using Dynamic Features of Speech Spectrum" in "IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-34, pp. 52–59, Feb. 1986.Google Scholar