Neural Network Language Model with Cache
In this paper we investigate whether a combination of statistical, neural network and cache language models can outperform a basic statistical model. These models have been developed, tested and exploited for a Czech spontaneous speech data, which is very different from common written Czech and is specified by a small set of the data available and high inflection of the words. As a baseline model we used a trigram model and after its training several cache models interpolated with the baseline model have been tested and measured on a perplexity. Finally, an evaluation of the model with the lowest perplexity has been performed on speech recordings of phone calls.
Keywordsneural networks language modelling automatic speech recognition
Unable to display preview. Download preview PDF.
- 1.Stolcke, A.: SRILM – an extensible language modeling toolkit. In: INTERSPEECH (2002)Google Scholar
- 2.Mikolov, T., Kopecký, J., Burget, L., Glembek, O., Černocký, J.: Neural network based language models for highly inflective languages. In: ICASSP, pp. 4725–4728 (2009)Google Scholar
- 3.Schwenk, H., Gauvain, J.: Training Neural Network Language Models on Very Large Corpora. In: HLT/EMNLP (2005)Google Scholar
- 4.Brown, P.F., Pietra, V.J.D., Souza, P.V.D., Lai, J.C., Mercer, R.L.: Class-Based n-gram Models of Natural Language. Computational Linguistics, 467–479 (1992)Google Scholar
- 6.Kuhn, R., De Mori, R.: A Cache-Based Natural Language Model for Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 570–583 (June 1990)Google Scholar
- 10.Bacchiani, M., Roark, B.: Unsupervised language model adaptation. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 224–227 (2003)Google Scholar
- 11.Psutka, J., Švec, J., Psutka, J.V., Vaněk, J., Pražák, A., Šmídl, L., Ircing, P.: System for Fast Lexical and Phonetic Spoken Term Detection in a Czech Cultural Heritage Archive. EURASIP Journal on Audio, Speech, and Music Processing (2011)Google Scholar