Abstract
Recognition of Hungarian conversational telephone speech is challenging due to the informal style and morphological richness of the language. Recurrent Neural Network Language Model (RNNLM) can provide remedy for the high perplexity of the task; however, two-pass decoding introduces a considerable processing delay. In order to eliminate this delay we investigate approaches aiming at the complexity reduction of RNNLM, while preserving its accuracy. We compare the performance of conventional back-off n-gram language models (BNLM), BNLM approximation of RNNLMs (RNN-BNLM) and RNN n-grams in terms of perplexity and word error rate (WER). Morphological richness is often addressed by using statistically derived subwords - morphs - in the language models, hence our investigations are extended to morph-based models, as well. We found that using RNN-BNLMs 40% of the RNNLM perplexity reduction can be recovered, which is roughly equal to the performance of a RNN 4-gram model. Combining morph-based modeling and approximation of RNNLM, we were able to achieve 8% relative WER reduction and preserve real-time operation of our conversational telephone speech recognition system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adel, H., Kirchhoff, K., Vu, N.T., Telaar, D., Schultz, T.: Comparing approaches to convert recurrent neural networks into backoff language models for efficient decoding. Interspeech 2014, 651–655 (2014)
Arisoy, E., Chen, S.F., Ramabhadran, B., Sethy, A.: Converting neural network language models into back-off language models for efficient decoding in automatic speech recognition. IEEE Trans. Audio, Speech Lang. Process. 22(1), 184–192 (2014)
Chelba, C., Norouzi, M., Bengio, S.: N-gram Language Modeling using Recurrent Neural Network Estimation. CoRR 1703.10724 (Mar 2017)
Chen, S.F., Goodman, J.: An empirical study of smoothing techniques for language modeling. Comput. Speech Lang. 13(4), 359–393 (1999)
Creutz, M., Lagus, K.: Unsupervised discovery of morphemes. In: Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning. vol. 6, pp. 21–30. Association for Computational Linguistics, Morristown, NJ, USA (2002)
Deoras, A., Mikolov, T., Kombrink, S., Karafiat, M., Khudanpur, S.: Variational approximation of long-span language models for lVCSR. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5532–5535. IEEE (may 2011)
Enarvi, S., Smit, P., Virpioja, S., Kurimo, M.: Automatic speech recognition with very large conversational finnish and estonian vocabularies. IEEE/ACM Trans. Audio Speech Lang. Process. 25(11), 2085–2097 (2017)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Kurimo, M., et al.: Unlimited vocabulary speech recognition for agglutinative languages. In: Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, pp. 487–494. Association for Computational Linguistics, Morristown, NJ, USA (2007)
Mihajlik, P., Tüske, Z., Tarján, B., Németh, B., Fegyó, T.: Improved recognition of spontaneous hungarian speech-morphological and acoustic modeling techniques for a less resourced task. IEEE Trans. Audio, Speech Lang. Process. 18(6), 1588–1600 (2010)
Mikolov, T., Karafiat, M., Burget, L., Cernocky, J., Khudanpur, S.: Recurrent neural network based language model. Interspeech 2010, 1045–1048 (2010)
Povey, D., et al.: The Kaldi speech recognition toolkit. In: IEEE 2011 Workshop on Automatic Speech Recognition and Understanding. IEEE Signal Processing Society (2011)
Singh, M., Smit, P., Virpioja, S., Kurimo, M.: First-pass decoding with n-gram approximation of RNNLM : The problem of rare words (2018), working paper
Smit, P., Virpioja, S., Kurimo, M.: Improved subword modeling for WFST-based speech recognition. In: Interspeech 2017. pp. 2551–2555. ISCA, ISCA (Aug 2017)
Stolcke, A.: SRILM - an extensible language modeling toolkit. In: Proceedings International Conference on Spoken Language Processing, pp. 901–904. Denver, US (2002)
Stolcke, A.: Entropy-based pruning of backoff language models. In: Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop. pp. 270–274 (2000)
Sundermeyer, M., Schlueter, R., Ney, H.: LSTM neural networks for language modeling. Interspeech 2012, 194–197 (2012)
Tarján, B., Mihajlik, P., Balog, A., Fegyó, T.: Evaluation of lexical models for Hungarian Broadcast speech transcription and spoken term detection. In: 2nd International Conference on Cognitive Infocommunications (CogInfoCom), pp. 1–5. Budapest, Hungary (2011)
Tarján, B., Sarosi, G., Fegyo, T., Mihajlik, P.: Improved recognition of Hungarian call center conversations. In: 2013 7th Conference on Speech Technology and Human - Computer Dialogue. SpeD 2013, pp. 1–6. IEEE, Cluj-Napoca, Romania (Oct 2013)
Tüske, Z., Schlüter, R., Ney, H.: Investigation on LSTM recurrent N-gram language models for speech recognition. In: Interspeech 2018, pp. 3358–3362. ISCA, ISCA (Sep 2018)
Virpioja, S., Smit, P., Grönroos, S.A., Kurimo, M.: Morfessor 2.0: Python Implementation and Extensions for Morfessor Baseline. Technical Report September, Aalto University (2013)
Zaremba, W., Sutskever, I., Vinyals, O.: Recurrent Neural Network Regularization. CoRR 1409, 2329 (2014)
Acknowledgments
The research was partly supported by the DANSPLAT (EUREKA_15_1_2016-0019) project.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tarján, B., Szaszák, G., Fegyó, T., Mihajlik, P. (2019). Investigation on N-Gram Approximated RNNLMs for Recognition of Morphologically Rich Speech. In: Martín-Vide, C., Purver, M., Pollak, S. (eds) Statistical Language and Speech Processing. SLSP 2019. Lecture Notes in Computer Science(), vol 11816. Springer, Cham. https://doi.org/10.1007/978-3-030-31372-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-31372-2_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-31371-5
Online ISBN: 978-3-030-31372-2
eBook Packages: Computer ScienceComputer Science (R0)