Abstract
In this paper we describe the RWTH Aachen keyword search (KWS) system developed in the course of the IARPA Babel program. We put focus on acoustic modeling with neural networks and evaluate the full pipeline with respect to the KWS performance. At the core of this study lie multilingual bottleneck features extracted from a deep neural network trained on all 28 languages available to the project articipants. We show that in a low-resource scenario, the multilingual features are crucial for achieving state-of-the-art performance.
Further highlights of this work include comparisons of tandem and hybrid acoustic models based on feed-forward and recurrent neural networks, keyword search pipelines based on lattice and time-marked word list representation and measuring the effect of adding large amounts of text data scraped from the web. The evaluation is performed on multiple languages of the last two project periods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
BP: base period, OP: option period.
- 2.
see Table 1 in [1] for details on the four additional corpora.
References
Alumäe, T., Karakos, D., Hartmann, W., Hsiao, R., Zhang, L., Nguyen, L., Tsakalidis, S., Schwartz, R.: The 2016 BBN Georgian telephone speech keyword spotting system. In: ICASSP, pp. 5755–5759 (2017)
Babel: US IARPA Project (2012–2016). http://www.iarpa.gov/Programs/ia/Babel/babel.html
Beulen, K., Ney, H.: Automatic question generation for decision tree based state tying. In: ICASSP, pp. 805–808 (1998)
Doetsch, P., Zeyer, A., Voigtlaender, P., Kulikov, I., Schlüter, R., Ney, H.: RETURNN: the RWTH extensible training framework for universal recurrent neural networks. In: ICASSP, pp. 5345–5349 (2017)
Fiscus, J.G., Ajot, J., Garofolo, J.S., Doddingtion, G.: Results of the 2006 spoken term detection evaluation. In: Proceedings of ACM SIGIR Workshop on Searching Spontaneous Conversational Speech, pp. 51–57 (2007)
Golik, P., Tüske, Z., Schlüter, R., Ney, H.: Multilingual features based keyword search for very low-resource languages. In: Interspeech, pp. 1260–1264 (2015)
Grézl, F., Karafiát, M., Kontár, S., Černocký, J.: Probabilistic and bottle-neck features for LVCSR of meetings. In: ICASSP, pp. 757–760 (2007)
Hermansky, H., Ellis, D., Sharma, S.: Tandem connectionist feature extraction for conventional HMM systems. ICASSP, vol. 3, pp. 1635–1638 (2000)
Hermansky, H., Fousek, P.: Multi-resolution RASTA filtering for TANDEM-based ASR. In: Interspeech, pp. 361–364 (2005)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Irie, K., Golik, P., Schlüter, R., Ney, H.: Investigations on byte-level convolutional neural networks for language modeling in low resource speech recognition. In: ICASSP, pp. 5740–5744 (2017)
Kanthak, S., Ney, H.: Context-dependent acoustic modeling using graphemes for large vocabulary speech recognition. In: ICASSP, pp. 845–848 (2002)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Knill, K.M., Gales, M.J.F., Rath, S.P., Woodland, P.C., Zhang, C., Zhang, S.X.: Investigation of multilingual deep neural networks for spoken term detection. In: ASRU, pp. 138–143 (2013)
Mamou, J., Cui, J., Cui, X., Gales, M., Kingsbury, B., Knill, K., Mangu, L., Nolden, D., Picheny, M., Ramabhadran, B., Schlüter, R., Sethy, A., Woodland, P.: System combination and score normalization for spoken term detection. In: ICASSP, pp. 8272–8276 (2013)
Mangu, L., Saon, G., Picheny, M., Kingsbury, B.: Order-free spoken term detection. In: ICASSP, pp. 5331–5335 (2015)
Mangu, L., Soltau, H., Kuo, H.K., Kingsbury, B., Saon, G.: Exploiting diversity for spoken term detection. In: ICASSP, pp. 8282–8286 (2013)
Mendels, G., Cooper, E., Hirschberg, J.: Babler - data collection from the web to support speech recognition and keyword search. In: ACL, pp. 72–81 (2016)
Povey, D., Woodland, P.: Minimum phone error and I-smoothing for improved discriminative training. In: ICASSP, pp. 105–108 (2002)
Rybach, D., Hahn, S., Lehnen, P., Nolden, D., Sundermeyer, M., Tüske, Z., Wiesler, S., Schlüter, R., Ney, H.: RASR - the RWTH Aachen university open source speech recognition toolkit. In: ASRU (2011)
Saraclar, M., Sethy, A., Ramabhadran, B., Mangu, L., Cui, J., Cui, X., Kingsbury, B., Mamou, J.: An empirical study of confusion modeling in keyword search for low resource languages. In: ASRU, pp. 464–469 (2013)
Scanzio, S., Laface, P., Fissore, L., Gemello, R., Mana, F.: On the use of a multilingual neural network front-end. In: Interspeech, pp. 2711–2714 (2008)
Schlüter, R., Bezrukov, I., Wagner, H., Ney, H.: Gammatone features and feature combination for large vocabulary speech recognition. In: ICASSP, pp. 649–652 (2007)
Sundermeyer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. In: Interspeech, pp. 194–197 (2012)
Tüske, Z., Golik, P., Nolden, D., Schlüter, R., Ney, H.: Data augmentation, feature combination, and multilingual neural networks to improve ASR and KWS performance for low-resource languages. In: Interspeech, pp. 1420–1424 (2014)
Tüske, Z., Nolden, D., Schlüter, R., Ney, H.: Multilingual MRASTA features for low-resource keyword search and speech recognition systems. In: ICASSP (2014)
Zhang, L., Karakos, D., Hartmann, W., Hsiao, R., Schwartz, R., Tsakalidis, S.: Enhancing low resource keyword spotting with automatically retrieved web documents. In: Interspeech, pp. 839–843 (2015)
Zolnay, A., Schlüter, R., Ney, H.: Robust speech recognition using a voiced-unvoiced feature. In: ICSLP, vol. 2 (2002)
Acknowledgments
Supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Defense U.S. Army Research Laboratory (DoD/ARL) contract no. W911NF-12-C-0012. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoD/ARL, or the U.S. Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Golik, P., Tüske, Z., Irie, K., Beck, E., Schlüter, R., Ney, H. (2017). The 2016 RWTH Keyword Search System for Low-Resource Languages. In: Karpov, A., Potapova, R., Mporas, I. (eds) Speech and Computer. SPECOM 2017. Lecture Notes in Computer Science(), vol 10458. Springer, Cham. https://doi.org/10.1007/978-3-319-66429-3_72
Download citation
DOI: https://doi.org/10.1007/978-3-319-66429-3_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66428-6
Online ISBN: 978-3-319-66429-3
eBook Packages: Computer ScienceComputer Science (R0)