The 2016 RWTH Keyword Search System for Low-Resource Languages

  • Pavel GolikEmail author
  • Zoltán Tüske
  • Kazuki Irie
  • Eugen Beck
  • Ralf Schlüter
  • Hermann Ney
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10458)


In this paper we describe the RWTH Aachen keyword search (KWS) system developed in the course of the IARPA Babel program. We put focus on acoustic modeling with neural networks and evaluate the full pipeline with respect to the KWS performance. At the core of this study lie multilingual bottleneck features extracted from a deep neural network trained on all 28 languages available to the project articipants. We show that in a low-resource scenario, the multilingual features are crucial for achieving state-of-the-art performance.

Further highlights of this work include comparisons of tandem and hybrid acoustic models based on feed-forward and recurrent neural networks, keyword search pipelines based on lattice and time-marked word list representation and measuring the effect of adding large amounts of text data scraped from the web. The evaluation is performed on multiple languages of the last two project periods.


Acoustic modeling Keyword search Graphemic Multilingual Neural networks Time-marked word list Recurrent LSTM 



Supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of Defense U.S. Army Research Laboratory (DoD/ARL) contract no. W911NF-12-C-0012. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoD/ARL, or the U.S. Government.


  1. 1.
    Alumäe, T., Karakos, D., Hartmann, W., Hsiao, R., Zhang, L., Nguyen, L., Tsakalidis, S., Schwartz, R.: The 2016 BBN Georgian telephone speech keyword spotting system. In: ICASSP, pp. 5755–5759 (2017)Google Scholar
  2. 2.
    Babel: US IARPA Project (2012–2016).
  3. 3.
    Beulen, K., Ney, H.: Automatic question generation for decision tree based state tying. In: ICASSP, pp. 805–808 (1998)Google Scholar
  4. 4.
    Doetsch, P., Zeyer, A., Voigtlaender, P., Kulikov, I., Schlüter, R., Ney, H.: RETURNN: the RWTH extensible training framework for universal recurrent neural networks. In: ICASSP, pp. 5345–5349 (2017)Google Scholar
  5. 5.
    Fiscus, J.G., Ajot, J., Garofolo, J.S., Doddingtion, G.: Results of the 2006 spoken term detection evaluation. In: Proceedings of ACM SIGIR Workshop on Searching Spontaneous Conversational Speech, pp. 51–57 (2007)Google Scholar
  6. 6.
    Golik, P., Tüske, Z., Schlüter, R., Ney, H.: Multilingual features based keyword search for very low-resource languages. In: Interspeech, pp. 1260–1264 (2015)Google Scholar
  7. 7.
    Grézl, F., Karafiát, M., Kontár, S., Černocký, J.: Probabilistic and bottle-neck features for LVCSR of meetings. In: ICASSP, pp. 757–760 (2007)Google Scholar
  8. 8.
    Hermansky, H., Ellis, D., Sharma, S.: Tandem connectionist feature extraction for conventional HMM systems. ICASSP, vol. 3, pp. 1635–1638 (2000)Google Scholar
  9. 9.
    Hermansky, H., Fousek, P.: Multi-resolution RASTA filtering for TANDEM-based ASR. In: Interspeech, pp. 361–364 (2005)Google Scholar
  10. 10.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  11. 11.
    Irie, K., Golik, P., Schlüter, R., Ney, H.: Investigations on byte-level convolutional neural networks for language modeling in low resource speech recognition. In: ICASSP, pp. 5740–5744 (2017)Google Scholar
  12. 12.
    Kanthak, S., Ney, H.: Context-dependent acoustic modeling using graphemes for large vocabulary speech recognition. In: ICASSP, pp. 845–848 (2002)Google Scholar
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)Google Scholar
  14. 14.
    Knill, K.M., Gales, M.J.F., Rath, S.P., Woodland, P.C., Zhang, C., Zhang, S.X.: Investigation of multilingual deep neural networks for spoken term detection. In: ASRU, pp. 138–143 (2013)Google Scholar
  15. 15.
    Mamou, J., Cui, J., Cui, X., Gales, M., Kingsbury, B., Knill, K., Mangu, L., Nolden, D., Picheny, M., Ramabhadran, B., Schlüter, R., Sethy, A., Woodland, P.: System combination and score normalization for spoken term detection. In: ICASSP, pp. 8272–8276 (2013)Google Scholar
  16. 16.
    Mangu, L., Saon, G., Picheny, M., Kingsbury, B.: Order-free spoken term detection. In: ICASSP, pp. 5331–5335 (2015)Google Scholar
  17. 17.
    Mangu, L., Soltau, H., Kuo, H.K., Kingsbury, B., Saon, G.: Exploiting diversity for spoken term detection. In: ICASSP, pp. 8282–8286 (2013)Google Scholar
  18. 18.
    Mendels, G., Cooper, E., Hirschberg, J.: Babler - data collection from the web to support speech recognition and keyword search. In: ACL, pp. 72–81 (2016)Google Scholar
  19. 19.
    Povey, D., Woodland, P.: Minimum phone error and I-smoothing for improved discriminative training. In: ICASSP, pp. 105–108 (2002)Google Scholar
  20. 20.
    Rybach, D., Hahn, S., Lehnen, P., Nolden, D., Sundermeyer, M., Tüske, Z., Wiesler, S., Schlüter, R., Ney, H.: RASR - the RWTH Aachen university open source speech recognition toolkit. In: ASRU (2011)Google Scholar
  21. 21.
    Saraclar, M., Sethy, A., Ramabhadran, B., Mangu, L., Cui, J., Cui, X., Kingsbury, B., Mamou, J.: An empirical study of confusion modeling in keyword search for low resource languages. In: ASRU, pp. 464–469 (2013)Google Scholar
  22. 22.
    Scanzio, S., Laface, P., Fissore, L., Gemello, R., Mana, F.: On the use of a multilingual neural network front-end. In: Interspeech, pp. 2711–2714 (2008)Google Scholar
  23. 23.
    Schlüter, R., Bezrukov, I., Wagner, H., Ney, H.: Gammatone features and feature combination for large vocabulary speech recognition. In: ICASSP, pp. 649–652 (2007)Google Scholar
  24. 24.
    Sundermeyer, M., Schlüter, R., Ney, H.: LSTM neural networks for language modeling. In: Interspeech, pp. 194–197 (2012)Google Scholar
  25. 25.
    Tüske, Z., Golik, P., Nolden, D., Schlüter, R., Ney, H.: Data augmentation, feature combination, and multilingual neural networks to improve ASR and KWS performance for low-resource languages. In: Interspeech, pp. 1420–1424 (2014)Google Scholar
  26. 26.
    Tüske, Z., Nolden, D., Schlüter, R., Ney, H.: Multilingual MRASTA features for low-resource keyword search and speech recognition systems. In: ICASSP (2014)Google Scholar
  27. 27.
    Zhang, L., Karakos, D., Hartmann, W., Hsiao, R., Schwartz, R., Tsakalidis, S.: Enhancing low resource keyword spotting with automatically retrieved web documents. In: Interspeech, pp. 839–843 (2015)Google Scholar
  28. 28.
    Zolnay, A., Schlüter, R., Ney, H.: Robust speech recognition using a voiced-unvoiced feature. In: ICSLP, vol. 2 (2002)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Pavel Golik
    • 1
    Email author
  • Zoltán Tüske
    • 1
  • Kazuki Irie
    • 1
  • Eugen Beck
    • 1
  • Ralf Schlüter
    • 1
  • Hermann Ney
    • 1
  1. 1.Human Language Technology and Pattern Recognition, Computer Science DepartmentRWTH Aachen UniversityAachenGermany

Personalised recommendations