Experiments with Cross-Language Speech Retrieval for Lower-Resource Languages

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12004)


Cross-language speech retrieval systems face a cascade of errors due to transcription and translation ambiguity. Using 1-best speech recognition and 1-best translation in such a scenario could adversely affect recall if those 1-best system guesses are not correct. Accurately representing transcription and translation probabilities could therefore improve recall, although possibly at some cost in precision. The difficulty of the task is exacerbated when working with languages for which limited resources are available, since both recognition and translation probabilities may be less accurate in such cases. This paper explores the combination of expected term counts from recognition with expected term counts from translation to perform cross-language speech retrieval in which the queries are in English and the spoken content to be retrieved is in Tagalog or Swahili. Experiments were conducted using two query types, one focused on term presence and the other focused on topical retrieval. Overall, the results show that significant improvements in ranking quality result from modeling transcription and recognition ambiguity, even in lower-resource settings, and that adapting the ranking model to specific query types can yield further improvements.


  1. 1.
    Can, D., Saraclar, M.: Lattice indexing for spoken term detection. IEEE Trans. Audio Speech Lang. Process. 19(8), 2338–2347 (2011)CrossRefGoogle Scholar
  2. 2.
    Chelba, C., et al.: Retrieval and browsing of spoken content. IEEE Signal Process. Mag. 25(3), 39–49 (2008)CrossRefGoogle Scholar
  3. 3.
    Chen, G., et al.: Using proxies for OOV keywords in the keyword search task. In: ASRU, pp. 416–421 (2013)Google Scholar
  4. 4.
    Darwish, K., Oard, D.: Probabilistic structured query methods. In: SIGIR, pp. 338–344 (2003)Google Scholar
  5. 5.
    Fiscus, J., Doddington, G.: Topic detection and tracking evaluation overview. In: Allan, J. (ed.) Topic Detection and Tracking. The Information Retrieval Series, vol. 12, pp. 17–31. Springer, Boston (2002)CrossRefGoogle Scholar
  6. 6.
    Hull, D.: Using structured queries for disambiguation in cross-language information retrieval. In: AAAI Symposium on Cross-Language Text and Speech Retrieval (1997)Google Scholar
  7. 7.
    Karakos, D., et al.: Score normalization and system combination for improved keyword spotting. In: ASRU, pp. 210–215 (2013)Google Scholar
  8. 8.
    Kim, S., et al.: Combining lexical and statistical translation evidence for cross-language information retrieval. JASIST 66(1), 23–39 (2015)Google Scholar
  9. 9.
    Lee, L.S., Chen, B.: Spoken document understanding and organization. IEEE Signal Process. Mag. 22(5), 42–60 (2005)CrossRefGoogle Scholar
  10. 10.
    Lee, L.S., Pan, Y.C.: Voice-based information retrieval—how far are we from the text-based information retrieval? In: ASRU, pp. 26–43 (2009)Google Scholar
  11. 11.
    Makhoul, J., et al.: Speech and language technologies for audio indexing and retrieval. Proc. IEEE 88(8), 1338–1353 (2000)CrossRefGoogle Scholar
  12. 12.
    Mamou, J., et al.: Developing keyword search under the IARPA Babel program. In: Afeka Speech Processing Conference (2013)Google Scholar
  13. 13.
    McNamee, P., Mayfield, J.: Comparing cross-language query expansion techniques by degrading translation resources. In: SIGIR, pp. 159–166 (2002)Google Scholar
  14. 14.
    Oard, D.W., et al.: Overview of the CLEF-2006 cross-language speech retrieval track. In: Peters, C., et al. (eds.) CLEF 2006. LNCS, vol. 4730, pp. 744–758. Springer, Heidelberg (2007). Scholar
  15. 15.
    Och, F., Ney, H.: A systematic comparison of various statistical alignment models. Comput. Linguist. 29(1), 19–51 (2003)CrossRefGoogle Scholar
  16. 16.
    Pecina, P., Hoffmannová, P., Jones, G.J.F., Zhang, Y., Oard, D.W.: Overview of the CLEF-2007 cross-language speech retrieval track. In: Peters, C., et al. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 674–686. Springer, Heidelberg (2008). Scholar
  17. 17.
    Pirkola, A.: The effects of query structure and dictionary setups in dictionary-based cross-language information retrieval. In: SIGIR, pp. 55–63 (1998)Google Scholar
  18. 18.
    Ragni, A., Gales, M.: Automatic speech recognition system development in the ‘wild’. In: ICSA, pp. 2217–2221 (2018)Google Scholar
  19. 19.
    Riedhammer, K., et al.: A study on LVCSR and keyword search for tagalog. In: INTERSPEECH, pp. 2529–2533 (2013)Google Scholar
  20. 20.
    Robertson, S.: Okapi at TREC-7: automatic ad hoc, filtering, VLC and interactive track. In: TREC (1998)Google Scholar
  21. 21.
  22. 22.
    Saraclar, M., Sproat, R.: Lattice-based search for spoken utterance retrieval. In: NAACL (2004)Google Scholar
  23. 23.
    Strohman, T., et al.: Indri: a language model-based search engine for complex queries. In: International Conference on Intelligence Analysis (2005)Google Scholar
  24. 24.
    Tur, G., De Mori, R.: Spoken Language Understanding: Systems for Extracting Semantic Information from Speech. Wiley, New York (2011)CrossRefGoogle Scholar
  25. 25.
    Wang, J., Oard, D.: Matching meaning for cross-language information retrieval. Inf. Process. Manag. 48(4), 631–653 (2012)CrossRefGoogle Scholar
  26. 26.
    Wegmann, S., et al.: The TAO of ATWV: probing the mysteries of keyword search performance. In: ASRU, pp. 192–197 (2013)Google Scholar
  27. 27.
    Weintraub, M.: Keyword-spotting using SRI’s DECIPHER large-vocabulary speech-recognition system. In: ICASSP, vol. 2, pp. 463–466 (1993)Google Scholar
  28. 28.
    White, R.W., Oard, D.W., Jones, G.J.F., Soergel, D., Huang, X.: Overview of the CLEF-2005 cross-language speech retrieval track. In: Peters, C., et al. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 744–759. Springer, Heidelberg (2006). Scholar
  29. 29.
    Xu, J., Weischedel, R.: Cross-lingual information retrieval using hidden Markov models. In: EMNLP, pp. 95–103 (2000)Google Scholar
  30. 30.
    Zbib, R., et al.: Neural-network lexical translation for cross-lingual IR from text and speech. In: SIGIR (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.University of MarylandCollege ParkUSA
  2. 2.University of CambridgeCambridgeUK
  3. 3.University of EdinburghEdinburghUK

Personalised recommendations