Overview of the CLEF-2006 Cross-Language Speech Retrieval Track

  • Douglas W. Oard
  • Jianqiang Wang
  • Gareth J. F. Jones
  • Ryen W. White
  • Pavel Pecina
  • Dagobert Soergel
  • Xiaoli Huang
  • Izhak Shafran
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4730)

Abstract

The CLEF-2006 Cross-Language Speech Retrieval (CL-SR) track included two tasks: to identify topically coherent segments of English interviews in a known-boundary condition, and to identify time stamps marking the beginning of topically relevant passages in Czech interviews in an unknown-boundary condition. Five teams participated in the English evaluation, performing both monolingual and cross-language searches of speech recognition transcripts, automatically generated metadata, and manually generated metadata. Results indicate that the 2006 English evaluation topics are more challenging than those used in 2005, but that cross-language searching continued to pose no unusual challenges when compared with monolingual searches of the same collection. Three teams participated in the monolingual Czech evaluation using a new evaluation measure based on differences between system-suggested and ground truth replay start times, with results that were broadly comparable to those observed for English.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    White, R.W., Oard, D.W., Jones, G.J.F., Soergel, D., Huang, X.: Overview of the clef-2005 cross-language speech retrieval track. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    López-Ostenero, F., Peinado, V., Sama, V., Verdejo, F.: UNED@CL-SR CLEF 2005: Mixing different strategies to retrieve automatic speech transcriptions. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Shafran, I., Byrne, W.: Task-specific minimum bayes-risk decoding using learned edit distance. In: Proceedings of INTERSPEECH 2004-ICSLP (2004)Google Scholar
  4. 4.
    Shafran, I., Hall, K.: Corrective models for speech recognition of inflected languages. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (2006)Google Scholar
  5. 5.
    Olsson, J.S., Oard, D.W., Hajic, J.: Cross-language text classification. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM Press, New York (2005)Google Scholar
  6. 6.
    Murray, G.C., Dorr, B.J., Lin, J., Hajic, J., Pecina, P.: Leveraging reusability: Cost-effective lexical acquisition for large-scale ontology translation. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (2006)Google Scholar
  7. 7.
    Liu, B., Oard, D.W.: One-sided measures for evaluating ranked retrieval effectiveness with spontaneous conversational speech. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM Press, New York (2006)Google Scholar
  8. 8.
    Kekalainen, J., Jarvelin, K.: Using graded relevance assessments in ir evaluation. Journal of the American Society for Information Science and Technology (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Douglas W. Oard
    • 1
  • Jianqiang Wang
    • 2
  • Gareth J. F. Jones
    • 3
  • Ryen W. White
    • 4
  • Pavel Pecina
    • 5
  • Dagobert Soergel
    • 6
  • Xiaoli Huang
    • 6
  • Izhak Shafran
    • 7
  1. 1.College of Information Studies and, Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742USA
  2. 2.Department of Library and Information Studies, State University of New York at Buffalo, Buffalo, NY 14260USA
  3. 3.School of Computing, Dublin City University, Dublin 9Ireland
  4. 4.Microsoft Research, One Microsoft Way, Redmond, WA 98052USA
  5. 5.MFF UK, Malostranske namesti 25, Room 422, Charles University, 118 00 Praha 1Czech Republic
  6. 6.College of Information Studies, University of Maryland, College Park, MD 20742USA
  7. 7.OGI School of Science & Engineering, Oregon Health and Sciences University, 20000 NW Walker Rd, Portland, OR 97006USA

Personalised recommendations