Advertisement

Overview of QAST 2008

  • Jordi Turmo
  • Pere R. Comas
  • Sophie Rosset
  • Lori Lamel
  • Nicolas Moreau
  • Djamel Mostefa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5706)

Abstract

This paper describes the experience of QAST 2008, the second time a pilot track of CLEF has been held aiming to evaluate the task of Question Answering in Speech Transcripts. Five sites submitted results for at least one of the five scenarios (lectures in English, meetings in English, broadcast news in French and European Parliament debates in English and Spanish). In order to assess the impact of potential errors of automatic speech recognition, for each task contrastive conditions are with manual and automatically produced transcripts. The QAST 2008 evaluation framework is described, along with descriptions of the five scenarios and their associated data, the system submissions for this pilot track and the official evaluation results.

Keywords

Question answering Spontaneous speech transcripts 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Galliano, S., Geoffrois, E., Gravier, G., Bonastre, J.F., Mostefa, D., Choukri, K.: Corpus description of the ESTER Evaluation Campaign for the Rich Transcription of French Broadcast News. In: Proceedings of LREC 2006, Genoa, pp. 315–320 (2006)Google Scholar
  2. 2.
    Gravier, G., Bonastre, J.F., Galliano, S., Geoffrois, E., McTait, K., Choukri, K.: The ESTER evaluation campaign of Rich Transcription of French Broadcast News. In: Proceedings of LREC 2004, Lisbon, pp. 885–888 (2004)Google Scholar
  3. 3.
    Peters, C., Clough, P., Gey, F.C., Karlgren, J., Magnini, B., Oard, D.W., de Rijke, M., Stempfhuber, M. (eds.): CLEF 2006. LNCS, vol. 4730. Springer, Heidelberg (2007)Google Scholar
  4. 4.
    TC-Star (2004-2008), http://www.tc-star.org
  5. 5.
    Turmo, J., Comas, P.R., Ayache, C., Mostefa, D., Rosset, S., Lamel, L.: Overview of QAST 2007. In: Peters, C., Jijkoun, V., Mandl, T., Müller, H., Oard, D.W., Peñas, A., Petras, V., Santos, D. (eds.) CLEF 2007. LNCS, vol. 5152, pp. 249–256. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Voorhees, E.M., Buckland, L.L. (eds.): The Fifteenth Text Retrieval Conference Proceedings, TREC 2006 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jordi Turmo
    • 1
  • Pere R. Comas
    • 1
  • Sophie Rosset
    • 2
  • Lori Lamel
    • 2
  • Nicolas Moreau
    • 3
  • Djamel Mostefa
    • 3
  1. 1.TALP Research CenterTechnical University of Catalonia (UPC)Spain
  2. 2.LIMSIParisFrance
  3. 3.ELDA/ELRAParisFrance

Personalised recommendations