Abstract
We present in this paper the three LIMSI question-answering systems on speech transcripts which participated to the QAst 2009 evaluation. These systems are based on a complete and multi-level analysis of both queries and documents. These systems use an automatically generated research descriptor. A score based on those descriptors is used to select documents and snippets. Three different methods are tried to extract and score candidate answers, and we present in particular a tree transformation based ranking method. We participated to all the tasks and submitted 30 runs (for 24 sub-tasks). The evaluation results for manual transcripts range from 27% to 36% for accuracy depending on the task and from 20% to 29% for automatic transcripts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bernard, G., Rosset, S., Adda-Decker, M., Galibert, O.: A question–answer distance measure to investigate QA system progress. In: LREC 2010 (2010)
Bernard, G., Rosset, S., Galibert, O., Bilinski, E., Adda, G.: The limsi participation to the qast 2009 track. In: Working Notes for the CLEF 2009 Workshop, Corfu, Greece (October 2009)
Déchelotte, D., Schwenk, H., Adda, G., Gauvain, J.-L.: Improved machine translation of speech-to-text outputs. In: Interspeech 2007, Antwerp, Belgium (2007)
Paroubek, P., Vilnat, A., Grau, B., Ayache, C.: Easy, evaluation of parsers of french: what are the results. In: European Language Resources Association (ELRA) (ed.) Proceedings of the Sixth International Language Resources and Evaluation (LREC’08), pp. 2480–2486, Marrakech, Morocco (2008)
Pradhan, S., Ward, W., Hacioglu, K., Martin, J.H., Jurafski, D.: Semantic role labeling using different syntactic views. In: Proceedings of the 43rd Annual Meeting of the ACL, Ann Arbor, USA, pp. 581–588 (2005)
Rosset, S., Galibert, O., Bernard, G., Bilinski, E., Adda, G.: The limsi participation to the qast track. In: Peters, C., Deselaers, T., Ferro, N., Gonzalo, J., Jones, G.J.F., Kurimo, M., Mandl, T., Peñas, A., Petras, V. (eds.) CLEF 2008. LNCS, vol. 5706, pp. 480–487. Springer, Heidelberg (2009)
Sakai, T., Saito, Y., Ichimura, Y., Koyama, M., Kokubu, T., Manabe, T.: Askmi: A japanese question answering system based on semantic role analysis. In: Proceedings of RIAO 2004, Avignon (2004)
Turmo, J., Comas, P., Rosset, S., Galibert, O., Moreau, N., Mostefa, D., Rosso, P., Buscaldi, D.: Overview of qast 2009 - question answering on speech transcriptions. In: CLEF 2009 Workshop, Greece, Corfu (2009) (to appear)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bernard, G., Rosset, S., Galibert, O., Adda, G., Bilinski, E. (2010). The LIMSI Participation in the QAst 2009 Track: Experimenting on Answer Scoring. In: Peters, C., et al. Multilingual Information Access Evaluation I. Text Retrieval Experiments. CLEF 2009. Lecture Notes in Computer Science, vol 6241. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15754-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-15754-7_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15753-0
Online ISBN: 978-3-642-15754-7
eBook Packages: Computer ScienceComputer Science (R0)