Abstract
This paper reports on an analysis of the CLEF 2001 topics. In particular, we investigated potential correlations between features of the topics and the performance of retrieval systems. Although there are some weak relations, we claim that the properties of the CLEF topics do not influence the results of the retrieval systems. We found just one correlation for the English topics. The more linguistic challenges contained in the topic texts, the better the systems performed. However, no correlation for the length of a topic could be found.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Voorhees, E.: Variations in relevance judgments and the measurement of retrieval effectiveness. In: Proc of the 21st Annual Intl ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) ACM Press, New York, NY, USA (1998) 223–315.
Sparck Jones, Karen: Reflections on TREC. Information Processing & Management 31(1995) 291–314
Kluck, M., Womser-Hacker, C: Inside the Evaluation Process of the Cross-Language Evaluation Forum (CLEF): Issues of Multilingual Topic Creation and Multilingual Relevance Assessment. In: 3rd International Conference on Language Resources and Evaluation, Las Palmas, Spain (2002) 573-576
Voorhees, E., Buckley, C.: The effect of topic set size on retrieval experiment error. In: Proc of the 25th Annual Intl ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) ACM Press, New York, NY, USA (2002) 316-323
Womser-Hacker, C.: Das MIMOR-Modell. Mehrfachindexierung zur dynamischen Methoden-Objekt-Relationierung im Information Retrieval. Habilitationsschrift. Universität Regensburg, Informationswissenschaft (1997)
Hawking, D., Craswell, N.: Overview of the TREC-2001 Web Track. In: Voorhees E., Harman, D. (eds.): The Ninth Text Retrieval Conference (TREC-9). NIST Special Publication. Gaithersburg, Maryland (2001) http://trec.nist.gov/pubs/
Kwok, K. L., Chan, M.: Improving Two-Stage Ad-Hoc Retrieval for Short Queries. In: Proc of the 21st Annual Intl ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) ACM Press, New York, NY, USA (1998) 250–256
Wilkinson, R., Zobel, J., Sacks-Davis, R.: Similarity Measures for Short Queries. In: Harman, D. (ed.) The Fourth Text REtrieval Conference (TREC-4). NIST Special Publication. Gaithersburg, Maryland (1995) http://trec.nist.gov/pubs/
Rorvig, M., Fitzpatri, S.: Visualization and Scaling of TREC Topic Document Sets. Information Processing and Management 34 (1998) 135–149
Womser-Hacker, C.: Multilingual Topic Generation within the CLEF 2001 Experiments. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds.): Evaluation of Cross-Language Information Retrieval Systems. Workshop of the Cross-Language Information Evaluation Forum (CLEF) Lecture Notes in Computer Science, Vol. 2406 Springer-Verlag, Berlin Heidelberg New York (2002) 389–393
Smeaton, A., Over, P., Taban, R.: The TREC-2001 Video Track Report. In: Voorhees E, Harman, D. (eds.): The Tenth Text Retrieval Conference (TREC-10). NIST Special Publication. Gaithersburg, Maryland (2002) http://trec.nist.gov/pubs/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Mandl, T., Womser-Hacker, C. (2003). Linguistic and Statistical Analysis of the CLEF Topics. In: Peters, C., Braschler, M., Gonzalo, J., Kluck, M. (eds) Advances in Cross-Language Information Retrieval. CLEF 2002. Lecture Notes in Computer Science, vol 2785. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45237-9_43
Download citation
DOI: https://doi.org/10.1007/978-3-540-45237-9_43
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40830-7
Online ISBN: 978-3-540-45237-9
eBook Packages: Springer Book Archive