CLEF 2013: Information Access Evaluation. Multilinguality, Multimodality, and Visualization pp 192-211 | Cite as
Cultural Heritage in CLEF (CHiC) 2013
Abstract
The Cultural Heritage in CLEF 2013 lab comprised three tasks: multilingual ad-hoc retrieval and semantic enrichment in 13 languages (Dutch, English, German, Greek, Finnish, French, Hungarian, Italian, Norwegian, Polish, Slovenian, Spanish, and Swedish), Polish ad-hoc retrieval and the interactive task, which studied user behavior via log analysis and questionnaires. For the multilingual and Polish sub-tasks, more than 170,000 documents were assessed for relevance on a tertiary scale. The multilingual task had 7 participants submitting 30 multilingual and 41 monolingual runs. The Polish task comprised 3 participating groups submitting manual and automatic runs. The interactive task had 4 participating research groups and 208 user participants in the study. For the multilingual task, results show that more participants are necessary in order to provide comparative analyses. The interactive task created a rich data set comprising of questionnaire of log data. Further analysis of the data is planned in the future.
Keywords
cultural heritage Europeana ad-hoc retrieval semantic enrichment multilingual retrieval Polish interactive user behaviorPreview
Unable to display preview. Download preview PDF.
References
- 1.Agosti, M., Ferro, N.: Towards an Evaluation Infrastructure for DL Performance Evaluation. In: Tsakonas, G., Papatheodorou, C. (eds.) Evaluation of Digital Libraries: An Insight to Useful Applications and Methods, pp. 93–120. Chandos Publishing, Oxford (2009)Google Scholar
- 2.Akasereh, M., Naji, N., Savoy, J.: UniNE at CLEF – CHIC 2013. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 3.Amati, G., van Rijsbergen, C.J.: Probabilistic Models of Information Retrieval Based on Measuring the Divergence from Randomness. ACM Transactions on Information Systems 20, 357–389 (2002)CrossRefGoogle Scholar
- 4.Dolamic, L., Savoy, J.: Indexing and Stemming Approaches for the Czech Language. Information Processing & Management 45, 714–720 (2009)CrossRefGoogle Scholar
- 5.Fernando, S., Hall, M.M., Agirre, E., Soroa, A., Clough, P., Stevenson, M.: Comparing taxonomies for organising collections of documents. In: Proceedings of COLING 2012: Technical Papers, pp. 879–894 (2012)Google Scholar
- 6.Hall, M.M., Toms, E.: Building a common framework for IIR evaluation. In: CLEF 2013. LNCS, vol. 8138, pp. 17–28. Springer, Heidelberg (2013)Google Scholar
- 7.Hall, M., Villa, R., Rutter, S., Bell, D., Clough, P., Toms, E.: Sheffield Submission to the CHiC Interactive Task: Exploring Digital Cultural Heritage. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 8.International Council of Museums, Scope Definition of the CIDOC Conceptual Reference Model (2003), http://www.cidoc-crm.org/scope.html
- 9.Larson, R.: Pseudo-Relevance Feedback for CLEF-CHiC Adhoc. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 10.Malak, P.: The Polish Task within Cultural Heritage in CLEF (CHiC) 2013. Torun runs. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 11.Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008)CrossRefMATHGoogle Scholar
- 12.Pawlowski, A.: Polish Monolingual Task within Cultural Heritage in CLEF (CHiC) 2013. Wroclaw Runs. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 13.Petras, V., Ferro, N., Gäde, M., Isaac, A., Kleineberg, M., Masiero, I., Nicchio, M., Stiller, J.: Cultural Heritage in CLEF (CHiC) Overview 2012. In: Proceedings CLEF-2012, Working Paper (2012)Google Scholar
- 14.Petras, V., Bogers, T., Ferro, N., Masiero, I.: CHiC Multilingual Task Overview and Analysis. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 15.Popescu, A.: CEA LIST’s participation at the CLEF CHiC 2013. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 16.Robertson, S.E., Walker, S., Beaulieu, M.: Experimentation as a Way of Life: Okapi at TREC. Information Processing & Management 36, 95–108 (2000)CrossRefGoogle Scholar
- 17.Skov, M., Bogers, T., Lund, H., Jensen, M., Wistrup, E., Larsen, B.: RSLIS/AAU at CHiC 2013. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 18.Stiller, J., Gäde, M., Petras, V.: Ambiguity of Queries and the Challenges for Query Language Detection. In: CLEF 2010 LABs and Workshops (2010), http://clef2010.org/resources/proceedings/clef2010labs_submission_41.pdf (retrieved)
- 19.Swan Oscar, E.: Polish Grammar in a Nutshell, http://polish.slavic.pitt.edu/firstyear/nutshell.pdf
- 20.Tan, K., Almasri, M., Chevallet, J., Mulhem, P., Berrut, C.: Multimedia Information Modeling and Retrieval(MRIM)/Laboratoire d’Informatique de Grenoble (LIG) at CHiC 2013. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 21.Tanase, D.: Using the Divergence Framework for Randomness: CHiC 2013 Lab Report. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar
- 22.Wilhelm-Stein, T., Schürer, B., Eibl, M.: Identifying the most suitable stemmer for the CHiC multilingual ad-hoc task. In: Proceedings CLEF 2013, Working Notes (2013)Google Scholar