Assessing the Scholarly Impact of ImageCLEF
Systematic evaluation has an important place in information retrieval research starting with the Cranfield tests and currently with TREC (Text REtrieval Conference) and other evaluation campaigns. Such benchmarks are often mentioned to have an important impact in advancing a research field and making techniques comparable. Still, their exact impact is hard to measure. This paper aims at assessing the scholarly impact of the ImageCLEF image retrieval evaluation initiative. To this end, the papers in the proceedings published after each evaluation campaign and their citations are analysed using Scopus and Google Scholar. A significant impact of ImageCLEF could be shown through this bibliometric analysis. The differences between the employed analysis methods, each with its advantages and limitations, are also discussed.
KeywordsInformation Retrieval Image Retrieval Citation Analysis Citation Count Bibliometric Analysis
Unable to display preview. Download preview PDF.
- 2.Cleverdon, C.W.: The evaluation of systems used in information retrieval. In: Proceedings of the International Conference on Scientific Information, vol. 1, pp. 687–698. National Academy of Sciences, National Research Council (1959)Google Scholar
- 3.Clough, P., Müller, H., Deselaers, T., Grubinger, M., Lehmann, T.M., Jensen, J., Hersh, W.R.: The CLEF 2005 cross–language image retrieval track. In: Peters, C., Gey, F.C., Gonzalo, J., Müller, H., Jones, G.J.F., Kluck, M., Magnini, B., de Rijke, M., Giampiccolo, D. (eds.) CLEF 2005. LNCS, vol. 4022, pp. 535–557. Springer, Heidelberg (2006)CrossRefGoogle Scholar
- 4.A.-W. Harzing. Citation analysis across disciplines: The impact of different data sources and citation metrics (2010), http://www.harzing.com/data_metrics_comparison.htm (retrieved)
- 6.Proceedings of the 20th International Conference on Pattern Recognition (ICPR 2010). IEEE Computer Society, Instanbul (2010)Google Scholar
- 11.Rowe, B.R., Wood, D.W., Link, A.N., Simoni, D.A.: Economic impact assessment of NIST’s Text REtrieval Conference (TREC) Program. Technical Report Project Number 0211875, RTI International (2010)Google Scholar
- 14.Ünay, D., Çataltepe, Z., Aksoy, S. (eds.): Proceedings of the 20th International Conference on Recognizing Patterns in Signals, Speech, Images, and Videos, ICPR Contest Reports. Springer, Heidelberg (2010)Google Scholar