Overview of ImageCLEF 2017: Information Extraction from Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10456)


This paper presents an overview of the ImageCLEF 2017 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) labs 2017. ImageCLEF is an ongoing initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to collections of images in various usage scenarios and domains. In 2017, the 15th edition of ImageCLEF, three main tasks were proposed and one pilot task: (1) a LifeLog task about searching in LifeLog data, so videos, images and other sources; (2) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based on the figure alone; (3) a tuberculosis task that aims at detecting the tuberculosis type from CT (Computed Tomography) volumes of the lung and also the drug resistance of the tuberculosis; and (4) a remote sensing pilot task that aims at predicting population density based on satellite images. The strong participation of over 150 research groups registering for the four tasks and 27 groups submitting results shows the interest in this benchmarking campaign despite the fact that all four tasks were new and had to create their own community.



This research was supported in part by the Intramural Research Program of the National Institutes of Health (NIH), National Library of Medicine (NLM), and Lister Hill National Center for Biomedical Communications (LHNCBC). It is also partly supported European Union’s Horizon 2020 Research and Innovation programme under the Grant Agreement no693210 (FabSpace 2.0).


  1. 1.
    Arenas, H., Baker, A., Bialczak, A., Bargiel, D., Becker, M., Gaildrat, V., Carbone, F., Heising, S., Islam, M.B., Lattes, P., Marantos, C., Menou, C., Mothe, J., Nzeh Ngong, A., Paraskevas, I.S., Penalver, M., Sciana, P., Soudris, D.: FabSpaces at population estimation (remote) task - ImageCLEF at the CLEF 2017 Labs. CLEF working notes, CEUR (2017)Google Scholar
  2. 2.
    Arenas, H., Islam, B., Mothe, J.: Overview of the ImageCLEF 2017 population estimation task. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings,, Dublin, Ireland (2017).
  3. 3.
    Balikas, G., Krithara, A., Partalas, I., Paliouras, G.: BioASQ: a challenge on large-scale biomedical semantic indexing and question answering. In: Müller, H., Jimenez del Toro, O.A., Hanbury, A., Langs, G., Foncubierta Rodríguez, A. (eds.) Multimodal Retrieval in the Medical Domain. LNCS, vol. 9059, pp. 26–39. Springer, Cham (2015). doi: 10.1007/978-3-319-24471-6_3 CrossRefGoogle Scholar
  4. 4.
    Barnston, A.G.: Correspondence among the correlation, RMSE, and heidke forecast verification measures; refinement of the heidke score. Weather Forecast. 7(4), 699–709 (1992)CrossRefGoogle Scholar
  5. 5.
    Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 597–613. Springer, Heidelberg (2005). doi: 10.1007/11519645_59 CrossRefGoogle Scholar
  6. 6.
    Dang-Nguyen, D.T., Piras, L., Riegler, M., Boato, G., Zhou, L., Gurrin, C.: Overview of ImageCLEFlifelog 2017: lifelog retrieval and summarization. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings,, Dublin, Ireland (2017).
  7. 7.
    Dicente Cid, Y., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings,, Dublin, Ireland (2017).
  8. 8.
    Dogariu, M., Ionescu, B.: A textual filtering of hog-based hierarchical clustering of lifelog data, pp. 11–14. CLEF working notes, CEUR (2017)Google Scholar
  9. 9.
    Eickhoff, C., Schwall, I., Seco, G., de Herrera, A., Müller, H.: Overview of ImageCLEF caption 2017 - image caption prediction and concept detection for biomedical images. In: CLEF 2017 Labs Working Notes. CEUR Workshop Proceedings,, Dublin, Ireland (2017).
  10. 10.
    Goeuriot, L., Kelly, L., Li, W., Palotti, J., Pecina, P., Zuccon, G., Hanbury, A., Jones, G.J.F., Müller, H.: Share/CLEF ehealth evaluation lab 2014, task 3: User-centred health information retrieval. In: Working Notes of CLEF 2014, CEUR (2014)Google Scholar
  11. 11.
    Hanbury, A., Müller, H., Balog, K., Brodt, T., Cormack, G.V., Eggel, I., Gollub, T., Hopfgartner, F., Kalpathy-Cramer, J., Kando, N., Krithara, A., Lin, J., Mercer, S., Potthast, M.: Evaluation-as-a-service: overview and outlook. ArXiv 1512.07454 (2015)Google Scholar
  12. 12.
    Seco, G., de Herrera, A., Schaer, R., Bromuri, S., Müller, H.: Overview of the ImageCLEF 2016 medical task. In: Working Notes of CLEF 2016 (Cross Language Evaluation Forum), September 2016Google Scholar
  13. 13.
    Islam, M.B., Becker, M., Bargiel, D., Ahmed, K.R., Duzak, P., Emana, N.G.: Sentinel-2 satellite imagery based population estimation strategies at FabSpace 2.0 Lab Darmstadt (2017)Google Scholar
  14. 14.
    Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.P., Lombardo, J.C., Planqué, R., Palazzo, S., Müller, H.: Lifeclef 2017 lab overview: multimedia species identification challenges. In: Proceedings of CLEF 2017 (2017)Google Scholar
  15. 15.
    Kalpathy-Cramer, J., García Seco de Herrera, A., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems: overview of the medical image retrieval task at ImageCLEF 2004–2014. Computerized Medical Imaging and Graphics 39, 55–61 (2015)CrossRefGoogle Scholar
  16. 16.
    Koutsouri, K., Skepetari, I., Anastasakis, K., Lappas, S.: Population estimation using satellite imagery. In: CLEF working notes, CEUR., Dublin, Ireland (2017).
  17. 17.
    Molino, A.G.D., Mandal, B., Lin, J., Lim, J.H., Subbaraju, V., Chandrasekhar, V.: VC-I2R@ImageCLEF2017: ensemble of deep learned features for lifelog video summarization, pp. 11–14. CLEF working notes, CEUR (2017)Google Scholar
  18. 18.
    Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF - Experimental Evaluation in Visual Information Retrieval, The Springer International Series on Information Retrieval, vol. 32. Springer, Heidelberg (2010)zbMATHGoogle Scholar
  19. 19.
    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)Google Scholar
  20. 20.
    Pomente, A., Aleandri, D.: Convolutional expectation maximization for population estimation, pp. 11–14. CLEF working notes, CEUR (2017)Google Scholar
  21. 21.
    Soldaini, L., Goharian, N.: QuickUMLS: a fast, unsupervised approach for medical concept extraction. In: MedIR Workshop, SIGIR (2016)Google Scholar
  22. 22.
    Tsikrika, T., Herrera, A.G.S., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 95–106. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23708-9_12 CrossRefGoogle Scholar
  23. 23.
    Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 1–12. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-40802-1_1 CrossRefGoogle Scholar
  24. 24.
    Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2015 labs. In: Mothe, J., et al. (eds.) CLEF 2015. LNCS, vol. 9283, pp. 444–461. Springer, Cham (2015). doi: 10.1007/978-3-319-24027-5_45 CrossRefGoogle Scholar
  25. 25.
    Villegas, M., et al.: General overview of ImageCLEF at the CLEF 2016 labs. CLEF 2016. LNCS, vol. 9822, pp. 267–285. Springer, Cham (2016). doi: 10.1007/978-3-319-44564-9_25 CrossRefGoogle Scholar
  26. 26.
    Zhou, L., Piras, L., Rieger, M., Boato, G., Dang-Nguyen, D.T., Gurrin, C.: Organizer team at ImageCLEFlifelog 2017: baseline approaches for lifelog retrieval and summarization, pp. 11–14. CLEF working notes, CEUR (2017)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University Politehnica of BucharestBucharestRomania
  2. 2.University of Applied Sciences Western Switzerland (HES-SO)SierreSwitzerland
  3. 3.SearchInkBerlinGermany
  4. 4.Institut de Recherche en Informatique de ToulouseToulouseFrance
  5. 5.University of TrentoTrentoItaly
  6. 6.Dublin City UniversityDublinIreland
  7. 7.ETH ZurichZurichSwitzerland
  8. 8.National Library of MedicineBethesdaUSA
  9. 9.Technische Universität DarmstadtDarmstadtGermany
  10. 10.United Institute of Informatics ProblemsMinskBelarus
  11. 11.University of CagliariCagliariItaly
  12. 12.Simula Research LaboratoryLysakerNorway

Personalised recommendations