International Conference of the Cross-Language Evaluation Forum for European Languages

Experimental IR Meets Multilinguality, Multimodality, and Interaction pp 444-461 | Cite as

General Overview of ImageCLEF at the CLEF 2015 Labs

  • Mauricio Villegas
  • Henning Müller
  • Andrew Gilbert
  • Luca Piras
  • Josiah Wang
  • Krystian Mikolajczyk
  • Alba G. Seco de Herrera
  • Stefano Bromuri
  • M. Ashraful Amin
  • Mahmood Kazi Mohammed
  • Burak Acar
  • Suzan Uskudarli
  • Neda B. Marvasti
  • José F. Aldana
  • María del Mar Roldán García
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9283)

Abstract

This paper presents an overview of the ImageCLEF 2015 evaluation campaign, an event that was organized as part of the CLEF labs 2015. ImageCLEF is an ongoing initiative that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to databases of images in various usage scenarios and domains. In 2015, the 13th edition of ImageCLEF, four main tasks were proposed: 1) automatic concept annotation, localization and sentence description generation for general images; 2) identification, multi-label classification and separation of compound figures from biomedical literature; 3) clustering of x-rays from all over the body; and 4) prediction of missing radiological annotations in reports of liver CT images. The x-ray task was the only fully novel task this year, although the other three tasks introduced modifications to keep up relevancy of the proposed challenges. The participation was considerably positive in this edition of the lab, receiving almost twice the number of submitted working notes papers as compared to previous years.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amin, M.A., Mohammed, M.K.: Overview of the ImageCLEF 2015 medical clustering task. In: CLEF2015 Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Toulouse, France, September 8–11, 2015Google Scholar
  2. 2.
    Caputo, B., Müller, H., Martinez-Gomez, J., Villegas, M., Acar, B., Patricia, N., Marvasti, N., Üsküdarlı, S., Paredes, R., Cazorla, M., Garcia-Varea, I., Morell, V.: ImageCLEF 2014: overview and analysis of the results. In: Kanoulas, E., Lupu, M., Clough, P., Sanderson, M., Hall, M., Hanbury, A., Toms, E. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 192–211. Springer, Heidelberg (2014). doi:10.1007/978-3-319-11382-1_18 Google Scholar
  3. 3.
    Denkowski, M., Lavie, A.: Meteor universal: language specific translation evaluation for any target language. In: Proceedings of the EACL 2014 Workshop on Statistical Machine Translation (2014)Google Scholar
  4. 4.
    Faruque, M.S.S., Banik, S., Mohammed, M.K., Hasan, M., Amin, M.A.: Teaching and learning system for diagnostic imaging phase i: X-ray image analysis and retrieval. In: Proceedings of the 6th International Conference on Computer Supported Education (2015)Google Scholar
  5. 5.
    Gilbert, A., Piras, L., Wang, J., Yan, F., Dellandrea, E., Gaizauskas, R., Villegas, M., Mikolajczyk, K.: Overview of the ImageCLEF 2015 scalable image annotation, localization and sentence generation task. In: CLEF2015 Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Toulouse, France, September 8–11, 2015Google Scholar
  6. 6.
    Kelly, L., Goeuriot, L., Suominen, H., Schreck, T., Leroy, G., Mowery, D.L., Velupillai, S., Chapman, W.W., Martinez, D., Zuccon, G., Palotti, J.: Overview of the ShARe/CLEF eHealth evaluation lab 2014. In: Kanoulas, E., Lupu, M., Clough, P., Sanderson, M., Hall, M., Hanbury, A., Toms, E. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 172–191. Springer, Heidelberg (2014) Google Scholar
  7. 7.
    García Seco de Herrera, A., Kalpathy-Cramer, J., Demner Fushman, D., Antani, S., Müller, H.: Overview of the ImageCLEF 2013 medical tasks. In: Working Notes of CLEF 2013 (Cross Language Evaluation Forum), September 2013. http://ceur-ws.org/Vol-1179/CLEF2013wn-ImageCLEF-SecoDeHerreraEt2013b.pdf
  8. 8.
    García Seco de Herrera, A., Müller, H., Bromuri, S.: Overview of the ImageCLEF 2015 medical classification task. In: Working Notes of CLEF 2015 (Cross Language Evaluation Forum). CEUR Workshop Proceedings, CEUR-WS.org, September 2015Google Scholar
  9. 9.
    Joly, A., Goëau, H., Glotin, H., Spampinato, C., Bonnet, P., Vellinga, W.-P., Planque, R., Rauber, A., Fisher, R., Müller, H.: LifeCLEF 2014: multimedia life species identification challenges. In: Kanoulas, E., Lupu, M., Clough, P., Sanderson, M., Hall, M., Hanbury, A., Toms, E. (eds.) CLEF 2014. LNCS, vol. 8685, pp. 229–249. Springer, Heidelberg (2014) Google Scholar
  10. 10.
    Kalpathy-Cramer, J., de Herrera, A.G.S., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems -an overview of the medical image retrieval task at imageclef 2004–2014. Computerized Medical Imaging and Graphics 39, 55–61 (2015). doi:10.1016/j.compmedimag.2014.03.004 CrossRefGoogle Scholar
  11. 11.
    Kokciyan, N., Turkay, R., Uskudarli, S., Yolum, P., Bakir, B., Acar, B.: Semantic description of liver ct images: An ontological approach. IEEE Journal of Biomedical and Health Informatics PP(99), 1–1 (2014). doi:10.1109/JBHI.2014.2298880 Google Scholar
  12. 12.
    Marvasti, N., Kökciyan, N., Türkay, R., Yazıcı, A., Yolum, P., Üsküdarlı, S., Acar, B.: ImageCLEF liver CT image annotation task 2014. In: CLEF 2014 Evaluation Labs and Workshop, Online Working Notes (2014). http://ceur-ws.org/Vol-1180/CLEF2014wn-Image-MarvastiEt2014.pdf
  13. 13.
    Marvasti, N.B., del Mar Roldán García, M., Uskudarli, S., Aldana, J.F., Acar, B.: Overview of the ImageCLEF 2015 liver CT annotation task. In: CLEF2015 Working Notes. CEUR Workshop Proceedings, CEUR-WS.org, Toulouse, France, September 8–11, 2015Google Scholar
  14. 14.
    Müller, H., Clough, P., Deselaers, T., Caputo, B.: ImageCLEF: experimental evaluation in visual information retrieval. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15181-1 CrossRefMATHGoogle Scholar
  15. 15.
    Müller, H., García Seco de Herrera, A., Kalpathy-Cramer, J., Demner Fushman, D., Antani, S., Eggel, I.: Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In: Working Notes of CLEF 2012 (Cross Language Evaluation Forum), September 2012. http://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-MullerEt2012.pdf
  16. 16.
    Pelka, O., Friedrich, C.M.: FHDO biomedical computer science group at medical classification task of ImageCLEF 2015. In: Working Notes of CLEF 2015 (Cross Language Evaluation Forum). CEUR Workshop Proceedings, CEUR-WS.org, September 2015Google Scholar
  17. 17.
    Rodríguez-Sánchez, A., Fontanella, S., Piater, J., Szedmak, S.: IIS at imageclef 2015: multi-label classification task. In: Working Notes of CLEF 2015 (Cross Language Evaluation Forum). CEUR Workshop Proceedings, CEUR-WS.org, September 2015Google Scholar
  18. 18.
    Santosh, K.C. and Xue, Z., Antani, S., Thoma, G.: NLM at imageCLEF2015: biomedical multipanel figure separation. In: Working Notes of CLEF 2015 (Cross Language Evaluation Forum). CEUR Workshop Proceedings, CEUR-WS.org, September 2015Google Scholar
  19. 19.
    Tsikrika, T., de Herrera, A.G.S., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., de Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 95–106. Springer, Heidelberg (2011). doi:10.1007/978-3-642-23708-9_12 CrossRefGoogle Scholar
  20. 20.
    Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 1–12. Springer, Heidelberg (2013) Google Scholar
  21. 21.
    Villegas, M., Paredes, R.: Overview of the ImageCLEF 2012 scalable web image annotation task. In: Forner, P., Karlgren, J., Womser-Hacker, C. (eds.) CLEF 2012 Evaluation Labs and Workshop, Online Working Notes. Rome, Italy, September 17–20, 2012. http://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-VillegasEt2012.pdf
  22. 22.
    Villegas, M., Paredes, R.: Overview of the ImageCLEF 2014 scalable concept image annotation task. In: CLEF2014 Working Notes. CEUR Workshop Proceedings. CEUR-WS.org, Sheffield, UK, vol. 1180, pp. 308–328, September 15–18, 2014. http://ceur-ws.org/Vol-1180/CLEF2014wn-Image-VillegasEt2014.pdf
  23. 23.
    Villegas, M., Paredes, R., Thomee, B.: Overview of the ImageCLEF 2013 scalable concept image annotation subtask. In: CLEF 2013 Evaluation Labs and Workshop, Online Working Notes. Valencia, Spain, September 23–26, 2013. http://ceur-ws.org/Vol-1179/CLEF2013wn-ImageCLEF-VillegasEt2013.pdf

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Mauricio Villegas
    • 1
  • Henning Müller
    • 2
  • Andrew Gilbert
    • 3
  • Luca Piras
    • 4
  • Josiah Wang
    • 5
  • Krystian Mikolajczyk
    • 3
  • Alba G. Seco de Herrera
    • 2
  • Stefano Bromuri
    • 2
  • M. Ashraful Amin
    • 6
  • Mahmood Kazi Mohammed
    • 7
  • Burak Acar
    • 8
  • Suzan Uskudarli
    • 8
  • Neda B. Marvasti
    • 8
  • José F. Aldana
    • 9
  • María del Mar Roldán García
    • 9
  1. 1.Universitat Politècnica de ValènciaValenciaSpain
  2. 2.University of Applied Sciences Western Switzerland (HES-SO)DelémontSwitzerland
  3. 3.University of SurreyGuildfordUK
  4. 4.University of CagliariCagliariItaly
  5. 5.University of SheffieldSheffieldUK
  6. 6.Independent UniversityDhakaBangladesh
  7. 7.Sir Salimullah Medical CollegeDhakaBangladesh
  8. 8.Bogazici UniversityIstanbulTurkey
  9. 9.University of MalagaMalagaSpain

Personalised recommendations