Skip to main content

Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications

  • Conference paper
  • First Online:
Experimental IR Meets Multilinguality, Multimodality, and Interaction (CLEF 2020)

Abstract

This paper presents an overview of the ImageCLEF 2020 lab that was organized as part of the Conference and Labs of the Evaluation Forum - CLEF Labs 2020. ImageCLEF is an ongoing evaluation initiative (first run in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2020, the 18th edition of ImageCLEF runs four main tasks: (i) a medical task that groups three previous tasks, i.e., caption analysis, tuberculosis prediction, and medical visual question answering and question generation, (ii) a lifelog task (videos, images and other sources) about daily activity understanding, retrieval and summarization, (iii) a coral task about segmenting and labeling collections of coral reef images, and (iv) a new Internet task addressing the problems of identifying hand-drawn user interface components. Despite the current pandemic situation, the benchmark campaign received a strong participation with over 40 groups submitting more than 295 runs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.imageclef.org/.

  2. 2.

    https://www.aicrowd.com/.

  3. 3.

    https://scholar.google.com/.

  4. 4.

    https://www.imageclef.org/2020/.

  5. 5.

    https://visualqa.org/.

  6. 6.

    https://medpix.nlm.nih.gov/.

  7. 7.

    http://www.cs.cmu.edu/~ark/mheilman/questions/.

  8. 8.

    https://visualqa.org/evaluation.html.

  9. 9.

    https://www.ncbi.nlm.nih.gov/pmc/.

  10. 10.

    Fitbit Fitness Tracker (FitBit Versa) - https://www.fitbit.com/.

  11. 11.

    Moves App for Android and iOS - http://www.moves-app.com/.

  12. 12.

    https://essexnlip.uk/marine-technology-research-unit/.

  13. 13.

    http://host.robots.ox.ac.uk/pascal/VOC/.

  14. 14.

    http://evansdata.com/.

  15. 15.

    http://appdevelopermagazine.com/.

  16. 16.

    https://github.com/philferriere/cocoapi/.

References

  1. Beltramelli, T.: pix2code: generating code from a graphical user interface screenshot. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, pp. 1–9 (2018)

    Google Scholar 

  2. Ben Abacha, A., Datla, V.V., Hasan, S.A., Demner-Fushman, D., Müller, H.: Overview of the VQA-med task at imageclef 2020: visual question answering and generation in the medical domain. In: CLEF 2020 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org, Thessaloniki (2020)

    Google Scholar 

  3. Ben Abacha, A., Hasan, S.A., Datla, V.V., Liu, J., Demner-Fushman, D., Müller, H.: VQA-Med: overview of the medical visual question answering task at imageclef 2019. In: CLEF2019 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  4. Birkeland, C.: Global status of coral reefs: in combination, disturbances and stressors become ratchets. In: World Seas: An Environmental Evaluation, pp. 35–56. Elsevier (2019)

    Google Scholar 

  5. Bodenreider, O.: The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 32(Database–Issue), 267–270 (2004). https://doi.org/10.1093/nar/gkh061

    Article  Google Scholar 

  6. Brander, L.M., Rehdanz, K., Tol, R.S., Van Beukering, P.J.: The economic impact of ocean acidification on coral reefs. Clim. Change Econ. 3(01), 1250002 (2012)

    Article  Google Scholar 

  7. Chamberlain, J., Campello, A., Wright, J.P., Clift, L.G., Clark, A., García Seco de Herrera, A.: Overview of ImageCLEFcoral 2019 task. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org (2019)

    Google Scholar 

  8. Chamberlain, J., Campello, A., Wright, J.P., Clift, L.G., Clark, A., García Seco de Herrera, A.: Overview of the ImageCLEFcoral 2020 task: automated coral reef image annotation. In: CLEF2020 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org (2020)

    Google Scholar 

  9. Chen, C., Su, T., Meng, G., Xing, Z., Liu, Y.: From UI design image to GUI skeleton : a neural machine translator to bootstrap mobile GUI implementation. In: International Conference on Software Engineering, vol. 6 (2018)

    Google Scholar 

  10. Clough, P., Müller, H., Sanderson, M.: The CLEF 2004 cross-language image retrieval track. In: Peters, C., Clough, P., Gonzalo, J., Jones, G.J.F., Kluck, M., Magnini, B. (eds.) CLEF 2004. LNCS, vol. 3491, pp. 597–613. Springer, Heidelberg (2005). https://doi.org/10.1007/11519645_59

    Chapter  Google Scholar 

  11. Clough, P., Sanderson, M.: The CLEF 2003 cross language image retrieval track. In: Peters, C., Gonzalo, J., Braschler, M., Kluck, M. (eds.) CLEF 2003. LNCS, vol. 3237, pp. 581–593. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30222-3_56

    Chapter  Google Scholar 

  12. Dang-Nguyen, D.T., et al.: Overview of ImageCLEFlifelog 2019: solve my life puzzle and lifelog moment retrieval. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019)

    Google Scholar 

  13. Dao, M.S., Vo, A.K., Phan, T.D., Zettsu, K.: BIDAL@imageCLEFlifelog2019: the role of content and context of daily activities in insights from lifelogs. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  14. Deka, B., et al.: Rico: a mobile app dataset for building data-driven design applications. In: UIST 2017 - Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, pp. 845–854 (2017). https://doi.org/10.1145/3126594.3126651

  15. Dicente Cid, Y., Jimenez-del-Toro, O., Depeursinge, A., Müller, H.: Efficient and fully automatic segmentation of the lungs in CT volumes. In: Goksel, O., Jimenez-del-Toro, O., Foncubierta-Rodriguez, A., Müller, H. (eds.) Proceedings of the VISCERAL Challenge at ISBI, CEUR Workshop Proceedings, pp. 31–35, no. 1390, April 2015

    Google Scholar 

  16. Dicente Cid, Y., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2017 - predicting tuberculosis type and drug resistances. In: CLEF2017 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Dublin (2017). http://ceur-ws.org

  17. Dicente Cid, Y., Liauchuk, V., Klimuk, D., Tarasau, A., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2019 - Automatic CT-based Report Generation and Tuberculosis Severity Assessment. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  18. Dicente Cid, Y., Liauchuk, V., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2018 - detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score. In: CLEF2018 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Avignon (2018). http://ceur-ws.org

  19. Dogariu, M., Ionescu, B.: Multimedia lab @ ImageCLEF 2019 lifelog moment retrieval task. In: CLEF2019 Working Notes. CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  20. Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010). https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  21. Fichou, D., et al.: Overview of ImageCLEFdrawnUI 2020: the detection and recognition of hand drawn website UIs task. In: CLEF2020 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Thessaloniki (2020). http://ceur-ws.org

  22. Gurrin, C., Joho, H., Hopfgartner, F., Zhou, L., Albatal, R.: Overview of NTCIR-12 lifelog task. In: NTCIR (2016)

    Google Scholar 

  23. Gurrin, C., et al.: Overview of NTCIR-13 lifelog-2 task (2017)

    Google Scholar 

  24. Gurrin, C., et al.: Overview of the NTCIR-14 lifelog-3 task (2019)

    Google Scholar 

  25. Hasan, S.A., Ling, Y., Farri, O., Liu, J., Lungren, M., Müller, H.: Overview of the ImageCLEF 2018 medical domain visual question answering task. In: CLEF2018 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Avignon (2018). http://ceur-ws.org

  26. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 386–397 (2020). https://doi.org/10.1109/TPAMI.2018.2844175

    Article  Google Scholar 

  27. He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)

    Google Scholar 

  28. García Seco de Herrera, A., Eickhoff, C., Andrearczyk, V., Müller, H.: Overview of the ImageCLEF 2018 caption prediction tasks. In: CLEF2018 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Avignon (2018). http://ceur-ws.org

  29. García Seco de Herrera, A., Schaer, R., Bromuri, S., Müller, H.: Overview of the ImageCLEF 2016 medical task. In: Working Notes of CLEF 2016 (Cross Language Evaluation Forum), September 2016

    Google Scholar 

  30. Ionescu, B., et al.: ImageCLEF 2019: multimedia retrieval in medicine, lifelogging, security and nature. In: Crestani, F., et al. (eds.) CLEF 2019. LNCS, vol. 11696, pp. 358–386. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-28577-7_28

    Chapter  Google Scholar 

  31. Kalpathy-Cramer, J., García Seco de Herrera, A., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems: overview of the medical image retrieval task at ImageCLEF 2004–2014. Comput. Med. Imaging Graph. 39, 55–61 (2015)

    Google Scholar 

  32. Kougia, V., Pavlopoulos, J., Androusopoulos, I.: AUEB NLP group at ImageCLEFMED caption 2019. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  33. Kozlovski, S., Liauchuk, V., Dicente Cid, Y., Tarasau, A., Kovalev, V., Müller, H.: Overview of ImageCLEFtuberculosis 2020 - automatic CT-based report generation. In: CLEF2020 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org, Thessaloniki (2020). http://ceur-ws.org

  34. Le, N.K., Nguyen, D.H., Nguyen, V.T., Tran, M.T.: Lifelog moment retrieval with advanced semantic extraction and flexible moment visualization for exploration. In: CLEF2019 Working Notes, CEUR Workshop Proceedings, CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  35. Liauchuk, V., Kovalev, V.: Imageclef 2017: supervoxels and co-occurrence for tuberculosis CT image classification. In: CLEF2017 Working Notes. CEUR Workshop Proceedings. CEUR-WS.org, Dublin (2017). http://ceur-ws.org

  36. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  37. Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): ImageCLEF - Experimental Evaluation in Visual Information Retrieval, The Springer International Series On Information Retrieval, vol. 32. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15181-1

    Book  MATH  Google Scholar 

  38. Ninh, V.T., et al.: Overview of ImageCLEF lifelog 2020: lifelog moment retrieval and sport performance lifelog. In: CLEF2020 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Thessaloniki (2020). http://ceur-ws.org

  39. Ninh, V.T., et al.: LIFER 2.0: discover personal lifelog insight by interactive lifelog retrieval system. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  40. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318. Association for Computational Linguistics (2002)

    Google Scholar 

  41. Pelka, O., Friedrich, C.M., García Seco de Herrera, A., Müller, H.: Overview of the ImageCLEFmed 2019 concept prediction task. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  42. Pelka, O., Friedrich, C.M., García Seco de Herrera, A., Müller, H.: Overview of the ImageCLEFmed 2020 concept prediction task: medical image understanding. In: CLEF2020 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Thessaloniki (2020)

    Google Scholar 

  43. Pelka, O., Koitka, S., Rückert, J., Nensa, F., Friedrich, C.M.: Radiology Objects in COntext (ROCO): a multimodal image dataset. In: Stoyanov, D., et al. (eds.) LABELS/CVII/STENT -2018. LNCS, vol. 11043, pp. 180–189. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01364-6_20

    Chapter  Google Scholar 

  44. Pelka, O., Nensa, F., Friedrich, C.M.: Adopting semantic information of grayscale radiographs for image classification and retrieval. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018), BIOIMAGING, Funchal, Madeira, Portugal, 19–21 January 2018, vol. 2, pp. 179–187 (2018). https://doi.org/10.5220/0006732301790187

  45. Rajpurkar, P., et al.: Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning. CoRR abs/1711.05225 (2017). http://arxiv.org/abs/1711.05225

  46. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  47. Ribeiro, R., Neves, A.J.R., Oliveira, J.L.: UAPTBioinformatics working notes at ImageCLEF 2019 lifelog moment retrieval (LMRT) task. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  48. Roberts, R.J.: PubMed central: the GenBank of the published literature. Proc. Nat. Acad. Sci. U.S.A. 98(2), 381–382 (2001). https://doi.org/10.1073/pnas.98.2.381

    Article  MathSciNet  Google Scholar 

  49. Russakovsky, O.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  50. Speers, A.E., Besedin, E.Y., Palardy, J.E., Moore, C.: Impacts of climate change and ocean acidification on coral reef fisheries: an integrated ecological-economic model. Ecol. Econ. 128, 33–43 (2016)

    Article  Google Scholar 

  51. Thambawita, V., et al.: PMData: a sports logging dataset (2020). https://doi.org/10.31219/osf.io/k2apb

  52. Tournadre, M., Dupont, G., Pauwels, V., Cheikh, B., Lmami, M., Ginsca, A.L.: A multimedia modular approach to lifelog moment retrieval. In: CLEF2019 Working Notes, CEUR Workshop Proceedings. CEUR-WS.org, Lugano (2019). http://ceur-ws.org

  53. Tsikrika, T., de Herrera, A.G.S., Müller, H.: Assessing the scholarly impact of ImageCLEF. In: Forner, P., Gonzalo, J., Kekäläinen, J., Lalmas, M., de Rijke, M. (eds.) CLEF 2011. LNCS, vol. 6941, pp. 95–106. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23708-9_12

    Chapter  Google Scholar 

  54. Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 1–12. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40802-1_1

    Chapter  Google Scholar 

  55. World Health Organization, et al.: Global tuberculosis report 2019 (2019)

    Google Scholar 

  56. Xu, Y., Mo, T., Feng, Q., Zhong, P., Lai, M., Chang, E.I.: Deep learning of feature representation with multiple instance learning for medical image analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, Florence, Italy, 4–9 May 2014, pp. 1626–1630 (2014). https://doi.org/10.1109/ICASSP.2014.6853873

  57. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40, 1452–1464 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

Data collection for the Tuberculosis task was supported by the National Institute of Allergy and Infectious Diseases, National Institutes of Health, US Department of Health and Human Services, CRDF project DAA9-19-65987-1.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bogdan Ionescu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ionescu, B. et al. (2020). Overview of the ImageCLEF 2020: Multimedia Retrieval in Medical, Lifelogging, Nature, and Internet Applications. In: Arampatzis, A., et al. Experimental IR Meets Multilinguality, Multimodality, and Interaction. CLEF 2020. Lecture Notes in Computer Science(), vol 12260. Springer, Cham. https://doi.org/10.1007/978-3-030-58219-7_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58219-7_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58218-0

  • Online ISBN: 978-3-030-58219-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics