Skip to main content
SpringerLink
Log in

Multi-Lingual Retrieval of Pictures in ImageCLEF

  • Chapter
  • First Online:
Information Retrieval Evaluation in a Changing World

Part of the book series: The Information Retrieval Series ((INRE,volume 41))

  • 687 Accesses

Abstract

CLEF first launched a multi-lingual visual information retrieval task in 2003 as part of the ImageCLEF track. Several such tasks subsequently followed, encompassing both the medical and non-medical domains. The main aim of such ad hoc image retrieval tasks was to investigate the effectiveness of retrieval approaches that exploit textual and visual evidence in the context of large and heterogeneous collections of images that are searched for by users with diverse information needs. This chapter presents an overview of the image retrieval activities within ImageCLEF from 2003 to 2011, focusing on the non-medical domain and, in particular, on the photographic retrieval and Wikipedia image retrieval tasks. We review the available test collections built in the context of these activities, present the main evaluation results, and summarise the contributions and lessons learned.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Agrawal R, Gollapudi S, Halverson A, Ieong S (2009) Diversifying search results. In: Proceedings of the second ACM international conference on web search and data mining, WSDM‘09. ACM, New York, pp 5–14. http://doi.acm.org/10.1145/1498759.1498766

    Chapter  Google Scholar 

  • Arni T, Clough P, Sanderson M, Grubinger M (2009) Overview of the ImageCLEFphoto 2008 photographic retrieval task. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones GJF, Kurimo M, Mandl T, Peñas A (eds) Evaluating systems for multilingual and multimodal information access: ninth workshop of the cross–language evaluation forum (CLEF 2008). Revised Selected Papers, Lecture notes in computer science (LNCS), vol 5706. Springer, Heidelberg, pp 500–511

    Chapter  Google Scholar 

  • Boididou C, Papadopoulos S, Kompatsiaris Y, Schifferes S, Newman N (2014) Challenges of computational verification in social multimedia. In: Proceedings of the 23rd international conference on world wide web, WWW‘14 companion. ACM, New York, pp 743–748. http://doi.acm.org/10.1145/2567948.2579323

    Chapter  Google Scholar 

  • Buckley C, Voorhees EM (2000) Evaluating evaluation measure stability. In: Proceedings of the 23rd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘00. ACM, New York, pp 33–40. http://doi.acm.org/10.1145/345508.345543

    Chapter  Google Scholar 

  • Buckley C, Voorhees EM (2004) Retrieval evaluation with incomplete information. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘04. ACM, New York, pp 25–32. http://doi.acm.org/10.1145/1008992.1009000

    Google Scholar 

  • Clarke CL, Kolla M, Cormack GV, Vechtomova O, Ashkan A, Büttcher S, MacKinnon I (2008) Novelty and diversity in information retrieval evaluation. In: Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘08. ACM, New York, pp 659–666. http://doi.acm.org/10.1145/1390334.1390446

    Chapter  Google Scholar 

  • Clough P, Sanderson M (2004) The CLEF 2003 cross language image retrieval track. In: Peters C, Braschler M, Gonzalo J, Kluck M (eds) Comparative evaluation of multilingual information access systems: fourth workshop of the cross–language evaluation forum (CLEF 2003), revised selected papers. Lecture notes in computer science (LNCS), vol 3237. Springer, Heidelberg, pp 581–593

    Chapter  Google Scholar 

  • Clough P, Müller H, Sanderson M (2005) The CLEF 2004 cross-language image retrieval track. In: Peters C, Clough P, Gonzalo J, Jones GJF, Kluck M, Magnini B (eds) Multilingual information access for text, speech and images: fifth workshop of the cross–language evaluation forum (CLEF 2004). Revised selected papers. Lecture notes in computer science (LNCS), vol 3491. Springer, Heidelberg, pp 597–613

    Chapter  Google Scholar 

  • Clough P, Grubinger M, Deselaers T, Hanbury A, Müller H (2006a) Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Nardi A, Peters C, Vicedo JL, Ferro N (eds) CLEF 2006 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1172/

  • Clough PD, Sanderson M, Reid N (2006b) The eurovision St Andrews collection of photographs. SIGIR Forum 40(1):21–30. http://doi.acm.org/10.1145/1147197.1147199

    Article  Google Scholar 

  • Clough P, Grubinger M, Deselaers T, Hanbury A, Müller H (2007) Overview of the ImageCLEF 2006 photographic retrieval and object annotation tasks. In: Peters C, Clough P, Gey FC, Karlgren J, Magnini B, Oard DW, de Rijke M, Stempfhuber M (eds) Evaluation of multilingual and multi-modal information retrieval: seventh workshop of the cross–language evaluation forum (CLEF 2006). Revised selected papers, Lecture notes in computer science (LNCS), vol 4730. Springer, Heidelberg, pp 223–256

    Chapter  Google Scholar 

  • Del Bimbo A (1999) Visual information retrieval. Morgan Kaufmann, San Francisco

    Google Scholar 

  • Grubinger M, Clough P, Leung C (2006) The IAPR TC-12 benchmark for visual information search. IAPR Newsl 28(2):10–12

    Google Scholar 

  • Grubinger M, Clough P, Hanbury A, Müller H (2008) Overview of the ImageCLEFphoto 2007 photographic retrieval task. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard DW, Peñas A, Petras V, Santos D (eds) Advances in multilingual and multimodal information retrieval: eighth workshop of the cross–language evaluation forum (CLEF 2007). Revised selected papers, Lecture notes in computer science (LNCS), vol 5152. Springer, Heidelberg, pp 433–444

    Chapter  Google Scholar 

  • Husain SS, Bober M (2017) Improving large-scale image retrieval through robust aggregation of local descriptors. IEEE Trans Pattern Anal Mach Intell 39(9):1783–1796. https://doi.org/10.1109/TPAMI.2016.2613873

    Article  Google Scholar 

  • Müller H, Deselaers T, Grubinger M, Clough P, Hanbury A, Hersh W (2007) Problems with running a successful multimedia retrieval benchmark. In: Proceedings of the third MUSCLE/ImageCLEF workshop on image and video retrieval evaluation

    Google Scholar 

  • Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, Khan CE, Jr, Hersh WR (2010) Overview of the CLEF 2009 medical image retrieval track. In: Peters C, Tsikrika T, Müller H, Kalpathy-Cramer J, Jones GJF, Gonzalo J, Caputo B (eds) Multilingual information access evaluation vol. ii multimedia experiments – tenth workshop of the cross–language evaluation forum (CLEF 2009). Revised selected papers, Lecture notes in computer science (LNCS). Springer, Heidelberg, pp 72–84

    Google Scholar 

  • Paramita ML, Grubinger M (2010) Photographic image retrieval. In: Müller H, Clough PD, Deselaers T, Caputo B (eds) ImageCLEF, experimental evaluation in visual information retrieval. Springer, Berlin, pp 141–162. https://doi.org/10.1007/978-3-642-15181-1_8

    Chapter  Google Scholar 

  • Paramita ML, Sanderson M, Clough P (2009a) Developing a test collection to support diversity analysis. In: Proceedings of redundancy, diversity, and IDR workshop-SIGIR, vol 9, pp 39–45

    Google Scholar 

  • Paramita ML, Sanderson M, Clough P (2009b) Diversity in photo retrieval: overview of the imageclefphoto task 2009. In: Workshop of the cross-language evaluation forum for European languages. Springer, Berlin pp 45–59

    Google Scholar 

  • Popescu A, Tsikrika T, Kludas J (2010) Overview of the wikipedia retrieval task at ImageCLEF 2010. In: Braschler M, Harman DK, Pianta E, Ferro N (eds) CLEF 2010 working notes, CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1176/

  • Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252. https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  • Santos RLT, Macdonald C, Ounis I (2015) Search result diversification. Found Trends Inf Retr 9(1):1–90. http://dx.doi.org/10.1561/1500000040

    Article  Google Scholar 

  • Tsikrika T, Kludas J (2009) Overview of the WikipediaMM task at ImageCLEF 2008. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones GJF, Kurimo M, Mandl T, Peñas A (eds) Evaluating systems for multilingual and multimodal information access: ninth workshop of the cross–language evaluation forum (CLEF 2008). Revised selected papers, Lecture notes in computer science (LNCS), vol 5706. Springer, Heidelberg, pp 539–550

    Chapter  Google Scholar 

  • Tsikrika T, Kludas J (2010) Overview of the WikipediaMM task at ImageCLEF 2009. In: Peters C, Tsikrika T, Müller H, Kalpathy-Cramer J, Jones GJF, Gonzalo J, Caputo B (eds) Multilingual information access evaluation vol. ii multimedia experiments – tenth workshop of the cross–language evaluation forum (CLEF 2009). Revised selected papers, Lecture notes in computer science (LNCS). Springer, Heidelberg, pp 60–71

    Google Scholar 

  • Tsikrika T, Garcia Seco de Herrera A, Müller H (2011a) Assessing the scholarly impact of ImageCLEF. In: Forner P, Gonzalo J, Kekäläinen J, Lalmas M, de Rijke M (eds) Multilingual and multimodal information access evaluation. Proceedings of the second international conference of the cross-language evaluation forum (CLEF 2011). Lecture notes in computer science (LNCS), vol 6941. Springer, Heidelberg, pp 95–106

    Chapter  Google Scholar 

  • Tsikrika T, Popescu A, Kludas J (2011b) Overview of the Wikipedia image retrieval task at ImageCLEF 2011. In: Petras V, Forner P, Clough P, Ferro N (eds) CLEF 2011 working notes. CEUR workshop proceedings (CEUR-WS.org). ISSN 1613-0073. http://ceur-ws.org/Vol-1177/

  • Tsikrika T, Kludas J, Popescu A (2012) Building reliable and reusable test collections for image retrieval: the wikipedia task at imageclef. IEEE MultiMedia 19(3):24–33

    Article  Google Scholar 

  • Tsikrika T, Larsen B, Müller H, Endrullis S, Rahm E (2013) The scholarly impact of CLEF (2000–2009). In: Forner P, Müller H, Paredes R, Rosso P, Stein B (eds) Information access evaluation meets multilinguality, multimodality, and visualization. Proceedings of the fourth international conference of the CLEF initiative (CLEF 2013). Lecture notes in computer science (LNCS), vol 8138. Springer, Heidelberg, pp 1–12

    Chapter  Google Scholar 

  • Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the 22Nd ACM international conference on multimedia, MM‘14. ACM, New York, pp 157–166. http://doi.acm.org/10.1145/2647868.2654948

    Google Scholar 

  • Westerveld T, van Zwol R (2007) Multimedia retrieval at inex 2006. ACM SIGIR Forum 41(1):58–63

    Article  Google Scholar 

  • Zhai CX, Cohen WW, Lafferty J (2003) Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval, SIGIR‘03. ACM, New York, pp 10–17. http://doi.acm.org/10.1145/860435.860440

    Google Scholar 

Download references

Acknowledgements

We thank all those involved in helping to coordinate and organise ImageCLEF over the years and all those who have engaged with the track, providing submissions and presenting at CLEF.

Author information

Authors and Affiliations

  1. Information School, University of Sheffield, Sheffield, UK

    Paul Clough

  2. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Theodora Tsikrika

Authors
  1. Paul Clough
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Theodora Tsikrika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul Clough .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell’Informazione, Università degli Studi di Padova , Padova, Italy

    Nicola Ferro

  2. Consiglio Nazionale delle Ricerche, Istituto di Scienza e Tecnologie dell’Informazione, Pisa, Italy

    Carol Peters

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Clough, P., Tsikrika, T. (2019). Multi-Lingual Retrieval of Pictures in ImageCLEF. In: Ferro, N., Peters, C. (eds) Information Retrieval Evaluation in a Changing World. The Information Retrieval Series, vol 41. Springer, Cham. https://doi.org/10.1007/978-3-030-22948-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-22948-1_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22947-4

  • Online ISBN: 978-3-030-22948-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation