Experimental IR Meets Multilinguality, Multimodality, and Interaction

11th International Conference of the CLEF Association, CLEF 2020, Thessaloniki, Greece, September 22–25, 2020, Proceedings

  • Avi Arampatzis
  • Evangelos Kanoulas
  • Theodora Tsikrika
  • Stefanos Vrochidis
  • Hideo Joho
  • Christina Lioma
  • Carsten Eickhoff
  • Aurélie Névéol
  • Linda Cappellato
  • Nicola Ferro
Conference proceedings CLEF 2020

Part of the Lecture Notes in Computer Science book series (LNCS, volume 12260)

Also part of the Information Systems and Applications, incl. Internet/Web, and HCI book sub series (LNISA, volume 12260)

Table of contents

  1. Front Matter
    Pages i-xvii
  2. Full Papers

    1. Front Matter
      Pages 1-1
    2. Pavel Efimov, Andrey Chertok, Leonid Boytsov, Pavel Braslavski
      Pages 3-15
    3. Marco Ferrante, Nicola Ferro, Luca Piazzon
      Pages 16-27
    4. Vivien Petras, Andreas Lüschow, Roland Ramthun, Juliane Stiller, Cristina España-Bonet, Sophie Henning
      Pages 28-42
    5. Camille Pradel, Damien Sileo, Álvaro Rodrigo, Anselmo Peñas, Eneko Agirre
      Pages 43-54
    6. Tilahun Yeshambel, Josiane Mothe, Yaregal Assabie
      Pages 55-66
  3. Short Papers

    1. Front Matter
      Pages 67-67
    2. Florian Dietz
      Pages 69-74
    3. Mahsa S. Shahshahani, Jaap Kamps
      Pages 75-81
  4. Best of CLEF 2019 Labs

    1. Front Matter
      Pages 83-83
    2. João Rafael Almeida, Olga Fajarda, José Luís Oliveira
      Pages 85-96
    3. Casper Hansen, Christian Hansen, Jakob Grue Simonsen, Christina Lioma
      Pages 124-130
    4. Dijana Kosmajac, Vlado Keselj
      Pages 141-153
    5. Vasiliki Kougia, John Pavlopoulos, Ion Androutsopoulos
      Pages 154-166
  5. CLEF 2020 Lab Overviews

    1. Front Matter
      Pages 167-167
    2. Richard Zanibbi, Douglas W. Oard, Anurag Agarwal, Behrooz Mansouri
      Pages 169-193
    3. Anastasios Nentidis, Anastasia Krithara, Konstantinos Bougiatiotis, Martin Krallinger, Carlos Rodriguez-Penagos, Marta Villegas et al.
      Pages 194-214
    4. Alberto Barrón-Cedeño, Tamer Elsayed, Preslav Nakov, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh et al.
      Pages 215-236
    5. Jiayuan He, Dat Quoc Nguyen, Saber A. Akhondi, Christian Druckenbrodt, Camilo Thorne, Ralph Hoessel et al.
      Pages 237-254
    6. Lorraine Goeuriot, Hanna Suominen, Liadh Kelly, Antonio Miranda-Escalada, Martin Krallinger, Zhengyang Liu et al.
      Pages 255-271
    7. David E. Losada, Fabio Crestani, Javier Parapar
      Pages 272-287
    8. Maud Ehrmann, Matteo Romanello, Alex Flückiger, Simon Clematide
      Pages 288-310
    9. Bogdan Ionescu, Henning Müller, Renaud Péteri, Asma Ben Abacha, Vivek Datla, Sadid A. Hasan et al.
      Pages 311-341
    10. Alexis Joly, Hervé Goëau, Stefan Kahl, Benjamin Deneu, Maximillien Servajean, Elijah Cole et al.
      Pages 342-363
    11. Philipp Schaer, Johann Schaible, Leyla Jael Garcia Castro
      Pages 364-371
    12. Janek Bevendorff, Bilal Ghanem, Anastasia Giachanou, Mike Kestemont, Enrique Manjavacas, Ilia Markov et al.
      Pages 372-383
    13. Alexander Bondarenko, Maik Fröbe, Meriem Beloucif, Lukas Gienapp, Yamen Ajjour, Alexander Panchenko et al.
      Pages 384-395
  6. Back Matter
    Pages 397-399

About these proceedings


This book constitutes the refereed proceedings of the 11th International Conference of the CLEF Association, CLEF 2020, held in Thessaloniki, Greece, in September 2020.*

The conference has a clear focus on experimental information retrieval with special attention to the challenges of multimodality, multilinguality, and interactive search ranging from unstructured to semi structures and structured data.

The 5 full papers and 2 short papers presented in this volume were carefully reviewed and selected from 9 submissions. This year, the contributions addressed the following challenges: a large-scale evaluation of translation effects in academic search, advancement of assessor-driven aggregation methods for efficient relevance assessments, and development of a new test dataset.

In addition to this, the volume presents 7 “best of the labs” papers which were reviewed as full paper submissions with the same review criteria. The 12 lab overview papers were accepted out of 15 submissions and represent scientific challenges based on new data sets and real world problems in multimodal and multilingual information access.

* The conference was held virtually due to the COVID-19 pandemic.


artificial intelligence computational linguistics computer systems computer vision data mining databases education Human-Computer Interaction (HCI) image analysis image processing image segmentation information retrieval information retrieval systems machine learning multimedia systems Natural Language Processing (NLP) natural languages search engines semantics text processing

Editors and affiliations

  1. 1.Department of Electrical and Computer EngineeringDemocritus University of ThraceXanthiGreece
  2. 2.University of AmsterdamAmsterdamThe Netherlands
  3. 3.Information Technologies InstituteCentre for Research and Technology HellasThessalonikiGreece
  4. 4.Information Technologies InstituteCentre for Research and Technology HellasThessalonikiGreece
  5. 5.Faculty of Library, Information and Media ScienceUniversity of TsukubaIbarakiJapan
  6. 6.Department of Computer ScienceUniversity of CopenhagenCopenhagenDenmark
  7. 7.Brown UniversityProvidenceUSA
  8. 8.LIMSI-CNRSOrsayFrance
  9. 9.Department of Information EngineeringUniversity of PadovaPaduaItaly
  10. 10.Department of Information EngineeringUniversity of PadovaPaduaItaly

Bibliographic information