Comparative Evaluation of Multilingual Information Access Systems

4th Workshop of the Cross-Language Evaluation Forum, CLEF 2003, Trondheim, Norway, August 21-22, 2003, Revised Selected Papers

  • Carol Peters
  • Julio Gonzalo
  • Martin Braschler
  • Michael Kluck
Conference proceedings CLEF 2003

Part of the Lecture Notes in Computer Science book series (LNCS, volume 3237)

Table of contents

  1. Front Matter
  2. Introduction

    1. Julio Gonzalo, Carol Peters
      Pages 1-6
    2. Martin Braschler, Carol Peters
      Pages 7-20
  3. Part I. Ad-hoc Text Retrieval Tracks

    1. Martin Braschler
      Pages 44-63
  4. Mainly Cross-Language Experiments

    1. Jacques Savoy
      Pages 64-73
    2. Eija Airio, Heikki Keskustalo, Turid Hedlund, Ari Pirkola
      Pages 74-84
    3. Paul McNamee, James Mayfield
      Pages 85-97
    4. Nicola Cancedda, Hervé Déjean, Éric Gaussier, Jean-Michel Renders, Alexei Vinokourov
      Pages 98-107
    5. Fernando Llopis, Rafael Muñoz
      Pages 122-132
    6. Jaap Kamps, Christof Monz, Maarten de Rijke, Börkur Sigurbjörnsson
      Pages 152-165
    7. René Hackl, Ralph Kölle, Thomas Mandl, Alexandra Ploedt, Jan-Hendrik Scheufen, Christa Womser-Hacker
      Pages 166-173
    8. Romaric Besançon, Gaël de Chalendar, Olivier Ferret, Christian Fluhr, Olivier Mesnard, Hubert Naets
      Pages 174-184
    9. Fernando Martínez-Santiago, Arturo Montejo-Ráez, Luis Alfonso Ureña-López, M. Carlos Díaz-Galiano
      Pages 192-201
    10. Wen-Cheng Lin, Hsin-Hsi Chen
      Pages 202-209

About these proceedings


The fourth campaign of the Cross-language Evaluation Forum (CLEF) for European languages was held from January to August 2003. Participation in this campaign showed a slight rise in the number of participants from the previous year, with 42 groups submitting results for one or more of the different tracks (compared with 37 in 2002), but a steep rise in the number of experiments attempted. A distinctive feature of CLEF 2003 was the number of new tracks and tasks that were offered as pilot experiments. The aim was to try out new ideas and to encourage the development of new evaluation methodologies, suited to the emerging requirements of both system developers and users with respect to today’s digital collections and to encourage work on many European languages rather than just those most widely used. CLEF is thus gradually pushing its participants towards the ultimate goal: the development of truly multilingual systems capable of processing collections in diverse media. The campaign culminated in a two-day workshop held in Trondheim, Norway, 21–22 August, immediately following the 7th European Conference on Digital Libraries (ECDL 2003), and attended by more than 70 researchers and system developers. The objective of the workshop was to bring together the groups that had participated in the CLEF 2003 campaign so that they could report on the results of their experiments.


DOM algorithms bilingual information retrieval cross-language queries cross-lingual text retrieval image retrieval information system interactive text retrieval systems machine translation multilingual information retrieval natural language processing ontology proving spoken document retrieval

Editors and affiliations

  • Carol Peters
    • 1
  • Julio Gonzalo
    • 2
  • Martin Braschler
    • 2
  • Michael Kluck
    • 3
  1. 1.ISTI-CNR, Area di RicercaPisaItaly
  2. 2.No Affiliations 
  3. 3.German Institute for International and Security AffairsStiftung Wissenschaft und Politik (SWP)BerlinGermany

Bibliographic information

  • DOI
  • Copyright Information Springer-Verlag Berlin Heidelberg 2004
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Springer Book Archive
  • Print ISBN 978-3-540-24017-4
  • Online ISBN 978-3-540-30222-3
  • Series Print ISSN 0302-9743
  • Series Online ISSN 1611-3349
  • About this book