Large Scale Evaluations of Multimedia Information Retrieval: The TRECVid Experience

  • Alan F. Smeaton
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3568)


Information Retrieval is a supporting technique which underpins a broad range of content-based applications including retrieval, filtering, summarisation, browsing, classification, clustering, automatic linking, and others. Multimedia information retrieval (MMIR) represents those applications when applied to multimedia information such as image, video, music, etc. In this presentation and extended abstract we are primarily concerned with MMIR as applied to information in digital video format. We begin with a brief overview of large scale evaluations of IR tasks in areas such as text, image and music, just to illustrate that this phenomenon is not just restricted to MMIR on video. The main contribution, however, is a set of pointers and a summarisation of the work done as part of TRECVid, the annual benchmarking exercise for video retrieval tasks.


Information Retrieval Search Task Automatic Speech Recognition Video Retrieval Topic Detection 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    The TREC Website. Last visited (April 2005), Available at
  2. 2.
    Kazai, G., Lalmas, M., de Vries, A.P.: The Overlap Problem in Content-Oriented XML Retrieval Evaluation. In: Proceedings of the 27th annual international conference on Research and development in information retrieval, SIGIR 2004, Sheffield, United Kingdom, pp. 72–79. ACM Press, New York (2004)CrossRefGoogle Scholar
  3. 3.
    MIREX 2005: 2nd Annual Music Information Retrieval Evaluation eXchange. Last visited (April 2005), Available at
  4. 4.
    The Benchathlon Network: Home of CBIR Benchmarking. Last visited (April 2005), Available at
  5. 5.
    TREC Video Retrieval Evaluation Online Proceedings. Last visited (April 2005), Available at
  6. 6.
    TREC Video Retrieval Evaluation Bibliography. Last visited (April 2005), Available at
  7. 7.
    Smeaton, A.F., Kraaij, W., Over, P.: The TREC Video Retrieval Evaluation (TRECVID): A Case Study and Status Report. In: Proceedings of RIAO 2004 - Coupling Approaches, Coupling Media and Coupling Languages for Information Retrieval, Avignon, France (April 26-28, 2004)Google Scholar
  8. 8.
    Kraaij, W., Smeaton, A.F., Over, P., Arlandis, J.: TRECVID 2004 - An Overview. In: Proceedings of TRECVid 2004, National Institute for Standards and Technology. Last visited (April 2005),
  9. 9.
    Fiscus, J.G., Doddington, G.R.: Topic Detection and Tracking Evaluation Overview. In: Topic Detection and Tracking: Event-Based Information Organization, pp. 17–31. Kluwer Academic Publishers, Norwell (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Alan F. Smeaton
    • 1
  1. 1.Adaptive Information Cluster, Center for Digital Video ProcessingDublin City UniversityGlasnevin, Dublin 9Ireland

Personalised recommendations