Skip to main content
SpringerLink
Log in

Multimedia Retrieval Evaluation

  • Reference work entry
Encyclopedia of Database Systems

  • 51 Accesses

Definition

Multimedia Retrieval Evaluation is the activity of measuring the effectiveness of one or more multimedia search techniques. A common way of evaluating multimedia retrieval systems is by comparing them to each other in community wide benchmarks. In such benchmarks participants are invited to submit their retrieval results for a given set of topics, the relevance of the submitted items is checked, and effectiveness measures for each of the submissions are reported. Multimedia retrieval evaluation measures the effectiveness of multimedia retrieval systems or techniques by looking at how well the information need as described by a topic is satisfied by the results retrieved by the system or technique. Efficiency of the techniques is typically not taken into account, but may be studied separately.

Historical Background

Until the mid-1990s, no commonly used evaluation methodology existed for multimedia retrieval. An important reason for this is that the field has merely been a...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 2,500.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Readings

  1. Cleverdon C.W. The cranfield tests on index languagr devices. Aslib Proc., 1967, pp. 173–192.

    Google Scholar 

  2. Draper S.W., Dunlop M.D., Ruthven I., and van Rijsbergen C.J. (eds.). In Proc. Mira 99: Evaluating Interactive Information Retrieval. Electronic Workshops in Computing, 1999.

    Google Scholar 

  3. Gunther N.J. and Beretta G. A benchmark for image retrieval using distributed systems over the internet: BIRDS-I. Technical Report HPL-2000-162, HP Laboratories, 2000.

    Google Scholar 

  4. Hull D. Using statistical testing in the evaluation of retrieval experiments. In Proc. 16th Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, 1993, pp. 329–338.

    Google Scholar 

  5. Jones K.S. Automatic indexing. J. Doc., 30:393–432, 1974.

    Article  Google Scholar 

  6. Leung C.H.C. and Ho-Shing Ip H. Benchmarking for content-based visual information search. In Advances in Visual Information Systems, Fourth Int. Conf., 2000, pp. 442–456.

    Google Scholar 

  7. Müller H., Müller W., McG. Squire D., Marchand-Maillet S., and Pun T. Performance evaluation in content-based image retrieval: overview and proposals. Pattern Recogn. Lett. (Special Issue on Image and Video Indexing), 22(5):593–601, 2001.

    Google Scholar 

  8. Narasimhalu A.D., Kankanhalli M.S., and Wu J. Benchmarking multimedia databases. Multimedia Tools Appl., 4(3):333–356, 1997. ISSN 1380-7501.

    Article  Google Scholar 

  9. Smeaton A.F., Over P., Costello C.J., de Vries A.P., Doermann D., Hauptmann A., Rorvig M.E., Smith J.R., and Wu L. The TREC-2001 video track: information retrieval on digital video information. In Research and Advanced Technology for Digital Libraries, Sixth European Conference, 2002, pp. 266–275.

    Google Scholar 

  10. Smith J.R. Image retrieval evaluation. In Proc. IEEE Workshop on Content-based Access of Image and Video Libraries, 1998, pp. 112–113.

    Google Scholar 

  11. Voorhees E.M. and Harman D.K. Overview of the eighth text retrieval conference (TREC-8). In Proc. The 8th Text Retrieval Conference, 2000.

    Google Scholar 

  12. Voorhees E.M. and Harman D.K. TREC: Experiment and Evaluation in Information Retrieval (Digital Libraries and Electronic Publishing). MIT, Cambridge, MA, 2005. ISBN 0262220733.

    Google Scholar 

  13. Westerveld T. Trecvid as a re-usable test-collection for video retrieval. In Proc. Multimedia Information Retrieval Workshop, 2005.

    Google Scholar 

  14. Zobel J. How reliable are the results of large-scale information retrieval experiments? In Proc. 21st Annual Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, 1998, pp. 307–314.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Teezir Search Solutions, Ede, The Netherlands

    Thijs Westerveld

  2. CWI, Amsterdam, The Netherlands

    Thijs Westerveld

Authors
  1. Thijs Westerveld
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Computing, Georgia Institute of Technology, 266 Ferst Drive, 30332-0765, Atlanta, GA, USA

    LING LIU (Professor) (Professor)

  2. Database Research Group David R. Cheriton School of Computer Science, University of Waterloo, 200 University Avenue West, N2L 3G1, Waterloo, ON, Canada

    M. TAMER ÖZSU (Professor and Director, University Research Chair) (Professor and Director, University Research Chair)

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer Science+Business Media, LLC

About this entry

Cite this entry

Westerveld, T. (2009). Multimedia Retrieval Evaluation. In: LIU, L., ÖZSU, M.T. (eds) Encyclopedia of Database Systems. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-39940-9_235

Download citation

Publish with us

Policies and ethics

Search

Navigation