Skip to main content
SpringerLink
Log in

Evaluation of content-based image descriptors by statistical methods

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Evaluation of visual information retrieval systems is usually performed by executing test queries and computing recall- and precision-like measures based on predefined media collections and ground truth information. This process is complex and time consuming. For the evaluation of feature transformations (transformation of visual media objects to feature vectors) it would be desirable to have simpler methods available as well. In this paper we introduce a supplementary evaluation procedure for features that is founded on statistical data analysis. A second novelty is that we make use of the existing visual MPEG-7 descriptors to judge the characteristics of feature transformations. The proposed procedure is divided into four steps: (1) feature extraction, (2) merging with MPEG-7 data and normalisation, (3) statistical data analysis and (4) visualisation and interpretation. Three types of statistical methods are used for evaluation: (1) univariate description (moments, etc.), (2) identification of similarities between feature elements (e.g. cluster analysis) and (3) identification of dependencies between variables (e.g. factor analysis). Statistical analysis provides beneficial insights into the structure of features that can be exploited for feature redesign. Application and advantages of the proposed approach are shown in a number of toy examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bober M (2001) MPEG-7 visual shape descriptors. Special issue on MPEG-7. IEEE Trans Circuits Syst Video Technol 11(6):716–719

    Article  Google Scholar 

  2. Breiteneder C, Eidenberger H (1999) Content-based image retrieval of coats of arms. In: Proc. IEEE International Workshop on Multimedia Signal Processing, Helsingör, pp 91–96

  3. Chang SF, Sikora T, Puri A (2001) Overview of the MPEG-7 standard. Special issue on MPEG-7. IEEE Trans Circuits Syst Video Technol 11(6):688–695

    Article  Google Scholar 

  4. Computer Vision Image Library, http://www-2.cs.cmu.edu/~cil/v-images.html (last visited 2005–03–29)

  5. Cowan G (1998) Statistical data analysis. Oxford University Press, Oxford, UK

    Google Scholar 

  6. Del Bimbo A (1999) Visual information retrieval. Morgan Kaufmann, San Francisco

    Google Scholar 

  7. Edwards JD, Riley KJ, Eakins JP (2003) A technique for mapping irregular sized vectors applied to image collections. In: Proc. SPIE visual communications and image processing conference, vol 5150. Lugano, pp 467–475

  8. Eidenberger H (2003) How good are the visual MPEG-7 Features? In: Proc. SPIE visual communications and image processing conference, vol 5150. Lugano, pp 476–488

  9. Eidenberger H (2004) A new method for visual descriptor evaluation. In: Proc. SPIE storage and retrieval methods and applications for multimedia, vol 5307. San Jose, pp 145–157

  10. Eidenberger H (2004) Statistical analysis of the MPEG-7 image descriptors. In: ACM Multimedia Systems Journal, vol 10, no 2. Springer, Berlin Heidelberg New York, pp 84–97

  11. Eidenberger H, Breiteneder C (2003) VizIR—a framework for visual information retrieval. J Vis Lang Comput, Elsevier 14(5):443–469

    Article  Google Scholar 

  12. Fuhr N (2001) Information retrieval methods for multimedia objects. In: Veltkamp RC, Burkhardt H, Kriegel, HP (eds) State-of-the-art in content-based image and video retrieval. Kluwer, Boston, pp 191–212

    Google Scholar 

  13. Izquierdo E, Casas JR, Leonardi R, Migliorati P, O’Connor NE, Kompatsiaris I, Strintzis MG (2003) Advanced content-based semantic scene analysis and information retrieval: the SCHEMA project. In: Proc. workshop on image analysis for multimedia interactive services, London, pp 519–528

  14. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Comput Surv 31(3):264–323

    Article  Google Scholar 

  15. Kohonen T (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  16. Kohonen T, Oja E, Simula O, Visa A, Kangas J (1996) Engineering applications of the self-organizing map. Proc IEEE 84(10):1358–1384

    Article  Google Scholar 

  17. Koikkalainen P, Oja E (1990) Self-organising hierarchical feature maps. In: Proc. neural networks conference. San Diego, pp 279–284

  18. Lew MS (ed) (2003) Principles of visual information retrieval. Springer, Berlin Heidelberg New York

  19. Loehlin JC (2001) Latent variable models: an introduction to factor. Path and structural analysis, 3rd edn. Erlbaum, Mahwah, NJ

    Google Scholar 

  20. Manjunath BS, Ohm JR, Vasudevan VV, Yamada A (2001) Color and texture descriptors. Special issue on MPEG-7. IEEE Trans Circuits Syst Video Technol 11(6):703–715

    Article  Google Scholar 

  21. Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG-7. Wiley, San Francisco

    Google Scholar 

  22. Marques O, Furht B (2002) Content-based image and video retrieval. Kluwer, Boston

    MATH  Google Scholar 

  23. Müller H, Müller W, Squire DMcG, Marchand-Maillet S, Pun T (2001) Performance evaluation in content-based image retrieval: overview and proposals. Special issue on image and video indexing. Pattern Recogn Lett 22(5):593–601

    Article  MATH  Google Scholar 

  24. Smeaton AF, Over P, Kraaij W (2004) TRECVID: evaluating the effectiveness of information retrieval tasks on digital video. In: Proceedings ACM Multimedia Conference, New York, pp 652–655

  25. Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380

    Article  Google Scholar 

  26. University of California Berkeley Corel Dataset Website, http://elib.cs.berkeley.edu/photos/corel/ (last visited 2005–03–29)

  27. Vesanto J, Himberg J, Alhoniemi E, Parhankangas J (1999) Self-organizing map in Matlab: the SOM toolbox. In: Proc. Matlab DSP Conference, Espoo, pp 35–40

Download references

Author information

Authors and Affiliations

  1. Institute of Software Technology and Interactive Systems, Vienna University of Technology, Favoritenstrasse 9-11, 1040, Vienna, Austria

    Horst Eidenberger

Authors
  1. Horst Eidenberger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Horst Eidenberger.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Eidenberger, H. Evaluation of content-based image descriptors by statistical methods. Multimed Tools Appl 35, 241–258 (2007). https://doi.org/10.1007/s11042-007-0106-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-007-0106-y

Keywords

Search

Navigation