Non-parametric Performance Comparison in Pictorial Query by Content Systems

  • Sergio Domínguez
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3211)


In this paper the author addresses the problem of performance comparison between CBIR systems. The main objective is the introduction of a new non-parametric method for this task. This method has different advantages that are herein explained, since performance evaluation can be stablished on a query-by-query or an averaged basis; it is straigthforwardly computed; can be easily shown in a graphic, including information for different sizes of the retrieval set; and can be easily interpreted, yielding a fast and clear idea of the comparative performance of the CBIR systems being analyzed. Moreover, it can overcome the drawbacks of the precision vs. recall graphics qualitative comparison.


Relevant Element Cardinality Measure CBIR System Retrieval Test IEEE CVPR 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hull, D.: Using statistical testing in the evaluation of retrieval experiments. In: Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval (1993)Google Scholar
  2. 2.
    Muller, H., Muller, W., Pun, T.: Automated Benchmarking in Content-Based Image Retrieval. Technical Report 01.01, Universite de Geneve, Centre Universitaire D’Informatique, Groupe Vision (January 2001) Google Scholar
  3. 3.
    Muller, H., Muller, W., Squire, D.M.C.G.: Performance Evaluation in Content-Based IMage Retrieval: Overview and Proposals. Technical Report 99.05, Universite de Geneve, Centre Universitaire D’Informatique, Groupe Vision (December 1999)Google Scholar
  4. 4.
    Muller, H., Muller, W., Squire, D.M.C.G.: Performance evaluation in content-based image retrieval: overview and proposals. Pattern Recognition Letters 22, 593–601 (2001)CrossRefGoogle Scholar
  5. 5.
    Smith, J.R.: Image retrieval evaluation (1998) Google Scholar
  6. 6.
    Wenyin, L., Su, Z., Li, S., Sun, Y.-F., Zhang, H.: A performance evaluation protocol for content-based image retrieval algorithms/systems. In: IEEE CVPR Workshop on Empirical Evaluation Methods in Computer Vision (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Sergio Domínguez
    • 1
  1. 1.DISAMUniversidad Politecnica de Madrid 

Personalised recommendations