Non-parametric Performance Comparison in Pictorial Query by Content Systems
In this paper the author addresses the problem of performance comparison between CBIR systems. The main objective is the introduction of a new non-parametric method for this task. This method has different advantages that are herein explained, since performance evaluation can be stablished on a query-by-query or an averaged basis; it is straigthforwardly computed; can be easily shown in a graphic, including information for different sizes of the retrieval set; and can be easily interpreted, yielding a fast and clear idea of the comparative performance of the CBIR systems being analyzed. Moreover, it can overcome the drawbacks of the precision vs. recall graphics qualitative comparison.
KeywordsRelevant Element Cardinality Measure CBIR System Retrieval Test IEEE CVPR
Unable to display preview. Download preview PDF.
- 1.Hull, D.: Using statistical testing in the evaluation of retrieval experiments. In: Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval (1993)Google Scholar
- 2.Muller, H., Muller, W., Pun, T.: Automated Benchmarking in Content-Based Image Retrieval. Technical Report 01.01, Universite de Geneve, Centre Universitaire D’Informatique, Groupe Vision (January 2001) Google Scholar
- 3.Muller, H., Muller, W., Squire, D.M.C.G.: Performance Evaluation in Content-Based IMage Retrieval: Overview and Proposals. Technical Report 99.05, Universite de Geneve, Centre Universitaire D’Informatique, Groupe Vision (December 1999)Google Scholar
- 5.Smith, J.R.: Image retrieval evaluation (1998) Google Scholar
- 6.Wenyin, L., Su, Z., Li, S., Sun, Y.-F., Zhang, H.: A performance evaluation protocol for content-based image retrieval algorithms/systems. In: IEEE CVPR Workshop on Empirical Evaluation Methods in Computer Vision (2001)Google Scholar