On Performance Characterization and Optimization for Image Retrieval
In content-based image retrieval (CBIR) performance characterization is easily being neglected. A major difficulty lies in the fact that ground truth and the definition of benchmarks are extremely user and application dependent. This paper proposes a two-stage CBIR framework which allows to predict the behavior of the retrieval system as well as to optimize its performance. In particular, it is possible to maximize precision, recall, or jointly precision and recall. The framework is based on the detection of high-level concepts in images. These concepts correspond to vocabulary users can query the database with. Performance optimization is carried out on the basis of the user query, the performance of the concept detectors, and an estimated distribution of the concepts in the database. The optimization is transparent to the user and leads to a set of internal parameters that optimize the succeeding retrieval. Depending only on the query and the desired concept, precision and recall of the retrieval can be increased by up to 40%. The paper discusses the theoretical and empirical results of the optimization as well as its dependency on the estimate of the concept distribution.
KeywordsPerformance Prediction Image Retrieval User Query Image Detector Vision Algorithm
Unable to display preview. Download preview PDF.
- 1.R. Haralick. Computer vision theory: The lack thereof. In IEEE Computer Society Third Workshop on Computer Vision: Representation and Control, (1985) 113–121Google Scholar
- 2.K.E. Price. I’ve seen your demo: so what? In IEEE Computer Society Third Workshop on Computer Vision: Representation and Control, (1985) 122–124Google Scholar
- 3.E.M. Voorhees and D.K. Harman, editors. NIST Special Publication 500-246: The Eighth Text REtrieval Conference (TREC 8). Department of Commerce, National Institute of Standards and Technology, (1999)Google Scholar
- 4.P.J. Phillips, H. Moon, P.J. Rauss, and S. Rizvi. The FERET evaluation methodology for face recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (2000).Google Scholar
- 5.First IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS’2000), Grenoble, France (2000)Google Scholar
- 6.H.I. Christensen, W. Förstner, and C.B. Madsen, editors. Workshop on Performance Characteristics of Vision Algorithms, Cambridge, United Kingdom (1996)Google Scholar
- 7.R. Haralick, R. Klette, S. Stiehl, and M. Viergever. Evaluation and validation of computer vision algorithms. http://www.dagstuhl.de/data/seminars/98/, Dagstuhl Seminar No. 98111 (1998).
- 8.K.W. Bowyer and P.J. Phillips, editors. Empirical Evaluation Techniques in Computer Vision. IEEE Computer Society Press (1998)Google Scholar
- 9.A. Clark and P Courtney, editors. Workshop on Performance Characterisation and Benchmarking of Vision Systems, Las Palmas, Spain (1999)Google Scholar
- 10.P. Courtney and N.A. Thacker. Performance characterisation in computer vision: The role of statistics in testing and design. In J. Blanc-Talon and D. Popescu, editors, Imaging and Vision Systems: Theory, Assessment and Applications. NOVA Science Books (2001)Google Scholar
- 11.W. Förstner. 10 pro’s and con’s against performance characterization of vision algorithms. In Workshop on Performance Characteristics of Vision Algorithms, Cambridge, United Kingdom (1996).Google Scholar
- 14.John R. Smith. Image retrieval evaluation. In IEEE Workshop on Content-based Access of Image and Video Libraries, Santa Barbara, California (1998)Google Scholar
- 16.B. Moghaddam and A. Pentland. Probabilistic visual learning for object representation. Pattern Recognition and Machine Intelligence, 19 (1997)Google Scholar
- 17.C. Papageorgiou, M. Oren, and T. Poggio. A general framework for object detection. In International Conference on Computer Vision ICCV, pages Bombay, India (1998) 555–562Google Scholar
- 18.C.P. Town and D. Sinclair. Content based image retrieval using semantic visual categories. Technical Report 2000.14, AT&T Laboratories Cambridge (2000).Google Scholar
- 19.B. Schiele and J. Vogel. Vocabulary-supported image retrieval. In First DELOS Workshop on Information Seeking, Searching and Querying in Digital Libraries, Zurich, Switzerland (2000)Google Scholar
- 20.J. Vogel and B. Schiele. Performance prediction for vocabulary-supported image retrieval. In International Conference on Image Processing ICIP (2001)Google Scholar