On Performance Characterization and Optimization for Image Retrieval

  • J. Vogel
  • B. Schiele
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2353)


In content-based image retrieval (CBIR) performance characterization is easily being neglected. A major difficulty lies in the fact that ground truth and the definition of benchmarks are extremely user and application dependent. This paper proposes a two-stage CBIR framework which allows to predict the behavior of the retrieval system as well as to optimize its performance. In particular, it is possible to maximize precision, recall, or jointly precision and recall. The framework is based on the detection of high-level concepts in images. These concepts correspond to vocabulary users can query the database with. Performance optimization is carried out on the basis of the user query, the performance of the concept detectors, and an estimated distribution of the concepts in the database. The optimization is transparent to the user and leads to a set of internal parameters that optimize the succeeding retrieval. Depending only on the query and the desired concept, precision and recall of the retrieval can be increased by up to 40%. The paper discusses the theoretical and empirical results of the optimization as well as its dependency on the estimate of the concept distribution.


Performance Prediction Image Retrieval User Query Image Detector Vision Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    R. Haralick. Computer vision theory: The lack thereof. In IEEE Computer Society Third Workshop on Computer Vision: Representation and Control, (1985) 113–121Google Scholar
  2. 2.
    K.E. Price. I’ve seen your demo: so what? In IEEE Computer Society Third Workshop on Computer Vision: Representation and Control, (1985) 122–124Google Scholar
  3. 3.
    E.M. Voorhees and D.K. Harman, editors. NIST Special Publication 500-246: The Eighth Text REtrieval Conference (TREC 8). Department of Commerce, National Institute of Standards and Technology, (1999)Google Scholar
  4. 4.
    P.J. Phillips, H. Moon, P.J. Rauss, and S. Rizvi. The FERET evaluation methodology for face recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (2000).Google Scholar
  5. 5.
    First IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS’2000), Grenoble, France (2000)Google Scholar
  6. 6.
    H.I. Christensen, W. Förstner, and C.B. Madsen, editors. Workshop on Performance Characteristics of Vision Algorithms, Cambridge, United Kingdom (1996)Google Scholar
  7. 7.
    R. Haralick, R. Klette, S. Stiehl, and M. Viergever. Evaluation and validation of computer vision algorithms., Dagstuhl Seminar No. 98111 (1998).
  8. 8.
    K.W. Bowyer and P.J. Phillips, editors. Empirical Evaluation Techniques in Computer Vision. IEEE Computer Society Press (1998)Google Scholar
  9. 9.
    A. Clark and P Courtney, editors. Workshop on Performance Characterisation and Benchmarking of Vision Systems, Las Palmas, Spain (1999)Google Scholar
  10. 10.
    P. Courtney and N.A. Thacker. Performance characterisation in computer vision: The role of statistics in testing and design. In J. Blanc-Talon and D. Popescu, editors, Imaging and Vision Systems: Theory, Assessment and Applications. NOVA Science Books (2001)Google Scholar
  11. 11.
    W. Förstner. 10 pro’s and con’s against performance characterization of vision algorithms. In Workshop on Performance Characteristics of Vision Algorithms, Cambridge, United Kingdom (1996).Google Scholar
  12. 12.
    Y. Rui, Th.S. Huang, and S. Chang. Image retrieval: Current techniques, promising directions and open issues. Journal of Visual Communication and Image Representation, 10 (1999) 39–62CrossRefGoogle Scholar
  13. 13.
    A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22 (2000) 1349–1380CrossRefGoogle Scholar
  14. 14.
    John R. Smith. Image retrieval evaluation. In IEEE Workshop on Content-based Access of Image and Video Libraries, Santa Barbara, California (1998)Google Scholar
  15. 15.
    H. Müller, W. Müller, D. Squire, St. Marchand-Maillet, and Th. Pun. Performance evaluation in content-based image retrieval: overview and proposals. Pattern Recognition Letters, 22 (2001) 593–601zbMATHCrossRefGoogle Scholar
  16. 16.
    B. Moghaddam and A. Pentland. Probabilistic visual learning for object representation. Pattern Recognition and Machine Intelligence, 19 (1997)Google Scholar
  17. 17.
    C. Papageorgiou, M. Oren, and T. Poggio. A general framework for object detection. In International Conference on Computer Vision ICCV, pages Bombay, India (1998) 555–562Google Scholar
  18. 18.
    C.P. Town and D. Sinclair. Content based image retrieval using semantic visual categories. Technical Report 2000.14, AT&T Laboratories Cambridge (2000).Google Scholar
  19. 19.
    B. Schiele and J. Vogel. Vocabulary-supported image retrieval. In First DELOS Workshop on Information Seeking, Searching and Querying in Digital Libraries, Zurich, Switzerland (2000)Google Scholar
  20. 20.
    J. Vogel and B. Schiele. Performance prediction for vocabulary-supported image retrieval. In International Conference on Image Processing ICIP (2001)Google Scholar
  21. 21.
    R.O. Duda, P.E. Hart, and D.G. Stork. Pattern Classification. John Wiley & Sons, New York (2001)zbMATHGoogle Scholar
  22. 22.
    J.Z. Wang, J. Li, and G. Wiederhold. SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 (2001) 947–963CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • J. Vogel
    • 1
  • B. Schiele
    • 1
  1. 1.Perceptual Computing and Computer Vision GroupETH ZurichSwitzerland

Personalised recommendations