A Visual Category Filter for Google Images

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3021)


We extend the constellation model to include heterogeneous parts which may represent either the appearance or the geometry of a region of the object. The parts and their spatial configuration are learnt simultaneously and automatically, without supervision, from cluttered images.

We describe how this model can be employed for ranking the output of an image search engine when searching for object categories. It is shown that visual consistencies in the output images can be identified, and then used to rank the images according to their closeness to the visual object category.

Although the proportion of good images may be small, the algorithm is designed to be robust and is capable of learning in either a totally unsupervised manner, or with a very limited amount of supervision.

We demonstrate the method on image sets returned by Google’s image search for a number of object categories including bottles, camels, cars, horses, tigers and zebras.


Training Image Unsupervised Learning Object Category Image Search Curve Segment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Amit, Y., Geman, D.: A computational model for visual selection. Neural Computation 11(7), 1691–1715 (1999)CrossRefGoogle Scholar
  2. 2.
    Bach, J., Fuller, C., Humphrey, R., Jain, R.: The virage image search engine: An open framework for image management. In: SPIE Conf. on Storage and Retrieval for Image and Video Databases, vol. 2670, pp. 76–87 (1996)Google Scholar
  3. 3.
    Borenstein, E., Ullman, S.: Class-specific, top-down segmentation. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 109–124. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Brin, S., Page, L.: The anatomy of a large-scale hypertextual web search engine. In: 7th Int. WWW Conference (1998)Google Scholar
  5. 5.
    Burl, M., Leung, T., Perona, P.: A probabilistic approach to object recognition using local photometry and global geometry. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, p. 628. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  6. 6.
    Canny, J.F.: A computational approach to edge detection. IEEE PAMI 8(6), 679–698 (1986)Google Scholar
  7. 7.
    Deselaers, T., Keysers, D., Ney, H.: Clustering visually similar images to improve image search engines. In: Informatiktage 2003 der Gesellschaft für Informatik, Bad Schussenried, Germany (2003)Google Scholar
  8. 8.
    Fei-Fei, L., Fergus, R., Perona, P.: A bayesian approach to unsupervised one-shot learning of object categories. In: Proceedings of the 9th International Conference on Computer Vision, Nice, France, pp. 1134–1141 (2003)Google Scholar
  9. 9.
    Felzenszwalb, P., Huttenlocher, D.: Pictorial structures for object recognition. In: Proc. CVPR, pp. 2066–2073 (2000)Google Scholar
  10. 10.
    Fergus, R., Perona, P., Zisserman, A.: Object class recognition by unsupervised scaleinvariant learning. In: Proc. CVPR (2003)Google Scholar
  11. 11.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24(6), 381–395 (1981)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Gevers, T., Smeulders, A.W.M.: Content-based image retrieval by viewpoint-invariant color indexing. Image and vision computing 17, 475–488 (1999)CrossRefGoogle Scholar
  13. 13.
    Heisele, B., Serre, T., Pontil, M., Vetter, T., Poggio, T.: Categorization by learning and combining object parts. In: Advances in Neural Information Processing Systems 14, Vancouver, Canada, vol. 2, pp. 1239–1245 (2002)Google Scholar
  14. 14.
    Kadir, T., Brady, M.: Scale, saliency and image description. IJCV 45(2), 83–105 (2001)zbMATHCrossRefGoogle Scholar
  15. 15.
    Leibe, B., Schiele, B.: Analyzing appearance and contour based methods for object categorization. In: Proc. CVPR (2003)Google Scholar
  16. 16.
    Riesenhuber, M., Poggio, T.: Hierarchical models of object recognition in cortex. Nature Neuroscience 2(11), 1019–1025 (1999)CrossRefGoogle Scholar
  17. 17.
    Rothwell, C., Zisserman, A., Forsyth, D., Mundy, J.: Planar object recognition using projective shape representation. IJCV 16(2) (1995)Google Scholar
  18. 18.
    Schmid, C.: Constructing models for content-based image retrieval. In: Proc. CVPR, vol. 2, pp. 39–45 (2001)Google Scholar
  19. 19.
    Tong, S., Chang, E.: Support vector machine active learning for image retrieval. ACM Multimedia (2001)Google Scholar
  20. 20.
    Vasconcelos, N., Lippman, A.: Learning from user feedback in image retrieval systems. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 33–47. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  21. 21.
    Veltkamp, R., Tanase, M.: Content-based image retrieval systems: A survey. Technical Report UU-CS-2000-34, Department of Computing Science, Utrecht University (2000)Google Scholar
  22. 22.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proc. CVPR, pp. 511–518 (2001)Google Scholar
  23. 23.
    Weber, M., Welling, M., Perona, P.: Unsupervised learning of models for recognition. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 18–32. Springer, Heidelberg (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  1. 1.Dept. of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.Dept. of Electrical EngineeringCalifornia Institute of TechnologyPasadenaU.S.A.

Personalised recommendations