Single-Histogram Class Models for Image Segmentation

  • F. Schroff
  • A. Criminisi
  • A. Zisserman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4338)


Histograms of visual words (or textons) have proved effective in tasks such as image classification and object class recognition. A common approach is to represent an object class by a set of histograms, each one corresponding to a training exemplar. Classification is then achieved by k-nearest neighbour search over the exemplars.

In this paper we introduce two novelties on this approach: (i) we show that new compact single histogram models estimated optimally from the entire training set achieve an equal or superior classification accuracy. The benefit of the single histograms is that they are much more efficient both in terms of memory and computational resources; and (ii) we show that bag of visual words histograms can provide an accurate pixel-wise segmentation of an image into object class regions. In this manner the compact models of visual object classes give simultaneous segmentation and recognition of image regions.

The approach is evaluated on the MSRC database [5] and it is shown that performance equals or is superior to previous publications on this database.


Image Segmentation Visual Word Training Image Object Class Colour Patch 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Borenstein, E., Ullman, S.: Class-specific, top-down segmentation. In: Proc. ECCV, pp. 109–124 (2002)Google Scholar
  2. 2.
    Borenstein, E., Ullman, S.: Learning to segment. In: Proc. ECCV, vol. 3, pp. 315–328 (2004)Google Scholar
  3. 3.
    Bosch, A., Zisserman, A., Munoz, X.: Scene classification via pLSA. In: Proc. ECCV (2006)Google Scholar
  4. 4.
    Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: Proc. ICCV, vol. 2, pp. 105–112 (2001)Google Scholar
  5. 5.
    Criminisi, A.: Microsoft research cambridge object recognition image database. version 1.0 (2004),
  6. 6.
    Csurka, G., Bray, C., Dance, C., Fan, L.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, pp. 1–22 (2004)Google Scholar
  7. 7.
    Dhillon, I., Mallela, S., Kumar, R.: A divisive information-theoretic feature clustering algorithm for text classification. J. Machine Learning Research (2003)Google Scholar
  8. 8.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: Proc. CVPR (June 2005)Google Scholar
  9. 9.
    He, X., Zemel, R.S., Carreira-Perpiñán, M.Á.: Multiscale conditional random fields for image labeling. In: Proc. CVPR (2004)Google Scholar
  10. 10.
    Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Machine Learning 43, 177–196 (2001)CrossRefGoogle Scholar
  11. 11.
    Kumar, M.P., Torr, P.H.S., Zisserman, A.: OBJ CUT. In: Proc. CVPR (2005)Google Scholar
  12. 12.
    Leibe, B., Schiele, B.: Interleaved object categorization and segmentation. In: BMVC 2003, vol. 2, pp. 264–271 (2003)Google Scholar
  13. 13.
    Leung, T., Malik, J.: Representing and recognizing the visual appearance of materials using three-dimensional textons. IJCV 43(1), 29–44 (2001)MATHCrossRefGoogle Scholar
  14. 14.
    Quelhas, P., Monay, F., Odobez, J.-M., Gatica, D., Tuytelaars, T., Van Gool, L.: Modeling scenes with local descriptors and latent aspects. In: Proc. ICCV, pp. 883–890 (2005)Google Scholar
  15. 15.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: interactive foreground extraction using iterated graph cuts. Proc. ACM SIGGRAPH 23(3), 309–314 (2004)CrossRefGoogle Scholar
  16. 16.
    Sharon, E., Brandt, A., Basri, R.: Segmentation and boundary detection using multiscale intensity measurements. In: Proc. CVPR, vol. 1, pp. 469–476. IEEE Computer Society Press, Los Alamitos (2001)Google Scholar
  17. 17.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: TextonBoost: Joint Appearance, Shape and context Modeling for Multi-Class Object Recognition and Segmentation. In: Proc. ECCV (2006)Google Scholar
  18. 18.
    Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering object categories in image collections. In: Proc. ICCV (2005)Google Scholar
  19. 19.
    Spellman, E., Vemuri, B.C., Rao, M.: Using the KL-center for Efficient and Accurate Retrieval of Distributions Arising from Texture Images. In: Proc. CVPR (2005)Google Scholar
  20. 20.
    Varma, M., Zisserman, A.: Texture classification: Are filter banks necessary? In: Proc. CVPR, June 2003, vol. 2, pp. 691–698 (2003)Google Scholar
  21. 21.
    Varma, M., Zisserman, A.: A statistical approach to texture classification from single images. IJCV 62(1–2), 61–81 (2005)Google Scholar
  22. 22.
    Winn, J., Criminisi, A., Minka., T.: Object Categorization by Learned Universal Visual Dictionary. In: Proc. ICCV (2005)Google Scholar
  23. 23.
    Zhang, J., Marszalek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classifcation of texture and object categories: An in-depth study. Technical Report RR-5737, INRIA Rhône-Alpes (November 2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • F. Schroff
    • 1
  • A. Criminisi
    • 2
  • A. Zisserman
    • 1
  1. 1.Dept. of Engineering ScienceUniversity of OxfordOxfordUK
  2. 2.Microsoft Research LtdCambridgeUK

Personalised recommendations