Advertisement

Semantic Texton Forests

  • Matthew Johnson
  • Jamie Shotton
Part of the Studies in Computational Intelligence book series (SCI, volume 285)

Abstract

The semantic texton forest is an efficient and powerful low-level feature which can be effectively employed in the semantic segmentation of images. As ensembles of decision trees that act directly on image pixels, semantic texton forests do not need the expensive computation of filter-bank responses or local descriptors. They are extremely fast to both train and test, especially compared with k-means clustering and nearest-neighbor assignment of feature descriptors. The nodes in the trees provide (i) an implicit hierarchical clustering into semantic textons, and (ii) an explicit local classification estimate. The bag of semantic textons combines a histogram of semantic textons over an image region with a region prior category distribution. The bag of semantic textons can be used by an SVM classifier to infer an image-level prior over categories, allowing the segmentation to emphasize those categories that the SVM believes to be present. We will examine the segmentation performance of semantic texton forests on two datasets including the VOC 2007 segmentation challenge.

Keywords

Support Vector Machine Computer Vision Leaf Node Visual Word Training Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9(7), 1545–1588 (1997)CrossRefGoogle Scholar
  2. 2.
    Bishop, C.: Pattern Recognition and Machine Learning. Springer-Verlag New York, Inc. (2006)Google Scholar
  3. 3.
    Bosch, A., Zisermann, A., Muñoz, X.: Image classification using random forests and ferns. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  4. 4.
    Breiman, L.: Random forests. Machine Learning 45(1), 5–32 (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Breiman, L., Friedman, J., Olshen, R.: Classification and Regression Trees. Wadsworth, Belmont (1984)zbMATHGoogle Scholar
  6. 6.
    Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Proceedings of the International Workshop on Statistical Learning in Computer Vision, ECCV (2004)Google Scholar
  7. 7.
    Elkan, C.: Using the triangle inequality to accelerate k-means. In: Proceedings of the International Conference on Machine Learning, pp. 147–153 (2003)Google Scholar
  8. 8.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL VOC Challenge (2007), http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html
  9. 9.
    Fei-Fei, L., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2005)Google Scholar
  10. 10.
    Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Machine Learning 36(1), 3–42 (2006)CrossRefGoogle Scholar
  11. 11.
    Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: Proceedings of the International Conference on Computer Vision (2005)Google Scholar
  12. 12.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, New Jersey (1989)zbMATHGoogle Scholar
  13. 13.
    Jurie, F., Triggs, B.: Creating efficient codebooks for visual recognition. In: Proceedings of the International Conference on Computer Vision, pp. 604–610 (2005)Google Scholar
  14. 14.
    Lasserre, J., Kannan, A., Winn, J.: Hybrid learning of large jigsaws. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, Minneapolis (2007)Google Scholar
  15. 15.
    Lepetit, V., Lagger, P., Fua, P.: Randomized trees for real-time keypoint recognition. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 775–781 (2005)Google Scholar
  16. 16.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Malik, J., Belongie, S., Leung, T., Shi, J.: Contour and texture analysis for image segmentation. International Journal of Computer Vision 43(1), 7–27 (2001)zbMATHCrossRefGoogle Scholar
  18. 18.
    Marée, R., Geurts, P., Piater, J., Wehenkel, L.: Random subwindows for robust image classification. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 34–40 (2005)Google Scholar
  19. 19.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. International Journal of Computer Vision 60(1), 63–86 (2004)CrossRefGoogle Scholar
  20. 20.
    Moosmann, F., Triggs, B., Jurie, F.: Fast discriminative visual codebooks using randomized clustering forests. In: Proceedings of the International Conference on Neural Information Processing Systems (2006)Google Scholar
  21. 21.
    Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  22. 22.
    Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: Proceedings of the International Conference on Computer Vision (2006)Google Scholar
  23. 23.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)zbMATHCrossRefGoogle Scholar
  24. 24.
    Oliva, A., Torralba, A.: Building the gist of a scene: The role of global image features in recognition. Visual Perception, Progress in Brain Research 155(1), 23–26 (2006)Google Scholar
  25. 25.
    Quelhas, P., Monay, F., Odobez, J.M., Gatica, D., Tuytelaars, T.: Modeling scenes with local descriptors and latent aspects. In: Proceedings of the International Conference on Computer Vision (2005)Google Scholar
  26. 26.
    Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., Belongie, S.: Objects in context. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  27. 27.
    Russell, B., Torralba, A., Murphy, K., Freeman, W.T.: Labelme: a database and web-based tool for image annotation. Journal of Computer Vision 77(1-3), 157–173 (2008)CrossRefGoogle Scholar
  28. 28.
    Russell, B.C., Efros, A.A., Sivic, J., Freeman, W.T., Zisserman, A.: Using multiple segmentations to discover objects and their extent in image collections. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  29. 29.
    Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, Minneapolis (2007)Google Scholar
  30. 30.
    Shotton, J., Winn, J., Rother, C., Criminisi, A.: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. International Journal of Computer Vision 81(1) (2009)Google Scholar
  31. 31.
    Sivic, J., Russel, B., Efros, A., Zisserman, A., Freeman, W.: Discovering objects and their localization in images. In: Proceedings of the International Conference on Computer Vision, Beijing, China, pp. 370–377 (2005)Google Scholar
  32. 32.
    Sivic, J., Zisserman, A.: Video Google: A text retrieval approach to object matching in videos. In: Proceedings of the International Conference on Computer Vision, vol. 2, pp. 1470–1477 (2003)Google Scholar
  33. 33.
    Swain, M., Ballard, D.: Color indexing. Int. J. Computer Vision 7, 11–32 (1991)CrossRefGoogle Scholar
  34. 34.
    Tuytelaars, T., Schmid, C.: Vector quantizing feature space with a regular lattice. In: Proceedings of the International Conference on Computer Vision (2007)Google Scholar
  35. 35.
    Varma, M., Zisserman, A.: A statistical approach to texture classification from single images. International Journal of Computer Vision 62(1-2), 61–81 (2005)CrossRefGoogle Scholar
  36. 36.
    Verbeek, J., Triggs, B.: Region classification with markov field aspect models. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  37. 37.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 511–518 (2001)Google Scholar
  38. 38.
    Winder, S., Brown, M.: Learning local image descriptors. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  39. 39.
    Winn, J., Criminisi, A., Minka, T.: Object categorization by learned universal visual dictionary. In: Proceedings of the International Conference on Computer Vision, Beijing, China, pp. 1800–1807 (2005)Google Scholar
  40. 40.
    Zhang, J., Marszałek, M., Lazebnik, S., Schmid, C.: Local features and kernels for classificaiton of texture and object categories: A comprehensive study. International Journal of Computer Vision 73(2), 213–238 (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Matthew Johnson
    • 1
  • Jamie Shotton
    • 2
  1. 1.NokiaSan FranciscoUSA
  2. 2.Microsoft ResearchCambridgeUK

Personalised recommendations