Skip to main content
SpringerLink
Log in

Decision Tree Fields: An Efficient Non-parametric Random Field Model for Image Labeling

  • Chapter
Decision Forests for Computer Vision and Medical Image Analysis

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

This chapter introduces a new random field model for discrete image labeling tasks, the Decision Tree Field (DTF), that combines and generalizes decision forests and conditional random fields (CRF) which have been widely used in computer vision.

In a typical CRF model the unary potentials are derived from sophisticated forest or boosting-based classifiers, however, the pairwise potentials are assumed to (1) have a simple parametric form with a pre-specified and fixed dependence on the image data, and (2) to be defined on the basis of a small and fixed neighborhood. In contrast, in DTF, local interactions between multiple variables are determined by means of decision trees evaluated on the image data, allowing the interactions to be adapted to the image content.

This results in powerful graphical models which are able to represent complex label structure.

Our key technical contribution is to show that the DTF model can be trained efficiently and jointly using a convex approximate likelihood function, enabling us to learn over a million free model parameters.

We show experimentally that for applications which have a rich and complex label structure, our model achieves excellent results. Parts of this chapter are reprinted, with permission, from Nowozin et al., Proc. IEEE Intl. Conf. on Computer Vision (ICCV) (2011), © 2011 IEEE.

Parts of this chapter are reprinted, with permission, from [271], © 2011 IEEE.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amit Y, Geman D (1997) Shape quantization and recognition with randomized trees. Neural Comput 9(7)

    Google Scholar 

  2. Anguelov D, Taskar B, Chatalbashev V, Koller D, Gupta D, Ng A (2005) Discriminative learning of Markov random fields for segmentation of 3D scan data. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  3. Batra D, Sukthankar R, Chen T (2008) Learning class-specific affinities for image labelling. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  4. Besag J (1977) Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika

    Google Scholar 

  5. Blake A, Rother C, Brown M, Perez P, Torr PHS (2004) Interactive image segmentation using an adaptive GMMRF model. In: Pajdla T, Matas J (eds) Proc European conf on computer vision (ECCV), Prague, Czech Republic, May 2004. LNCS, vol 3021. Springer, Berlin

    Google Scholar 

  6. Boykov Y, Jolly M-P (2001) Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In: Proc IEEE intl conf on computer vision (ICCV), Vancouver, Canada, July 2001, vol 1

    Google Scholar 

  7. Breiman L (2001) Random forests. Mach Learn 45(1)

    Google Scholar 

  8. Cho TS, Joshi N, Zitnick CL, Kang SB, Szeliski R, Freeman WT (2010) A content-aware image prior. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  9. Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6

    Google Scholar 

  10. Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 36(1)

    Google Scholar 

  11. Glesner S, Koller D (1995) Constructing flexible dynamic belief networks from first-order probabilistic knowledge bases. In: ECSQARU

    Google Scholar 

  12. Gould S, Fulton R, Koller D (2009) Decomposing a scene into geometric and semantically consistent regions. In: Proc IEEE intl conf on computer vision (ICCV)

    Google Scholar 

  13. He X, Zemel RS, Carreira-Perpiñán MÁ (2004) Multiscale conditional random fields for image labeling. In: Proc IEEE conf computer vision and pattern recognition (CVPR), June 2004, vol 2

    Google Scholar 

  14. Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge

    Google Scholar 

  15. Kolmogorov V (2006) Convergent tree-reweighted message passing for energy minimization. IEEE Trans Pattern Anal Mach Intell 28(10)

    Google Scholar 

  16. Kolmogorov V, Boykov Y (2005) What metrics can be approximated by geo-cuts, or global optimization of length/area and flux. In: Proc IEEE intl conf on computer vision (ICCV)

    Google Scholar 

  17. Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proc intl conf on machine learning (ICML)

    Google Scholar 

  18. Lee H, Grosse R, Ranganath R, Ng AY (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proc intl conf on machine learning (ICML)

    Google Scholar 

  19. Li SZ (1995) Markov random field modeling in computer vision. Springer, Berlin

    Google Scholar 

  20. Nowozin S, Lampert CH (2009) Global connectivity potentials for random field models. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  21. Nowozin S, Lampert CH (2011) Structured learning and prediction in computer vision. Found Trends Comput Graph Vis 6(3–4)

    Google Scholar 

  22. Nowozin S, Rother C, Bagon S, Sharp T, Yao B, Kohli P (2011) Decision tree fields. In: Proc IEEE intl conf on computer vision (ICCV)

    Google Scholar 

  23. Payet N, Todorovic S (2010) (RF)2—random forest random field. In: Advances in neural information processing systems (NIPS)

    Google Scholar 

  24. Prasad M, Zisserman A, Fitzgibbon AW, Kumar MP, Torr PHS (2006) Learning class-specific edges for object detection and segmentation. In: ICVGIP

    Google Scholar 

  25. Roth S, Black MJ (2007) Steerable random fields. In: Proc IEEE intl conf on computer vision (ICCV)

    Google Scholar 

  26. Rother C, Kolmogorov V, Blake A (2004) GrabCut—interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3)

    Google Scholar 

  27. Schnitzspan P, Roth S, Schiele B (2010) Automatic discovery of meaningful object parts with latent CRFs. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  28. Sharp T (2008) Implementing decision trees and forests on a GPU. In: Proc European conf on computer vision (ECCV). Springer, Berlin

    Google Scholar 

  29. Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  30. Shotton J, Winn JM, Rother C, Criminisi A (2009) TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81(1)

    Google Scholar 

  31. Shotton J, Fitzgibbon AW, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from a single depth image. In: Proc IEEE conf computer vision and pattern recognition (CVPR)

    Google Scholar 

  32. Sutton C, McCallum A (2006) An introduction to conditional random fields for relational learning. MIT Press, Cambridge. Chap 4

    Google Scholar 

  33. Szeliski R, Zabih R, Scharstein D, Veksler O, Kolmogorov V, Agarwala A, Tappen ML, Rother C (2008) A comparative study of energy minimization methods for Markov random fields with smoothness-based priors. IEEE Trans Pattern Anal Mach Intell 30(7)

    Google Scholar 

  34. Szummer M, Kohli P, Hoiem D (2008) Learning CRFs using graph cuts. In: Proc European conf on computer vision (ECCV). Springer, Berlin

    Google Scholar 

  35. Taskar B, Chatalbashev V, Koller D, Guestrin C (2005) Learning structured prediction models: a large margin approach. In: Proc intl conf on machine learning (ICML)

    Google Scholar 

  36. Tu Z, Bai X (2010) Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans Pattern Anal Mach Intell 32(10)

    Google Scholar 

  37. Vishwanathan SVN, Schraudolph NN, Schmidt MW, Murphy KP (2006) Accelerated training of conditional random fields with stochastic gradient methods. In: Proc intl conf on machine learning (ICML)

    Google Scholar 

  38. Wainwright MJ, Jordan MI (2008) Graphical models, exponential families, and variational inference. Found Trends Mach Learn 1(1–2)

    Google Scholar 

  39. Zhu C, Byrd RH, Lu P, Nocedal J (1997) Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw 23(4)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, Cambridge, UK

    S. Nowozin, C. Rother, T. Sharp & P. Kohli

  2. Weizmann Institute of Science, Rehovot, Israel

    S. Bagon

  3. Stanford University, Stanford, USA

    B. Yao

Authors
  1. S. Nowozin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Rother
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Bagon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. T. Sharp
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. B. Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. P. Kohli
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Microsoft Research Ltd., 7 J.J. Thomson Avenue, Cambridge, CB3 0FB, United Kingdom

    A. Criminisi

  2. Microsoft Research Ltd., 7 J.J. Thomson Avenue, Cambridge, CB3 0FB, United Kingdom

    J. Shotton

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Nowozin, S., Rother, C., Bagon, S., Sharp, T., Yao, B., Kohli, P. (2013). Decision Tree Fields: An Efficient Non-parametric Random Field Model for Image Labeling. In: Criminisi, A., Shotton, J. (eds) Decision Forests for Computer Vision and Medical Image Analysis. Advances in Computer Vision and Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-4471-4929-3_20

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4929-3_20

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4928-6

  • Online ISBN: 978-1-4471-4929-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation