Implementing Decision Trees and Forests on a GPU

  • Toby Sharp
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5305)


We describe a method for implementing the evaluation and training of decision trees and forests entirely on a GPU, and show how this method can be used in the context of object recognition.

Our strategy for evaluation involves mapping the data structure describing a decision forest to a 2D texture array. We navigate through the forest for each point of the input data in parallel using an efficient, non-branching pixel shader. For training, we compute the responses of the training data to a set of candidate features, and scatter the responses into a suitable histogram using a vertex shader. The histograms thus computed can be used in conjunction with a broad range of tree learning algorithms.

We demonstrate results for object recognition which are identical to those obtained on a CPU, obtained in about 1% of the time.

To our knowledge, this is the first time a method has been proposed which is capable of evaluating or training decision trees on a GPU. Our method leverages the full parallelism of the GPU.

Although we use features common to computer vision to demonstrate object recognition, our framework can accommodate other kinds of features for more general utility within computer science.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9(7), 1545–1588 (1997)CrossRefGoogle Scholar
  2. 2.
    Breiman, L.: Random forests. ML Journal 45(1), 5–32 (2001)MATHGoogle Scholar
  3. 3.
    Lepetit, V., Fua, P.: Keypoint recognition using randomized trees. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006)CrossRefGoogle Scholar
  4. 4.
    Ozuysal, M., Fua, P., Lepetit, V.: Fast keypoint recognition in ten lines of code. In: IEEE CVPR (2007)Google Scholar
  5. 5.
    Winn, J., Criminisi, A.: Object class recognition at a glance. In: IEEE CVPR, video track (2006)Google Scholar
  6. 6.
    Shotton, J., Johnson, M., Cipolla, R.: Semantic texton forests for image categorization and segmentation. In: IEEE CVPR, Anchorage (2008)Google Scholar
  7. 7.
    Yin, P., Criminisi, A., Winn, J.M., Essa, I.A.: Tree-based classifiers for bilayer video segmentation. In: CVPR (2007)Google Scholar
  8. 8.
    Bosh, A., Zisserman, A., Munoz, X.: Image classification using random forests and ferns. In: IEEE ICCV (2007)Google Scholar
  9. 9.
    Apostolof, N., Zisserman, A.: Who are you? - real-time person identification. In: BMVC (2007)Google Scholar
  10. 10.
    Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: CVPR, vol. (1), pp. 211–220 (2003)Google Scholar
  11. 11.
    Brunton, A., Shu, C., Roth, G.: Belief propagation on the gpu for stereo vision. In: CRV, p. 76 (2006)Google Scholar
  12. 12.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. International Journal of Computer Vision 70(1), 41–54 (2006)CrossRefGoogle Scholar
  13. 13.
    Steinkraus, D., Buck, I., Simard, P.: Using gpus for machine learning algorithms. In: Proceedings of Eighth International Conference on Document Analysis and Recognition, 2005, 29 August-1 September 2005, vol. 2, pp. 1115–1120 (2005)Google Scholar
  14. 14.
    Winn, J.M., Criminisi, A., Minka, T.P.: Object categorization by learned universal visual dictionary. In: ICCV, pp. 1800–1807 (2005)Google Scholar
  15. 15.
    Viola, P.A., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  16. 16.
    Shotton, J., Winn, J.M., Rother, C., Criminisi, A.: TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 1–15. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Deselaers, T., Criminisi, A., Winn, J.M., Agarwal, A.: Incorporating on-demand stereo for real time recognition. In: CVPR (2007)Google Scholar
  18. 18.
    James, G., O’Rorke, J.: Real-time glow. In: GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics, pp. 343–362. Addison-Wesley, Reading (2004)Google Scholar
  19. 19.
    Blelloch, G.E.: Prefix sums and their applications. Technical Report CMU-CS-90-190, School of Computer Science, Carnegie Mellon University (November 1990)Google Scholar
  20. 20.
    Hensley, J., Scheuermann, T., Coombe, G., Singh, M., Lastra, A.: Fast summed-area table generation and its applications. Comput. Graph. Forum 24(3), 547–555 (2005)CrossRefGoogle Scholar
  21. 21.
    Quinlan, J.R.: Induction of decision trees. Machine Learning 1(1), 81–106 (1986)Google Scholar
  22. 22.
    Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, California (1992)Google Scholar
  23. 23.
    Scheuermann, T., Hensley, J.: Efficient histogram generation using scattering on gpus. In: SI3D, pp. 33–37 (2007)Google Scholar
  24. 24.

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Toby Sharp
    • 1
  1. 1.Microsoft ResearchCambridgeUK

Personalised recommendations