Advertisement

Thresholding a Random Forest Classifier

  • Florian Baumann
  • Fangda Li
  • Arne Ehlers
  • Bodo Rosenhahn
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8888)

Abstract

The original Random Forest derives the final result with respect to the number of leaf nodes voted for the corresponding class. Each leaf node is treated equally and the class with the most number of votes wins. Certain leaf nodes in the topology have better classification accuracies and others often lead to a wrong decision. Also the performance of the forest for different classes differs due to uneven class proportions. In this work, a novel voting mechanism is introduced: each leaf node has an individual weight. The final decision is not determined by majority voting but rather by a linear combination of individual weights leading to a better and more robust decision. This method is inspired by the construction of a strong classifier using a linear combination of small rules of thumb (AdaBoost). Small fluctuations which are caused by the use of binary decision trees are better balanced. Experimental results on several datasets for object recognition and action recognition demonstrate that our method successfully improves the classification accuracy of the original Random Forest algorithm.

Keywords

Leaf Node Action Recognition AdaBoost Algorithm Random Forest Algorithm Binary Decision Tree 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)CrossRefMATHGoogle Scholar
  2. 2.
    Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings of the Thirteenth International Conference on Machine Learning, pp. 148–156. IEEE (1996)Google Scholar
  3. 3.
    Stallkamp, J., Schlipsing, M., Salmen, J., Igel, C.: Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Networks (2012)Google Scholar
  4. 4.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  5. 5.
    Weinland, D., Özuysal, M., Fua, P.: Making action recognition robust to occlusions and viewpoint changes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 635–648. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Weinland, D., Ronfard, R., Boyer, E.: Free viewpoint action recognition using motion history volumes. In: Computer Vision and Image Understanding (CVIU) (2006)Google Scholar
  7. 7.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: a local svm approach. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR) (2004)Google Scholar
  8. 8.
    Blank, M., Gorelick, L., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. In: 10th International Conference on Computer Vision (ICCV), pp. 1395–1402 (2005)Google Scholar
  9. 9.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 29, 2247–2253 (2007)CrossRefGoogle Scholar
  10. 10.
    Esuli, A., Fagni, T., Sebastiani, F.: TreeBoost.MH: A boosting algorithm for multi-label hierarchical text categorization. In: Crestani, F., Ferragina, P., Sanderson, M. (eds.) SPIRE 2006. LNCS, vol. 4209, pp. 13–24. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Grossmann, E.: Adatree 2: boosting to build decision trees or improving adatree with soft splitting rules. unpublished work done at the Center for Visualisation and Virtual Environments, University of Kentucky (2004)Google Scholar
  12. 12.
    Grossmann, E.: Adatree: Boosting a weak classifier into a decision tree. In: Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW 2004), vol. 6, p. 105. IEEE Computer Society, Washington, DC (2004)CrossRefGoogle Scholar
  13. 13.
    Roe, B.P., Yang, H.J., Zhu, J., Liu, Y., Stancu, I., McGregor, G.: Boosted decision trees as an alternative to artificial neural networks for particle identification. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 543, 577–584 (2005)CrossRefGoogle Scholar
  14. 14.
    Zhang, Z., Xie, X.: Research on adaboost. m1 with random forest. In: 2010 2nd International Conference on Computer Engineering and Technology (ICCET), vol. 1, pp. V1–V647 (2010)Google Scholar
  15. 15.
    Zhou, S.K.: A binary decision tree implementation of a boosted strong classifier. In: Zhao, W., Gong, S., Tang, X. (eds.) AMFG 2005. LNCS, vol. 3723, pp. 198–212. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  16. 16.
    Seyedhosseini, M., Paiva, A.R., Tasdizen, T.: Fast adaboost training using weighted novelty selection. In: The 2011 International Joint Conference on Neural Networks (IJCNN), pp. 1245–1250. IEEE (2011)Google Scholar
  17. 17.
    Schulter, S., Roth, P.M., Bischof, H.: Ordinal random forests for object detection. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 261–270. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  18. 18.
    Schulter, S., Wohlhart, P., Leistner, C., Saffari, A., Roth, P.M., Bischof, H.: Alternating decision forests. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  19. 19.
    Bernard, S., Adam, S., Heutte, L.: Dynamic random forests. Pattern Recognition Letters 33, 1580–1586 (2012)CrossRefGoogle Scholar
  20. 20.
    Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57, 137–154 (2004)CrossRefGoogle Scholar
  21. 21.
    Breiman, L.: Bagging predictors. In: Machine Learning, vol. 24, pp. 123–140 (1996)Google Scholar
  22. 22.
    Ho, T.K.: Random decision forests. In: Proceedings of the Third International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE (1995)Google Scholar
  23. 23.
    Ho, T.K.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 832–844 (1998)CrossRefGoogle Scholar
  24. 24.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9, 1545–1588 (1997)CrossRefGoogle Scholar
  25. 25.
    Bernard, S., Adam, S., Heutte, L.: Using random forests for handwritten digit recognition. In: Ninth International Conference on Document Analysis and Recognition, ICDAR 2007, vol. 2, pp. 1043–1047. IEEE (2007)Google Scholar
  26. 26.
    Bernard, S., Heutte, L., Adam, S.: On the selection of decision trees in random forests. In: International Joint Conference on Neural Networks, IJCNN 2009, pp. 302–307. IEEE (2009)Google Scholar
  27. 27.
    Fan, G., Wang, Z., Wang, J.: Cw-ssim kernel based random forest for image classification. In: Visual Communications and Image Processing 2010, pp. 774425–774425. International Society for Optics and Photonics (2010)Google Scholar
  28. 28.
    O’Hara, S., Draper, B.A.: Scalable action recognition with a subspace forest. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  29. 29.
    Lui, Y.M., Beveridge, J., Kirby, M.: Action classification on product manifolds. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  30. 30.
    Schindler, K., Van Gool, L.: Action snippets: How many frames does human action recognition require? In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  31. 31.
    Li, W., Yu, Q., Sawhney, Vasconcelos, N.: Recognizing activities via bag of words for attribute dynamics. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2587–2594 (2013)Google Scholar
  32. 32.
    Tian, Y., Sukthankar, R., Shah, M.: Spatiotemporal deformable part models for action detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  33. 33.
    Yu, T.H., Kim, T.K., Cipolla, R.: Unconstrained monocular 3d human pose estimation by action detection and cross-modality regression forest. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3642–3649 (2013)Google Scholar
  34. 34.
    Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologically inspired system for action recognition. In: 11th International Conference on Computer Vision (ICCV), pp. 1–8. IEEE (2007)Google Scholar
  35. 35.
    Lin, Z., Jiang, Z., Davis, L.S.: Recognizing actions by shape-motion prototype trees. In: 12th International Conference on Computer Vision (ICCV), pp. 444–451 (2009)Google Scholar
  36. 36.
    Yeffet, L., Wolf, L.: Local trinary patterns for human action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  37. 37.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  38. 38.
    Kihl, O., Picard, D., Gosselin, P.H., et al.: Local polynomial space-time descriptors for actions classification. In: International Conference on Machine Vision Applications (2013)Google Scholar
  39. 39.
    Baumann, F., Liao, J., Ehlers, A., Rosenhahn, B.: Motion binary patterns for action recognition. In: 3rd International Conference on Pattern Recognition Applications and Methods (ICPRAM) (2014)Google Scholar
  40. 40.
    Wang, Z., Wang, J., Xiao, J., Lin, K.-H., Huang, T.: Substructure and boundary modeling for continuous action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  41. 41.
    Wu, X., Xu, D., Duan, L., Luo, J.: Action recognition using context and appearance distribution features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  42. 42.
    Li, R., Zickler, T.: Discriminative virtual views for cross-view action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Florian Baumann
    • 1
  • Fangda Li
    • 2
  • Arne Ehlers
    • 1
  • Bodo Rosenhahn
    • 1
  1. 1.Institut für Informationsverarbeitung (TNT)Leibniz Universität HannoverGermany
  2. 2.Electrical and Computer EngineeringPurdue UniversityUnited States

Personalised recommendations