Skip to main content
SpringerLink
Log in

M-ary Random Forest - A new multidimensional partitioning approach to Random Forest

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Random Forest (RF) is composed of decision trees as base classifiers. In general, a decision tree recursively partitions the feature space into two disjoint subspaces using a single feature as axis-parallel splits for each internal node. The oblique decision tree uses a linear combination of features (to form a hyperplane) to partition the feature space in two subspaces. The later approach is an NP-hard problem to compute the best-suited hyperplane. In this work, we propose to use multiple features at a node for splitting the data as in axis parallel method. Each feature independently divides into two subspaces and this process is done by multiple features at one node. Hence, the given space is divided into multiple subspaces simultaneously, and in turn, to construct the M-ary trees. Hence, the forest formed is named as M-ary Random Forest (MaRF). To measure the performance of the task in MaRF, we have extended the notion of tree strength of the regression tree. We empirically prove that the performance of the MaRF improves due to the improvement in the strength of the M-ary trees. We have shown the performance to wide range of datasets ranging from UCI datasets, Hyperspectral dataset, MNIST dataset, Caltech 101 and Caltech 256 datasets. The efficiency of the MaRF approach is found satisfactory as compared to state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Referred to as conventional random forest throughout the text

References

  1. Akash PS, Kadir ME, Ali AA, Tawhid MNA, Shoyaib M (2019) Introducing confidence as a weight in random forest. In: 2019 International conference on robotics, electrical and signal processing techniques (ICREST). IEEE, pp 611–616

  2. Amaratunga D, Cabrera J, Lee YS (2008) Enriched random forests. Bioinformatics 24(18):2010–2014

    Article  Google Scholar 

  3. Asuncion A, Newman D (2007) Uci machine learning repository

  4. Biau G, Devroye L, Lugosi G (2008) Consistency of random forests and other averaging classifiers. J Mach Learn Res 9(Sep):2015–2033

    MathSciNet  MATH  Google Scholar 

  5. Biau G (2012) Analysis of a random forests model. J Mach Learn Res 13:1063–1095

    MathSciNet  MATH  Google Scholar 

  6. Bosch A, Zisserman A, Munoz X (2007) Image classification using random forests and ferns. In: 2007 IEEE 11Th international conference on computer vision. IEEE, pp 1–8

  7. Breiman L (1994) Bagging predictors, Technical Report. UC Berkeley

  8. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  9. Denil M, Matheson D, De Freitas N (2014) Narrowing the gap: Random forests in theory and in practice. In: International conference on machine learning, pp 665–673

  10. Fei-Fei L, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: 2004 Conference on computer vision and pattern recognition workshop. IEEE, pp 178–178

  11. Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42

    Article  Google Scholar 

  12. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

  13. He Z, Liu L, Deng R, Shen Y (2016) Low-rank group inspired dictionary learning for hyperspectral image classification. Signal Process 120:209–221

    Article  Google Scholar 

  14. Hyperspectral dataset. http://lesun.weebly.com/hyperspectral-data-set.html. Online Accessed: 25-September-2019

  15. Hyperspectral benchmark dataset on soil moisture. https://zenodo.org/record/1227837#.XmYNw_fhVuQ.html. Online Accessed: 20-January-2020

  16. Ishwaran H (2015) The effect of splitting on random forests. Mach Learn 99(1):75–118

    Article  MathSciNet  Google Scholar 

  17. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37

    Article  Google Scholar 

  18. Jain V, Phophalia A (2019) M-ary random forest. In: International conference on pattern recognition and machine intelligence. Springer, pp 161–169

  19. Jain V, Sharma J, Singhal K, Phophalia A (2019) Exponentially weighted random forest. In: International conference on pattern recognition and machine intelligence. Springer, pp 170–178

  20. Ji R, Gao Y, Hong R, Liu Q, Tao D, Li X (2014) Spectral-spatial constraint hyperspectral image classification. IEEE Trans Geosci Remote Sens 52(3):1811–1824

    Article  Google Scholar 

  21. Landgrebe DA (2005) Signal theory methods in multispectral remote sensing, vol 29. Wiley, New York

  22. LeCun Y, Bottou L, Bengio Y, Haffner P, et al. (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  23. Louppe G (2014) Understanding random forests: From theory to practice. arXiv:1407.7502

  24. Menze BH, Kelm BM, Splitthoff DN, Koethe U, Hamprecht FA (2011) On oblique random forests. In: Joint european conference on machine learning and knowledge discovery in databases. Springer, pp 453–469

  25. Mishina Y, Murata R, Yamauchi Y, Yamashita T, Fujiyoshi H (2015) Boosted random forest. IEICE Trans Inf Syst 98(9):1630–1636

    Article  Google Scholar 

  26. Murthy SK, Kasif S, Salzberg S (1994) A system for induction of oblique decision trees. J Artif Intell Res 2:1–32

    Article  Google Scholar 

  27. Oshiro TM, Perez PS, Baranauskas JA (2012) How many trees in a random forest?. In: International workshop on machine learning and data mining in pattern recognition. Springer, pp 154–168

  28. Paul A, Mukherjee DP (2014) Enhanced random forest for mitosis detection. In: Proceedings of the 2014 Indian conference on computer vision graphics and image processing. ACM, pp 85

  29. Paul A, Mukherjee DP (2016) Reinforced random forest. In: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing. ACM, pp 1

  30. Paul A, Mukherjee DP (2019) Reinforced quasi-random forest. Pattern Recogn 94:13–24

    Article  Google Scholar 

  31. Paul A, Mukherjee DP, Das P, Gangopadhyay A, Chintha AR, Kundu S (2018) Improved random forest for classification. IEEE Transactions on Image Processing

  32. Wang Y, Xia ST, Tang Q, Wu J, Zhu X (2018) A novel consistent random forest framework: Bernoulli random forests IEEE transactions on neural networks and learning systems

  33. Wang L, Zhao C (2016) Hyperspectral image processing. Springer, Berlin

  34. Wickramarachchi D, Robertson B, Reale M, Price C, Brown J (2016) Hhcart: an oblique decision tree. Comput Stat Data Anal 96:12–23

    Article  MathSciNet  Google Scholar 

  35. Winham SJ, Freimuth RR, Biernacka JM (2013) A weighted random forests approach to improve predictive performance. Stat Anal Data Min ASA Data Sci J 6(6):496–505

    Article  MathSciNet  Google Scholar 

  36. Yang J, Jiang YG, Hauptmann AG, Ngo CW (2007) Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the international workshop on Workshop on multimedia information retrieval. ACM, pp 197–206

  37. Zhang L, Suganthan PN (2014) Oblique decision tree ensemble via multisurface proximal support vector machine. IEEE Trans Cybern 45(10):2165–2176

    Article  Google Scholar 

  38. Zhang Y, Cao G, Li X, Wang B (2018) Cascaded random forest for hyperspectral image classification. IEEE J Sel Top Appl Earth Observ Remote Sens 11(4):1082–1094

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology Vadodara, Gujarat, 382028, India

    Vikas Jain & Ashish Phophalia

Authors
  1. Vikas Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ashish Phophalia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vikas Jain.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jain, V., Phophalia, A. M-ary Random Forest - A new multidimensional partitioning approach to Random Forest. Multimed Tools Appl 80, 35217–35238 (2021). https://doi.org/10.1007/s11042-020-10047-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10047-9

Keywords

Search

Navigation