Neural Computing and Applications

, Volume 27, Issue 8, pp 2279–2288 | Cite as

A fine-grained Random Forests using class decomposition: an application to medical diagnosis

  • Eyad Elyan
  • Mohamed Medhat GaberEmail author
Predictive Analytics Using Machine Learning


Class decomposition describes the process of segmenting each class into a number of homogeneous subclasses. This can be naturally achieved through clustering. Utilising class decomposition can provide a number of benefits to supervised learning, especially ensembles. It can be a computationally efficient way to provide a linearly separable data set without the need for feature engineering required by techniques like support vector machines and deep learning. For ensembles, the decomposition is a natural way to increase diversity, a key factor for the success of ensemble classifiers. In this paper, we propose to adopt class decomposition to the state-of-the-art ensemble learning Random Forests. Medical data for patient diagnosis may greatly benefit from this technique, as the same disease can have a diverse of symptoms. We have experimentally validated our proposed method on a number of data sets that are mainly related to the medical domain. Results reported in this paper show clearly that our method has significantly improved the accuracy of Random Forests.


Machine learning Random Forests Clustering Ensemble learning 


  1. 1.
    Abdallah ZS, Gaber MM (2011) Kb-cb-n classification: towards unsupervised approach for supervised learning. In: Computational intelligence and data mining (CIDM), 2011 IEEE symposium on IEEE, pp 283–290Google Scholar
  2. 2.
    Abdallah ZS, Gaber MM, Srinivasan B, Krishnaswamy S (2015) Adaptive mobile activity recognition system with evolving data streams. Neurocomputing 150:304–317CrossRefGoogle Scholar
  3. 3.
    Amaratunga D, Cabrera J, Lee Y-S (2008) Enriched random forests. Bioinformatics 24(18):2010–2014CrossRefGoogle Scholar
  4. 4.
    Amit Y, Geman D (1997) Shape quantization and recognition with randomized trees. Neural Comput. 9(7):1545–1588CrossRefGoogle Scholar
  5. 5.
    Bader-El-Den M, Gaber M (2012) Garf: towards self-optimised random forests. In: Neural information processing. Springer, pp 506–515Google Scholar
  6. 6.
    Breiman L (1996) Bagging predictors. Mach. Learn. 24(2):123–140MathSciNetzbMATHGoogle Scholar
  7. 7.
    Breiman L (2001) Random forests. Mach. Learn. 45(1):5–32MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Dietterich TG, Bakiri G (1991) Error-correcting output codes: a general method for improving multiclass inductive learning programs. In: AAAI, Citeseer, pp 572–577Google Scholar
  9. 9.
    Drucker H, Cortes C, Jackel LD, LeCun Y, Vapnik V (1994) Boosting and other ensemble methods. Neural Comput. 6(6):1289–1301CrossRefzbMATHGoogle Scholar
  10. 10.
    Elter M, Schulz-Wendtland R, Wittenberg T (2007) The prediction of breast cancer biopsy outcomes using two CAD approaches that both emphasize an intelligible decision process. Med. Phys. 34:4164CrossRefGoogle Scholar
  11. 11.
    Fawagreh K, Gaber MM, Elyan E (2014) Diversified random forests using random subspaces. In: Intelligent data engineering and automated learning–IDEAL 2014. Springer, pp 85–92Google Scholar
  12. 12.
    Fawagreh K, Gaber MM, Elyan E (2014) Random forests: from early developments to recent advancements. Syst Sci Control Eng Open Access J 2(1):602–609CrossRefGoogle Scholar
  13. 13.
    Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15:3133–3181MathSciNetzbMATHGoogle Scholar
  14. 14.
    Freund Y (1995) Boosting a weak learning algorithm by majority. Inf Comput 121(2):256–285MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Friedman JH (2002) Stochastic gradient boosting. Comput Stat Data Anal 38(4):367–378MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Ho TK (1995) Random decision forests. In: Proceedings of the third international conference on document analysis and recognition, vol 1, pp 278–282Google Scholar
  17. 17.
    Ho TK (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):832–844CrossRefGoogle Scholar
  18. 18.
    Hong Z-Q, Yang J-Y (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recognit 24(4):317–324MathSciNetCrossRefGoogle Scholar
  19. 19.
    Jain AK, Dubes RC et al (1988) Algorithms for clustering data, vol 6. Prentice hall, Englewood CliffszbMATHGoogle Scholar
  20. 20.
    Latinne P, Debeir O, Decaestecker C (2001) Limiting the number of trees in random forests. In: Multiple classifier systems. Springer, pp 178–187Google Scholar
  21. 21.
    Liaw A, Wiener M (2002) Classification and regression by randomforest. R News 2(3):18–22Google Scholar
  22. 22.
    Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA.
  23. 23.
    Little MA, McSharry PE, Roberts SJ, Costello DA, Moroz IM (2007) Exploiting nonlinear recurrence and fractal scaling properties for voice disorder detection. BioMed Eng OnLine 6(1):23. CrossRefGoogle Scholar
  24. 24.
    MacQueen JB (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth berkeley symposium on math, statistics, and probability, vol 1, pp 281–297Google Scholar
  25. 25.
    Mangasarian OL, Street WN, Wolberg WH (1995) breast cancer diagnosis and prognosis via linear programming. Oper Res 43:570–577MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Polaka I (2013) Clustering algorithm specifics in class decomposition. In: International conference on applied information and communication technologies (AICT2013), 25–26 April 2013, Jelgava, LatviaGoogle Scholar
  27. 27.
    Polikar R (2006) Ensemble based systems in decision making. IEEE Circuits Syst Mag 6(3):21–45CrossRefGoogle Scholar
  28. 28.
    Repository U (1996) Heart Disease dataset. Accessed Dec 2014
  29. 29.
    Robnik-Šikonja M (2004) Improving random forests. In: Machine learning: ECML 2004. Springer, pp 359–370Google Scholar
  30. 30.
    Tsymbal A, Pechenizkiy M, Cunningham P (2006) Dynamic integration with random forests. In: Machine learning: ECML 2006. Springer, pp 801–808Google Scholar
  31. 31.
    Vilalta R, Achari M-K, Eick CF (2003) Class decomposition via clustering: a new framework for low-variance classifiers. In: Data mining, 2003, ICDM 2003, third IEEE international conference on IEEE, pp 673–676Google Scholar
  32. 32.
    Wolpert DH (1992) Stacked generalization. Neural Netw 5(2):241–259MathSciNetCrossRefGoogle Scholar
  33. 33.
    Woolson RF (2008) Wilcoxon signed-rank test. Wiley encyclopedia of clinical trials, pp 1–3. doi: 10.1002/9780471462422.eoct979

Copyright information

© The Natural Computing Applications Forum 2015

Authors and Affiliations

  1. 1.School of Computing Science and Digital MediaRobert Gordon UniversityAberdeenUnited Kingdom

Personalised recommendations