Skip to main content

Advertisement

SpringerLink
Log in

A novel framework of fuzzy oblique decision tree construction for pattern classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In this paper, some significant efforts on fuzzy oblique decision tree (FODT) have been done to improve classification accuracy and decrease tree size. Firstly, to eliminate data redundancy and improve classification efficiency, a forward greedy fast feature selection algorithm based on neighborhood rough set (NRS_FS_FAST) is introduced. Then, a new fuzzy rule generation algorithm (FRGA) is proposed to generate fuzzy rules. These fuzzy rules are used to construct leaf nodes for each class in each layer of the FODT. Different from the traditional axis-parallel decision trees and oblique decision trees, the FODT takes dynamic mining fuzzy rules as decision functions. Moreover, the parameter δ, which can control the size of the tree, is optimized by genetic algorithm. Finally, a series of comparative experiments are carried out with five traditional decision trees (C4.5, Best First Tree (BFT), amulti-class alternating decision tree (LAD), Simple Cart (SC), Naive Bayes Tree (NBT)), and recently proposed decision trees (FRDT, HHCART, and FMMDT-HB) on UCI machine learning datasets. The experimental results demonstrate that the FODT exhibits better performance on classification accuracy and tree size than the chosen benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Cover T, Hart P (2002) Nearest neighbor pattern classification[J]. IEEE Trans Inf Theory 13(1):21–27

    MATH  Google Scholar 

  2. Medin DL, Schaffer MM (2016) Context theory of classification learning[J]. Psychol Rev 85(3):207–238

    Google Scholar 

  3. Tsang ECC, Chen D, Yeung DS et al (2008) Attributes reduction using fuzzy rough sets[J]. IEEE Trans Fuzzy Syst 16(5):1130–1141

    Google Scholar 

  4. Jing Y, Li T, Fujita H et al (2017) An incremental attribute reduction approach based on knowledge granularity with a multi-granulation view[J]. Inf Sci 411:23–38

    MathSciNet  Google Scholar 

  5. Zhang DW, Wang P, Qiu JQ et al (2010) An improved approach to feature selection[C]. In: Machine learning and cybernetics (ICMLC), 2010 international conference on, vol 1. IEEE, pp 488–493

  6. Liu ZG, Pan Q, Dezert J (2014) A belief classification rule for imprecise data[J]. Appl Intell 40(2):214–228

    Google Scholar 

  7. Afify AA (2016) A fuzzy rule induction algorithm for discovering classification rules[J]. J Intell Fuzzy Syst 30(6):3067–3085

    Google Scholar 

  8. Park SB, Zhang BT, Kim YT (2003) Word sense disambiguation by learning decision trees from unlabeled data[J]. Appl Intell 19(1–2):27–38

    MATH  Google Scholar 

  9. Li XB, Sweigart J, Teng J et al (2001) A dynamic programming based pruning method for decision trees[J]. INFORMS J Comput 13(4):332–344

    MATH  Google Scholar 

  10. Wu CC, Chen YL, Liu YH et al (2016) Decision tree induction with a constrained number of leaf nodes[J]. Appl Intell 45(3):673–685

    Google Scholar 

  11. Quinlan JR (2014) C4.5: programs for machine learning[M]. Elsevier, New York

    Google Scholar 

  12. Liang C, Zhang Y, Shi P et al (2015) Learning accurate very fast decision trees from uncertain data streams[J]. Int J Syst Sci 46(16):3032–3050

    MATH  Google Scholar 

  13. Sok HK, Ooi MPL, Kuang YC et al (2016) Multivariate alternating decision trees[J]. Pattern Recogn 50:195–209

    Google Scholar 

  14. Kumar PSJ, Yung Y, Huan TL (2017) Neural network based decision trees using machine learning for Alzheimer’s diagnosis[J]. Int J Comput Inform Sci 4(11):63–72

    Google Scholar 

  15. Shukla SK, Tiwari MK (2012) GA guided cluster based fuzzy decision tree for reactive ion etching modeling: a data mining approach[J]. IEEE Trans Semicond Manuf 25(1):45–56

    Google Scholar 

  16. Liu X, Feng X, Pedrycz W (2013) Extraction of fuzzy rules from fuzzy decision trees: an axiomatic fuzzy sets (AFS) approach[J]. Data Knowl Eng 84:1–25

    Google Scholar 

  17. Wang X, Liu X, Pedrycz W et al (2015) Fuzzy rule based decision trees[J]. Pattern Recogn 48(1):50–59

    Google Scholar 

  18. Segatori A, Marcelloni F, Pedrycz W (2018) On distributed fuzzy decision trees for big data[J]. IEEE Trans Fuzzy Syst 26(1):174–192

    Google Scholar 

  19. Tan PJ, Dowe DL (2006) Decision forests with oblique decision trees[C]. In: Mexican international conference on artificial intelligence. Springer, Berlin, pp 593–603

  20. Barros RC, Jaskowiak PA, Cerri R et al (2014) A framework for bottom-up induction of oblique decision trees[J]. Neurocomputing 135:3–12

    Google Scholar 

  21. Do TN, Lenca P, Lallich S (2015) Classifying many-class high-dimensional fingerprint datasets using random forest of oblique decision trees[J]. Vietnam journal of computer science 2(1):3–12

    Google Scholar 

  22. Wickramarachchi DC, Robertson BL, Reale M et al (2016) HHCART: An oblique decision tree[J]. Comput Stat Data Anal 96:12–23

    MathSciNet  MATH  Google Scholar 

  23. Rivera-Lopez R, Canul-Reich J, Gómez JA et al (2017) OC1-DE: a differential evolution based approach for inducing oblique decision trees[C]. In: International conference on artificial intelligence and soft computing. Springer, Cham, pp 427–438

  24. Rivera-Lopez R, Canul-Reich J (2017) A global search approach for inducing oblique decision trees using differential evolution[C]. In: Canadian conference on artificial intelligence. Springer, Cham, pp 27–38

  25. Narayanan SJ, Paramasivam I, Bhatt RB et al (2015) A study on the approximation of clustered data to parameterized family of fuzzy membership functions for the induction of fuzzy decision trees[J]. Cybernetics and Information Technologies 15(2):75–96

    Google Scholar 

  26. Han NM, Hao NC (2016) An algorithm to building a fuzzy decision tree for data classification problem based on the fuzziness intervals matching[J]. Journal of Computer Science and Cybernetics 32(4):367–380

    Google Scholar 

  27. Sardari S, Eftekhari M, Afsari F (2017) Hesitant fuzzy decision tree approach for highly imbalanced data classification[J]. Appl Soft Comput 61:727–741

    Google Scholar 

  28. Ludwig SA, Picek S, Jakobovic D (2018) Classification of Cancer data: analyzing gene expression data using a fuzzy decision tree algorithm[M]. In: Operations research applications in health care management. Springer, Cham, pp 327–347

  29. Manwani N, Sastry PS (2012) Geometric decision tree[J]. IEEE Trans Syst Man Cybern B Cybern 42(1):181–192

    Google Scholar 

  30. Patil SP, Badhe SV (2015) Geometric approach for induction of oblique decision tree[J]. International Journal of Computer Science and Information Technologies 5(1):197–201

    Google Scholar 

  31. Cant-Paz E, Kamath C (2000) Using evolutionary algorithms to induce oblique decision trees[C]. In: Proceedings of the 2nd annual conference on genetic and evolutionary computation. Morgan Kaufmann Publishers Inc, pp 1053–1060

  32. Skowron A (2000) Rough sets in KDD[J]. Special invited speaking, WCC pp 1–17

  33. Zhang-Yan XU, Hou W, Song W et al (2009) Efficient heuristic attribute reduction algorithm based on information entropy[J]. Journal of Chinese Computer Systems 30(9):1805–1810

    Google Scholar 

  34. Yang SM, Jie M, Zhang ZB et al (2015) An improved heuristic attribute reduction algorithm based on information entropy in rough set[J]. Open Cybernetics & Systemics Journal 9(1):2774–2779

    Google Scholar 

  35. Guang-Yuan FU, Han-Zhao WU, Yang XG (2013) Attribute reduction algorithm of rough set based on the combined compatibility and importance of attributes[J]. Science Technology & Engineering

  36. Zadeh LA (1997) Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic[J]. Fuzzy Sets Syst 90(2):111–127

    MathSciNet  MATH  Google Scholar 

  37. Agrawal R, Imielinski T, Swami A (1993) Database mining: a performance perspective[J]. IEEE Trans Knowl Data Eng 5(6):914–925

    Google Scholar 

  38. Wang X, Liu X, Pedrycz W et al (2012) Mining axiomatic fuzzy set association rules for classification problems[J]. Eur J Oper Res 218(1):202–210

    MathSciNet  MATH  Google Scholar 

  39. Shi H (2007) Best-first decision tree learning[D]. The University of Waikato

  40. Holmes G, Pfahringer B, Kirkby R et al (2002) Multiclass alternating decision trees[C]. In: European conference on machine learning. Springer, Berlin, pp 161–172

  41. Breiman LI, Friedman JH, Olshen RA et al (1984) Classification and regression trees (CART)[J]. Encyclopedia of Ecology 40(3):582–588

    Google Scholar 

  42. Kohavi R (1996) Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid[C]. In: KDD 96, pp 202–207

  43. Mirzamomen Z, Kangavari MR (2017) A framework to induce more stable decision trees for pattern classification[J]. Pattern Anal Applic 20(4):991–1004

    MathSciNet  Google Scholar 

  44. Asuncion A, Newman D (2007) UCI machine learning repository[J]. http://www.ics.uci.edu/mlearn/MLRepository.html. Accessed 26 Jan 2018

  45. http://stanford.edu/marinka/nimfa/nimfa.datasets.html. Accessed 28 July 2018

  46. Witten LH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques, third edition[J]. ACM SIGMOD Rec 31(1):76–77

    Google Scholar 

  47. Hhn JC, Hllermeier E (2009) FR3: a fuzzy rule learner for inducing reliable classifiers[J]. IEEE Trans Fuzzy Syst 17(1):138–149

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (61627809, 61433004, 61621004), and Liaoning Revitalization Talents Program (XLYC1801005).

Author information

Authors and Affiliations

  1. State Key Laboratory of Synthetical Automation for Process Industries, or College of Information Science and Engineering, Northeastern University, Shenyang, Liaoning, China

    Yuliang Cai, Huaguang Zhang & Jie Duan

  2. College of Computer Science and Engineering, Northeastern University, Shenyang, Liaoning, China

    Qiang He

Authors
  1. Yuliang Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaguang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huaguang Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, Y., Zhang, H., He, Q. et al. A novel framework of fuzzy oblique decision tree construction for pattern classification. Appl Intell 50, 2959–2975 (2020). https://doi.org/10.1007/s10489-020-01675-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01675-7

Keywords

Search

Navigation