Skip to main content
SpringerLink
Log in

OC1-DE: A Differential Evolution Based Approach for Inducing Oblique Decision Trees

  • Conference paper
  • First Online:
Artificial Intelligence and Soft Computing (ICAISC 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10245))

Included in the following conference series:

Abstract

This paper describes the application of a Differential Evolution based approach for inducing oblique decision trees in a recursive partitioning strategy. Considering that: (1) the task of finding an optimal hyperplane with real-valued coefficients is a complex optimization problem in a continuous space, and (2) metaheuristics have been successfully applied for solving this type of problems, in this work a differential evolution algorithm is applied with the aim of finding near-optimal hyperplanes that will be used as test conditions of an oblique decision tree. The experimental results show that this approach induces more accurate classifiers than those produced by other proposed induction methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A dipole is a pair of instances in training set represented as vectors.

  2. 2.

    Highest values for each dataset are in bold.

References

  1. Agapitos, A., O’Neill, M., Brabazon, A., Theodoridis, T.: Maximum margin decision surfaces for increased generalisation in evolutionary decision tree learning. In: Silva, S., Foster, J.A., Nicolau, M., Machado, P., Giacobini, M. (eds.) EuroGP 2011. LNCS, vol. 6621, pp. 61–72. Springer, Heidelberg (2011). doi:10.1007/978-3-642-20407-4_6

    Chapter  Google Scholar 

  2. Bennett, K.P., Cristianini, N., Shawe-Taylor, J., Wu, D.: Enlarging the margins in perceptron decision trees. Mach. Learn. 41(3), 295–313 (2000). doi:10.1023/A:1007600130808

    Article  MATH  Google Scholar 

  3. Bot, M.C.J., Langdon, W.B.: Improving induction of linear classification trees with genetic programming. In: Whitley, L.D., Goldberg, D.E., Cantú-Paz, E., Spector, L., Parmee, I.C., Beyer, H.G. (eds.) GECCO-2000, pp. 403–410. Morgan Kaufmann (2000)

    Google Scholar 

  4. Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and Regression Trees. Taylor & Francis, Abington (1984)

    MATH  Google Scholar 

  5. Cantú-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. Evol. Comput. 7(1), 54–68 (2003). doi:10.1109/TEVC.2002.806857

    Article  Google Scholar 

  6. Chai, B.B., Zhuang, X., Zhao, Y., Sklansky, J.: Binary linear decision tree with genetic algorithm. In: ICPR 1996, vol. 4, pp. 530–534 IEEE (1996). doi:10.1109/ICPR.1996.547621

  7. Das, S., Suganthan, P.N.: Differential evolution: A survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011). doi:10.1109/TEVC.2010.2059031

    Article  Google Scholar 

  8. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  9. Dumitrescu, D., András, J.: Generalized decision trees built with evolutionary techniques. Stud. Inf. Control 14(1), 15–22 (2005)

    MATH  Google Scholar 

  10. Gama, J., Brazdil, P.: Linear tree. Intell. Data Anal. 3(1), 1–22 (1999). doi:10.1016/S1088-467X(99)00002-5

    Article  MATH  Google Scholar 

  11. García, S., Derrac, J., Triguero, I., Carmona, C.J., Herrera, F.: Evolutionary-based selection of generalized instances for imbalanced classification. Knowl. Based Syst. 25(1), 3–12 (2012). doi:10.1016/j.knosys.2011.01.012

    Article  Google Scholar 

  12. García, S., Fernández, A., Luengo, J., Herrera, F.: A study of statistical techniques and performance measures for genetics-based machine learning: Accuracy and interpretability. Soft Comput. 13(10), 959 (2008). doi:10.1007/s00500-008-0392-y

    Article  Google Scholar 

  13. Geetha, K., Baboo, S.S.: An empirical model for thyroid disease classification using evolutionary multivariate bayseian prediction method. Glob. J. Comput. Sci. Technol. 16(1), 1–9 (2016)

    Google Scholar 

  14. Gendreau, M., Potvin, J.Y.: Handbook of Metaheuristics, vol. 2. Springer, Heidelberg (2010). doi:10.1007/978-1-4419-1665-5

    Book  MATH  Google Scholar 

  15. Gray, J.B., Fan, G.: Classification tree analysis using TARGET. Comput. Stat. Data Anal. 52(3), 1362–1372 (2008). doi:10.1016/j.csda.2007.03.014

    Article  MathSciNet  MATH  Google Scholar 

  16. Heath, D.G.: A geometric framework for machine learning. Ph.D. thesis, Johns Hopkins University (1993)

    Google Scholar 

  17. Heath, D.G., Kasif, S., Salzberg, S.: Induction of oblique decision trees. In: Bajcsy, R., et al. (ed.) IJCAI 1993, pp. 1002–1007 (1993)

    Google Scholar 

  18. Hyafil, L., Rivest, R.L.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5(1), 15–17 (1976). doi:10.1016/0020-0190(76)90095-8

    Article  MathSciNet  MATH  Google Scholar 

  19. Jankowski, D., Jackowski, K.: Evolutionary algorithm for decision tree induction. In: Saeed, K., Snášel, V. (eds.) CISIM 2014. LNCS, vol. 8838, pp. 23–32. Springer, Heidelberg (2014). doi:10.1007/978-3-662-45237-0_4

    Chapter  Google Scholar 

  20. Krȩtowski, M.: An evolutionary algorithm for oblique decision tree induction. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS, vol. 3070, pp. 432–437. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24844-6_63

    Chapter  Google Scholar 

  21. Krętowski, M., Grześ, M.: Evolutionary learning of linear trees with embedded feature selection. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Żurada, J.M. (eds.) ICAISC 2006. LNCS, vol. 4029, pp. 400–409. Springer, Heidelberg (2006). doi:10.1007/11785231_43

    Chapter  Google Scholar 

  22. Kushida, J.I., Hara, A., Takahama, T.: A novel tree differential evolution using inter-symbol distance. In: IWCIA 2014, pp. 107–112. IEEE (2014). doi:10.1109/IWCIA.2014.6988087

  23. Leema, N., Nehemiah, H.K., Kannan, A.: Neural network classifier optimization using differential evolution with global information and back propagation algorithm for clinical datasets. Appl. Soft Comput. 49, 834–844 (2016). doi:10.1016/j.asoc.2016.08.001

    Article  Google Scholar 

  24. Levi, D.: Hereboy: A fast evolutionary algorithm. In: Lohn, J., et al. (ed.) EH 2000, pp. 17–24. IEEE (2000). doi:10.1109/EH.2000.869338

  25. Li, J., Ding, L., Li, B.: Differential evolution-based parameters optimisation and feature selection for support vector machine. Int. J. Comput. Sci. Eng. 13(4), 355–363 (2016). doi:10.1504/ijcse.2016.080212

    Article  Google Scholar 

  26. Li, X.B., Sweigart, J.R., Teng, J.T.C., Donohue, J.M., Thombs, L., Wang, S.M.: Multivariate decision trees using linear discriminants and tabu search. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 33(2), 194–205 (2003). doi:10.1109/TSMCA.2002.806499

    Article  Google Scholar 

  27. Lichman, M.: UCI Machine Learning Repository. University of California, Irvine (2013). http://archive.ics.uci.edu/ml

  28. Liu, K.H., Xu, C.G.: A genetic programming-based approach to the classification of multiclass microarray datasets. Bioinformatics 25(3), 331–337 (2009). doi:10.1093/bioinformatics/btn644

    Article  Google Scholar 

  29. Lopes, R.A., Freitas, A.R.R., Silva, R.C.P., Guimarães, F.G.: Differential evolution and perceptron decision trees for classification tasks. In: Yin, H., Costa, J.A.F., Barreto, G. (eds.) IDEAL 2012. LNCS, vol. 7435, pp. 550–557. Springer, Heidelberg (2012). doi:10.1007/978-3-642-32639-4_67

    Chapter  Google Scholar 

  30. Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Intell. Res. 2(1), 1–32 (1994). doi:10.1613/jair.63

    MATH  Google Scholar 

  31. Murthy, S.K., Kasif, S., Salzberg, S., Beigel, R.: OC1: A randomized algorithm for building oblique decision trees. In: Proceedings of AAAI 93, vol. 93, pp. 322–327 (1993)

    Google Scholar 

  32. Orsenigo, C., Vercellis, C.: Discrete support vector decision trees via tabu search. Comput. Stat. Data Anal. 47(2), 311–322 (2004). doi:10.1016/j.csda.2003.11.005

    Article  MathSciNet  MATH  Google Scholar 

  33. Pangilinan, J.M., Janssens, G.K.: Pareto-optimality of oblique decision trees from evolutionary algorithms. J. Glob. Optim. 51(2), 301–311 (2011). doi:10.1007/s10898-010-9614-9

    Article  MathSciNet  MATH  Google Scholar 

  34. Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986). doi:10.1007/BF00116251

    Google Scholar 

  35. Quinlan, J.R.: Simplifying decision trees. Int. J. Hum. Comput. Stud. 27(3), 221–234 (1987). doi:10.1006/ijhc.1987.0321

    Google Scholar 

  36. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)

    Google Scholar 

  37. Shali, A., Kangavari, M.R., Bina, B.: Using genetic programming for the induction of oblique decision trees. In: Arif-Wani, M. (ed.) ICMLA 2007, pp. 38–43. IEEE (2007). doi:10.1109/ICMLA.2007.66

  38. Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11(4), 341–359 (1997). doi:10.1023/A:1008202821328

    Article  MathSciNet  MATH  Google Scholar 

  39. Struharik, R., Vranjkovic, V., Dautovic, S., Novak, L.: Inducing oblique decision trees. In: SISY-2014, pp. 257–262. IEEE (2014). doi:10.1109/SISY.2014.6923596

  40. Tušar, T.: Optimizing accuracy and size of decision trees. In: ERK-2007, pp. 81–84 (2007)

    Google Scholar 

  41. Utgoff, P.E., Brodley, C.E.: Linear machine decision trees. University of Massachusetts, Amherst, MA, USA, Technical report (1991)

    Google Scholar 

  42. Veenhuis, C.B.: Tree based differential evolution. In: Vanneschi, L., Gustafson, S., Moraglio, A., Falco, I., Ebner, M. (eds.) EuroGP 2009. LNCS, vol. 5481, pp. 208–219. Springer, Heidelberg (2009). doi:10.1007/978-3-642-01181-8_18

    Chapter  Google Scholar 

  43. Vukobratović, B., Struharik, R.: Evolving full oblique decision trees. In: CINTI 2015, pp. 95–100. IEEE (2015). doi:10.1109/CINTI.2015.7382901

  44. Wang, P., Tang, K., Weise, T., Tsang, E.P.K., Yao, X.: Multiobjective genetic programming for maximizing ROC performance. Neurocomputing 125, 102–118 (2014). doi:10.1016/j.neucom.2012.06.054

    Article  Google Scholar 

  45. Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  46. Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Philip, S.Y.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2008). doi:10.1007/s10115-007-0114-2

    Article  Google Scholar 

  47. Zhang, K., Xu, Z., Buckles, B.P.: Oblique decision tree induction using multimembered evolution strategies. In: Dasarathy, B.V. (ed.) SPIE 2005, vol. 5812, pp. 263–270. SPIE (2005). doi:10.1117/12.596766

Download references

Acknowledgments

This work has been supported in part by the Mexican Government (CONACyT FOMIX-DICC project No. TAB-2014-C01-245876 and the PROMEP-SEP project No. DSA/103.5/15/6409).

Author information

Authors and Affiliations

  1. Instituto Tecnológico de Veracruz, Veracruz, Mexico

    Rafael Rivera-Lopez

  2. Universidad Juárez Autónoma de Tabasco, Cunduacán, Mexico

    Juana Canul-Reich

  3. Universidad de Castilla-La Mancha, Albacete, Spain

    José A. Gámez & José M. Puerta

Authors
  1. Rafael Rivera-Lopez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juana Canul-Reich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. José A. Gámez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José M. Puerta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juana Canul-Reich .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  3. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  4. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  5. University of California, Berkeley, California, USA

    Lotfi A. Zadeh

  6. University of Louisville, Louisville, Kentucky, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Rivera-Lopez, R., Canul-Reich, J., Gámez, J.A., Puerta, J.M. (2017). OC1-DE: A Differential Evolution Based Approach for Inducing Oblique Decision Trees. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2017. Lecture Notes in Computer Science(), vol 10245. Springer, Cham. https://doi.org/10.1007/978-3-319-59063-9_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59063-9_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59062-2

  • Online ISBN: 978-3-319-59063-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation