Skip to main content
SpringerLink
Log in

More Interpretable Decision Trees

  • Conference paper
  • First Online:
Hybrid Artificial Intelligent Systems (HAIS 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12886))

Included in the following conference series:

Abstract

We present a new Decision Tree Classifier (DTC) induction algorithm that produces vastly more interpretable trees in many situations. These understandable trees are highly relevant for explainable artificial intelligence, fair automatic classification, and human-in-the-loop learning systems. Our method is an improvement over the Nested Cavities (NC) algorithm. That is, we profit from the parallel-coordinates visualisation of high dimensional datasets. However, we build a hybrid with other decision tree heuristics to generate node-expanding splits. The rules in the DTCs learnt using our algorithm have a straightforward representation and, thus, are readily understood by a human user, even though our algorithm constructs rules whose nodes can involve multiple attributes. We compare our algorithm to the well-known decision tree induction algorithm C4.5, and find that our methods produce similar accuracy with significantly smaller trees. When coupled with a human-in-the-loop-learning (HILL) system, our approach can be highly effective for inferring understandable patterns in datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Implementations available: https://github.com/eugene-gilmore/SwiftDecisionTrees.

References

  1. Ala-Pietilä, P., et al.: Ethics guidelines for trustworthy AI. Technical report, European Commission – AI HLEG, B-1049 Brussels (2019)

    Google Scholar 

  2. Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: the role of humans in interactive machine learning. AI Mag. 35(4), 105–120 (2014)

    Google Scholar 

  3. Ankerst, M., Elsen, C., Ester, M., Kriegel, H.P.: Visual classification: an interactive approach to decision tree construction. In: 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’99, NY, USA, pp. 392–396 (1999)

    Google Scholar 

  4. Ankerst, M., Ester, M., Kriegel, H.P.: Towards an effective cooperation of the user and the computer for classification. In: 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’00, NY, USA, pp. 179–188 (2000)

    Google Scholar 

  5. Basgalupp, M.P., Barros, R.C., Podgorelec, V.: Evolving decision-tree induction algorithms with a multi-objective hyper-heuristic. In: 30th ACM Symposium on Applied Computing, pp. 110–117. ACM (2015)

    Google Scholar 

  6. Bouckaert, R.R., et al.: WEKA Manual V 3-6-2. University of Waikato (2010)

    Google Scholar 

  7. Breiman, L., Friedman, J., Stone, C., Olshen, R.: Classification and Regression Trees. Wadsworth, Monterrey (1984)

    MATH  Google Scholar 

  8. Cantú-Paz, E., Kamath, C.: Inducing oblique decision trees with evolutionary algorithms. IEEE Trans. Evol. Comput. 7(1), 54–68 (2003)

    Article  Google Scholar 

  9. Cohen, P.R., Feigenbaum, E.A.: The Handbook of Artificial Intelligence, vol. III. HeurisTech Press, Stanford (1982)

    MATH  Google Scholar 

  10. Estivill-Castro, V.: Collaborative knowledge acquisition with a genetic algorithm. In: 9th International Conference on Tools with Artificial Intelligence, ICTAI ’97, pp. 270–277. IEEE Computer Society, Newport Beach (1997)

    Google Scholar 

  11. Estivill-Castro, V., Gilmore, E., Hexel, R.: Human-in-the-loop construction of decision tree classifiers with parallel coordinates. In: 2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC, pp. 3852–3859. IEEE (2020)

    Google Scholar 

  12. Fails, J.A., Olsen, D.R.: Interactive machine learning. In: 8th International Conference on Intelligent User Interfaces. IUI ’03, pp. 39–45. ACM (2003)

    Google Scholar 

  13. Freitas, A.A.: Comprehensible classification models: a position paper. SIGKDD Explor. 15(1), 1–10 (2013)

    Article  Google Scholar 

  14. Heath, D. G. et al. : Induction of oblique decision trees. In: 13th International Joint Conference on Artificial Intelligence, pp. 1002–1007. Morgan Kaufmann (1993)

    Google Scholar 

  15. Hunt, E.: Concept Learning – An Information Processing Problem, 2nd edn. Wiley, New York (1962)

    Book  Google Scholar 

  16. Hunt, E., Martin, J., Stone, P.: Experiments in Induction. Academic Press, New York (1966)

    Google Scholar 

  17. Inselberg, A.: Parallel Coordinates: Visual Multidimensional Geometry and its Applications. Springer, New York (2009). https://doi.org/10.1007/978-0-387-68628-8

    Book  MATH  Google Scholar 

  18. Inselberg, A., Avidan, T.: Classification and visualization for high-dimensional data. In: 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, pp. 370–374 (2000)

    Google Scholar 

  19. Kotsiantis, S.B.: Decision trees: a recent overview. Artif. Intell. Rev. 39(4), 261–283 (2013). https://doi.org/10.1007/s10462-011-9272-4

    Article  Google Scholar 

  20. Lai, P.L., Liang, Y.J., Inselberg, A.: Geometric divide and conquer classification for high-dimensional data. In: DATA International Conference on Data Technologies and Applications, pp. 79–82. SciTePress (2012)

    Google Scholar 

  21. Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/ml

  22. Maadi, M., Akbarzadeh Khorshidi, H., Aickelin, U.: A review on human-AI interaction in machine learning and insights for medical applications. Int. J. Environ. Res. Public Health 18(4), 2121 (2021)

    Article  Google Scholar 

  23. Mantas, C.J., Abellán, J.: Credal decision trees to classify noisy data sets. In: Polycarpou, M., de Carvalho, A.C.P.L.F., Pan, J.-S., Woźniak, M., Quintian, H., Corchado, E. (eds.) HAIS 2014. LNCS (LNAI), vol. 8480, pp. 689–696. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07617-1_60

    Chapter  Google Scholar 

  24. Murthy, S.K.: Automatic construction of decision trees from data: a multi-disciplinary survey. Data Min. Knowl. Discov. 2(4), 345–389 (1998). https://doi.org/10.1023/A:1009744630224

    Article  Google Scholar 

  25. Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Int. Res. 2(1), 1–32 (1994)

    MATH  Google Scholar 

  26. Pedraza, J.A., García-Martínez, C., Cano, A., Ventura, S.: Classification rule mining with iterated greedy. In: Polycarpou, M., de Carvalho, A.C.P.L.F., Pan, J.-S., Woźniak, M., Quintian, H., Corchado, E. (eds.) HAIS 2014. LNCS (LNAI), vol. 8480, pp. 585–596. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07617-1_51

    Chapter  Google Scholar 

  27. Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  28. Rivera-Lopez, R., Canul-Reich, J., Gámez, J.A., Puerta, J.M.: OC1-DE: a differential evolution based approach for inducing oblique decision trees. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2017. LNCS (LNAI), vol. 10245, pp. 427–438. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59063-9_38

    Chapter  Google Scholar 

  29. Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - a survey. Trans. Syst. Man Cyber Part C 35(4), 476–487 (2005)

    Article  Google Scholar 

  30. Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019)

    Article  Google Scholar 

  31. Safavian, S.R., Landgrebe, D.A.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991)

    Article  MathSciNet  Google Scholar 

  32. Soukup, T., Davidson, I.: Visual Data Mining: Techniques and Tools for Data Visualization and Mining. Wiley, New York (2002)

    Google Scholar 

  33. Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley, Reading (2006)

    Google Scholar 

  34. Nguyen, T.D., Ho, T.B., Shimodaira, H.: Interactive visualization in mining large decision trees. In: Terano, T., Liu, H., Chen, A.L.P. (eds.) PAKDD 2000. LNCS (LNAI), vol. 1805, pp. 345–348. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45571-X_40

    Chapter  Google Scholar 

  35. Utgoff, P.E., Brodley, C.E.: An incremental method for finding multivariate splits for decision trees. In: 7th International Conference on Machine Learning, pp. 58–65. Morgan Kaufmann (1990)

    Google Scholar 

  36. Ware, M., et al.: Interactive machine learning: letting users build classifiers. Int. J. Hum.-Comput. Stud. 55(3), 281–292 (2001)

    Article  Google Scholar 

  37. Webb, G.I.: Integrating machine learning with knowledge acquisition. In: Expert Systems, vol. 3, pp. 937–959. Academic Press, San Diego (2002)

    Google Scholar 

  38. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, Burlington (1999)

    Google Scholar 

  39. Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. In: 30th International Conference on Machine Learning, ICML, vol 28, pp. 325–333 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Griffith University, Nathan, QLD, 4111, Australia

    Eugene Gilmore & René Hexel

  2. Universitat Pompeu Fabra, 08018, Barcelona, Spain

    Vladimir Estivill-Castro

Authors
  1. Eugene Gilmore
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vladimir Estivill-Castro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. René Hexel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Estivill-Castro .

Editor information

Editors and Affiliations

  1. University of Deusto, Bilbao, Spain

    Hugo Sanjurjo González

  2. University of Deusto, Bilbao, Spain

    Iker Pastor LĂłpez

  3. University of Deusto, Bilbao, Spain

    Pablo GarcĂ­a Bringas

  4. University of A Coruña, A Coruña, Spain

    Héctor Quintián

  5. University of Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gilmore, E., Estivill-Castro, V., Hexel, R. (2021). More Interpretable Decision Trees. In: Sanjurjo González, H., Pastor López, I., García Bringas, P., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2021. Lecture Notes in Computer Science(), vol 12886. Springer, Cham. https://doi.org/10.1007/978-3-030-86271-8_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86271-8_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86270-1

  • Online ISBN: 978-3-030-86271-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation