Abstract
Deep neural networks, known for their superior learning capabilities, excel in identifying complex relationships between inputs and outputs, leveraging hierarchical, distributed data processing. Despite their impressive performance, these networks often resemble ’black boxes’ due to their highly intricate internal structure and representation, raising challenges in terms of safety, ethical standards, and social norms. Decompositional rule extraction techniques have sought to address these issues by delving into the latent space and retrieving a broad set of symbolic rules. However, the interpretability of these rules is often hampered by their size and complexity. In this paper, we introduce EDICT (Extracting Deep Interpretable Concepts using Trees), a novel approach for rule extraction which employs a hierarchy of decision trees to mine concepts learned in a neural network, thereby generating highly interpretable rules. Evaluations across multiple datasets reveal that our method extracts rules with greater speed and interpretability compared to existing decompositional rule extraction techniques. Simultaneously, our approach demonstrates competitive performance in classification accuracy and model fidelity.
This work has been partially funded by the German Federal Ministry for Economic Affairs and Climate Action within the project “KI Wissen” (19A20020J).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst. 8(6), 373–389 (1995). https://doi.org/10.1016/0950-7051(96)81920-4
Benitez, J., Castro, J., Requena, I.: Are artificial neural networks black boxes? IEEE Trans. Neural Netw. 8(5), 1156–1164 (1997). https://doi.org/10/cr3mjs
Blanc, G., Lange, J., Tan, L.Y.: Top-down induction of decision trees: rigorous guarantees and inherent limitations. arXiv preprint arXiv:1911.07375 (2019). https://doi.org/10.48550/arXiv.1911.07375
Bock, R.: MAGIC Gamma Telescope. UCI Machine Learning Repository (2007). https://doi.org/10.24432/C52C8B
Breiman, L., Friedman, J., Olshen, R., Stone, C.: Classification and regression trees. Wadsworth Int. Group 37(15), 237–251 (1984). https://doi.org/10.1201/9781315139470
Craven, M., Shavlik, J.: Extracting tree-structured representations of trained networks. In: Touretzky, D., Mozer, M., Hasselmo, M. (eds.) Advances in Neural Information Processing Systems, vol. 8. MIT Press (1995)
Frey, P.W., Slate, D.J.: Letter recognition using Holland-style adaptive classifiers. Mach. Learn. 6(2), 161–182 (1991). https://doi.org/10.1007/BF00114162
Fürnkranz, J., Gamberger, D., Lavrač, N.: Foundations of Rule Learning. Cognitive Technologies, Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-540-75197-7
Fu, L.: Rule generation from neural networks. IEEE Trans. Syst. Man Cybern. 24(8), 1114–1124 (1994). https://doi.org/10.1109/21.299696
Hastie, T., Tibshirani, R., Friedman, J.H., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, vol. 2. Springer, New York (2009). https://doi.org/10.1007/978-0-387-84858-7
He, C., Ma, M., Wang, P.: Extract interpretability-accuracy balanced rules from artificial neural networks: a review. Neurocomputing 387, 346–358 (2020). https://doi.org/10.1016/j.neucom.2020.01.036
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539
Muggleton, S., Buntine, W.: Machine invention of first-order predicates by inverting resolution. In: Laird, J. (ed.) Machine Learning Proceedings, pp. 339–352. Morgan Kaufmann (1988). https://doi.org/10.1016/B978-0-934613-64-4.50040-2
Sato, M., Tsukimoto, H.: Rule extraction from neural networks via decision tree induction. In: International Joint Conference on Neural Networks Proceedings, vol. 3, pp. 1870–1875. IEEE (2001). https://doi.org/10.1109/IJCNN.2001.938448
Schmitz, G., Aldrich, C., Gouws, F.: ANN-DT: an algorithm for extraction of decision trees from artificial neural networks. IEEE Trans. Neural Netw. 10(6), 1392–1401 (1999). https://doi.org/10/bzvfs2
Sethi, K.K., Mishra, D.K., Mishra, B.: KDRuleEx: a novel approach for enhancing user comprehensibility using rule extraction. In: 2012 Third International Conference on Intelligent Systems Modelling and Simulation, Kota Kinabalu, Malaysia, pp. 55–60. IEEE (2012). https://doi.org/10/gks4jz
Setiono, R., Leow, W.K.: FERNN: an algorithm for fast extraction of rules from neural networks. Appl. Intell. 11 (2000). https://doi.org/10.1023/A:1008307919726
Shams, Z., et al.: REM: an integrative rule extraction methodology for explainable data analysis in healthcare. medRxiv preprint (2021). https://doi.org/10.1101/2021.01.25.21250459
Taha, I., Ghosh, J.: Symbolic interpretation of artificial neural networks. IEEE Trans. Knowl. Data Eng. 11(3), 448–463 (1999). https://doi.org/10/bpjz4s
Thrun, S.B.: Extracting provably correct rules from artificial neural networks. Technical report, University of Bonn (1993). https://dl.acm.org/doi/book/10.5555/895610
Towell, G.G., Shavlik, J.W.: Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13(1), 71–101 (1993). https://doi.org/10.1007/BF00993103
Tsukimoto, H.: Extracting rules from trained neural networks. IEEE Trans. Neural Netw. 11(2), 377–389 (2000). https://doi.org/10/c43brd
Wörmann, J., Bogdoll, D., Bührle, E., Chen, H., Chuo: Knowledge augmented machine learning with applications in autonomous driving: a survey. arXiv preprint (2022). https://doi.org/10.48550/arXiv.2205.04712
Zarlenga, M.E., Shams, Z., Jamnik, M.: Efficient decompositional rule extraction for deep neural networks. arXiv preprint (2022). http://arxiv.org/abs/2111.12628
Zhou, Z.H., Chen, S.F., Chen, Z.Q.: A statistics based approach for extracting priority rules from trained neural networks. In: Proceedings of the International Joint Conference on Neural Networks, vol. 3, pp. 401–406 (2000). https://doi.org/10.1109/IJCNN.2000.861337
Zilke, J.R., Loza Mencía, E., Janssen, F.: DeepRED – rule extraction from deep neural networks. In: Calders, T., Ceci, M., Malerba, D. (eds.) DS 2016. LNCS (LNAI), vol. 9956, pp. 457–473. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46307-0_29
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Paschke, A. (2023). Extracting Interpretable Hierarchical Rules from Deep Neural Networks’ Latent Space. In: Fensel, A., Ozaki, A., Roman, D., Soylu, A. (eds) Rules and Reasoning. RuleML+RR 2023. Lecture Notes in Computer Science, vol 14244. Springer, Cham. https://doi.org/10.1007/978-3-031-45072-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-45072-3_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45071-6
Online ISBN: 978-3-031-45072-3
eBook Packages: Computer ScienceComputer Science (R0)