Pedagogical Method for Extraction of Symbolic Knowledge
Conference paper
First Online:
Abstract
This paper addresses the extraction of symbolic knowledge from trained artificial neural networks. Specifically, for that purpose the so-called pedagogical approach is incorporated, where the trained network is used as an oracle when inducing the symbolic description. We present an essential extension of the Trepan algorithm proposed originally by Craven and Shavlik [4][5]. The crucial modification concerns the way of generating artificial training instances. The paper ends with an empirical verification of the proposed method on popular machine learning benchmarks and comparison with the original Trepan.
Keywords
Decision Tree Class Label Trained Network Knowledge Extraction Essential Extension
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Preview
Unable to display preview. Download preview PDF.
References
- 1.Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks (1984)Google Scholar
- 2.Partridge, D., Wilks, Y. (eds.): The Foundations of Artificial Intelligence. A Sourcebook. Cambridge Univ. Press (1990)Google Scholar
- 3.Craven, M.W., Shavlik, J.W.: Using Sampling and Queries to Extract Rules from Trained Neural Networks. In: Proc. Eleventh Int. Conf. on Machine Learning, Morgan Kaufmann (1994)Google Scholar
- 4.Craven, M.W., Shavlik, J.W.: Extracting Tree-Structured Representations of Trained Networks. In: Touretzky, D., Mozer, M., Hasselmo, M. (eds.): Advances in Neural Information Processing Systems (vol. 8), MIT Press, Cambridge (1996)Google Scholar
- 5.Craven, M.W.: Extracting Comprehensible Models from Trained Neural Networks. Ph.D. Thesis, Comp. Sci. Dept., Univ. of Wisconsin-Madison (1996)Google Scholar
- 6.Diederich, J.: Explanation and artificial neural networks. Int. J. Man-Machine Studies 37 (1992) 335–355CrossRefGoogle Scholar
- 7.Dinsmore, J. (ed.): The Symbolic and Connectionist Paradigms: Closing the Gap. Lawrence Erlbaum Associates (1992)Google Scholar
- 8.Fu, L.: Rule Generation from Neural Networks. IEEE Trans. SMC-24 (1994)Google Scholar
- 9.Gallant, S.I.: Connectionist expert systems. Comm. of the ACM 31 (1988) 152–169CrossRefGoogle Scholar
- 10.Hayashi, Y.: A neural expert system with automated extraction of fuzzy if-then rules and its application to medical diagnosis. In: Advances in Neural Information Processing Systems (vol. 3). Morgan Kaufmann (1990)Google Scholar
- 11.Merz, C.J., Murphy, P.M.: Repository of machine learning databases. Univ. of California, Dept. of Information and Comp. Sci. (1996)Google Scholar
- 12.Murphy, P.M., Pazzani, M.J.: ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proc. Eighth Int. Machine Learning Workshop, Morgan Kaufmann (1991)Google Scholar
- 13.Quinlan, J.R.: Induction of Decision Trees. Machine Learning 1 (1986) 81–106Google Scholar
- 14.Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)Google Scholar
- 15.Riedmiller, M., Braun, H.: RPROP-A Fast Adaptive Learning Algorithm. Technical Report, Univ. Karlsruhe (1992)Google Scholar
- 16.Słowiński, R., Stefanowski, J., Susmaga, R.: Rough set analysis of attribute dependencies in technical diagnosis. Proc. 4th Intl. Workshop on Rough Sets, Fuzzy Sets, and Machine Discovery, Univ. of Tokyo (1996)Google Scholar
- 17.Szcześniak, I.: Acquisition of symbolic knowledge from neural networks. M.Sc. Thesis, Inst. of Comp. Sci., Poznań Univ. of Tech. (1997)Google Scholar
- 18.Thrun, S.B.: Extracting provably correct rules from artificial neural networks. Technical Report IAI-TR-93-5, Institut für Informatik, Univ. Bonn (1993)Google Scholar
- 19.Towell, G.G., Shavlik, J.W.: Knowledge-Based Artificial Neural Networks. Artificial Intelligence 70 (1994) 119–165MATHCrossRefGoogle Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 1998