Pedagogical Method for Extraction of Symbolic Knowledge

  • Krzysztof Krawiec
  • Roman Słowiński
  • Irmina Szcześniak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1424)


This paper addresses the extraction of symbolic knowledge from trained artificial neural networks. Specifically, for that purpose the so-called pedagogical approach is incorporated, where the trained network is used as an oracle when inducing the symbolic description. We present an essential extension of the Trepan algorithm proposed originally by Craven and Shavlik [4][5]. The crucial modification concerns the way of generating artificial training instances. The paper ends with an empirical verification of the proposed method on popular machine learning benchmarks and comparison with the original Trepan.


Decision Tree Class Label Trained Network Knowledge Extraction Essential Extension 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth and Brooks (1984)Google Scholar
  2. 2.
    Partridge, D., Wilks, Y. (eds.): The Foundations of Artificial Intelligence. A Sourcebook. Cambridge Univ. Press (1990)Google Scholar
  3. 3.
    Craven, M.W., Shavlik, J.W.: Using Sampling and Queries to Extract Rules from Trained Neural Networks. In: Proc. Eleventh Int. Conf. on Machine Learning, Morgan Kaufmann (1994)Google Scholar
  4. 4.
    Craven, M.W., Shavlik, J.W.: Extracting Tree-Structured Representations of Trained Networks. In: Touretzky, D., Mozer, M., Hasselmo, M. (eds.): Advances in Neural Information Processing Systems (vol. 8), MIT Press, Cambridge (1996)Google Scholar
  5. 5.
    Craven, M.W.: Extracting Comprehensible Models from Trained Neural Networks. Ph.D. Thesis, Comp. Sci. Dept., Univ. of Wisconsin-Madison (1996)Google Scholar
  6. 6.
    Diederich, J.: Explanation and artificial neural networks. Int. J. Man-Machine Studies 37 (1992) 335–355CrossRefGoogle Scholar
  7. 7.
    Dinsmore, J. (ed.): The Symbolic and Connectionist Paradigms: Closing the Gap. Lawrence Erlbaum Associates (1992)Google Scholar
  8. 8.
    Fu, L.: Rule Generation from Neural Networks. IEEE Trans. SMC-24 (1994)Google Scholar
  9. 9.
    Gallant, S.I.: Connectionist expert systems. Comm. of the ACM 31 (1988) 152–169CrossRefGoogle Scholar
  10. 10.
    Hayashi, Y.: A neural expert system with automated extraction of fuzzy if-then rules and its application to medical diagnosis. In: Advances in Neural Information Processing Systems (vol. 3). Morgan Kaufmann (1990)Google Scholar
  11. 11.
    Merz, C.J., Murphy, P.M.: Repository of machine learning databases. Univ. of California, Dept. of Information and Comp. Sci. (1996)Google Scholar
  12. 12.
    Murphy, P.M., Pazzani, M.J.: ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. Proc. Eighth Int. Machine Learning Workshop, Morgan Kaufmann (1991)Google Scholar
  13. 13.
    Quinlan, J.R.: Induction of Decision Trees. Machine Learning 1 (1986) 81–106Google Scholar
  14. 14.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)Google Scholar
  15. 15.
    Riedmiller, M., Braun, H.: RPROP-A Fast Adaptive Learning Algorithm. Technical Report, Univ. Karlsruhe (1992)Google Scholar
  16. 16.
    Słowiński, R., Stefanowski, J., Susmaga, R.: Rough set analysis of attribute dependencies in technical diagnosis. Proc. 4th Intl. Workshop on Rough Sets, Fuzzy Sets, and Machine Discovery, Univ. of Tokyo (1996)Google Scholar
  17. 17.
    Szcześniak, I.: Acquisition of symbolic knowledge from neural networks. M.Sc. Thesis, Inst. of Comp. Sci., Poznań Univ. of Tech. (1997)Google Scholar
  18. 18.
    Thrun, S.B.: Extracting provably correct rules from artificial neural networks. Technical Report IAI-TR-93-5, Institut für Informatik, Univ. Bonn (1993)Google Scholar
  19. 19.
    Towell, G.G., Shavlik, J.W.: Knowledge-Based Artificial Neural Networks. Artificial Intelligence 70 (1994) 119–165zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Krzysztof Krawiec
    • 1
  • Roman Słowiński
    • 1
  • Irmina Szcześniak
    • 1
  1. 1.Institute of Computing SciencePoznań University of TechnologyPoznańPoland

Personalised recommendations