Designing a Decompositional Rule Extraction Algorithm for Neural Networks

  • Jen-Cheng Chen
  • Jia-Sheng Heh
  • Maiga Chang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3971)


The neural networks are successfully applied to many applications in different domains. However, due to the results made by the neural networks are difficult to explain the decision process of neural networks is supposed as a black box. The explanation of reasoning is important to some applications such like credit approval application and medical diagnosing software. Therefore, the rule extraction algorithm is becoming more and more important in explaining the extracted rules from the neural networks. In this paper, a decompositional algorithm is analyzed and designed to extract rules from neural networks. The algorithm is simple but efficient; can reduce the extracted rules but improve the efficiency of the algorithm at the same time. Moreover, the algorithm is compared to the other two algorithms, M-of-N and Garcez, by solving the MONK’s problem.


Neural Network Artificial Neural Network Expert System Boolean Logic Decompositional Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andrews, R., Diederich, J., Tickle, A.B.: Survey and Critique of Techniques for Extracting rules from Trained Artificial Neural Networks. Knowledge Base Systems 8(6), 373–389 (1995)CrossRefGoogle Scholar
  2. 2.
    Andrews, R., Geva, S.: Inserting and Extracting Knowledge from Constrained Error Back Propagation Networks. In: Proc. of 6th Australian Conference on Neural Networks, Sydney, NSW (1995)Google Scholar
  3. 3.
    Andrews, R., Geva, S.: Rule Extraction from Local Cluster Neural Nets. Neurocomputing 47, 1–20 (2002)MATHCrossRefGoogle Scholar
  4. 4.
    Chang, M., Chen, J.C., Chang, J.W., Heh, J.S.: Advanced Process Control Expert System of CVD Membrane Thickness Based on Neural Network. Materials Science Forum 505-507, 313–318 (2006)CrossRefGoogle Scholar
  5. 5.
    Chen, J.C., Liu, T.S., Weng, C.S., Heh, J.S.: An Expert System of Coronary Artery Disease in Chinese and Western Medicine. In: Proc. of 6th Asian-Pacific Conference on Medical and Biological Engineering, Tsukuba, Japan, April 24-27 (2005) PA-3-06Google Scholar
  6. 6.
    Craven, M.W., Shavlik, J.W.: Using Sampling and Queries to Extract Rules from Trained Neural Networks. In: Proc. of 11th International Conference on Machine Learning, New Brunswick, pp. 37–45 (1994)Google Scholar
  7. 7.
    Freeman, J.A., Skapura, D.M.: Neural Networks. Addison-Wesley, New York (1992)Google Scholar
  8. 8.
    Fu, L.M.: Rule Generation from Neural Networks. IEEE Transactions on Systems, Man, and Cybernetics 28(8), 1114–1124 (1994)Google Scholar
  9. 9.
    Garcez, A.S., d’Avila, B.K., Gabbay, D.M.: Symbolic Knowledge Extraction from Trained Neural Networks: A Sound Approach. Artificial Intelligence 125, 155–207 (2001)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Jackson, P.: Introduction to Expert Systems. Addison-Wesley, New York (1999)Google Scholar
  11. 11.
    Krishnan, R., Sivakumar, G., Bhattacharya, P.: A Search Technique for Rule Extraction from Trained Neural Networks. Pattern Recognition Letters 20, 273–280 (1999)MATHCrossRefGoogle Scholar
  12. 12.
    Pop, E., Hayward, R., Diedrich, A.: RULENEG: Extracting Rules from a Trained Neural Network Using Stepwise Negation. Technical Report, Neurocomputing Research Centre, Queensland University of Technology (1994)Google Scholar
  13. 13.
    Setiono, R.: A Penalty Function Approach for Pruning Feedforward Neural Networks. Neural Computation 9(1), 185–204 (1997)MATHCrossRefGoogle Scholar
  14. 14.
    Setiono, R.: Extracting Rules from Neural Networks by Pruning and Hidden-unit Splitting. Neural Computation 9(1), 205–225 (1997)MATHCrossRefGoogle Scholar
  15. 15.
    Taha, I.A., Ghosh, J.: Symbolic Interpretation of Artificial Neural Networks. IEEE transactions on Knowledge and Data Engineering 11(3), 448–463 (1999)CrossRefGoogle Scholar
  16. 16.
    Thrun, S.B. (ed.): The MONK’s Problems: A Performance Comparison of Different Learning Algorithms. Technical Report CMU-CS-91-197. Carnegie Mellon University (1991)Google Scholar
  17. 17.
    Thrun, S.B.: Extracting Provably Correct Rules from Artificial Neural Networks. Technical Report IAI-TR-93-5, Institut für Informatik III. University of Bonn, Germany (1994)Google Scholar
  18. 18.
    Tickle, A.B., Andrews, R., Golea, M., Diederich, J.: The Truth Will Come to Light: Directions and Challenges in Extracting the Knowledge Embedded within Trained Artificial Neural Networks. IEEE Trans. Neural Networks 9(6), 1057–1068 (1998)CrossRefGoogle Scholar
  19. 19.
    Towell, G.G., Shavlik, J.W.: The Extraction of Refined Rules from Knowledge-Based Neural Networks. Machine Learning 13, 71–101 (1993)Google Scholar
  20. 20.
    Towell, G.G., Shavlik, J.W.: Knowledge-Based Artificial Neural Networks. Artificial Intelligence 70, 119–165 (1994)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jen-Cheng Chen
    • 1
  • Jia-Sheng Heh
    • 2
  • Maiga Chang
    • 3
  1. 1.Department of Electronic EngineeringChung Yuan Christian University, Chung Li, Taiwan, R.O.C.Chung-LiTaiwan
  2. 2.Department of Information and Computer EngineeringChung Yuan Christian University, Chung Li, Taiwan, R.O.C.Chung-LiTaiwan
  3. 3.National Science and Technology Program for e-LearningChung-LiTaiwan

Personalised recommendations