Rules and Generalization Capacity Extraction from ANN with GP

  • Juan R. Rabuñal
  • Julián Dorado
  • Alejandro Pazos
  • Daniel Rivero
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2686)


Different techniques for extracting Artificial Neural Networks (ANN) rules have been used up to the present time, but most of them have focused on certain types of networks and their training. However, there are practically no methods which deal with ANN rule-discovery as systems that are independent from their architecture, training, and internal distribution of weights, connections, and activation functions. This paper proposes a method based on Genetic Programming (GP) with the purpose of achieving the generalization capacity characteristic of ANNs, by means of symbolic rules which can be understood by human beings.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tickle, A.B., Andrews, R. Golea, M. Diederich, J. “The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks”. IEEE Transaction on Neural Networks, vol. 9, n° 6, pag. 1057–1068, 1998.CrossRefGoogle Scholar
  2. 2.
    Cramer, N.L. “A Representation for the Adaptive Generation of Simple Sequential Programs”, Grefenstette: Proceedings of First International Conference on Genetic Algorithms, 1985.Google Scholar
  3. 3.
    Fujiki C. “Using the Genetic Algorithm to Generate Lisp Source Code to Solve the Prisoner’s Dilemma”, International Conf on GAs, pp. 236–240, 1987.Google Scholar
  4. 4.
    Friedberg R.M. “A learning machine: Part I”, IBM Journal of Research and Development, 2(1) pag. 2–13, 1958.MathSciNetCrossRefGoogle Scholar
  5. 5.
    Koza J. “Genetic Programming. On the Programming of Computers by means of Natural Selection”. The Mit Press, Cambridge, Massachusetts, 1992.MATHGoogle Scholar
  6. 6.
    Jang J., Sun C. “Functional equivalence between radial basis function networks and fuzzy inference systems”. IEEE Transactions on Neural Networks, vol. 4, páginas 156–158, 1992.CrossRefGoogle Scholar
  7. 7.
    Buckley J.J., Hayashi Y., Czogala E. “On the equivalence of neural nets and fuzzy expert systems”, Fuzzy Sets Systems, N° 53, pag. 129–134, 1993.Google Scholar
  8. 8.
    Benítez J. M., Castro J. L., Requena I. “Are artificial neural networks black boxes?“. IEEE Transactions on Neural Networks, vol. 8, n° 5, pag. 1156–1164, 1997.CrossRefGoogle Scholar
  9. 9.
    Andrews R. Diederich J. & Tickle A. “A Survey and Critique of Techniques For Extracting Rules From Trained Artificial Neural Networks”. Knowledge Based Systems vol. 8, pag. 373–389, 1995.CrossRefGoogle Scholar
  10. 10.
    Towell G., Shavlik J. W. “Knowledge-Based Artificial Neural Networks”. Artificial Intelligence, 70, pag. 119–165.Google Scholar
  11. 11.
    Thrun S. “Extracting rules from networks with distributed representations”. Advances in Neural Information Processing Systems (NIPS) 7, G. Tesauro, D. Touretzky, T. Leen (eds), MIT Press.Google Scholar
  12. 12.
    Pop E., Hayward R., Diederich J. “RULENEG: Extracting Rules from a Trained ANN by Stepwise Negation”. Queensland University of Technology, Neurocomputing Research Centre. QUT NRC Technical report, 1994.Google Scholar
  13. 13.
    Tickle A. B., Orlowski M., Diedrich J. “DEDEC: A methodology for extracting rules from trained artificial neural networks”. Queensland University of Technology, Neurocomputing Research Centre. QUT NRC Technical report, 1996.Google Scholar
  14. 14.
    Chalup S., Hayward R., Diedrich J. “Rule extraction from artificial neural networks trained on elementary number classification task”. Queensland University of Technology, Neurocomputing Research Centre. QUT NRC Technical report, 1998.Google Scholar
  15. 15.
    Visser U., Tickle A., Hayward R., Andrews R. “Rule-Extraction from trained neural networks: Different techniques for the determination of herbicides for the plant protection advisory system PRO_PLANT”. Proceedings of the rule extraction from trained artificial neural networks workshop, Brighton, UK, pag. 133–139. 1996.Google Scholar
  16. 16.
    Keedwell E., Narayanan A., Savic D. “Creating rules from trained neural networks using genetic algorithms”. In the International Journal of Computers, Systems and Signals (IJCSS), vol. 1, N° 1, pag. 30–42. 2000.Google Scholar
  17. 17.
    Wong M.L., Leung K.S. “Data Mining using Grammar Based Genetic Programming and Applications”, Kluwer Academic Publishers, 2000.Google Scholar
  18. 18.
    Engelbrecht A.P., Rouwhorst S.E., Schoeman L. “A Building Block Approach to Genetic Programming for Rule Discovery”, Data Mining: A Heuristic Approach, Abbass, R. Sarkar, C. Newton editors, Idea Group Publishing, 2001.Google Scholar
  19. 19.
    Fisher R.A. “The use of multiple measurements in taxonomic problems”, Annals of Eugenics, vol. 7, pag. 179–188, 1936.Google Scholar
  20. 20.
    Rivero D., Rabuñal J., Dorado J., Pazos A., Pedreira N. ”Extracting knowledge from databases with Genetic Programming: Tris flower classification problem”, Proceedings of CIMCA and TAWTIC, 2003Google Scholar
  21. 21.
    Duch W., Adamczak R., Grabczewski K. “A new methodology of extraction, optimisation and application of crisp and fuzzy logical rules”, IEEE Transactions on Neural Networks, vol. 11, n° 2, 2000.Google Scholar
  22. 22.
    Martínez A., Goddard J. “Definitión de una red neuronal para clasificación por medio de un programa evolutivo”, revista mexicana de ingeniería biomédica, vol. 22, 1, pp. 4–11, 2001.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Juan R. Rabuñal
    • 1
  • Julián Dorado
    • 1
  • Alejandro Pazos
    • 1
  • Daniel Rivero
    • 1
  1. 1.Facultad Informática, Campus ElviñaUniv. da CoruñaA CoruñaSpain

Personalised recommendations