Partially-Connected Artificial Neural Networks Developed by Grammatical Evolution for Pattern Recognition Problems

  • Olga Quiroz-Ramírez
  • Andrés EspinalEmail author
  • Manuel Ornelas-Rodríguez
  • Alfonso Rojas-Domínguez
  • Daniela Sánchez
  • Héctor Puga-Soberanes
  • Martin Carpio
  • Luis Ernesto Mancilla Espinoza
  • Janet Ortíz-López
Part of the Studies in Computational Intelligence book series (SCI, volume 749)


Evolutionary Artificial Neural Networks (EANNs) are a special case of Artificial Neural Networks (ANNs) for which Evolutionary Algorithms (EAs) are used to modify or create them. EANNs adapt their defining components ad hoc for solving a particular problem with little or no intervention of human expert. Grammatical Evolution (GE) is an EA that has been used to indirectly develop ANNs, among other design problems. This is achieved by means of three elements: a Context-Free Grammar (CFG) which includes the ANNs defining components, a search engine that drives the search process and a mapping process. The last component is a heuristic for transforming each GE’s individual from its genotypic form into its phenotypic form (a functional ANN). Several heuristics have been proposed as mapping processes in the literature; each of them may transform a specific individual’s genotypic form into a very different phenotypic form. In this paper, partially-connected ANNs are automatically developed by means of GE. A CFG is proposed to define the topologies, a Genetic Algorithm (GA) is the search engine and three mapping processes are tested for this task; six well-known pattern recognition benchmarks are used to statistically compare them. The aim of this work for using and comparing different mapping process is to analyze them for setting the basis of a generic framework to automatically create ANNs.


Evolutionary artificial neuronal networks Grammatical evolution Mapping process Pattern recognition 



We are grateful to the National Council for Science and Technology (CONACYT) of Mexico for the support provided by means of the Scholarship for Postgraduate Studies: 703036 (O. Quiroz) and Research Grant: CÁTEDRAS-2598 (A. Rojas) as well as to the National Technology Institute of Mexico.


  1. 1.
    K. Soltanian, F.A. Tab, F.A. Zar, I. Tsoulos, Artificial neural networks generation using grammatical evolution, in 21st Iranian Conference on Electrical Engineering (ICEE), pp. 1–5 (2013)Google Scholar
  2. 2.
    C.M. Bishop, Neural networks for pattern recognition. J. Am. Stat. Assoc. 92, 482 (1995)MathSciNetzbMATHGoogle Scholar
  3. 3.
    G.P. Zhang, Neural networks for classification: a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 30(4), 451–462 (2000)Google Scholar
  4. 4.
    B.A. Garro, H. Sossa, R.A. Vazquez, Design of artificial neural networks using a modified particle swarm optimization algorithm, in 2009 International Joint Conference on Neural Networks, pp. 938–945 (2009)Google Scholar
  5. 5.
    F. Ahmadizar, K. Soltanian, F. Akhlaghiantab, I. Tsoulos, Engineering applications of artificial intelligence artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Eng. Appl. Artif. Intell. 39, 1–13 (2015)CrossRefGoogle Scholar
  6. 6.
    S. Kulluk, L. Ozbakir, A. Baykasoglu, Training neural networks with harmony search algorithms for classification problems. Eng. Appl. Artif. Intell. 25(1), 11–19 (2012)CrossRefGoogle Scholar
  7. 7.
    D. Elizondo, E. Fiesler, A survey of partially connected neural networks. Int. J. Neural Syst. 8, 535–558 (1997)Google Scholar
  8. 8.
    E. Cantu-Paz, C. Kamath, An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. 35, 915–927 (2005)Google Scholar
  9. 9.
    D.J. Montana, L. Davis, Training feedforward neural networks using genetic algorithms, in Proceedings of 11th International Joint Conference Artificial Intelligence, vol. 1, vol. 89, pp. 762–767 (1989)Google Scholar
  10. 10.
    M. Gardner, S. Dorling, Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmos. Environ. 32(14–15), 2627–2636 (1998)CrossRefGoogle Scholar
  11. 11.
    D. Simon, Evolutionary Algorithms Biologically-Inspired and Population-Based Approaches to Computer Intelligence (Wiley, Hoboken, New Jersey, 2013)zbMATHGoogle Scholar
  12. 12.
    J. Branke, Evolutionary algorithms for neural network design and training, in Workshop on Genetic Algorithms and its Applications, pp. 1–21 (1995)Google Scholar
  13. 13.
    S. Ding, H. Li, C. Su, J. Yu, Evolutionary artificial neural networks: a review. Artif. Intell. Rev. 39(3) (2011)Google Scholar
  14. 14.
    X. Yao, Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  15. 15.
    L. Wang, Y. Zeng, T. Chen, Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst. Appl. 42(2), 855–863 (2015)CrossRefGoogle Scholar
  16. 16.
    M. O’Neill, C. Ryan, Grammatical Evolution: Evolutionary Automatic Programming in a Arbitrary Language, vol. 4 (Springer, US, 2003)CrossRefzbMATHGoogle Scholar
  17. 17.
    M. O’Neill, C. Ryan, Grammatical evolution. IEEE Trans. Evol. Comput. 5(4), 349–358 (2001)CrossRefGoogle Scholar
  18. 18.
    C. Ryan, J.J. Collins, M. Neill, Grammatical Evolution: Evolving Programs for an Arbitrary Language (Springer, Berlin Heidelberg, 1998)Google Scholar
  19. 19.
    I. Tsoulos, D. Gavrilis, E. Glavas, Neural network construction and training using grammatical evolution. Neurocomputing 72(1–3), 269–277 (2008)CrossRefGoogle Scholar
  20. 20.
    D. Fagan, “Analysing the genotype-phenotype map in grammatical evolution,” for the degree of Ph.D. at the School of Computer Science and Informatics College of Science (2013)Google Scholar
  21. 21.
    T. Bäck, 1996—Back—Evolutionary Algorithms in Theory And Practice.pdf (Oxford University Press, Inc., 1996)Google Scholar
  22. 22.
    M. Mitchell, L.D. Davis, Handbook of genetic algorithms. Artif. Intell. 100(1–2), 325–330 (1998)CrossRefzbMATHGoogle Scholar
  23. 23.
    J.P.M. De Sa, Pattern Recognition: Concepts Methods and Applications (Springer, 2001)Google Scholar
  24. 24.
    J.H. Holland, Adaptation in Natural and Artificial Systems: An introductory Analysis with Applications to Biology, Control and Artificial Intelligence (MIT Press, 1975), p. 183Google Scholar
  25. 25.
    J.D. Schaffer, L.J. Eshelman, Combinations of genetic algorithms and neural networks: a survey of the state of the art, in International Workshop on Combinations of Genetic Algorithms Neural Networks, 1992, COGANN-92 June 6, 1992, Balt. Maryland/Cat. No. 92Th0435-8 E-b., pp. 1–37 (1992)Google Scholar
  26. 26.
    J. Arifovic, R. Gencay, Using genetic algorithms to select architecture of a feedforward artificial neural network. Phys. A Stat. Mech. Appl. 289(3–4), 574–594 (2001)CrossRefzbMATHGoogle Scholar
  27. 27.
    P.E. Valencia, Optimización Mediante Algoritmos Genéticos, Anales del Instituto de Ingenieros de Chile, vol. 109, no. 2, pp. 83–92 (1997)Google Scholar
  28. 28.
    T. Bäck, D.B. Fogel, Z. Michalewicz, Evolutionary Computation 1: Basic Algorithms and Operators, 1st edn. (CRC Press, 2000)Google Scholar
  29. 29.
    X. Yao, Evolutionary artificial neural networks. Int. J. Neural Syst. 4, 203–222 (1993)CrossRefGoogle Scholar
  30. 30.
    X. Yaot, A review of evolutionary artificial neural networks. Common. Sci. Ind. Res. Organ. 8, 539–567 (1993)Google Scholar
  31. 31.
    B.A. Garro, H. Sossa, R.A. Vazquez, Design of artificial neural networks using differential evolution algorithm, in Proceedings of 17th International Conference Neural Information Processing Models and Applications, vol. Part II, pp. 201–208 (2010)Google Scholar
  32. 32.
    B.A. Garro, R.A. Vázquez, Swarm optimization algorithms. Comput. Intell. Neurosci. 2015, 20 (2015)CrossRefGoogle Scholar
  33. 33.
    D. Whitley, An overview of evolutionary algorithms: practical issues and common pitfalls. Inf. Softw. Technol. 43, 817–831 (2001)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Olga Quiroz-Ramírez
    • 1
  • Andrés Espinal
    • 2
    Email author
  • Manuel Ornelas-Rodríguez
    • 1
  • Alfonso Rojas-Domínguez
    • 1
  • Daniela Sánchez
    • 3
  • Héctor Puga-Soberanes
    • 1
  • Martin Carpio
    • 1
  • Luis Ernesto Mancilla Espinoza
    • 1
  • Janet Ortíz-López
    • 4
  1. 1.Tecnológico Nacional de México-Instituto Tecnológico de LeónLeónMexico
  2. 2.División de Ciencias Económico AdministrativasUniversidad de GuanajuatoGuanajuatoMexico
  3. 3.Tecnológico Nacional de México-Instituto Tecnológico de TijuanaTijuanaMexico
  4. 4.Escuela Internacional de DoctoradoUniversidad de VigoVigoSpain

Personalised recommendations