Evolutionary Computation Applied to the Automatic Design of Artificial Neural Networks and Associative Memories

  • Humberto Sossa
  • Beatriz A. Garro
  • Juan Villegas
  • Gustavo Olague
  • Carlos Avilés
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 175)


In this paper we describe how evolutionary computation can be used to automatically design artificial neural networks (ANNs) and associative memories (AMs). In the case of ANNs, Particle Swarm Optimization (PSO), Differential Evolution (DE), and Artificial Bee Colony (ABC) algorithms are used, while Genetic Programming is adopted for AMs. The derived ANNs and AMs are tested with several examples of well-known databases.


Particle Swarm Optimization Genetic Programming Differential Evolution Automatic Design Synaptic Weight 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, J.A.: A simple neural network generating an interactive memory. Mathematical Biosciences 14, 197–220 (1972)zbMATHCrossRefGoogle Scholar
  2. 2.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Design of Artificial Neural Networks using a Modified Particle Swarm Optimization Algorithm. In: International Joint Conference on Neural Networks (IJCNN 2009), Atlanta, GE, USA, June 14-19, pp. 938–945 (2009)Google Scholar
  3. 3.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Design of Artificial Neural Networks Using Differential Evolution Algorithm. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) ICONIP 2010, Part II. LNCS, vol. 6444, pp. 201–208. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Evolving Neural Networks: A Comparison between Differential Evolution and Particle Swarm Optimization. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) ICSI 2011, Part I. LNCS, vol. 6728, pp. 447–454. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Artificial Neural Network Synthesis by means of Artificial Bee Colony (ABC) Algorithm. In: CEC 2011, New Orleans, June 5-8 (2011)Google Scholar
  6. 6.
    Garro, B.A., Sossa, H., Vázquez, R.A.: Back-Propagation vs Particle Swarm Optimization Algorithm: which Algorithm is better to adjust the Synaptic Weights of a Feed-Forward ANN? International Journal of Artificial Intelligence 7(11), 208–218 (2011)Google Scholar
  7. 7.
    Gurney, K.: An Introduction to Neural Networks. Taylor and Francis Group (1997)Google Scholar
  8. 8.
    Karaboga, D., Basturk, B.: A powerful and efficient algorithm for numerical function optimization: Artificial Bee Colony (ABC) algorithm. Journal of Global Optimization 39(3), 459–471 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Kennedy, J., Eberhart, R.: Particle Swarm Optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. IV, pp. 1942–1948 (1995)Google Scholar
  10. 10.
    Kohonen, T.: Correlation matrix memories. IEEE Transactions on Computers C-21(4), 353–359 (1972)CrossRefGoogle Scholar
  11. 11.
    Koza, J.R.: Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems, Stanford University Computer Science Department technical report (1990)Google Scholar
  12. 12.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press (1992)Google Scholar
  13. 13.
    Koza, J.R.: Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press (1994)Google Scholar
  14. 14.
    Koza, J.R., Bennett, F.H., Andre, D., Keane, M.A.: Genetic Programming III: Darwinian Invention and Problem Solving. Morgan Kaufmann (1999)Google Scholar
  15. 15.
    McCulloch, W., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics 7, 115–133 (1943)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Murphy, P.M., Aha, D.W.: UCI Repository of machine learning databases. University of California, Department of Information and Computer Science, Irvine, CA, US., Technical Report (1994)Google Scholar
  17. 17.
    Ritter, G., Dìaz, J.: Morphological associative memories. IEEE Transactions on Neural Networks 9(2), 281–293 (1998)CrossRefGoogle Scholar
  18. 18.
    Rosenblatt, F.: The Perceptron-a perceiving and recognizing automaton. Report 85-460-1, Cornell Aeronautical Laboratory (1957)Google Scholar
  19. 19.
    Rojas, R.: Neural networks - A systematic introduction. Chapter 7: The back propagation algorithm (1996)Google Scholar
  20. 20.
    Steinbuch, K.: Die Lernmatrix. Kybernetik 1(1), 36–45 (1961)zbMATHCrossRefGoogle Scholar
  21. 21.
    Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11, 341–359 (1997)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Vazquez, R.A.: Izhikevich Neuron Model and its Application in Pattern Recognition. Australian Journal of Intelligent Information Processing Systems 11(1), 53–60 (2010)Google Scholar
  23. 23.
    Vázquez, R.A.: Pattern Recognition Using Spiking Neurons and Firing Rates. In: Kuri-Morales, A., Simari, G.R. (eds.) IBERAMIA 2010. LNCS, vol. 6433, pp. 423–432. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Vazquez, R.A.: A computational approach for modelling the biological olfactory system during an odour discrimination task using spiking neuron. BMC Neuroscience 12(supp.1), 360 (2011)CrossRefGoogle Scholar
  25. 25.
    Vázquez, R.A., Garro, B.A.: Training Spiking Neurons by Means of Particle Swarm Optimization. In: Tan, Y., Shi, Y., Chai, Y., Wang, G. (eds.) ICSI 2011, Part I. LNCS, vol. 6728, pp. 242–249. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  26. 26.
    Villegas, J., Sossa, H., Avilés, C., Olague, G.: Automatic Synthesis of Associative Memories by genetic Programming, a First Approach. Research in Computing Science 42, 91–102 (2009)Google Scholar
  27. 27.
    Villegas, J., Sossa, H., Avilés, C., Olague, G.: Automatic Synthesis of Associative Memories through Genetic Programming: a co-evolutionary approach. Revista Mexicana de Fsica 57(2), 110–116 (2011)Google Scholar
  28. 28.
    Yao, X.: A review of evolutionary artificial neural networks. Int. J. Intell. Syst. 8(4), 539–567 (1993)CrossRefGoogle Scholar
  29. 29.
    Yao, X.: Evolutionary artificial neural networks. In: Kent, A., Williams, J.G. (eds.) Encyclopedia of Computer Science and Technology, vol. 33, pp. 137–170. Marcel Dekker, New York (1995)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Humberto Sossa
    • 1
  • Beatriz A. Garro
    • 1
  • Juan Villegas
    • 2
  • Gustavo Olague
    • 3
  • Carlos Avilés
    • 2
  1. 1.CIC-IPNMexico CityMexico
  2. 2.UAM-AzcapotzalcoMexico CityMexico
  3. 3.CICESE, Carretera Ensenada-TijuanaEnsenadaMexico

Personalised recommendations