Skip to main content
SpringerLink
Log in

Evolution of internal dynamics for neural network nodes

  • Research Paper
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Most artificial neural networks have nodes that apply a simple static transfer function, such as a sigmoid or gaussian, to their accumulated inputs. This contrasts with biological neurons, whose transfer functions are dynamic and driven by a rich internal structure. Our artificial neural network approach, which we call state-enhanced neural networks, uses nodes with dynamic transfer functions based on n-dimensional real-valued internal state. This internal state provides the nodes with memory of past inputs and computations. The state update rules, which determine the internal dynamics of a node, are optimized by an evolutionary algorithm to fit a particular task and environment. We demonstrate the effectiveness of the approach in comparison to certain types of recurrent neural networks using a suite of partially observable Markov decision processes as test problems. These problems involve both sequence detection and simulated mice in mazes, and include four advanced benchmarks proposed by other researchers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Aberdeen D (2003) A (revised) survey of approximate methods for solving partially observable markov decision processes. Technical report, National ICT Australia, Canberra

  2. Abraham A (2004) Meta learning evolutionary artificial neural networks. Neurocomputing 56:1–38

    Article  Google Scholar 

  3. Aihara K, Takabe T, Toyoda M (1990) Chaotic neural networks. Phys Lett A 144(6):333–340

    Article  MathSciNet  Google Scholar 

  4. Astor J, Adami C (2000) A developmental model for the evolution of artificial neural networks. Artif Life 6(3):189–218

    Article  Google Scholar 

  5. Asuncion A, Newman D (2007) UCI machine learning repository. http://www.ics.uci.edu/∼mlearn/MLRepository.html

  6. Bakker B, Zhumatiy V, Gruener G, Schmidhuber J (2003) A robot that reinforcement-learns to identify and memorize important previous observations. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS2003), pp 430–435

  7. Baxter J (1992) The evolution of learning algorithms for artificial neural networks. In: Green D, Bossomaier T (eds) Complex systems. IOS Press, Amsterdam

    Google Scholar 

  8. Blynel J, Floreano D (2003) Exploring the t-maze: evolving learning-like robot behaviors using ctrnns. In: Proceedings of the second European workshop on evolutionary robotics, pp 593–604

  9. Bongard J (2002) Evolving modular genetic regulatory networks. In: Proceedings of the 2002 congress on evolutionary computation, pp 1872–1877

  10. Cangelosi A, Parisi D, Nolfi S (1994) Cell division and migration in a ’genotype’ for neural networks. Network 5:497–515

    Article  MATH  Google Scholar 

  11. Chalmers D (1990) The evolution of learning: an experiment in genetic connectionism. In: Proceedings of the 1990 Connectionist Models Summer School, pp 81–90

  12. Dellaert F, Beer R (1996) A developmental model for the evolution of complete autonomous agents. In: Proceedings of the fourth international conference on simulation of adaptive behavior, pp 393–401

  13. D’Silva TRJ, Chrien M, Stanley K, Miikkulainen R (2005). Retaining learned behavior during real-time neuroevolution. In: Proceedings of the artificial intelligence and interactive digital entertainment conference (AIIDE)

  14. Eggenberger-Hotz P, Gomez G, Pfeifer R (2002) Evolving the morphology of a neural network for controlling a foveating retina and its test on a real robot. In: Proceedings of the eighth international workshop on the synthesis and simulation of living systems (Artificial Life VIII), pp 243–251

  15. Floreano D, Durr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62

    Article  Google Scholar 

  16. Floreano D, Mattiussi C (2001) Evolution of spiking neural controllers for autonomous vision-based robots. In: Gomi T (ed) Evolutionary robotics: from intelligent robotics to artificial life. Springer, Tokyo

    Google Scholar 

  17. Funahashi K, Nakamura Y (1993) Approximation of dynamical systems by continuous time recurrent neural networks. Neural Netw 6(6):801–806

    Article  Google Scholar 

  18. Gers F, Schmidhuber J (2001) Lstm recurrent networks learn simple context free and context sensitive languages. IEEE Trans Neural Netw 12(6):1333–1340

    Article  Google Scholar 

  19. Gerstner W, Kistler W (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, Cambridge

  20. Gomez F, Miikkulainen R (1999) Solving non-markovian control tasks with neuroevolution. In: Proceedings of the international joint conference on artificial intelligence, pp 1356–1361

  21. Gomez F, Schmidhuber J (2005) Co-evolving recurrent neurons learn deep memory POMDPs. In: Proceedings of the genetic and evolutionary computation conference (GECCO)

  22. Gruau F (1995) Automatic definition of modular neural networks. Adapt Behav 3(2):151–183

    Article  Google Scholar 

  23. Harp S, Samad T, Guha A (1989) Towards the genetic synthesis of neural networks. In: Proceedings of the third international conference on genetic algorithms, pp 360–369

  24. Haykin S (1999) Neural networks: a comprehensive foundation. Prentice-Hall, Englewood Cliffs

  25. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  26. Hussain T (2004) Generic neural markup language: facilitating the design of theoretical neural network models. In: Proceedings of the IEEE international joint conference on neural networks, pp 235–242

  27. Ichinose N, Aihara K, Kotani M (1993) An analysis on dynamics of pulse propagation networks. In: Proceedings of the international joint conference on neural networks, pp 2315–2318

  28. Izhikevich E (2004) Which model to use for cortical spiking neurons?. IEEE Trans Neural Netw 15(5):1063–1070

    Article  Google Scholar 

  29. Jakobi N (1995) Harnessing morphogenesis. In: International conference on information processing in cells and tissues, pp 29–41

  30. Jakobi N (1998) Evolutionary robotics and the radical envelope-of-noise hypothesis. Adapt Behav 6(2):325–368

    Article  Google Scholar 

  31. Kitano H (1990) Designing neural networks using genetic algorithms with graph generation system. Complex Syst 4:461–476

    MATH  Google Scholar 

  32. Kumar S, Bentley P (2003) Computational embryology: past, present and future. Springer, New York, pp 461–477

    Google Scholar 

  33. Luke S (2002) An evolutionary computation and genetic programming system. http://cs.gmu.edu/eclab/projects/ecj/docs/

  34. McCallum A (1993) Overcoming incomplete perception with utile distinction memory. In: International conference on machine learning, pp 190–196

  35. Montana D, Davis L (1989) Training feedforward neural networks using genetic algorithms. In: Proceedings of the international joint conference on artificial intelligence (IJCAI), pp 762–767

  36. Montana D, VanWyk E, Brinn M, Montana J, Milligan S (2006) Genomic computing networks learn complex POMDPs. In: Proceedings of the genetic and evolutionary computation conference (GECCO), pp 233–234

  37. Nolfi S, Miglino O, Parisi D (1994) Phenotypic plasticity in evolving neural networks. In: Proceedings of the internation conference from perception to action, pp 146–157

  38. Osana Y, Hattori M, Hagiwara M (1996) Chaotic bidirectional associative memory. In: IEEE international conference on neural networks, pp 816–821

  39. Pearlmutter B (1995) Gradient calculation for dynamic recurrent neural networks: a survey. IEEE Trans Neural Netw 6(5):1212–1228

    Article  Google Scholar 

  40. Pollack M, Ringuette M (1990) Introducing the Tileworld: experimentally evaluating agent architectures. In: Proceedings of the eighth national conference on artificial intelligence, pp 183–189

  41. Russel S, Norvig P (1995) Artificial intelligence: a modern approach. Prentice-Hall, Englewood Cliffs

  42. Schmidhuber J, Wierstra D, Galiolo M, Gomez F (2007) Training recurrent networks by evolino. Neural Comput 19(3):757–779

    Article  MATH  Google Scholar 

  43. Schmidhuber J, Wierstra D, Gomez F (2005) Evolino: hybrid neuroevolution/optimal linear search for sequence learning. In: Proceedings of the 19th international joint conference on artificial intelligence (IJCAI), pp 853–858

  44. Stanley K, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127

    Article  Google Scholar 

  45. Stanley K, Miikkulainen R (2003) A taxonomy for artificial embryogeny. Artif Life 9(2):93–130

    Article  Google Scholar 

  46. Watkins C, Dayan P (1992) Q-learning. Mach Learn 8(3):279–292

    MATH  Google Scholar 

  47. Whitley D (1989) Applying genetic algorithms to neural network learning. In: Proceedings of the seventh conference of the society of artificial intelligence and simulation of behavior, pp 137–144

  48. Yamauchi B, Beer R (1994) Sequential behavior and learning in evolved dynamical neural networks. Adapt Behav 2(3):219–246

    Article  Google Scholar 

  49. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. BBN Technologies, 10 Moulton Street, Cambridge, MA, 02138, USA

    David Montana, Eric VanWyk, Marshall Brinn, Joshua Montana & Stephen Milligan

Authors
  1. David Montana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric VanWyk
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marshall Brinn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joshua Montana
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephen Milligan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Montana.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Montana, D., VanWyk, E., Brinn, M. et al. Evolution of internal dynamics for neural network nodes. Evol. Intel. 1, 233–251 (2009). https://doi.org/10.1007/s12065-009-0017-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-009-0017-0

Keywords

Search

Navigation