Advertisement

Indirectly Encoding Neural Plasticity as a Pattern of Local Rules

  • Sebastian Risi
  • Kenneth O. Stanley
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6226)

Abstract

Biological brains can adapt and learn from past experience. In neuroevolution, i.e. evolving artificial neural networks (ANNs), one way that agents controlled by ANNs can evolve the ability to adapt is by encoding local learning rules. However, a significant problem with most such approaches is that local learning rules for every connection in the network must be discovered separately. This paper aims to show that learning rules can be effectively indirectly encoded by extending the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) method. Adaptive HyperNEAT is introduced to allow not only patterns of weights across the connectivity of an ANN to be generated by a function of its geometry, but also patterns of arbitrary learning rules. Several such adaptive models with different levels of generality are explored and compared. The long-term promise of the new approach is to evolve large-scale adaptive ANNs, which is a major goal for neuroevolution.

Keywords

Learning Rule Local Rule Neural Plasticity High Reward Iterate Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Floreano, D., Urzelai, J.: Evolutionary robots with online self-organization and behavioral fitness. Neural Networks 13, 431–443 (2000)CrossRefGoogle Scholar
  2. 2.
    Niv, Y., Joel, D., Meilijson, I., Ruppin, E.: Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adaptive Behavior 10(1), 5–24 (2002)CrossRefGoogle Scholar
  3. 3.
    Soltoggio, A., Bullinaria, J.A., Mattiussi, C., Dürr, P., Floreano, D.: Evolutionary Advantages of Neuromodulated Plasticity in Dynamic, Reward-based Scenarios. In: Artificial Life XI, pp. 569–576. MIT Press, Cambridge (2008)Google Scholar
  4. 4.
    Bentley, P.J., Kumar, S.: Three ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999), pp. 35–43 (1999)Google Scholar
  5. 5.
    Bongard, J.C.: Evolving modular genetic regulatory networks. In: Proceedings of the 2002 Congress on Evolutionary Computation (2002)Google Scholar
  6. 6.
    Gauci, J., Stanley, K.O.: Autonomous evolution of topographic regularities in artificial neural networks. Neural Computation (to appear 2010)Google Scholar
  7. 7.
    Hornby, G.S., Pollack, J.B.: Creating high-level components with a generative representation for body-brain evolution. Artificial Life 8(3) (2002)Google Scholar
  8. 8.
    Stanley, K.O.: Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems 8(2), 131–162 (2007)CrossRefMathSciNetGoogle Scholar
  9. 9.
    Stanley, K.O., Miikkulainen, R.: A taxonomy for artificial embryogeny. Artificial Life 9(2), 93–130 (2003)CrossRefGoogle Scholar
  10. 10.
    Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  11. 11.
    Gauci, J., Stanley, K.O.: A case study on the critical role of geometric regularity in machine learning. In: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI 2008). AAAI Press, Menlo Park (2008)Google Scholar
  12. 12.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life 15(2), 185–212 (2009)CrossRefGoogle Scholar
  13. 13.
    Clune, J., Beckmann, B.E., Ofria, C., Pennock, R.T.: Evolving coordinated quadruped gaits with the hyperneat generative encoding. In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2009) Special Section on Evolutionary Robotics. IEEE Press, Piscataway (2009)Google Scholar
  14. 14.
    Blynel, J., Floreano, D.: Exploring the T-Maze: Evolving Learning-Like Robot Behaviors using CTRNNs. In: Raidl, G.R., Cagnoni, S., Cardalda, J.J.R., Corne, D.W., Gottlieb, J., Guillot, A., Hart, E., Johnson, C.G., Marchiori, E., Meyer, J.-A., Middendorf, M. (eds.) EvoIASP 2003, EvoWorkshops 2003, EvoSTIM 2003, EvoROB/EvoRobot 2003, EvoCOP 2003, EvoBIO 2003, and EvoMUSART 2003. LNCS, vol. 2611, pp. 593–604. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  15. 15.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10, 99–127 (2002)CrossRefGoogle Scholar
  16. 16.
    Stanley, K.O., Miikkulainen, R.: Competitive coevolution through evolutionary complexification 21, 63–100 (2004)Google Scholar
  17. 17.
    Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evolutionary Intelligence 1(1), 47–62 (2008)CrossRefGoogle Scholar
  18. 18.
    Soltoggio, A.: Neural Plasticity and Minimal Topologies for Reward-Based Learning. In: Proceedings of the 2008 8th International Conference on Hybrid Intelligent Systems, pp. 637–642. IEEE Computer Society, Los Alamitos (2008)CrossRefGoogle Scholar
  19. 19.
    Risi, S., Vanderbleek, S.D., Hughes, C.E., Stanley, K.O.: How novelty search escapes the deceptive trap of learning to learn. In: GECCO 2009: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 153–160. ACM, New York (2009)CrossRefGoogle Scholar
  20. 20.
    Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Bullock, S., Noble, J., Watson, R., Bedau, M. (eds.) Proceedings of the Eleventh International Conference on Artificial Life (Alife XI). MIT Press, Cambridge (2008)Google Scholar
  21. 21.
    Green, C.: SharpNEAT homepage (2003–2006), http://sharpneat.sourceforge.net/

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sebastian Risi
    • 1
  • Kenneth O. Stanley
    • 1
  1. 1.School of Electrical Engineering and Computer ScienceUniversity of Central FloridaOrlando

Personalised recommendations