Advertisement

The Evolution of Self-taught Neural Networks in a Multi-agent Environment

  • Nam LeEmail author
  • Anthony Brabazon
  • Michael O’Neill
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11454)

Abstract

Evolution and learning are two different forms of adaptation by which the organism can change their behaviour to cope with problems posed by the environment. The second form of adaptation occurs when individuals exhibit plasticity in response to environmental conditions that may strengthen their survival. Learning has been shown to be beneficial to the evolutionary process through the Baldwin Effect. This line of thought has also been employed in evolving adaptive neural networks, in which learning algorithms, such as Backpropagation, can be used to enhance the adaptivity of the population. Most work focuses on evolving learning agents in separate environments, this means each agent experiences its own environment (mostly similar), and has no interactive effect on others (e.g., the more one gains, the more another loses). The competition for survival in such settings is not that strong, if being compared to that of a multi-agent (or shared) environment. This paper investigates an evolving population of self-taught neural networks – networks that can teach themselves – in a shared environment. Experimental results show that learning presents an effect in increasing the performance of the evolving multi-agent system. Indications for future work on evolving neural networks are also presented.

Keywords

The Baldwin effect Neural networks Neuroevolution Meta-learning Self-learning 

Notes

Acknowledgments

This research is funded by Science Foundation Ireland under Grant No. 13/IA/1850.

References

  1. 1.
    Baldwin, J.M.: A new factor in evolution. Am. Nat. 30(354), 441–451 (1896)CrossRefGoogle Scholar
  2. 2.
    Morgan, C.L.: On modification and variation. Science 4(99), 733–740 (1896)CrossRefGoogle Scholar
  3. 3.
    Hinton, G.E., Nowlan, S.J.: How learning can guide evolution. Complex Syst. 1, 495–502 (1987)zbMATHGoogle Scholar
  4. 4.
    Nolfi, S., Parisi, D., Elman, J.L.: Learning and evolution in neural networks. Adapt. Behav. 3(1), 5–28 (1994)CrossRefGoogle Scholar
  5. 5.
    Stanley, K.O., Bryant, B.D., Miikkulainen, R.: Evolving adaptive neural networks with and without adaptive synapses. In: Proceedings of the 2003 Congress on Evolutionary Computation. IEEE, Piscataway (2003)Google Scholar
  6. 6.
    Parisi, D., Nolfi, S., Cecconi, F.: Learning, behavior, and evolution (1991)Google Scholar
  7. 7.
    Le, N.: How the baldwin effect can guide evolution in dynamic environments. In: 7th International Conference on the Theory and Practice of Natural Computing. IEEE Press, 12–14 December 2018Google Scholar
  8. 8.
    Mery, F., Kawecki, T.J.: The effect of learning on experimental evolution of resource preference in drosophila melanogaster. Evolution 58(4), 757 (2004)CrossRefGoogle Scholar
  9. 9.
    Holland, J.H.: Adaptation in Natural and Artificial Systems, 2nd edn. University of Michigan Press, Ann Arbor (1992). 1975Google Scholar
  10. 10.
    Kauffman, S.A., Weinberger, E.D.: The NK model of rugged fitness landscapes and its application to maturation of the immune response. J. Theoret. Biol. 141(2), 211–245 (1989)CrossRefGoogle Scholar
  11. 11.
    Mayley, G.: Guiding or hiding: explorations into the effects of learning on the rate of evolution. In: Proceedings of the Fourth European Conference on Artificial Life. MIT Press, pp. 135–144 (1997)Google Scholar
  12. 12.
    Bull, L.: On the baldwin effect. Artif. Life 5(3), 241–246 (1999)CrossRefGoogle Scholar
  13. 13.
    Watson, J., Wiles, J.: The rise and fall of learning: a neural network model of the genetic assimilation of acquired traits. In: Proceedings of the 2002 Congress on Evolutionary Computation. IEEEGoogle Scholar
  14. 14.
    Todd, P.M., Miller, G.F.: Exploring adaptive agency ii: Simulating the evolution of associative learning. In: Proceedings of the First International Conference on Simulation of Adaptive Behavior on From Animals to Animats, pp. 306–315. MIT Press, Cambridge (1990)Google Scholar
  15. 15.
    Hebb, D.: The Organization of Behavior: A Neuropsychological Theory, ser. A Wiley book in clinical psychology. Wiley, New York (1949)Google Scholar
  16. 16.
    Harvey, I.: Is there another new factor in evolution? Evol. Comput. 4(3), 313–329 (1996)CrossRefGoogle Scholar
  17. 17.
    Ackley, D., Littman, M.: Interactions between learning and evolution. In: Langton, C.G., Taylor, C., Farmer, C.D., Rasmussen, S. (Eds.) Artificial Life II, SFI Studies in the Sciences of Complexity. Addison-Wesley, vol. X, pp. 487–509, Reading (1992)Google Scholar
  18. 18.
    Downing, K.L.: The baldwin effect in developing neural networks. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, ser. GECCO 2010, pp. 555–562. ACM, New York (2010)Google Scholar
  19. 19.
    Soltoggio, A., Stanley, K.O., Risi, S.: Born to learn: the inspiration, progress, and future of evolved plastic artificial neural networks. Neural Netw. 108, 48–67 (2018)CrossRefGoogle Scholar
  20. 20.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Neurocomputing: foundations of research. In: Anderson, J.A., Rosenfeld, E. (Eds.) ch. Learning Representations by Back-propagating Errors, pp. 696–699. MIT Press, Cambridge (1988)Google Scholar
  21. 21.
    Yao, X.: Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  22. 22.
    Le, N., O’Neill, M., Brabazon, A.: The baldwin effect reconsidered through the prism of social learning. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8, July 2018Google Scholar
  23. 23.
    Rendell, L., Fogarty, L., Hoppitt, W.J., Morgan, T.J., Webster, M.M., Laland, K.N.: Cognitive culture: theoretical and empirical insights into social learning strategies. Trends Cogn. Sci. 15(2), 68–76 (2011)CrossRefGoogle Scholar
  24. 24.
    Le, N., O’Neill, M., Brabazon, A.: Adaptive advantage of learning strategies: a study through dynamic landscape. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11102, pp. 387–398. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-99259-4_31
  25. 25.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Natural Computing Research and Applications GroupUniversity College DublinDublinIreland

Personalised recommendations