Abstract
Applied to certain problems, neuroevolution frequently gets stuck in local optima with very low fitness; in particular, this is true for some reinforcement learning problems where the input to the controller is a high-dimensional and/or ill-chosen state description. Evidently, some controller inputs are “poisonous”, and their inclusion induce such local optima. Previously, we proposed the memetic climber, which evolves neural network topology and weights at different timescales, as a solution to this problem. In this paper, we further explore the memetic climber, and introduce its population-based counterpart: the memetic ES. We also explore which types of inputs are poisonous for two different reinforcement learning problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lucas, S.M., Togelius, J.: Point-to-point car racing: an initial study of evolution versus temporal difference learning. In: Proceedings of the IEEE Symposium on Computational Intelligence and Games (2007)
Igel, C.: Neuroevolution for reinforcement learning using evolution strategies. In: Proceedings of the Congress on Evolutionary Computation (CEC) (2003)
De Nardi, R., Togelius, J., Holland, O., Lucas, S.M.: Evolution of neural networks for helicopter control: Why modularity matters. In: Proceedings of the IEEE Congress on Evolutionary Computation (2006)
Togelius, J., Gomez, F., Schmidhuber, J.: Learning what to ignore: memetic climbing in weight and topology space. In: Congress on Evolutionary Computation (CEC) (to be presented, 2008)
Yao, X.: Evolving artificial neural networks. Proceedings 1447, 87(9) (1999)
Siebel, N.T., Sommer, G.: Evolutionary reinforcement learning of artificial neural networks. International Journal of Hybrid Intelligent Systems 4(3), 171–183 (2007)
Krasnogor, N., Pacheco, A.A.: Memetic algorithms. In: Metaheuristics in Neural Networks Learning, pp. 225–247. Springer, Heidelberg (2006)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127 (2002)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)
Togelius, J.: Optimization, Imitation and Innovation: Computational Intelligence and Games. PhD thesis, Department of Computing and Electronic Systems, University of Essex, Colchester, UK (2007)
Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Togelius, J., Schaul, T., Schmidhuber, J., Gomez, F. (2008). Countering Poisonous Inputs with Memetic Neuroevolution. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds) Parallel Problem Solving from Nature – PPSN X. PPSN 2008. Lecture Notes in Computer Science, vol 5199. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87700-4_61
Download citation
DOI: https://doi.org/10.1007/978-3-540-87700-4_61
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87699-1
Online ISBN: 978-3-540-87700-4
eBook Packages: Computer ScienceComputer Science (R0)