Advertisement

Measuring and Optimizing Behavioral Complexity for Evolutionary Reinforcement Learning

  • Faustino J. Gomez
  • Julian Togelius
  • Juergen Schmidhuber
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5769)

Abstract

Model complexity is key concern to any artificial learning system due its critical impact on generalization. However, EC research has only focused phenotype structural complexity for static problems. For sequential decision tasks, phenotypes that are very similar in structure, can produce radically different behaviors, and the trade-off between fitness and complexity in this context is not clear. In this paper, behavioral complexity is measured explicitly using compression, and used as a separate objective to be optimized (not as an additional regularization term in a scalar fitness), in order to study this trade-off directly.

Keywords

Genetic Program Pareto Front Recurrent Neural Network Phenotype Space Minimum Description Length Principle 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Baronchelli, A., Caglioti, E., Loreto, V.: Artificial sequences and complexity measures. Journal of Statistical Mechanics (2005)Google Scholar
  2. 2.
    De Jong, E.D., Pollack, J.B.: Multi-objective methods for tree size control. Genetic Programming and Evolvable Machines 4(3), 211–233 (2003)CrossRefGoogle Scholar
  3. 3.
    De Jong, E.D., Watson, R.A., Pollack, J.B.: Reducing bloat and promoting diversity using multi-objective methods. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, pp. 11–18. Morgan Kaufmann, San Francisco (2001)Google Scholar
  4. 4.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transaction on Evolutionary Computation 6, 182–197 (2002)CrossRefGoogle Scholar
  5. 5.
    Gomez, F.: Sustaining diversity using behavioral information distance. In: Proceedings of the Genetic and Evolutionary Computation Conference (to appear, 2009)Google Scholar
  6. 6.
    Iba, H., Garis, H.D., Sato, T.: Genetic programming using a minimum description length principle. In: Advances in Genetic Programming, pp. 265–284. MIT Press, Cambridge (1994)Google Scholar
  7. 7.
    Li, M., Vitányi, P.M.B.: An introduction to Kolmogorov complexity and its applications. In: van Leeuwen, J. (ed.) Handbook of Theoretical Computer Science, pp. 188–254. Elsevier Science Publishers B.V., Amsterdam (1990)Google Scholar
  8. 8.
    Rissanen, J.: Modeling by shortest data description. Automatica, 465–471 (1978)Google Scholar
  9. 9.
    Teller, A.: Advances in Genetic Programming, ch. 9. MIT Press, Cambridge (1994)Google Scholar
  10. 10.
    Toffolo, A., Benini, E.: Genetic diversity as an objective in multi-objective evolutionary algorithms. Evolutionary Computation 11(2), 151–167 (2003)CrossRefGoogle Scholar
  11. 11.
    Togelius, J.: Optimization, Imitation and Innovation: Computational Intelligence and Games. PhD thesis, Department of Computing and Electronic Systems, University of Essex, Colchester, UK (2007)Google Scholar
  12. 12.
    Zhang, B.-T., Muhlenbein, H.: Evolving optimal neural networks using genetic algorithms with occam’s razor. Complex Systems 7, 199–220 (1993)Google Scholar
  13. 13.
    Zhang, B.-T., Muhlenbein, H.: Balancing accuracy and parsimony in genetic programming. Evolutionary Computation 3, 17–38 (1995)CrossRefGoogle Scholar
  14. 14.
    Zhang, B.-T., Mühlenbein, H.: MDL-based fitness functions for learning parsimonious programs. In: Siegel, E.V., Koza, J.R. (eds.) Working Notes for the AAAI Symposium on Genetic Programming, November 10–12, pp. 122–126. MIT, Cambridge (1995) AAAIGoogle Scholar
  15. 15.
    Zhang, B.-T., Ohm, P., Mühlenbein, H.: Evolutionary induction of sparse neural trees. Evolutionary Computation 5(2), 213–236 (1997)CrossRefGoogle Scholar
  16. 16.
    Ziv, J., Lempel, A.: Compression of individual sequences via variable-rate coding. IEEE Transactions on Information Theory (September 1978)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Faustino J. Gomez
    • 1
  • Julian Togelius
    • 1
  • Juergen Schmidhuber
    • 1
  1. 1.IDSIAManno-LuganoSwitzerland

Personalised recommendations