Bidirectional Relation between CMA Evolution Strategies and Natural Evolution Strategies

  • Youhei Akimoto
  • Yuichi Nagata
  • Isao Ono
  • Shigenobu Kobayashi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6238)

Abstract

This paper investigates the relation between the covariance matrix adaptation evolution strategy and the natural evolution strategy, the latter of which is recently proposed and is formulated as a natural gradient based method on the expected fitness under the mutation distribution. To enable to compare these algorithms, we derive the explicit form of the natural gradient of the expected fitness and transform it into the forms corresponding to the mean vector and the covariance matrix of the mutation distribution. We show that the natural evolution strategy can be viewed as a variant of covariance matrix adaptation evolution strategies using Cholesky update and also that the covariance matrix adaptation evolution strategy can be formulated as a variant of natural evolution strategies.

References

  1. 1.
    Wierstra, D., Schaul, T., Peters, J., Schmidhuber, J.: Natural evolution strategies. In: Proceedings of CEC 2008, pp. 3381–3387 (2008)Google Scholar
  2. 2.
    Sun, Y., Wierstra, D., Schaul, T., Schmidhuber, J.: Efficient natural evolution strategies. In: Proceedings of GECCO 2009, pp. 539–545 (2009)Google Scholar
  3. 3.
    Amari, S.: Natural gradient works efficiently in learning. Neural Computation 10(2), 251–276 (1998)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Hansen, N., Ostermeier, A.: Completely deranomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)CrossRefGoogle Scholar
  5. 5.
    Hansen, N., Müller, S., Koumoutsakos, P.: Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation 11(1), 1–18 (2003)CrossRefGoogle Scholar
  6. 6.
    Amari, S., Nagaoka, H.: Methods of Information Geometry. American Mathematical Society (2007)Google Scholar
  7. 7.
    Amari, S., Douglas, S.: Why natural gradient? In: Proceedings of ICASSP, pp. 1213–1216 (1998)Google Scholar
  8. 8.
    Harville, D.A.: Matrix Algebra From a Statistician’s Perspective. Springer, Heidelberg (2008)MATHCrossRefGoogle Scholar
  9. 9.
    Murray, M., Rice, J.: Differential geometry and statistics. Chapman & Hall/CRC, Boca Raton (1993)MATHGoogle Scholar
  10. 10.
    Suttorp, T., Hansen, N., Igel, C.: Efficient covariance matrix update for variable metric evolution strategies. Machine Learning 75(2), 167–197 (2009)CrossRefGoogle Scholar
  11. 11.
    Peters, J., Schaal, S.: Natural actor-critic. Neurocomputing 71, 1180–1190 (2008)CrossRefGoogle Scholar
  12. 12.
    Jastrebski, G.A., Arnold, D.V.: Improving evolution strategies through active covariance matrix adaptation. In: Proceedings of CEC 2006, pp. 9719–9726 (2006)Google Scholar
  13. 13.
    Kuan, C.M., Hornik, K.: Convergence of learning algorithms with constant learning rates. IEEE Transactions on Neural Networks 2(5), 484–489 (1991)CrossRefGoogle Scholar
  14. 14.
    Kushner, H.J., Yin, G.G.: Stochastic approximation and recursive algorithms and applications. Springer, Heidelberg (2003)MATHGoogle Scholar
  15. 15.
    Gonzàlez, A., Dorronsoro, J.R.: Natural conjugate gradient training of multilayer perceptrons. Neurocomputing 71(13-15), 2499–2506 (2008)CrossRefGoogle Scholar
  16. 16.
    Honkela, A., Tornio, M., Raiko, T., Karhunen, J.: Natural conjugate gradient in variational inference. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds.) ICONIP 2007, Part II. LNCS, vol. 4985, pp. 305–314. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Nishimori, Y., Akaho, S., Plumbley, M.D.: Natural conjugate gradient on complex flag manifolds for complex independent subspace analysis. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008, Part I. LNCS, vol. 5163, pp. 165–174. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Youhei Akimoto
    • 1
    • 2
  • Yuichi Nagata
    • 1
  • Isao Ono
    • 1
  • Shigenobu Kobayashi
    • 1
  1. 1.Tokyo Institute of TechnologyYokohamaJapan
  2. 2.Research Fellow of the Japan Society for the Promotion of Science

Personalised recommendations