Advertisement

Generalized Compressed Network Search

  • Rupesh Kumar Srivastava
  • Jürgen Schmidhuber
  • Faustino Gomez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7491)

Abstract

This paper presents initial results of Generalized Compressed Network Search (GCNS), a method for automatically identifying the important frequencies for neural networks encoded as Fourier-type coefficients (i.e. “compressed” networks [7]). GCNS is a general search procedure in this coefficient space – both the number of frequencies and their value are automatically determined by employing the use of variable-length chromosomes, inspired by messy genetic algorithms. The method achieves better compression than our previous approach, and promises improved generalization for evolved controllers. Results for a high-dimensional Octopus arm control problem show that a high fitness 3680-weight network can be encoded using less than 10 coefficients using the frequencies identified by GCNS.

Keywords

Discrete Cosine Transform Weight Matrice Inverse Discrete Cosine Transform Genetic Neural Network Messy Genetic Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Dürr, P., Mattiussi, C., Floreano, D.: Neuroevolution with analog genetic encoding. In: PPSN, pp. 671–680 (2006)Google Scholar
  2. 2.
    Goldberg, D., Korb, B., Deb, K.: Messy genetic algorithms: Motivation, analysis, and first results. Complex systems 3(5), 493–530 (1989)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Goldberg, D.E., Deb, K., Kargupta, H., Harik, G.: Rapid, accurate optimization of difficult problems using fast messy genetic algorithms. In: Proc. of the Fifth International Conference on Genetic Algorithms, pp. 56–64. Morgan Kaufmann (1993)Google Scholar
  4. 4.
    Gomez, F., Koutník, J., Schmidhuber, J.: Compressed Network Complexity Search. In: Coello Coello, C.A., et al. (eds.) PPSN 2012, Part I. LNCS, vol. 7491, pp. 316–326. Springer, Heidelberg (2012)Google Scholar
  5. 5.
    Gruau, F.: Cellular encoding of genetic neural networks. Technical Report RR-92-21, Ecole Normale Superieure de Lyon, Institut IMAG, Lyon, France (1992)Google Scholar
  6. 6.
    Kitano, H.: Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4, 461–476 (1990)zbMATHGoogle Scholar
  7. 7.
    Koutník, J., Gomez, F., Schmidhuber, J.: Evolving neural networks in compressed weight space. In: Proc. of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 619–626. ACM (2010)Google Scholar
  8. 8.
    Koutník, J., Gomez, F., Schmidhuber, J.: Searching for minimal neural networks in fourier space. In: Proc. of the 4th Conf. on Artificial General Intelligence (2010)Google Scholar
  9. 9.
    Schmidhuber, J.: Discovering neural nets with low Kolmogorov complexity and high generalization capability. Neural Networks 10(5), 857–873 (1997)CrossRefGoogle Scholar
  10. 10.
    Yekutieli, Y., Sagiv-Zohar, R., Aharonov, R., Engel, Y., Hochner, B., Flash, T.: Dynamic model of the octopus arm. I. Biomechanics of the octopus reaching movement. Journal of Neurophysiology 94(2), 1443–1458 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Rupesh Kumar Srivastava
    • 1
  • Jürgen Schmidhuber
    • 1
  • Faustino Gomez
    • 1
  1. 1.IDSIAUSI-SUPSIManno-LuganoSwitzerland

Personalised recommendations