Advertisement

Evolving Modular Architectures for Neural Networks

  • Andrea Di Ferdinando
  • Raffaele Calabretta
  • Domenico Parisi
Part of the Perspectives in Neural Computing book series (PERSPECT.NEURAL)

Abstract

Neural networks that learn the What and Where task perform better if they possess a modular architecture for separately processing the identity and spatial location of objects. In previous simulations the modular architecture either was hardwired or it developed during an individual’s life based on a preference for short connections given a set of hardwired unit locations. We present two sets of simulations in which the network architecture is genetically inherited and it evolves in a population of neural networks in two different conditions: (1) both the architecture and the connection weights evolve; (2) the network architecture is inherited and it evolves but the connection weights are learned during life. The best results are obtained in condition (2). Condition (1) gives unsatisfactory results because (a) adapted sets of weights can suddenly become maladaptive if the architecture changes, (b) evolution fails to properly assign computational resources (hidden units) to the two tasks, (c) genetic linkage between sets of weights for different modules can result in a favourable mutation in one set of weights being accompanied by an unfavourable mutation in another set of weights.

Keywords

Genetic Algorithm Network Architecture Connection Weight Hide Unit Output Unit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Belew, R K., McInerney, J., & Schraudolph, N. (1991). Evolving networks: using the genetic algorithm with connectionist learning. In C. G. Langton, C. Taylor, J. D. Farmer, & S. Rasmussen (eds), Artificial Life II. Addison-Wesley, Reading, MAGoogle Scholar
  2. 2.
    Belew, R K. & Mitchell, M. (1996). Adaptive Individuals in Evolving Populations. Addison-Wesley, Reading, MAGoogle Scholar
  3. 3.
    Calabretta, R., Nolfi, S., Parisi, D. & Wagner, G. P. (2000). Duplication of modules facilitates the evolution of functional specialization. Artificial Life 6:69–84.Google Scholar
  4. 4.
    Cangelosi A., Parisi D. & Nolfi S. (1994). Cell division and migration in a’ genotype’ for neural networks. Network 5:497–515.Google Scholar
  5. 5.
    Elman, J. L., Bates, E. A, Johnson, M. H., Karmiloff-Smith, A., Parisi, D. & Plunkett, K. (1996). Rethinking innateness. A connectionist perspective on development. The MIT Press, Cambridge, MAGoogle Scholar
  6. 6.
    Floreano, D. & Urzelai, J. (2000). Evolutionary robots with on-line selforganization and behavioral fitness. Neural Networks 13:431–443.Google Scholar
  7. 7.
    Jacobs, R. A., Jordan, M.I. & Barto, A. G. (1991). Task decomposition through competition in a modular connectionist architecture: The what and where vision tasks. Cognitive Science 15:219–250.Google Scholar
  8. 8.
    Jacobs, R. A & Jordan, M. I. (1992). Computational consequences of a bias toward short connections. Journal of Cognitive Neuroscience 4:323–335.Google Scholar
  9. 9.
    Kolen J. F. & Pollack, J. B. (1990). Back-propagation is sensitive to initial conditions. Complex Systems 4:269–280.Google Scholar
  10. 10.
    Murre, J. M. J. (1992). Learning and categorization in modular neural networks. Harvester, New York, NY.Google Scholar
  11. 11.
    Plaut D. C. & Hinton, G. E. (1987). Learning sets of filters using backpropagation. Computer Speech and Language 2:35–61.Google Scholar
  12. 12.
    Reed, R. D. & Marks II, R. J. (1999). Neural Smithing. Supervised Learning in Feedforward Artificial Neural Networks. The MIT Press, Cambridge, MAGoogle Scholar
  13. 13.
    Rueckl, J. G., Cave, K. R & Kosslyn, S. M. (1989). Why are “what” and “where” processed by separate cortical visual systems? A computational investigation. Journal of Cognitive Neuroscience 1:171–186.Google Scholar
  14. 14.
    Ungerleider, L. G. & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale & R J. W. Mansfield (Eds.), The Analysis of Visual Behavior. The MIT Press, Cambridge, MAGoogle Scholar

Copyright information

© Springer-Verlag London 2001

Authors and Affiliations

  • Andrea Di Ferdinando
  • Raffaele Calabretta
  • Domenico Parisi

There are no affiliations available

Personalised recommendations