Automatic Design of Modular Neural Networks Using Genetic Programming

  • Naser NourAshrafoddin
  • Ali R. Vahdat
  • Mohammad Mehdi Ebadzadeh
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4668)

Abstract

Traditional trial-and-error approach to design neural networks is time consuming and does not guarantee yielding the best neural network feasible for a specific application. Therefore automatic approaches have gained more importance and popularity. In addition, traditional (non-modular) neural networks can not solve complex problems since these problems introduce wide range of overlap which, in turn, causes a wide range of deviations from efficient learning in different regions of the input space, whereas a modular neural network attempts to reduce the effect of these problems via a divide and conquer approach. In this paper we are going to introduce a different approach to autonomous design of modular neural networks. Here we use genetic programming for automatic modular neural networks design; their architectures, transfer functions and connection weights. Our approach offers important advantages over existing methods for automated neural network design. First it prefers smaller modules to bigger modules, second it allows neurons even in the same layer to use different transfer functions, and third it is not necessary to convert each individual into a neural network to obtain the fitness value during the evolution process. Several tests were performed with problems based on some of the most popular test databases. Results show that using genetic programming for automatic design of neural networks is an efficient method and is comparable with the already existing techniques.

Keywords

Modular neural networks evolutionary computing genetic programming automatic design 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Murre, J.M.J.: Transputers and neural networks: An analysis of implementation constraints and performance. IEEE Transactions on Neural Networks 4, 284–292 (1993)CrossRefGoogle Scholar
  2. 2.
    Eccles, J.C.: Neuroscience 6, 1839–1855 (1981)Google Scholar
  3. 3.
    Murre, J.: Learning and Categorization in Modular Neural Networks, Harvester–Wheatcheaf (1992)Google Scholar
  4. 4.
    Auda, G.: Cooperative modular neural network classifiers, PhD thesis, University of Waterloo, Systems Design Engineering Department, Canada (1996)Google Scholar
  5. 5.
    Yao, X.: Evolving Artificial Neural Networks. Proceedings of the IEEE 87(9) (1999)Google Scholar
  6. 6.
    Back, T., Fogel, D.B., Michalewicz, Z.: Evolutionary Computation 1: Basic Algorithms and Operators, and Evolutionary Computation 2: Advanced Algorithms and Operators. IOP Publishing Ltd (2000)Google Scholar
  7. 7.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA (1992)MATHGoogle Scholar
  8. 8.
    Koza, J.R.: Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems. Stanford University Computer Science Department Technical Report STAN-CS-90-1314 (June 1990)Google Scholar
  9. 9.
    Koza, J.R., Rice, J.P.: Genetic Generation of Both Weights and Architecture for a Neural Network. In: Proceedings of the International Joint Conference on Neural Networks, vol. II, pp. 397–404 (1991)Google Scholar
  10. 10.
    Gruau, F.: Automatic definition of modular neural networks. Adaptive Behaviour 3(2), 151–183 (1995)CrossRefGoogle Scholar
  11. 11.
    Luke, S., Spector, L.: Evolving graphs and networks with edge encoding: Preliminary report. In: Koza, J.R. (ed.) Late Breaking Papers at the Genetic Programming Conference Stanford University, July 28-31, pp. 117–124 (1996)Google Scholar
  12. 12.
    Ritchie, M.D., White, B.C., Parker, J.S., Hahn, L.W., Moore, J.H.: Optimization of neural network architecture using genetic programming improves detection and modeling of gene-gene interactions in studies of human diseases. BMC Bioinformatics 2003 4(28) (2003)Google Scholar
  13. 13.
    Davis, L.: Adapting operator probabilities in genetic algorithms. In: Schaffer, J.D. (ed.) Proceedings of the Third International Conference on Genetic Algorithms, pp. 61–69. Morgan Kaufmann, San Mateo, CA (1989)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Naser NourAshrafoddin
    • 1
  • Ali R. Vahdat
    • 1
  • Mohammad Mehdi Ebadzadeh
    • 1
  1. 1.Amirkabir University of Technology (Tehran Polytechnic) 

Personalised recommendations