Automatic Design of Modular Neural Networks Using Genetic Programming
Traditional trial-and-error approach to design neural networks is time consuming and does not guarantee yielding the best neural network feasible for a specific application. Therefore automatic approaches have gained more importance and popularity. In addition, traditional (non-modular) neural networks can not solve complex problems since these problems introduce wide range of overlap which, in turn, causes a wide range of deviations from efficient learning in different regions of the input space, whereas a modular neural network attempts to reduce the effect of these problems via a divide and conquer approach. In this paper we are going to introduce a different approach to autonomous design of modular neural networks. Here we use genetic programming for automatic modular neural networks design; their architectures, transfer functions and connection weights. Our approach offers important advantages over existing methods for automated neural network design. First it prefers smaller modules to bigger modules, second it allows neurons even in the same layer to use different transfer functions, and third it is not necessary to convert each individual into a neural network to obtain the fitness value during the evolution process. Several tests were performed with problems based on some of the most popular test databases. Results show that using genetic programming for automatic design of neural networks is an efficient method and is comparable with the already existing techniques.
KeywordsModular neural networks evolutionary computing genetic programming automatic design
Unable to display preview. Download preview PDF.
- 2.Eccles, J.C.: Neuroscience 6, 1839–1855 (1981)Google Scholar
- 3.Murre, J.: Learning and Categorization in Modular Neural Networks, Harvester–Wheatcheaf (1992)Google Scholar
- 4.Auda, G.: Cooperative modular neural network classifiers, PhD thesis, University of Waterloo, Systems Design Engineering Department, Canada (1996)Google Scholar
- 5.Yao, X.: Evolving Artificial Neural Networks. Proceedings of the IEEE 87(9) (1999)Google Scholar
- 6.Back, T., Fogel, D.B., Michalewicz, Z.: Evolutionary Computation 1: Basic Algorithms and Operators, and Evolutionary Computation 2: Advanced Algorithms and Operators. IOP Publishing Ltd (2000)Google Scholar
- 8.Koza, J.R.: Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems. Stanford University Computer Science Department Technical Report STAN-CS-90-1314 (June 1990)Google Scholar
- 9.Koza, J.R., Rice, J.P.: Genetic Generation of Both Weights and Architecture for a Neural Network. In: Proceedings of the International Joint Conference on Neural Networks, vol. II, pp. 397–404 (1991)Google Scholar
- 11.Luke, S., Spector, L.: Evolving graphs and networks with edge encoding: Preliminary report. In: Koza, J.R. (ed.) Late Breaking Papers at the Genetic Programming Conference Stanford University, July 28-31, pp. 117–124 (1996)Google Scholar
- 12.Ritchie, M.D., White, B.C., Parker, J.S., Hahn, L.W., Moore, J.H.: Optimization of neural network architecture using genetic programming improves detection and modeling of gene-gene interactions in studies of human diseases. BMC Bioinformatics 2003 4(28) (2003)Google Scholar
- 13.Davis, L.: Adapting operator probabilities in genetic algorithms. In: Schaffer, J.D. (ed.) Proceedings of the Third International Conference on Genetic Algorithms, pp. 61–69. Morgan Kaufmann, San Mateo, CA (1989)Google Scholar