Abstract
For many practical problem domains the use of neural networks has led to very satisfactory results. Nevertheless the choice of an appropriate, problem specific network architecture still remains a very poorly understood task. Given an actual problem, one can choose a few different architectures, train the chosen architectures a few times and finally select the architecture with the best behaviour. But, of course, there may exist totally different and much more suited topologies. In this paper we present a genetic algorithm driven network generator that evolves neural feedforward network architectures for specific problems. Our system ENZO1 optimizes both the network topology and the connection weights at the same time, thereby saving an order of magnitude in necessary learning time. Together with our new concept to solve the crucial neural network problem of permuted internal representations this approach provides an efficient and successfull crossover operator. This makes ENZO very appropriate to manage the large networks needed in application oriented domains. In experiments with three different applications our system generated very successful networks. The generated topologies possess distinct improvements referring to network size, learning time, and generalization ability.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
D. Nguyen and B. Widrow, The truck backerupper, Proc. Internat. Network Conf. (Kluwer Academic Publishers, Dordrecht, 1990)
H. Braun, J. Feulner, V. Ullrich, Learning strategies for solving the problem of planning using backpropagation, in: Proc. Fourth Intern. Conf. Neural Networks (Nimes, 1991)
R.K. Belew, J. McInerney, N.N. Schraudolph, Evolving networks: using the genetic algorithm with connectionist learning, CSE Technical Report #CS90-174 (University of California, San Diego)
D.J. Chalmers, The evolution of learning: an experiment in genetic connectionism, in: Proc. of the 1990 Connectionist Models Summer School (Morgan-Kaufmann, San Mateo, CA, 1990)
D.B. Fogel, L.J. Fogel, V.W. Porto, Evolving neural networks, in: Biological Cybernetics 63 (Springer, Berlin, 1990)
P.J.B. Hancock, L.S. Smith, GANNET: Genetic design of a neural net for face recognition, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)
S. Harp, T. Samad, A. Guha, Towards the genetic synthesis of neural networks, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)
K.U. Hoeffgen, H.P. Siemon, A. Ultsch, Genetic improvements of feedforward nets for approximating functions, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)
G. Miller, P. Todd, S. Hedge, Designing neural networks using genetic algorithms, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)
H. Muehlenbein, Limitations of multi-layer perceptron networks — steps towards genetic neural networks, in: Parallel Computing 14 (1990)
W. Schiffmann, M. Joost, R. Werner, Performance evaluation of evolutionary created neural network topologies, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)
M. Scholz, A learning strategy for neural networks based on a modified evolutionary strategy, in: Parallel Problem Solving from Nature (Springer, Berlin, 1990)
D. Whitley, T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)
D. Whitley, T. Starkweather, C. Bogart, Genetic algorithms and neural networks: optimizing connections and connectivity, in: Parallel Computing 14 (1990)
D. Whitley, S. Dominic, R. Das, Genetic reinforcement learning with multilayer neural networks, in: Proc. Fourth Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1991)
J.H. Holland, Adaption in natural and artificial systems, (Ann Arbor: University of Michigan Press, 1975)
H. Braun, Massiv parallele Algorithmen fuer kombinatorische Optimierungsprobleme und ihre Implementierung auf einem Parallelrechner, Dissertation TH Karlsruhe, Fakultaet fuer Informatik, 1990
H. Braun, On solving traveling salesman problems by genetic algor ithms, in: Parallel Problem Solving from Nature, LNCS 496 (Berlin, 1991)
J. Weisbrod, Einsatz Genetischer Algorithmen zur Optimierung der Topologie mehrschichtiger Feedforward-Netzwerke, Diplomarbeit TH Karlsruhe, Fakultät für Informatik (1992)
J. Weisbrod, Untersuchung der Einsatzmöglichkeiten Neuronaler Netze zur visuellen Mustererkennung gestanzter Ziffern, Studienarbeit TH Karlsruhe, Fakultät für Elektrotechnik (1989)
V. Ullrich, Erlernen von Spielstrategien für Mühle durch Neuronale Netze, Diplomarbeit TH Karlsruhe, Fakultät für Informatik (1991)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1993 Springer-Verlag/Wien
About this paper
Cite this paper
Braun, H., Weisbrod, J. (1993). Evolving Neural Feedforward Networks. In: Albrecht, R.F., Reeves, C.R., Steele, N.C. (eds) Artificial Neural Nets and Genetic Algorithms. Springer, Vienna. https://doi.org/10.1007/978-3-7091-7533-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-7091-7533-0_5
Publisher Name: Springer, Vienna
Print ISBN: 978-3-211-82459-7
Online ISBN: 978-3-7091-7533-0
eBook Packages: Springer Book Archive