Abstract
The problems associated with training feedforward artificial neural networks (ANNs) such as the multilayer perceptron (MLP) network and radial basis function (RBF) network have been well documented. The solutions to these problems have inspired a considerable amount of research, one particular area being the application of evolutionary search algorithms such as the genetic algorithm (GA). To date, the vast majority of GA solutions have been aimed at the MLP network. This paper begins with a brief overview of feedforward ANNs and GAs followed by a review of the current state of research in applying evolutionary techniques to training RBF networks.
Similar content being viewed by others
References
Park J, Sandberg IW (1991) Universal approximation using radial basis function networks. Neur Comput 3(2):246–257
Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neur Netw 2:359–366
Niranjan M, Fallside F (1998) Neural networks and radial basis functions in classifying static speech patterns. Report CUED/FINFENG/TR22, University Engineering Department, Cambridge University, UK
Minsky ML, Papert SA (1990) Perceptrons. MIT Press, Cambridge, MA
Rumelhart DE, McClelland JL (eds) (1986) Parallel distributed processing: explorations in the microstructure of cognition, vol 1. MIT Press, Cambridge, MA
Broomhead DS, Lowe D (1988) Multivariable function interpolation and adaptive networks. Comp Sys 2:321–355
Moody JE, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neur Comput 1(2):281–294
Powell MJD (1987) Radial basis functions for multivariable interpolation: a review. In: Mason JC, Cox MG (eds) Algorithms for approximation, Clarendon Press, Oxford, UK
Batchelor BG, Wilkins BR (1969) Method for location of clusters of patterns to initialise a learning machine. Elect Lett 5:481–483
Kohonen T (1982) Self-organised formation of topologically correct feature maps. Biol Cybern 43:59–69
Holland J (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, MI
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Reading, MA
De Jong KA (1975) An analysis of the behaviour of a class of genetic adaptive systems. Dissertation, University of Michigan
Schaffer JD, Caruana RA, Eshelman LJ, Das R (1989) A study of control parameters affecting online performance of genetic algorithms for function optimisation. In: Schaffer JD (ed) Proceedings of the 11th International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Mateo, CA
Michalewicz Z (1996) Genetic algorithms + data structures = evolution programs, 3rd edition. Springer, Berlin Heidelberg New York
Coley DA (1999) An introduction to genetic algorithms for engineers and scientists. World Scientific, New York
Back T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, New York
Reed R (1993) Pruning algorithms—a survey. IEEE Trans Neur Netw 4:740–746
Kwok TY, Yeung Y (1997) Constructive algorithms for structure learning in feedforward neural networks for regression problems. IEEE Trans Neur Netw 8(3):630–645
Fahlman SE, Lebiere C (1990) The cascade-correlation learning architecture. In: Touretzky DS (ed) Advances in neural information processing systems, vol 2, Morgan Kaufmann, Los Altos, CA
Chudova DI, Dolenko SA, Orlov YuV, Pavlov DYu, Persiantsev IG (1998) Benchmarking of different modifications of the cascade correlation algorithm. In: Parmee I (ed) Proceedings of the 3rd International Conference on Adaptive Computing in Design and Manufacture. Springer, Berlin Heidelberg New York
Chen S, Billings SA, Cowan CFN, Grant PW (1990) Practical identification of narmax models using radial basis functions. Int J Contr 52(6):1327–1350
Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: Petrov BN, Csaki F (eds) Proceedings of the 2nd International Symposium on Information Theory, Tsahkadsov, Armenia
Holcomb T, Morari M (1991) Local training for radial basis function networks: towards solving the hidden unit problem. In: Proceedings of the American Control Conference, Boston, MA, June 1991
Lee S, Kil RM (1991) A Gaussian potential function network with hierarchically self-organising learning. Neur Netw 4:207–224
Musavi MT, Ahmed W, Chan KH, Faris KB, Hummels DM (1992) On the training of radial basis function classifiers. Neur Netw 5:595–603
Sundararajan N, Saratchandran P, Lu Ying Wei (1999) Radial basis function neural networks with sequential learning: MRAN and its applications. World Scientific, New York
Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimisation by simulated annealing. Science 220(4598):671–680
Schaffer JD, Whitley D, Eschleman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Proceedings of the International Workshop on Combinations of Genetic Algorithms and neural Networks (COGANN-92), IEEE Computer Society Press, Los Alamitos, CA
Yao X (1993) A review of evolutionary artificial neural networks. Int J Intellig Sys 8(4):539–567
Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447
Balakrishnan K, Honavar V (1995) Evolutionary design of neural architecture—a preliminary taxonomy and guide to literature. Technical report, AI Research Group, CS-TR 95–01
Whitley D (1995) Genetic algorithms and neural networks. In: Winter G, Periaux J, Galan M, Cuesta P (eds) Genetic algorithms in engineering and computer science, Wiley, New York
Masters T (1993) Practical neural network recipes in C++. Academic Press, New York
Rooiji AJF, Jain LC, Johnson RP (1997) Neural network training using genetic algorithms. Series in machine perception and artificial intelligence, vol. 26, World Scientific Publishing, New York
Reeves CR, Taylor SJ (1998) Selection of training data for neural networks by a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York
Hancock PJB (1992) Genetic algorithms and permutation problems: a comparison of recombination operators for neural net structure specification. In: Schaffer JD, Whitley D. and Eschleman LJ (eds) Proceedings of the International Workshop on Combinations of Genetic Algorithms and Neural Networks, (COGANN-92), IEEE Computer Society Press, Los Alamitos, CA
Whitehead BA, Choate DC (1994) Evolving space-filling curves to distribute radial basis functions over an input space. IEEE Trans Neur Netw 5(1):15–23
Goldberg DE (1991) Real coded genetic algorithms, virtual alphabets and blocking. Compl Sys 5:139–167
Mackey MC, Glass L (1977) Oscillation and chaos in physiological control systems. Science 197:287–289
Whitehead BA, Choate DC (1996) Cooperative-competitive genetic evolution of radial basis function centres and widths for time series prediction. IEEE Trans Neur Netw 7:869–880
Deb K, Goldberg DE (1989) An investigation of niche and species formation in genetic function optimization. In: Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann, San Francisco
Whitehead BA (1996) Genetic evolution of radial basis function coverage using orthogonal niches. IEEE Trans Neur Netw 7(6):1525–1528
Billings SA, Zheng GL (1995) Radial basis function network configuration using genetic algorithms. Neur Netw 8(6):877–890
Lucasius CB, Kateman G (1992) Towards solving subset selection problems with the aid of the genetic algorithm. In: Manner R, Manderick B (eds) Parallel problem solving from nature, vol 2, Elsevier, Amsterdam
Neruda R (1995) Functional equivalence and genetic learning of RBF networks. In: Pearson DW, Steele NC, Albrecht RF (eds) Artificial neural networks and genetic algorithms, Springer, Berlin Heidelberg New York
Hecht-Nielsen R (1990) On the algebraic structure of feed-forward network weight spaces. In: Advanced neural computers, Elsevier, Amsterdam
Carse B, Pipe AG, Fogarty TC, Hill T (1995) Evolving radial basis function neural networks using a genetic algorithm. In: Proceedings of the 2nd IEEE International Conference on Evolutionary Computation, Perth, Australia, 29 November-1 December 1995
Carse B, Fogarty TC, Munro A (1996) Evolving fuzzy rule based controllers using genetic algorithms. Int J Fuzz Set Sys 80(3):273–293
Jang JR, Sun T (1993) Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans Neur Netw 4:156–159
Carse B, Fogarty TC (1996) Tackling the “curse of dimensionality” of radial basis function networks using a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York
Carse B, Fogarty TC (1996) Fast evolutionary learning of minimal radial basis function neural networks using a genetic algorithm. Lecture Notes in Computer Science, Springer, Berlin Heidelberg New York
Burdsall B, Giraud-Carrier C (1997) GA-RBF: a self optimising RBF network. In: Proceedings of the Third International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA’97), Springer, Berlin Heidelberg New York
Burdsall B, Giraud-Carrier C (1997) Evolving fuzzy prototypes for efficient data clustering. In: Proceedings of the Second International ICSC Symposium on Fuzzy Logic and Applications (ISFL’97), ICSC Academic Press, New York
Kuncheva LI (1997) Initializing of an RBF network by a genetic algorithm. Neurocomput 14(3):273–288
Beasley D, Bull DR, Martin RR (1993) An overview of genetic algorithms: Part 1, fundamentals. Univer Comput 15:59–69
Beasley D, Bull DR, Martin RR (1993) An overview of genetic algorithms: Part 2, research topics. Univer Comput 15:170–181
Chaiyaratana N, Zalzala AMS (1998) Evolving hybrid RBF-MLP networks using combined genetic/unsupervised/supervised learning. In: Proceedings of the UKACC International Conference on CONTROL ‘98, Swansea, UK, September 1998
Sergeev SA, Mahotilo KV, Voronovsky GK, Petrashev SN (1998) Genetic algorithm for training dynamical object emulator based on RBF neural network. Int J Appl Electro Mech 9(1): 65–74
Xue Y, Watton J (1998) Dynamics modelling of fluid power systems applying a global error descent algorithm to a self-organising radial basis function network. Mechatronics 8(7): 727–745
Vesin JM, Gruter R (1999) Model selection using a simplex reproduction genetic algorithm. Sig Process (78):321–327
Rissanen J (1989) Stochastic complexity in statistical enquiry, World Scientific, Singapore
Tong H (1990) Non-linear time series, Oxford University Press, Oxford, UK
Chen S, Cowan CFN, Grant PM (1991) Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans Neur Netw 2(2):302–309
Moechtar M, Farag AS, Hu L, Cheng TC (1999) Combined genetic algorithms and neural network approach for power-system transient stability evaluation. Europ Trans Elect Power 9(2):115–122
Dawson CW, Wilby RL, Harpham C, Brown MR, Cranston, E, Darby EJ (2000) Modelling Ranunculus presence in the Rivers test and Itchen using artificial neural networks. The Fifth International Conference on GeoComputation, Greenwich, UK, August 2000
Sumathi S, Sivanandam SN, Ravindran R (2001) Design of a soft computing hybrid model classifier for data mining applications. Engin Intell Sys Electr Engin Comm 9(1):33–56
Leung H, Dubash N. Xie N (2002) Detection of small objects in clutter using a GA-RBF neural network. IEEE Trans Aero Electr Sys 38(1): 98–118
Leung H, Lo T (1993) Chaotic radar signal processing over the sea. IEEE J Ocean Eng 18:287–295
Sheta AF, De Jong K (2001) Time-series forecasting using GA-tuned radial basis functions. Info Sci 133(3–4):221–228
Chen S, Wu Y, Luk BL (1999) Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Trans Neur Netw 10(5):1239–1243
Chen S, Wu Y, Alkadhimi K (1995) A two-layer learning method for radial basis function networks using combined genetic and regularised OLS algorithms. In: Proceedings of the 1st IEE/IEEE International Conference on Genetic Algorithms in Engineering Systems: Innovations and Applications, Piscataway, NJ, September 1995
Jiang N, Zhao ZY, Ren LQ (2003) Design of structural modular neural networks with genetic algorithm. Adv Eng Soft (1):17–24
Liu Y, Yao X (1996) Evolutionary design of artificial neural networks with different nodes, evolutionary computation. Proc IEEE Int Conf 1996, 670–675
Mor JJ (1977) The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, GA (ed) Numerical analysis, Lecture Notes in Mathematics, Springer, Berlin Heidelberg New York
Aiguo S, Jiren L (1998) Evolving Gaussian RBF network for nonlinear time series modelling and prediction. Electr Lett 34(12):1241–1243
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Harpham, C., Dawson, C.W. & Brown, M.R. A review of genetic algorithms applied to training radial basis function networks. Neural Comput & Applic 13, 193–201 (2004). https://doi.org/10.1007/s00521-004-0404-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-004-0404-5