# CO^{2}RBFN: an evolutionary cooperative–competitive RBFN design algorithm for classification problems

- 145 Downloads
- 16 Citations

## Abstract

This paper presents a new evolutionary cooperative–competitive algorithm for the design of Radial Basis Function Networks (RBFNs) for classification problems. The algorithm, CO^{2}RBFN, promotes a cooperative–competitive environment where each individual represents a radial basis function (RBF) and the entire population is responsible for the final solution. The proposal considers, in order to measure the credit assignment of an individual, three factors: contribution to the output of the complete RBFN, local error and overlapping. In addition, to decide the operators’ application probability over an RBF, the algorithm uses a Fuzzy Rule Based System. It must be highlighted that the evolutionary algorithm considers a distance measure which deals, without loss of information, with differences between nominal features which are very usual in classification problems. The precision and complexity of the network obtained by the algorithm are compared with those obtained by different soft computing methods through statistical tests. This study shows that CO^{2}RBFN obtains RBFNs with an appropriate balance between accuracy and simplicity, outperforming the other methods considered.

## Keywords

Radial Basis Function Networks Evolutionary Algorithms Cooperative–Competitive Evolutionary Design Classification Fuzzy Rule Base Systems## Notes

### Acknowledgments

This work has been partially supported by the CICYT Spanish Projects TIN2005-04386-C05-03, TIN2007-60587 and the Andalusian Research Plan TIC-3928.

## References

- Alcalá-Fdez J, Sánchez L, García S, Del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM, Fernández JC, Herrera F (2009) KEEL: a software tool to assess evolutionary algorithms to data mining problems. Soft Comput 13(3): 307–318Google Scholar
- Ampazis N, Perantonis SJ (2002) Two highly efficient second-order algorithms for training feedforwards networks. IEEE Trans Neural Netw 13(3):1064–1074CrossRefGoogle Scholar
- Asuncion A, Newman DJ (2007) UCI machine learning repository. School of Information and Computer Science, University of California, Irvine, CA. http://www.ics.uci.edu/~mlearn/MLRepository.html
- Bäck T, Hammel U, Schwefel H (1997) Evolutionary computation: comments on the history and current state. IEEE Trans Evol Comput 1(1):3–17CrossRefGoogle Scholar
- Balakrishnan K, Honavar V (1995) Evolutionary design of neural architecture—a preliminary taxonomy and guide to literature. Technical report, AI Research Group, CS-TR 95–01Google Scholar
- Billings SA, Zheng GL (1995) Radial basis function network configuration using genetic algorithms. Neural Netw 8(6):877–890CrossRefGoogle Scholar
- Broomhead D, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355zbMATHMathSciNetGoogle Scholar
- Buchtala O, Klimek M, Sick B (2005) Evolutionary optimization of radial basis function classifiers for data mining applications. IEEE Trans Syst Man Cybern B 35(5):928–947CrossRefGoogle Scholar
- Burdsall B, Giraud-Carrier C (1997) GA-RBF: a self optimising RBF network. In: Proceedings of conference on artificial neural networks and genetic algorithms. Springer, BerlinGoogle Scholar
- Chaiyaratana N, Zalzala AMS (1998) Evolving hybrid RBF-MLP networks using combined genetic/unsupervised/supervised learning. In: Proceedings of UKACC international conference on control, Swansea, UKGoogle Scholar
- Chen S, Billings SA, Cowan CFN, Grant PW (1990) Practical identification of narmax models using radial basis functions. Int J Control 52(6):1327–1350zbMATHCrossRefMathSciNetGoogle Scholar
- Chen S, Cowan C, Grant P (1991) Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans Neural Netw 2:302–309CrossRefGoogle Scholar
- Chen S, Wu Y, Luk BL (1999) Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Trans Neural Netw 10(5):1239–1243CrossRefGoogle Scholar
- Cheng V, Li CH, Kwok JT, Li CK (2004) Dissimilarity learning for nominal data. Pattern Recogn 37:1471–1477CrossRefGoogle Scholar
- Dawson CW, Wilby RL, Harpham C, Brown MR, Cranston, E, Darby EJ (2000) Modelling Ranunculus presence in the Rivers test and Itchen using artificial neural networks. In: Proceedings of international conference on geocomputation, Greenwich, UKGoogle Scholar
- Deb K (2001) Multi-objective optimization using evolutionary algorithms, 1st edn. Wiley, New YorkGoogle Scholar
- Demšar J (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. J Mach Learn Res 7:1–30MathSciNetGoogle Scholar
- Du H, Zhang N (2008) Time series prediction using evolving radial basis function networks with new enconding scheme. Neurocomputing 71:1388–1400CrossRefGoogle Scholar
- Er MJ, Chen W, Wu S (2005) High-speed face recognition base on discrete cosine transform and RBF neural networks. IEEE Trans Neural Netw 16(3):679–691CrossRefGoogle Scholar
- Esposito F, Malerba D, Tamma V, Bock HH (2000a) Classical resemblance measures. In: Bock H-H, Diday E (eds) Analysis of symbolic data. Exploratory methods for extracting statistical information from complex data, Series: studies in classification, data analysis, and knowledge organization, vol 15. Springer, Berlin, pp 139–152Google Scholar
- Esposito A, Marinaro M, Oricchio D, Scarpetta S (2000b) Approximation of continuous and discontinuous mappings by a growing neural RBF-based algorithm. Neural Netw 13(6):651–665CrossRefGoogle Scholar
- Ferreira PM, Ruano AE, Fonseca CM (2003) Genetic assisted selection of RBF model structures for greenhouse inside air temperature prediction. IEEE Control Appl 1:576–581Google Scholar
- Fogel LJ, Owens AJ, Walsh MJ (1966) Artificial Intelligence through simulation evolution. Wiley, New YorkGoogle Scholar
- Fu X, Wang L (2002) A GA-based novel RBF classifier with class-dependent features. Proc Congr Evol Comput 2:1964–1969Google Scholar
- Fu X, Wang L (2003) Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance. IEEE Trans Syst Man Cybern B 33(3):399–409CrossRefGoogle Scholar
- Goldberg D (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, ReadingzbMATHGoogle Scholar
- Goldberg D, Richardson J (1987) Genetic algorithms with sharing for multimodal function optimization. In: Grefenstette (ed) Proceedings of second international conference on genetic algorithms. Lawrence Erlbaum Associates, pp 41–49Google Scholar
- Golub G, Van Loan C (1996) Matrix computations, 3rd edn. J. Hopkins University Press, BaltimoreGoogle Scholar
- González J, Rojas I, Ortega J, Pomares H, Fernández FJ, Díaz AF (2003) Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Trans Neural Netw 14(6):1478–1495CrossRefGoogle Scholar
- Guillén A, Pomares H, Rojas I, González J, Herrera LJ, Rojas F, Valenzuela O (2007) Output value-based initialization for radial basis function neural networks. Neural Process Lett. doi: 10.1007/s11063-007-9039-8
- Harpham C, Dawson C, Brown M (2004) A review of genetic algorithms applied to training radial basis function networks. Neural Comput Appl 13:193–201CrossRefGoogle Scholar
- Holcomb T, Morari M (1991) Local training for radial basis function networks: towards solving the hidden unit problem. In: Proceedings of American control conference, BostonGoogle Scholar
- Holland JH (1975) Adaptation in natural and artificial systems. The University of Michigan PressGoogle Scholar
- Huang SN, Tan KK, Lee TH (2008) Adaptive neural network algorithm for control design of rigid-link electrically driven robots. Neurocomputing 71(4–6):885–894CrossRefGoogle Scholar
- Jang JSR, Sun CT (1993) Functional equivalence between radial basis functions and fuzzy inference systems. IEEE Trans Neural Netw 4:156–158CrossRefGoogle Scholar
- Jiang N, Zhao ZY, Ren LQ (2003) Design of structural modular neural networks with genetic algorithm. Adv Eng Soft 1:17–24CrossRefGoogle Scholar
- Jin Y, Sendhoff B (2003) Extracting interpretable fuzzy rules from RBF networks. Neural Process Lett 17(2):149–164zbMATHCrossRefGoogle Scholar
- Lacerda E, Carvalho A, Braga A, Ludermir T (2005) Evolutionary radial functions for credit assessment. Appl Intell 22:167–181CrossRefGoogle Scholar
- Lee S, Kil RM (1991) A Gaussian potential function network with hierarchically seft-organising learning. Neural Netw 4:207–224CrossRefGoogle Scholar
- Leung H, Dubash N, Xie N (2002) Detection of small objects in clutter using a GA-RBF neural network. IEEE Trans Aero Electr Sys 38(1):98–118CrossRefGoogle Scholar
- Li M, Tian J, Chen F (2008) Improving multiclass pattern recognition with a co-evolutionary RBFNN. Pattern Recogn Lett 29(4):392–406CrossRefGoogle Scholar
- Maglogiannis I, Sarimveis H, Kiranoudis CT, Chatziioannou AA, Oikonomou N, Aidinis V (2008) Radial basis function neural networks classification for the recognition of idiopathic pulmonary fibrosis in microscopic images. IEEE Trans Inf Technol B 12(1):42–54CrossRefGoogle Scholar
- Mandani E, Assilian S (1975) An experiment in linguistic synthesis with a fuzzy logic controller. Int J Man Mach Stud 7(1):1–13CrossRefGoogle Scholar
- Marcos JV, Hornero R, Álvarez D, Del Campo F, López M, Zamarrón C (2008) Radial basis function classifiers to help in the diagnosis of the obstructive sleep apnoea syndrome from nocturnal oximetry. Med Biol Eng Comput 46:323–332CrossRefGoogle Scholar
- Moechtar M, Farag AS, Hu L, Cheng TC (1999) Combined genetic algorithms and neural network approach for power system transient stability evaluation. Europ Trans Elect Power 9(2):115–122Google Scholar
- Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1:281–294CrossRefGoogle Scholar
- Musavi MT, Ahmed W, Chan KH, Faris KB, Hummels DM (1992) On the training of radial basis function classifiers. Neural Netw 5:595–603CrossRefGoogle Scholar
- Neruda R, Kudová P (2005) Learning methods for radial basis function networks. Future Gener Comp Sy 21(7):1131–1142CrossRefGoogle Scholar
- Orr MJL (1995) Regularization on the selection of radial basis function centers. Neural Comput 7:606–623CrossRefGoogle Scholar
- Park J, Sandberg I (1991) Universal approximation using radial-basis function networks. Neural Comput 3:246–257CrossRefGoogle Scholar
- Park J, Sandberg I (1993) Universal approximation and radial basis function network. Neural Comput 5(2):305–316CrossRefGoogle Scholar
- Pedrycz W (1998) Conditional fuzzy clustering in the design of radial basis function neural networks. IEEE Trans Neural Netw 9(4):601–612CrossRefGoogle Scholar
- Peng JX, Li K, Huang DS (2006) A hybrid forward algorithm for RBF neural network construction. IEEE Trans Neural Netw 17(6):1439–1451CrossRefGoogle Scholar
- Pérez-Godoy MD, Rivera AJ, del Jesus MJ, Rojas I (2007) CoEvRBFN: an approach to solving the classification problem with a hybrid cooperative–coevolutive algorithm. In: Proceedings of international workshop on artificial neural networks, pp 324–332Google Scholar
- Pérez-Godoy MD, Aguilera JJ, Berlanga FJ, Rivas VM, Rivera AJ (2008) A preliminary study of the effect of feature selection in evolutionary RBFN design. In: Proceedings of information processing and management of uncertainty in knowledge-based system, pp 1151–1158Google Scholar
- Plat J (1991) A resource allocating network for function interpolation. Neural Comput 3(2):213–225Google Scholar
- Potter M, De Jong K (2000) Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol Comput 8(1):1–29CrossRefGoogle Scholar
- Powell M (1985) Radial basis functions for multivariable interpolation: a review. In: IMA Proceedings of conference on algorithms for the approximation of functions and data, pp 143–167Google Scholar
- Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kauffman, Menlo ParkGoogle Scholar
- Rivas VM, Merelo JJ, Castillo PA, Arenas MG, Castellanos JG (2004) Evolving RBF neural networks for time-series forecasting with EvRBF. Inf Sci 165(3–4):207–220CrossRefGoogle Scholar
- Rivera AJ, Ortega J, Prieto A (2001) Design of RBF networks by cooperative/competitive evolution of units. In: Proceedings of international conference on artificial neural networks and genetic algorithms (ICANNGA 2001), pp 375–378Google Scholar
- Rivera AJ, Rojas I, Ortega J, del Jesus MJ (2007) A new hybrid methodology for cooperative–coevolutionary optimization of radial basis function networks. Soft Comput. doi: 10.1007/s00500-006-0128-9
- Rojas R, Feldman J (1996) Neural networks: a systematic introduction. Springer, BerlinGoogle Scholar
- Rojas I, Valenzuela O, Prieto A (1997) Statisctical analysis of the main parameters in the definition of radial basis function networks. Lect Notes Comput Sci 1240:882–891CrossRefGoogle Scholar
- Sanchez VD (2002) A searching for a solution to the automatic RBF network design problem. Neurocomputing 42:147–170zbMATHCrossRefGoogle Scholar
- Schaffer JD, Whitley D, Eschleman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Proceedings of international workshop on combinations of genetic algorithms and neural networksGoogle Scholar
- Sergeev SA, Mahotilo KV, Voronovsky GK, Petrashev SN (1998) Genetic algorithm for training dynamical object emulator based on RBF neural network. Int J Appl Electro Mech 9(1):65–74Google Scholar
- Sheta AF, De Jong K (2001) Time-series forecasting using GA-tuned radial basis functions. Info Sci 133(3–4):221–228zbMATHCrossRefGoogle Scholar
- Stanfill C, Waltz D (1986) Towards memory-based reasoning, Commun. ACM 29(12):1213–1228CrossRefGoogle Scholar
- Sumathi S, Sivanandam SN, Ravindran R (2001) Design of a soft computing hybrid model classifier for data mining applications. Engin Intell Sys Electr Engin Comm 9(1):33–56Google Scholar
- Sun YF, Liang YC, Zhang WL, Lee HP, Lin WZ, Cao LJ (2005) Optimal partition algorithm of the RBF neural network and its application to financial time series forecasting. Neural Comput Appl 14(1):36–44CrossRefGoogle Scholar
- Sundararajan N, Saratchandran P, Yingwei L (1999) Radial basis function neural network with sequential learning: MRAN and its application. World Scientifics, New YorkGoogle Scholar
- Teixeira CA, Ruano MG, Ruano AE, Pereira WCA (2008) A soft-computing methodology for noninvasive time-spatial temperature estimation. IEEE Trans Bio Med Eng 55(2):572–580CrossRefGoogle Scholar
- Topchy A, Lebedko O, Miagkikh V, Kasabov N (1997) Adaptive training of radial basis function networks based on co-operative evolution and evolutionary programming. In: Proceedings of international conference neural information processing (ICONIP), pp 253–258Google Scholar
- Vesin JM, Gruter R (1999) Model selection using a simplex reproduction genetic algorithm. Sig Process 78:321–327zbMATHCrossRefGoogle Scholar
- Whitehead B, Choate T (1996) Cooperative–competitive genetic evolution of radial basis function centers and widths for time series prediction. IEEE Trans Neural Netw 7(4):869–880CrossRefGoogle Scholar
- Widrow B, Lehr MA (1990) 30 Years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEEE 78(9):1415–1442CrossRefGoogle Scholar
- Wilson DR, Martinez TR (1997) Improved heterogeneous distance functions. J Artif Intell Res 6(1):1–34zbMATHMathSciNetGoogle Scholar
- Xue Y, Watton J (1998) Dynamics modelling of fluid power systems applying a global error descent algorithm to a selforganising radial basis function network. Mechatronics 8(7):727–745CrossRefGoogle Scholar
- Yao X (1993) A review of evolutionary artificial neural networks. Int J Intell Syst 8(4):539–567CrossRefGoogle Scholar
- Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447CrossRefGoogle Scholar
- Yen GG (2006) Multi-Objective evolutionary algorithm for radial basis function neural network design. Stud Comput Intell 16:221–239CrossRefGoogle Scholar