Soft Computing

, Volume 11, Issue 7, pp 655–668 | Cite as

A new hybrid methodology for cooperative-coevolutionary optimization of radial basis function networks

  • A. J. Rivera
  • I. Rojas
  • J. Ortega
  • M. J. del Jesus
Original Article

Abstract

This paper presents a new multiobjective cooperative–coevolutive hybrid algorithm for the design of a Radial Basis Function Network (RBFN). This approach codifies a population of Radial Basis Functions (RBFs) (hidden neurons), which evolve by means of cooperation and competition to obtain a compact and accurate RBFN. To evaluate the significance of a given RBF in the whole network, three factors have been proposed: the basis function’s contribution to the network’s output, the error produced in the basis function radius, and the overlapping among RBFs. To achieve an RBFN composed of RBFs with proper values for these quality factors our algorithm follows a multiobjective approach in the selection process. In the design process, a Fuzzy Rule Based System (FRBS) is used to determine the possibility of applying operators to a certain RBF. As the time required by our evolutionary algorithm to converge is relatively small, it is possible to get a further improvement of the solution found by using a local minimization algorithm (for example, the Levenberg–Marquardt method). In this paper the results of applying our methodology to function approximation and time series prediction problems are also presented and compared with other alternatives proposed in the bibliography.

Keywords

Cooperative–coevolution Soft-computing techniques RBF networks Fuzzy rule based system Function approximation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Angeline P, Saunders G, Pollack J (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5(1):54–65CrossRefGoogle Scholar
  2. Bersini H, Duchateau A, Bradshaw N (1997) Using incremental learning algorithms in the search for minimal and effective models. In: Proceedings of the 6th IEEE international conference on fuzzy system. IEEE Computer Society Press, Barcelona, pp 1417–1422Google Scholar
  3. Broomhead D, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355MATHMathSciNetGoogle Scholar
  4. Buchtala O, Klimek M, Sick B (2005) Evolutionary optimization of radial basis function classifiers for data mining applications. IEEE Trans Syst Man Cybern B 35(5):928–947CrossRefGoogle Scholar
  5. Castillo P (2000) Optimización de perceptrones multicapa mediante algoritmos evolutivos. Ph.D. dissertation, University of Granada, SpainGoogle Scholar
  6. Chen S, Woo Y, Luk B (1999) Combined genetic algorithms optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Trans Neural Netw 10(5):1239–1243CrossRefGoogle Scholar
  7. Cherkassky V, Lari-Najafi H (1991) Constrained topological mapping for non-parametric regression analysis. Neural Netw 4(1):27–40CrossRefGoogle Scholar
  8. Cherkassky V, Gehring D, Mulier F (1996) Comparison of adaptive methods for function estimation from samples. IEEE Trans Neural Netw 7(4):969–984CrossRefGoogle Scholar
  9. Coello CA, Van Veldhuizen DA, Lamont GB (2002) Evolutionary algorithms for solving multi-objective problems. Kluwer, New YorkMATHGoogle Scholar
  10. De Falco I, Della Cioppa A, Iazzetta A, Natale P, Tarantino E (1998) Optimizing neural networks for time series prediction. In: Roy R et al (eds) Proceedings of the 3rd on line world conference on soft computing (WSC3). Advances in Soft Computing – Engeenering design and Manufacturing InternetGoogle Scholar
  11. Dickerson J, Kosko B (1996) Fuzzy function approximation with ellipsoidal rules. IEEE Trans Syst Man Cybern B 26(4):542–560CrossRefGoogle Scholar
  12. Friedman J (1981) Projection pursuit regression. J Am Stat Assoc 76:817–823CrossRefGoogle Scholar
  13. Friedman J (1991) Multivariate adaptive regression splines (with discussion). Ann Stat 19:1–141MATHGoogle Scholar
  14. García N, Hervás C, Muñoz J (2002) Multi-objective cooperative coevolution of artificial neural networks. Neural Netw 15:1259–1278CrossRefGoogle Scholar
  15. Goldberg D, Richardson J (1987) Genetic algorithms with sharing for multimodal function optimization. In: Grefenstette J (ed) Proceedings of the second international conference on genetic algorithms. Lawrence Erlbaum Associates, pp 41–49Google Scholar
  16. Golub G, Van Loan C (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, BaltimoreMATHGoogle Scholar
  17. González J, Rojas I, Ortega J, Pomares H, Fernández F, Diaz A (2003) Multiobjective evolutionary optimization of the size, shape and position parameters of radial basis function networks for function approximation. IEEE Trans Neural Netw 14(6):1478–1495CrossRefGoogle Scholar
  18. Harpham C, Dawson C, Brown M (2004) A review of genetic algorithms applied to training radial basis function networks. Neural Comput Appl 13:193–201CrossRefGoogle Scholar
  19. Haykin S (1999) Neural Networks. A comprehensive foundation, 2nd edn. Prentice-Hall, Upper Saddle RiverMATHGoogle Scholar
  20. Howlett RJ, Jain LD (2001a) Radial basis function networks 1—recent developments in theory and applications. Studies in fuzzyness and soft computing. Physica-Verlag, New YorkGoogle Scholar
  21. Howlett RJ, Jain LD (2001b) Radial basis function networks 2—New avances in design. Studies in fuzzyness and soft computing. Physica-Verlag, New YorkGoogle Scholar
  22. Lacerda E, de Carvalho A, Ludermir T (2001). Evolutionary optimization of RBF networks. In: Howlett RJ, Jain LC (eds). Radial basis. function networks 1—Recent developments in theory and applications, vol 66. Studies in fuzzyness and soft computing. Physica-Verlag, New York, pp. 281–309Google Scholar
  23. Lacerda E, Carvalho A, Padua A, Bernarda T (2005) Evolutionary radial basis functions for credit assessment. Appl Intell 22:167–181CrossRefGoogle Scholar
  24. Lipmann R (1987) An introduction to computing with neural nets. IEEE Trans Acoust Speech Signal Process 2(2):4–22Google Scholar
  25. Mackey M, Glass L (1977) Oscillation and chaos in physiological control system. Science 197:287–289CrossRefGoogle Scholar
  26. Mandani E, Assilian S (1975) An experiment in linguistic synthesis with a fuzzy logic controller. Int J Man-Mach Stud 7(1):1–13CrossRefGoogle Scholar
  27. Marquardt DW (1963) An algorithm for least-squares estimation of non-linear inequalities. SIAM J Appl Math 11:431–441MATHCrossRefMathSciNetGoogle Scholar
  28. Mendel J (1995) Fuzzy logic system for engineering: a tutorial. Proc IEEE 83(3): 345–377CrossRefGoogle Scholar
  29. Moody J, Darken CJ (1989) Fast learning in networks of locally-tuned processing units. Neural Comput 1:281–294Google Scholar
  30. Moriarty D, Miikkulainen R (1997) Forming neural networks through efficient and adaptive coevolution. Evol Comput 5(4):373–399Google Scholar
  31. Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3:213–225MathSciNetGoogle Scholar
  32. Pomares H, Rojas I, Ortega J, González J, Prieto A (2000) A systematic approach to self-generating fuzzy rule-table for function approximation. IEEE Trans Syst Man Cybern B 30(3):431–447CrossRefGoogle Scholar
  33. Potter M, De Jong K (2000) Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol Comput 8(1):1–29CrossRefGoogle Scholar
  34. Powell M (1985) Radial basis functions for multivariable interpolation: a review. In: IMA. Conference on Algorithms for the approximation of functions and data, pp 143–167Google Scholar
  35. Rechenberg I (1973) Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog, StuttgartGoogle Scholar
  36. Rivas V, Castillo PA, Merelo-Guervós JJ (2002) Evolved rbf networks for time-series forecasting and function approximation. LNCS 2439:505Google Scholar
  37. Rivera AJ, Ortega J, Prieto A (2001) Design of rbf networks by cooperative/competitive evolution of units. In: International conference on artificial neural networks and genetic algorithms. ICANNGA 2001, pp 375–378Google Scholar
  38. Rivera AJ, Ortega J, Rojas I, del Jesus M (2003) Co-evolutionary algorithm for RBF by self-organizing population of neurons. LNCS 2686:470–477Google Scholar
  39. Rivera AJ (2003) Diseño y optimización de redes de funciones de base radial mediante técnicas bioinspiradas. Ph.D. dissertation, University Granada, SpainGoogle Scholar
  40. Rojas I, Valenzuela O, Prieto A (1997) Statistical analysis of the main parameters in the definition of radial basic function networks. LNCS 1240:882–891Google Scholar
  41. Rosin C, Belew R (1997) New methods for competitive coevolution. Evol comput 5(1):1–29Google Scholar
  42. Rosipal R, Koska M, Farkaš I (1998) Prediction of chaotic time series with a resource allocating RBF networks. Neural Process Lett 7:185–197CrossRefGoogle Scholar
  43. Schwefel HP (1995) Evolution and optimum seeking. Wiley, New YorkGoogle Scholar
  44. Smalz R, Conrad M (1994) Combining evolution with credit apportionment: a new learning algorithm for neural nets. Neural Netw 7(2):341–351CrossRefGoogle Scholar
  45. Tettamanzi A, Tomassini M (2001) Soft computing. Integrating evolutionary, neural and fuzzy system. Springer, Berlin Heidelberg New YorkGoogle Scholar
  46. Whitehead B, Choate T (1996) Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction. IEEE Trans Neural Netw 7(4):869–880CrossRefGoogle Scholar
  47. Widrow B, Lehr MA (1990) 30 Years of adaptive neural networks: perceptron, madaline and backpropagation. Proc IEEE 78(9):1415–1442CrossRefGoogle Scholar
  48. Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • A. J. Rivera
    • 1
  • I. Rojas
    • 2
  • J. Ortega
    • 2
  • M. J. del Jesus
    • 1
  1. 1.Department of Computer ScienceUniversity of JaénJaenSpain
  2. 2.Department of Computer Architecture and Computer TechnologyUniversity of GranadaGranadaSpain

Personalised recommendations