Advertisement

Parallel Multi-objective Memetic RBFNNs Design and Feature Selection for Function Approximation Problems

  • Alberto Guillén
  • Héctor Pomares
  • Jesús González
  • Ignacio Rojas
  • L. J. Herrera
  • A. Prieto
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4507)

Abstract

The design of Radial Basis Function Neural Networks (RBFNNs) still remains as a difficult task when they are applied to classification or to regression problems. The difficulty arises when the parameters that define an RBFNN have to be set, these are: the number of RBFs, the position of their centers and the length of their radii. Another issue that has to be faced when applying these models to real world applications is to select the variables that the RBFNN will use as inputs. The literature presents several methodologies to perform these two tasks separately, however, due to the intrinsic parallelism of the genetic algorithms, a parallel implementation will allow the algorithm proposed in this paper to evolve solutions for both problems at the same time. The parallelization of the algorithm not only consists in the evolution of the two problems but in the specialization of the crossover and mutation operators in order to evolve the different elements to be optimized when designing RBFNNs. The subjacent Genetic Algorithm is the Non-Sorting Dominated Genetic Algorithm II (NSGA-II) that helps to keep a balance between the size of the network and its approximation accuracy in order to avoid overtraining networks. Another of the novelties of the proposed algorithm is the incorporation of local search algorithms in three stages of the algorithm: initialization of the population, evolution of the individuals, and final optimization of the Pareto front. The initialization of the individuals is performed hybridizing clustering techniques with the Mutual Information theory (MI) to select the input variables. As the experiment will show, the synergy of the different paradigms and techniques combined by the presented algorithm allow to obtain very accurate models using the most significant input variables.

Keywords

Genetic Algorithm Feature Selection Local Search Mutual Information Pareto Front 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Broomhead, D.S., Lowe, D.: Multivariate functional interpolation and adaptive networks. Complex Systems 2, 321–355 (1988)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Deb, K., Goel, T.: Controlled Elitist Non-dominated Sorting Genetic Algorithms for Better Convergence. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 67–81. Springer, Heidelberg (2001)Google Scholar
  4. 4.
    Eshelman, L.J., Caruana, A., Schaffer, J.D.: Realcoded genetic algorithms and interval schemata. In: Schaffer, J.D. (ed.) Foundation of Genetic Algorithms 2, pp. 187–202. Morgan Kaufmann, San Francisco (1993)Google Scholar
  5. 5.
    Gandibleux, X., Morita, H., Katoh, N.: The Supported Solutions Used as a Genetic Information in a Population Heuristic. In: Zitzler, E., Deb, K., Thiele, L., Coello Coello, C.A., Corne, D.W. (eds.) EMO 2001. LNCS, vol. 1993, pp. 429–442. Springer, Heidelberg (2001)Google Scholar
  6. 6.
    González, J., Rojas, I., Ortega, J., Pomares, H., Fernández, F.J., Díaz, A.: Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation. IEEE Transactions on Neural Networks 14(6), 1478–1495 (2003)CrossRefGoogle Scholar
  7. 7.
    Guillén, A.: Writing programs in MATLAB using any implementation of MPI. (2005), http://atc.ugr.es/~aguillen
  8. 8.
    Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Paechter, B.: Improving the Performance of Multi-objective Genetic Algorithm for Function Approximation Through Parallel Islands Specialisation. In: Sattar, A., Kang, B.-h. (eds.) AI 2006. LNCS (LNAI), vol. 4304, pp. 1127–1132. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Fernández, F.: Multiobjective RBFNNs Designer for Function Approximation: An Application for Mineral Reduction. In: Jiao, L., Wang, L., Gao, X.-b., Liu, J., Wu, F. (eds.) ICNC 2006. LNCS, vol. 4221, pp. 511–520. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Prieto, A.G.: A Fuzzy-Possibilistic Fuzzy Ruled Clustering Algorithm for RBFNNs Design. In: Greco, S., Hata, Y., Hirano, S., Inuiguchi, M., Miyamoto, S., Nguyen, H.S., Słowiński, R. (eds.) RSCTC 2006. LNCS (LNAI), vol. 4259, pp. 647–656. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Valenzuela, O., Prieto, A.G.: A Possibilistic Approach to RBFN Centers Initialization. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W., Hu, X. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3642, pp. 174–183. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Guillén, A., Rojas, I., González, J., Pomares, H., Herrera, L.J., Valenzuela, O., Prieto, A.G.: Improving Clustering Technique for Functional Approximation Problem Using Fuzzy Logic: ICFA Algorithm. In: Cabestany, J., Prieto, A.G., Sandoval, F. (eds.) IWANN 2005. LNCS, vol. 3512, pp. 272–279. Springer, Heidelberg (2005)Google Scholar
  13. 13.
    Herrera, F., Lozano, M.: Gradual distributed real-coded genetic algorithms. IEEE-EC 4(1), 43 (2000)Google Scholar
  14. 14.
    Karayiannis, N.B., Balasubramanian, M., Malki, H.A.: Evaluation of cosine radial basis function neural networks on electric power load forecasting. In: Proceedings of the International Joint Conference on Neural Networks, July 2003, vol. 3, pp. 2100–2105 (2003)Google Scholar
  15. 15.
    Moscato, P.: A memetic approach for the travelling salesman problem implementation of a computational ecology for combinatorial optimization on message-passing systems. In: Parallel Computing and Transputer Applications, pp. 177–186 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Alberto Guillén
    • 1
  • Héctor Pomares
    • 2
  • Jesús González
    • 2
  • Ignacio Rojas
    • 2
  • L. J. Herrera
    • 2
  • A. Prieto
    • 2
  1. 1.Department of Informatics, University of JaenSpain
  2. 2.Department of Computer Technology and Architecture, University of GranadaSpain

Personalised recommendations