Advertisement

Saw-Tooth Algorithm Guided by the Variance of Best Individual Distributions for Designing Evolutionary Neural Networks

  • Pedro Antonio Gutiérrez
  • César Hervás
  • Manuel Lozano
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4881)

Abstract

This paper proposes a diversity generating mechanism for an evolutionary algorithm that determines the basic structure of Multilayer Perceptron (MLP) classifiers and simultaneously estimates the coefficients of the models. We apply a modified version of a recently proposed diversity enhancement mechanism [1], that uses a variable population size and periodic partial reinitializations of the population in the form of a saw-tooth function. Our improvement on this standard scheme consists of guiding saw-tooth reinitializations by considering the variance of the best individuals in the population, performing the population restart when the difference of variance between two consecutive generations is lower than a percentage of the previous variance. The empirical results over six benchmark datasets show that the proposed mechanism outperforms the standard saw-tooth algorithm. Moreover, results are very promising in terms of classification accuracy, yielding a state-of-the-art performance.

Keywords

Evolutionary algorithm population reinitializations saw-tooth algorithm neural networks 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Koumousis, V.K., Katsaras, C.P.: A saw-tooth genetic algorithm combining the effects of variable population size and reinitialization to enhance performance. IEEE Transactions on Evolutionary Computation 10(1), 19–28 (2006)CrossRefGoogle Scholar
  2. 2.
    Fogel, L.: Artificial Intelligence through Simulated Evolution, 1st edn. John Wiley & Sons, New York (1996)Google Scholar
  3. 3.
    Yao, X.: Evolving artificial neural networks. Proc. of the IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar
  4. 4.
    Goldberg, D., Deb, K., Clark, J.: Genetic algorithms, noise, and the sizing of populations. Complex Systems 6(4), 333–362 (1992)zbMATHGoogle Scholar
  5. 5.
    Smith, R.: Adaptively resizing populations: An algorithm and analysis. In: Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 653–653 (1993)Google Scholar
  6. 6.
    Cobb, H., Grefenstette, J.: Genetic algorithms for tracking changing environments. In: Proceedings of the 5th International Conf. on Genetic Algorithms, pp. 523–530 (1993)Google Scholar
  7. 7.
    Eshelman, L.: The chc adaptive search algorithm: How to have safe search when engaging in nontraditional genetic recombination. In: Proc. Foundations Genetic Algorithms-1, pp. 257–266 (1991)Google Scholar
  8. 8.
    Goldberg, D.: Sizing populations for serial and parallel genetic algorithms. In: Proceedings of the Third International Conference on Genetic Algorithms, pp. 70–79 (1989)Google Scholar
  9. 9.
    Hervás, C., Martínez-Estudillo, F.J., Gutiérrez, P.A.: Classification by means evolutionary product-unit neural networks. In: Proc. of the 2006 International Joint Conference on Neural Networks, Vancouver, Canada, pp. 2834–2842 (2006)Google Scholar
  10. 10.
    Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
  11. 11.
    Angeline, P.J., Saunders, G.M., Pollackm, J.P.: An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. on Neu. Net. 5(1), 54–65 (1994)CrossRefGoogle Scholar
  12. 12.
    Yao, X., Liu, Y.: Evolving artificial neural networks through evolutionary programming. In: Fogel, L., Angeline, P., Bäck, T. (eds.) Proc. of the Fifth Annual Conference on Evolutionary Programming V, pp. 257–266 (1996)Google Scholar
  13. 13.
    Martínez-Estudillo, A.C., Hervás-Martínez, C., Martínez-Estudillo, F.J., García-Pedrajas, N.: Hybridization of evolutionary algorithms and local search by means of a clustering method. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 36(3), 534–545 (2006)CrossRefGoogle Scholar
  14. 14.
    Martínez-Estudillo, A.C., Martínez-Estudillo, F.J., Hervás-Martínez, C., García-Pedrajas, N.: Evolutionary product unit based neural networks for regression. Neural Networks 19(4), 477–486 (2006)zbMATHCrossRefGoogle Scholar
  15. 15.
    Ventura, S., Romero, C., Zafra, A., Delgado, J., Hervás, C.: JCLEC: a java framework for evolutionary computation. Soft Computing (published online, 2007)Google Scholar
  16. 16.
    Landwehr, N., Hall, M., Frank, E.: Logistic model trees. Mach. Learn. 59(1-2), 161–205 (2005)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Pedro Antonio Gutiérrez
    • 1
  • César Hervás
    • 1
  • Manuel Lozano
    • 2
  1. 1.Dept. of Computer Science and Numerical Analysis, University of Córdoba, Campus de Rabanales, building C2, 14004 - CórdobaSpain
  2. 2.Dept. of Computer Science and Artificial Intelligence, University of Granada, E.T.S. Ingeniería Informática, 18071 - GranadaSpain

Personalised recommendations