Local search in speciation-based bloat control for genetic programming

  • Perla Juárez-Smith
  • Leonardo TrujilloEmail author
  • Mario García-Valdez
  • Francisco Fernández de Vega
  • Francisco Chávez


This work presents a unique genetic programming (GP) approach that integrates a numerical local search method and a bloat-control mechanism to address some of the main issues with traditional GP. The former provides a directed search operator to work in conjunction with standard syntax operators that perform more exploration in design space, while the latter controls code growth by maintaining program diversity through speciation. The system can produce highly parsimonious solutions, thus reducing the cost of performing the local optimization process. The proposal is extensively evaluated using real-world problems from diverse domains, and the behavior of the search is analyzed from several different perspectives, including how species evolve, the effect of the local search process and the interpretability of the results. Results show that the proposed approach compares favorably with a standard approach, and that the hybrid algorithm can be used as a viable alternative for solving real-world symbolic regression problems.


Genetic programming Bloat NEAT Local search 



This work was funded by CONACYT (Mexico) project no. FC-2015-2/944 Aprendizaje evolutivo a gran escala, and TecNM (Mexico) project no. 6826-18-p. Second author was supported by CONACYT doctoral scholarship 332554. The authors would like to thank Spanish Ministry of Economy, Industry and Competitiveness and European Regional Development Fund (FEDER) under projects TIN2014-56494-C4-4-P (Ephemec) and TIN2017-85727-C4-4-P (DeepBio); Junta de Extremadura Project IB16035 Regional Government of Extremadura, Consejeria of Economy and Infrastructure, FEDER.


  1. 1.
    M. Affenzeller, S.M. Winkler, B. Burlacu, G. Kronberger, M. Kommenda, S. Wagner, Dynamic observation of genotypic and phenotypic diversity for different symbolic regression GP variants, in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’17 (ACM, New York, 2017), pp. 1553–1558Google Scholar
  2. 2.
    N. Agarwal, B. Bullins, E. Hazan, Second-order stochastic optimization for machine learning in linear time. J. Mach. Learn. Res. 18(1), 4148–4187 (2017)MathSciNetzbMATHGoogle Scholar
  3. 3.
    S. Angra, S. Ahuja, Machine learning and its applications: a review, in 2017 International Conference on Big Data Analytics and Computational Intelligence (ICBDAC) (2017), pp. 57–60Google Scholar
  4. 4.
    D. Applegate, B. Mayfield, An analysis of exchanging fitness cases with population size in symbolic regression genetic programming with respect to the computational model. in 2013 IEEE Congress on Evolutionary Computation (2013), pp. 3111–3116Google Scholar
  5. 5.
    R.M.A. Azad, C. Ryan, A simple approach to lifetime learning in genetic programming-based symbolic regression. Evol. Comput. 22(2), 287–317 (2014)CrossRefGoogle Scholar
  6. 6.
    S. Bleuler, J. Bader, E. Zitzler, Reducing Bloat in GP with Multiple Objectives (Springer, Berlin, 2008), pp. 177–200Google Scholar
  7. 7.
    R.H. Byrd, R.B. Schnabel, G.A. Shultz, A trust region algorithm for nonlinearly constrained optimization. SIAM J. Numer. Anal. 24(5), 1152–1170 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    M. Castelli, L. Trujillo, L. Vanneschi, A. Popovič, Prediction of energy performance of residential buildings: a genetic programming approach. Energy Build. 102, 67–74 (2015)CrossRefGoogle Scholar
  9. 9.
    S. Chand, M. Wagner, Evolutionary many-objective optimization: a quick-start guide. Surv. Oper. Res. Manag. Sci. 20(2), 35–42 (2015)MathSciNetGoogle Scholar
  10. 10.
    X. Chen, Y.-S. Ong, M.-H. Lim, K.C. Tan, A multi-facet survey on memetic computation. IEEE Trans. Evol. Comput. 15(5), 591–607 (2011)CrossRefGoogle Scholar
  11. 11.
    J.-S. Chou, C.-F. Tsai, Concrete compressive strength analysis using a combined classification and regression technique. Autom. Constr. 24, 52–60 (2012)CrossRefGoogle Scholar
  12. 12.
    V.V. de Melo, W. Banzhaf, Improving the prediction of material properties of concrete using kaizen programming with simulated annealing. Neurocomputing 246, 25–44 (2017)CrossRefGoogle Scholar
  13. 13.
    S. Dignum, R. Poli, Operator equalisation and bloat free GP, in Genetic Programming: 11th European Conference, EuroGP 2008 (Springer, Berlin, 2008), pp. 110–121Google Scholar
  14. 14.
    J. Enríquez-Zárate, L. Trujillo, S. de Lara, M. Castelli, E. Z-Flores, L. Muñoz, A. Popovič, Automatic modeling of a gas turbine using genetic programming: an experimental study. Appl. Soft Comput. 50, 212–222 (2017)CrossRefGoogle Scholar
  15. 15.
    O.F. Ertuĝrul, A novel type of activation function in artificial neural networks: trained activation function. Neural Netw. 99, 148–157 (2018)CrossRefGoogle Scholar
  16. 16.
    F.-A. Fortin, F.-M. De Rainville, M.-A. Gardner, M. Parizeau, C. Gagné, DEAP: evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012)MathSciNetzbMATHGoogle Scholar
  17. 17.
    J.E. Hernández-Beltran, V.H. Díaz-Ramirez, L. Trujillo, P. Legrand, Design of estimators for restoration of images degraded by haze using genetic programming. Swarm Evol. Comput. 44, 49–63 (2019)CrossRefGoogle Scholar
  18. 18.
    T.K. Ho, Random decision forests, in Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 1 (1995), pp. 278–282Google Scholar
  19. 19.
    P. Juárez-Smith, L. Trujillo, Integrating local search within neat-GP, in Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, GECCO ’16 Companion (ACM, New York, 2016), pp. 993–996Google Scholar
  20. 20.
    S.S. Kim, K.C. Kwak, Development of quantum-based adaptive neuro-fuzzy networks. IEEE Trans. Syst. Man Cybern. B (Cyberne.) 40(1), 91–100 (2010)CrossRefGoogle Scholar
  21. 21.
    M. Kommenda, G. Kronberger, S.M. Winkler, M. Affenzeller, S. Wagner, Effects of constant optimization by nonlinear least squares minimization in symbolic regression, in Genetic and Evolutionary Computation Conference, GECCO ’13, Amsterdam, The Netherlands, July 6–10, 2013, Companion Material Proceedings (2013), pp. 1121–1128Google Scholar
  22. 22.
    J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, Cambridge, 1992)zbMATHGoogle Scholar
  23. 23.
    J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evolvable Mach. 11(3–4), 251–284 (2010)CrossRefGoogle Scholar
  24. 24.
    W.B. Langdon, P. Riccardo, Foundations of Genetic Programming (Springer, Berlin, 2002)CrossRefzbMATHGoogle Scholar
  25. 25.
    D. Medernach, J. Fitzgerald, R.M.A. Azad, C. Ryan, A new wave: a dynamic approach to genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO ’16 (ACM, New York, 2016), pp. 757–764Google Scholar
  26. 26.
    A. Moraglio, K. Krawiec, C.G. Johnson, Geometric semantic genetic programming, in Proceedings of the 12th International Conference on Parallel Problem Solving from Nature—Volume Part I (Springer, Berlin, 2012), pp. 21–31Google Scholar
  27. 27.
    G. Olague, L. Trujillo, Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011)CrossRefGoogle Scholar
  28. 28.
    I. Ortigosa, R. López, J. Garcia, A neural networks approach to residuary resistance of sailing yachts prediction, in Proceedings of the International Conference on Marine Engineering MARINE (2007), p. 250Google Scholar
  29. 29.
    R. Poli, W.B. Langdon, S. Dignum, On the limiting distribution of program sizes in tree-based genetic programming, in Genetic Programming (Springer, Berlin, 2007), pp. 193–204Google Scholar
  30. 30.
    R. Poli, W.B. Langdon, N.F. McPhee, A Field Guide to Genetic Programming (Lulu Enterprises, Morrisville, 2008)Google Scholar
  31. 31.
    J.R. Quinlan, Combining instance-based and model-based learning, in Machine Learning, Proceedings of the Tenth International Conference, University of Massachusetts, Amherst, MA, USA, June 27–29, 1993 (1993), pp. 236–243Google Scholar
  32. 32.
    S.S. Roy, R. Roy, V.E. Balas, Estimating heating load in buildings using multivariate adaptive regression splines, extreme learning machine, a hybrid model of MARS and ELM. Renew. Sustain. Energy Rev. 82, 4256–4268 (2018)CrossRefGoogle Scholar
  33. 33.
    Y.-H. Shao, C.-H. Zhang, Z.-M. Yang, L. Jing, N.-Y. Deng, An \(\epsilon \)-twin support vector machine for regression. Neural Comput. Appl. 23(1), 175–185 (2013)CrossRefGoogle Scholar
  34. 34.
    S. Silva, Reassembling operator equalisation: a secret revealed, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO ’11 (ACM, New York, 2011), pp. 1395–1402Google Scholar
  35. 35.
    S. Silva, E. Costa, Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genet. Program. Evolvable Mach. 10(2), 141–179 (2009)CrossRefGoogle Scholar
  36. 36.
    D. Sorensen, Newton’s method with a model trust region modification. SIAM J. Numer. Anal. 16, 409–426 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)CrossRefGoogle Scholar
  38. 38.
    L. Trujillo, P. Legrand, G. Olague, J. LéVy-VéHel, Evolving estimators of the pointwise Hölder exponent with genetic programming. Inf. Sci. 209, 61–79 (2012)CrossRefGoogle Scholar
  39. 39.
    L. Trujillo, L. Muñoz, E. Galván-López, S. Silva, Neat genetic programming: controlling bloat naturally. Inf. Sci. 333, 21–43 (2016)CrossRefGoogle Scholar
  40. 40.
    A. Tsanas, A. Xifara, Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)CrossRefGoogle Scholar
  41. 41.
    E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evol. Comput. 13(2), 333–349 (2009)CrossRefGoogle Scholar
  42. 42.
    I.-C. Yeh, Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)CrossRefGoogle Scholar
  43. 43.
    E. Z-Flores, L. Trujillo, O. Schütze, P. Legrand, A local search approach to genetic programming for binary classification, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15 (ACM, New York, 2015), pp. 1151–1158Google Scholar
  44. 44.
    E. Z-Flores, L. Trujillo, O. Schütze, P. Legrand, EVOLVE—A Bridge Between Probability, Set Oriented Numerics, and Evolutionary Computation V. Chapter, Evaluating the Effects of Local Search in Genetic Programming (Springer, New York, 2014)Google Scholar
  45. 45.
    E. Z-Flores, M. Abatal, A. Bassam, L. Trujillo, P. Juárez-Smith, Y.E. Hamzaoui, Modeling the adsorption of phenols and nitrophenols by activated carbon using genetic programming. J. Clean. Prod. 161, 860–870 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Tecnológico Nacional de México/Instituto Tecnológico de TijuanaTijuanaMexico
  2. 2.Universidad de ExtremaduraBadajozSpain

Personalised recommendations