Evaluating the Effects of Local Search in Genetic Programming

  • Emigdio Z-Flores
  • Leonardo Trujillo
  • Oliver Schütze
  • Pierrick Legrand
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 288)


Genetic programming (GP) is an evolutionary computation paradigm for the automatic induction of syntactic expressions. In general, GP performs an evolutionary search within the space of possible program syntaxes, for the expression that best solves a given problem. The most common application domain for GP is symbolic regression, where the goal is to find the syntactic expression that best fits a given set of training data. However, canonical GP only employs a syntactic search, thus it is intrinsically unable to efficiently adjust the (implicit) parameters of a particular expression. This work studies a Lamarckian memetic GP, that incorporates a local search (LS) strategy to refine GP individuals expressed as syntax trees. In particular, a simple parametrization for GP trees is proposed, and different heuristic methods are tested to determine which individuals should be subject to a LS, tested over several benchmark and real-world problems. The experimental results provide necessary insights in this insufficiently studied aspect of GP, suggesting promising directions for future work aimed at developing new memetic GP systems.


Genetic Programming Local Search Memetic Algorithms 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chen, X., Ong, Y.-S., Lim, M.-H., Tan, K.C.: A multi-facet survey on memetic computation. Trans. Evol. Comp. 15(5), 591–607 (2011)CrossRefGoogle Scholar
  2. 2.
    Coello, C.A.C., Lamont, G.B., Veldhuizen, D.A.V.: Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
  3. 3.
    Coleman, T.F., Li, Y.: On the convergence of reflective Newton methods for large-scale nonlinear minimization subject to bounds (1992)Google Scholar
  4. 4.
    Coleman, T.F., Li, Y.: An interior trust region approach for nonlinear minimization subject to bounds. Technical report, Ithaca, NY, USA (1993)Google Scholar
  5. 5.
    De Jong, K.: Evolutionary Computation: A Unified Approach. Bradford Book. Mit Press (2006)Google Scholar
  6. 6.
    Dunn, E., Olague, G., Lutton, E.: Parisian camera placement for vision metrology. Pattern Recogn. Lett. 27(11), 1209–1219 (2006)CrossRefGoogle Scholar
  7. 7.
    Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer (2003)Google Scholar
  8. 8.
    Emmerich, M., Grötzner, M., Schütz, M.: Design of graph-based evolutionary algorithms: A case study for chemical process networks. Evol. Comput. 9(3), 329–354 (2001)CrossRefGoogle Scholar
  9. 9.
    Eskridge, B., Hougen, D.: Imitating success: A memetic crossover operator for genetic programming. In: Proceedings of the 2004 IEEE Congress on Evolutionary Computation, June 20-23, pp. 809–815. IEEE Press, Portland (2004)Google Scholar
  10. 10.
    Gill, P.E., Murray, W., Wright, M.H.: Practical optimization. Academic Press Inc. (Harcourt Brace Jovanovich Publishers), London (1981)Google Scholar
  11. 11.
    Graff, M., Pea, R., Medina, A.: Wind speed forecasting using genetic programming. In: IEEE Congress on Evolutionary Computation, pp. 408–415. IEEE (2013)Google Scholar
  12. 12.
    Hornby, G.S., Lohn, J.D., Linden, D.S.: Computer-automated evolution of an x-band antenna for nasa’s space technology 5 mission. Evol. Comput. 19(1), 1–23 (2011)CrossRefGoogle Scholar
  13. 13.
    Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 70–82. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Korns, M.F.: Accuracy in symbolic regression. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic Programming Theory and Practice IX, May 12-14. Genetic and Evolutionary Computation, ch. 8, pp. 129–151. Springer, Ann Arbor (2011)Google Scholar
  15. 15.
    Koza, J.: Human-competitive results produced by genetic programming. Genetic Programming and Evolvable Machines 11(3), 251–284 (2010)CrossRefGoogle Scholar
  16. 16.
    Koza, J.R.: Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge (1992)Google Scholar
  17. 17.
    Lawson, C.L., Hanson, R.J.: Solving Least Squares Problems. Society for Industrial and Applied Mathematics (1995)Google Scholar
  18. 18.
    Lohmann, R.: Application of Evolution Strategy in Parallel Populations. In: Schwefel, H.-P., Männer, R. (eds.) PPSN 1990. LNCS, vol. 496, pp. 198–208. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  19. 19.
    Luke, S.: Essentials of Metaheuristics, 2nd edn. Lulu (2013),
  20. 20.
    McConaghy, T.: FFX: Fast, scalable, deterministic symbolic regression technology. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic Programming Theory and Practice IX. Genetic and Evolutionary Computation, ch. 13, pp. 235–260. Springer, Ann Arbor (2011)Google Scholar
  21. 21.
    McDermott, J., White, D.R., Luke, S., Manzoni, L., Castelli, M., Vanneschi, L., Jaskowski, W., Krawiec, K., Harper, R., De Jong, K., O’Reilly, U.-M.: Genetic programming needs better benchmarks. In: Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference, GECCO 2012, pp. 791–798. ACM, New York (2012)Google Scholar
  22. 22.
    Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Scientific and Statistical Computing 4, 553–572 (1983)CrossRefzbMATHGoogle Scholar
  23. 23.
    Olague, G., Trujillo, L.: Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vision Comput 29(7), 484–498 (2011)CrossRefGoogle Scholar
  24. 24.
    Pagie, L., Hogeweg, P.: Evolutionary consequences of coevolving targets. Evolutionary Computation 5, 401–418 (1998)CrossRefGoogle Scholar
  25. 25.
    Poli, R., Langdon, W.B., McPhee, N.F.: A Field Guide to Genetic Programming. Lulu Enterprises, UK Ltd. (2008)Google Scholar
  26. 26.
    Shultz, G., Schnabel, R., Byrd, R., Colorado Univ. at Boulder Dept of Computer Science: A Family of Trust Region Based Algorithms for Unconstrained Minimization with Strong Global Convergence Properties. Defense Technical Information Center (1982)Google Scholar
  27. 27.
    Silva, S., Almeida, J.: Gplab–a genetic programming toolbox for matlab. In: Gregersen, L. (ed.) Proceedings of the Nordic MATLAB Conference, pp. 273–278 (2003)Google Scholar
  28. 28.
    Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Programming and Evolvable Machines 10(2), 141–179 (2009)CrossRefMathSciNetGoogle Scholar
  29. 29.
    Smart, W., Zhang, M.: Continuously evolving programs in genetic programming using gradient descent. In: Mckay, R.I., Cho, S.-B. (eds.) Proceedings of The Second Asian-Pacific Workshop on Genetic Programming, p. 16. Cairns, Australia (2004)Google Scholar
  30. 30.
    Sorensen, D.: Newton’s Method with a Model Trust Region Modification. Defense Technical Information Center (1982)Google Scholar
  31. 31.
    Spector, L.: Automatic Quantum Computer Programming: A Genetic Programming Approach (Genetic Programming). Springer-Verlag New York, Inc., Secaucus (2006)Google Scholar
  32. 32.
    Steihaug, T.: The Conjugate Gradient Method and Trust Regions in Large Scale Optimization. SIAM Journal on Numerical Analysis 20(3), 626–637 (1983)CrossRefzbMATHMathSciNetGoogle Scholar
  33. 33.
    Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), July 7-11, pp. 155–162. Morgan Kaufmann (2001)Google Scholar
  34. 34.
    Trujillo, L., Naredo, E., Martínez, Y.: Preliminary study of bloat in genetic programming with behavior-based search. In: Emmerich, M., et al. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics,and Evolutionary Computation IV. AISC, vol. 227, pp. 293–305. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  35. 35.
    Uy, N.Q., Hoai, N.X., O’Neill, M., Mckay, R.I., Galván-López, E.: Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genetic Programming and Evolvable Machines 12(2), 91–119 (2011)CrossRefGoogle Scholar
  36. 36.
    Vladislavleva, E.J., Smits, G.F., Den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. Trans. Evol. Comp. 13(2), 333–349 (2009)CrossRefGoogle Scholar
  37. 37.
    Wagner, S., Kronberger, G.: Algorithm and experiment design with heuristic lab: An open source optimization environment for research and education. In: Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference Companion, GECCO Companion 2012, pp. 1287–1316. ACM, New York (2012)CrossRefGoogle Scholar
  38. 38.
    Wang, P., Tang, K., Tsang, E.P.K., Yao, X.: A memetic genetic programming with decision tree-based local search for classification problems. In: IEEE Congress on Evolutionary Computation, pp. 917–924. IEEE (2011)Google Scholar
  39. 39.
    White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B., Kronberger, G., Ja’skowski, W., O’Reilly, U.-M., Luke, S.: Better gp benchmarks: community survey results and proposals. Genetic Programming and Evolvable Machines 14(1), 3–29 (2013)CrossRefGoogle Scholar
  40. 40.
    Worm, T., Chiu, K.: Prioritized grammar enumeration: Symbolic regression by dynamic programming. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, GECCO 2013, pp. 1021–1028. ACM, New York (2013)CrossRefGoogle Scholar
  41. 41.
    Yuan, J.Y.: Numerical methods for generalized least squares problems. Journal of Computational and Applied Mathematics 66(1-2), 571–584 (1996)CrossRefzbMATHMathSciNetGoogle Scholar
  42. 42.
    Zhang, M., Smart, W.: Genetic programming with gradient descent search for multiclass object classification. In: Keijzer, M., O’Reilly, U.-M., Lucas, S., Costa, E., Soule, T. (eds.) EuroGP 2004. LNCS, vol. 3003, pp. 399–408. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Emigdio Z-Flores
    • 1
  • Leonardo Trujillo
    • 1
  • Oliver Schütze
    • 2
  • Pierrick Legrand
    • 3
    • 4
  1. 1.TREE-LAB, Doctorado en Ciencias de la Ingeniería, Departamento de Ingeniería Eléctrica y ElectrónicaInstituto Tecnológico de TijuanaTijuanaMéxico
  2. 2.Computer Science DepartmentCINVESTAV-IPNMexico CityMéxico
  3. 3.UMR CNRS 5251Université Victor Segalen Bordeaux 2 and The Institut de Mathématiques de BordeauxBordeauxFrance
  4. 4.ALEA TeamINRIA Bordeaux Sud-OuestTalenceFrance

Personalised recommendations