Skip to main content

Local Search is Underused in Genetic Programming

  • Chapter
  • First Online:
Genetic Programming Theory and Practice XIV

Abstract

There are two important limitations of standard tree-based genetic programming (GP). First, GP tends to evolve unnecessarily large programs, what is referred to as bloat. Second, GP uses inefficient search operators that focus on modifying program syntax. The first problem has been studied extensively, with many works proposing bloat control methods. Regarding the second problem, one approach is to use alternative search operators, for instance geometric semantic operators, to improve convergence. In this work, our goal is to experimentally show that both problems can be effectively addressed by incorporating a local search optimizer as an additional search operator. Using real-world problems, we show that this rather simple strategy can improve the convergence and performance of tree-based GP, while also reducing program size. Given these results, a question arises: Why are local search strategies so uncommon in GP? A small survey of popular GP libraries suggests to us that local search is underused in GP systems. We conclude by outlining plausible answers for this question and highlighting future work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 54.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We will use the terms “standard GP” and “tree-based GP” interchangeably in this work, referring to the basic GP algorithm that relies on a tree representation and subtree genetic operators.

  2. 2.

    http://www.tree-lab.org/index.php/resources-2/downloads/open-source-tools/item/145-neat-gp.

References

  1. Azad, R.M.A., Ryan, C.: A simple approach to lifetime learning in genetic programming based symbolic regression. Evol. Comput. 22(2), 287–317 (2014). https://doi.org/10.1162/EVCO_a_00111. http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00111

    Article  Google Scholar 

  2. Castelli, M., Silva, S., Vanneschi, L.: A C++ framework for geometric semantic genetic programming. Genet. Program. Evolvable Mach. 16(1), 73–81 (2015). https://doi.org/10.1007/s10710-014-9218-0

    Article  Google Scholar 

  3. Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of energy performance of residential buildings: a genetic programming approach. Energy Build. 102(1), 67–74 (2015). https://doi.org/10.1016/j.enbuild.2015.05.013. http://www.sciencedirect.com/science/article/pii/S0378778815003849

    Article  Google Scholar 

  4. Castelli, M., Trujillo, L., Vanneschi, L., Popovic, A.: Prediction of relative position of CT slices using a computational intelligence system. Appl. Soft Comput. (2015). https://10.1016/j.asoc.2015.09.021. http://www.sciencedirect.com/science/article/pii/S1568494615005931

  5. Castelli, M., Trujillo, L., Vanneschi, L., Silva, S., Z-Flores, E., Legrand, P.: Geometric semantic genetic programming with local search. In: Silva, S., Esparcia-Alcazar, A.I., Lopez-Ibanez, M., Mostaghim, S., Timmis, J., Zarges, C., Correia, L., Soule, T., Giacobini, M., Urbanowicz, R., Akimoto, Y., Glasmachers, T., Fernandez de Vega, F., Hoover, A., Larranaga, P., Soto, M., Cotta, C., Pereira, F.B., Handl, J., Koutnik, J., Gaspar-Cunha, A., Trautmann, H., Mouret, J.B., Risi, S., Costa, E., Schuetze, O., Krawiec, K., Moraglio, A., Miller, J.F., Widera, P., Cagnoni, S., Merelo, J., Hart, E., Trujillo, L., Kessentini, M., Ochoa, G., Chicano, F., Doerr, C. (eds.) GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 999–1006. ACM, Madrid (2015). http://doi.acm.org/10.1145/2739480.2754795

  6. Castelli, M., Vanneschi, L., Trujillo, L.: Energy consumption forecasting using semantics based genetic programming with local search optimizer. Comput. Intell. Neurosci. 2015 (2015). https://doi.org/10.1155/2015/971908. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4464001/

    Article  Google Scholar 

  7. Castelli, M., Sormani, R., Trujillo, L., Popovic, A.: Predicting per capita violent crimes in urban areas: an artificial intelligence approach. J. Ambient. Intell. Humaniz. Comput. 8(1), 29–36 (2017). https://doi.org/10.1007/s12652-015-0334-3

    Article  Google Scholar 

  8. Chen, D.Y., Chuang, T.R., Tsai, S.C.: JGAP: a java-based graph algorithms platform. Softw. Pract. Exp. 31(7), 615–635 (2001)

    Article  Google Scholar 

  9. Chen, X., Ong, Y.S., Lim, M.H., Tan, K.C.: A multi-facet survey on memetic computation. IEEE Trans. Evol. Comput. 15(5), 591–607 (2011)

    Article  Google Scholar 

  10. De Rainville, F.M., Fortin, F.A., Gardner, M.A., Parizeau, M., Gagne, C.: Deap: a python framework for evolutionary algorithms. In: Wagner, S., Affenzeller, M. (eds.) GECCO 2012 Evolutionary Computation Software Systems (EvoSoft), pp. 85–92. ACM, Philadelphia, PA (2012). https://doi.org/10.1145/2330784.2330799

    Google Scholar 

  11. Dignum, S., Poli, R.: Operator equalisation and bloat free GP. In: O’Neill, M., Vanneschi, L., Gustafson, S., Esparcia Alcazar, A.I., De Falco, I., Della Cioppa, A., Tarantino, E. (eds.) Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008. Lecture Notes in Computer Science, vol. 4971, pp. 110–121. Springer, Naples (2008). https://doi.org/10.1007/978-3-540-78671-9_10

    Google Scholar 

  12. Emmerich, M., Grötzner, M., Schütz, M.: Design of graph-based evolutionary algorithms: a case study for chemical process networks. Evol. Comput. 9(3), 329–354 (2001)

    Article  Google Scholar 

  13. Gagné, C., Parizeau, M.: Open BEAGLE: A new C++ evolutionary computation framework. In: Langdon, W.B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., V. Honavar, G. Rudolph, J. Wegener, L. Bull, M.A. Potter, A.C. Schultz, J.F. Miller, E. Burke, N. Jonoska (eds.) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, p. 888. Morgan Kaufmann Publishers, New York (2002). http://www.cs.bham.ac.uk/~wbl/biblio/gecco2002/GP272.pdf

  14. Graff, M., Pena, R., Medina, A.: Wind speed forecasting using genetic programming. In: de la Fraga, L.G. (ed.) 2013 IEEE Conference on Evolutionary Computation, vol. 1, pp. 408–415. Cancun, Mexico (2013). https://doi.org/10.1109/CEC.2013.6557598

    Chapter  Google Scholar 

  15. Keijzer, M., Merelo, J.J., Romero, G., Schoenauer, M.: Evolving objects: a general purpose evolutionary computation library. In: EA-01, Evolution Artificielle, 5th International Conference in Evolutionary Algorithms, pp. 231–244 (2001). http://www.lri.fr/~marc/EO/EO-EA01.ps.gz

    Google Scholar 

  16. Kommenda, M., Kronberger, G., Winkler, S., Affenzeller, M., Wagner, S.: Effects of constant optimization by nonlinear least squares minimization in symbolic regression. In: Proceedings of the 15th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 1121–1128. ACM, New York (2013)

    Google Scholar 

  17. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA (1992). http://mitpress.mit.edu/books/genetic-programming

    MATH  Google Scholar 

  18. Koza, J.R.: Human-competitive results produced by genetic programming. Genet. Program. Evolvable Mach. 11(3/4), 251–284 (2010). https://doi.org/10.1007/s10710-010-9112-3. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.297.6227. Tenth Anniversary Issue: Progress in Genetic Programming and Evolvable Machines

    Article  Google Scholar 

  19. La Cava, W., Spector, L.: Inheritable epigenetics in genetic programming. In: Riolo, R., Worzel, W.P., Kotanchek, M. (eds.) Genetic Programming Theory and Practice XII, Genetic and Evolutionary Computation, pp. 37–51. Springer, Ann Arbor (2014). https://doi.org/10.1007/978-3-319-16030-6_3

    Google Scholar 

  20. Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer, Berlin (2002). https://doi.org/10.1007/978-3-662-04726-2. http://www.cs.ucl.ac.uk/staff/W.Langdon/FOGP/

    Book  Google Scholar 

  21. Lara, A., Sanchez, G., Coello, C.A.C., Schutze, O.: Hcs: A new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 14(1), 112–132 (2010)

    Article  Google Scholar 

  22. Lohmann, R.: Application of evolution strategy in parallel populations. In: International Conference on Parallel Problem Solving from Nature, pp. 198–208. Springer, Berlin (1990)

    Google Scholar 

  23. McConaghy, T.: Ffx: fast, scalable, deterministic symbolic regression technology. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, chap. 13, pp. 235–260. Springer, Ann Arbor (2011). https://doi.org/10.1007/978-1-4614-1770-5_13. http://trent.st/content/2011-GPTP-FFX-paper.pdf

    Google Scholar 

  24. Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: International Conference on Parallel Problem Solving from Nature, pp. 21–31. Springer, Berlin (2012)

    Google Scholar 

  25. Neri, F., Cotta, C., Moscato, P.: Handbook of Memetic Algorithms, vol. 379. Springer, Berlin (2012)

    Google Scholar 

  26. Olague, G., Trujillo, L.: Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011). https://doi.org/10.1016/j.imavis.2011.03.004. http://www.sciencedirect.com/science/article/B6V09-52GXV83-1/2/1462102339b445428fa4f2702939a41e

    Article  Google Scholar 

  27. Ortigosa, I., Lopez, R., Garcia, J.: A neural networks approach to residuary resistance of sailing yachts prediction. In: Proceedings of the International Conference on Marine Engineering MARINE, vol. 2007, p. 250 (2007)

    Google Scholar 

  28. Poli, R.: TinyGP. See TinyGP GECCO 2004 competition at http://cswww.essex.ac.uk/staff/sml/gecco/TinyGP.html (2004)

  29. Poli, R., McPhee, N.F.: General schema theory for genetic programming with subtree-swapping crossover: Part I. Evol. Comput. 11(1), 53–66 (2003). https://doi.org/10.1162/106365603321829005. http://cswww.essex.ac.uk/staff/rpoli/papers/ecj2003partI.pdf

    Article  Google Scholar 

  30. Poli, R., McPhee, N.F.: General schema theory for genetic programming with subtree-swapping crossover: Part II. Evol. Comput. 11(2), 169–206 (2003). https://doi.org/10.1162/106365603766646825. http://cswww.essex.ac.uk/staff/rpoli/papers/ecj2003partII.pdf

    Article  Google Scholar 

  31. Poli, R., McPhee, N.: Parsimony pressure made easy. In: Keijzer, M., Antoniol, G., Congdon, C.B., Deb, K., Doerr, B., Hansen, N., Holmes, J.H., Hornby, G.S., Howard, D., Kennedy, J., Kumar, S., Lobo, F.G., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Pollack, J., Sastry, K., Stanley, K., Stoica, A., Talbi, E.G., Wegener, I. (eds.) GECCO ’08: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 1267–1274. ACM, Atlanta (2008). https://doi.org/10.1145/1389095.1389340. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2008/docs/p1267.pdf

  32. Punch, B., Zongker, D.: LiL-GP 1.1. a genetic programming system (1998)

    Google Scholar 

  33. Quinlan, J.R.: Combining instance-based and model-based learning. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 236–243 (1993)

    Chapter  Google Scholar 

  34. Silva, S.: Reassembling operator equalisation: a secret revealed. In: Krasnogor, N., Lanzi, P.L., Engelbrecht, A., Pelta, D., Gershenson, C., Squillero, G., Freitas, A., Ritchie, M., Preuss, M., Gagne, C., Ong, Y.S., Raidl, G., Gallager, M., Lozano, J., Coello-Coello, C., Silva, D.L., Hansen, N., Meyer-Nieberg, S., Smith, J., Eiben, G., Bernado-Mansilla, E., Browne, W., Spector, L., Yu, T., Clune, J., Hornby, G., Wong, M.L., Collet, P., Gustafson, S., Watson, J.P., Sipper, M., Poulding, S., Ochoa, G., Schoenauer, M., Witt, C., Auger A. (eds.) GECCO ’11: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 1395–1402. ACM, Dublin (2011) https://doi.org/10.1145/2001576.2001764

  35. Silva, S., Almeida, J.: Gplab-a genetic programming toolbox for matlab. In: Proceedings of the Nordic MATLAB Conference, pp. 273–278 (2003)

    Google Scholar 

  36. Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories. Genetic Program. Evolvable Mach. 10(2), 141–179 (2009). https://doi.org/10.1007/s10710-008-9075-9

    Article  Google Scholar 

  37. Silva, S., Vanneschi, L.: The importance of being flat-studying the program length distributions of operator equalisation. In: Riolo, R., Vladislavleva, E., Moore J.H. (eds.) Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, chap. 12, pp. 211–233. Springer, Ann Arbor (2011). https://doi.org/10.1007/978-1-4614-1770-5_12

    Google Scholar 

  38. Silva, S., Dignum, S., Vanneschi, L.: Operator equalisation for bloat free genetic programming and a survey of bloat control methods. Genetic Programming and Evolvable Machines 13(2), 197–238 (2012). https://doi.org/10.1007/s10710-011-9150-5

    Article  Google Scholar 

  39. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  40. Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 155–162. Morgan Kaufmann, San Francisco, CA (2001). http://garage.cse.msu.edu/papers/GARAGe01-07-01.pdf

    Google Scholar 

  41. Trujillo, L., Legrand, P., Olague, G., Levy-Vehel, J.: Evolving estimators of the pointwise hoelder exponent with genetic programming. Inf. Sci. 209, 61–79 (2012). https://doi.org/10.1016/j.ins.2012.04.043. http://www.sciencedirect.com/science/article/pii/S0020025512003386

    Article  Google Scholar 

  42. Trujillo, L., Munoz, L., Naredo, E., Martinez, Y.: Neat, there’s no bloat. In: Nicolau, M., Krawiec, K., Heywood, M.I., Castelli, M., Garcia-Sanchez, P., Merelo, J.J., Rivas Santos, V.M., Sim K. (eds.) 17th European Conference on Genetic Programming. Lecture Notes in Computer Science, vol. 8599, pp. 174–185. Springer, Granada (2014). https://doi.org/10.1007/978-3-662-44303-3_15

    Google Scholar 

  43. Trujillo, L., Munoz, L., Galvan-Lopez, E., Silva, S.: Neat genetic programming: controlling bloat naturally. Inf. Sci. 333, 21–43 (2016). https://doi.org/10.1016/j.ins.2015.11.010. http://www.sciencedirect.com/science/article/pii/S0020025515008038

    Article  Google Scholar 

  44. Tsanas, A., Xifara, A.: Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)

    Article  Google Scholar 

  45. Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming. Genetic Program. Evolvable Mach. 15(2), 195–214 (2014). https://doi.org/10.1007/s10710-013-9210-0. http://link.springer.com/article/10.1007/s10710-013-9210-0

    Article  Google Scholar 

  46. Wagner, S., Affenzeller, M.: Heuristiclab: A generic and extensible optimization environment. In: Adaptive and Natural Computing Algorithms, pp. 538–541. Springer, Berlin, (2005)

    Google Scholar 

  47. White, D.R.: Software review: the ECJ toolkit. Genet. Program. Evolvable Mach. 13(1), 65–67 (2012). https://doi.org/10.1007/s10710-011-9148-z

    Article  Google Scholar 

  48. Worm, T., Chiu, K.: Prioritized grammar enumeration: symbolic regression by dynamic programming. In: Blum, C., Alba, E., Auger, A., Bacardit, J., Bongard, J., Branke, J., Bredeche, N., Brockhoff, D., Chicano, F., Dorin, A., Doursat, R., Ekart, A., Friedrich, T., Giacobini, M., Harman, M., Iba, H., Igel, C., Jansen, T., Kovacs, T., Kowaliw, T., Lopez-Ibanez, M., Lozano, J.A., Luque, G., McCall, J., Moraglio, A., Motsinger-Reif, A., Neumann, F., Ochoa, G., Olague, G., Ong, Y.S., Palmer, M.E., Pappa, G.L., Parsopoulos, K.E., Schmickl, T., Smith, S.L., Solnon, C., Stuetzle, T., Talbi, E.G., Tauritz, D., Vanneschi L. (eds.) GECCO ’13: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference, pp. 1021–1028. ACM, Amsterdam (2013). https://doi.org/10.1145/2463372.2463486

  49. Z-Flores, E., Trujillo, L., Schuetze, O., Legrand, P.: Evaluating the effects of local search in genetic programming. In: Tantar, A.A., Tantar, E., Sun, J.Q., Zhang, W., Ding, Q., Schuetze, O., Emmerich, M., Legrand, P., Del Moral, P., Coello Coello, C.A. (eds.) EVOLVE - A Bridge between Probability, Set Oriented Numerics, and Evolutionary Computation V. Advances in Intelligent Systems and Computing, vol. 288, pp. 213–228. Springer, Peking (2014). https://doi.org/10.1007/978-3-319-07494-8_15. https://hal.inria.fr/hal-01060315

    MATH  Google Scholar 

  50. Z-Flores, E., Trujillo, L., Schuetze, O., Legrand, P.: A local search approach to genetic programming for binary classification. In: Silva, S., Esparcia-Alcazar, A.I., Lopez-Ibanez, M., Mostaghim, S., Timmis, J., Zarges, C., Correia, L., Soule, T., Giacobini, M., Urbanowicz, R., Akimoto, Y., Glasmachers, T., Fernandez de Vega, F., Hoover, A., Larranaga, P., Soto, M., Cotta, C., Pereira, F.B., Handl, J., Koutnik, J., Gaspar-Cunha, A., Trautmann, H., Mouret, J.B., Risi, S., Costa, E., Schuetze, O., Krawiec, K., Moraglio, A., Miller, J.F., Widera, P., Cagnoni, S., Merelo, J., Hart, E., Trujillo, L., Kessentini, M., Ochoa, G., Chicano, F., Doerr, C. (eds.) GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1151–1158. ACM, Madrid (2015). http://doi.acm.org/10.1145/2739480.2754797

  51. Zhang, M., Smart, W.: Genetic programming with gradient descent search for multiclass object classification. In: Keijzer, M., O’Reilly, U.M., Lucas, S.M., Costa, E., Soule T. (eds.) Genetic Programming 7th European Conference, EuroGP 2004, Proceedings, Lecture Notes in Computer Science, vol. 3003, pp. 399–408. Springer, Coimbra (2004). https://doi.org/10.1007/978-3-540-24650-3_38. http://www.springerlink.com/openurl.asp?genre=article&issn=0302-9743&volume=3003&spage=399

    Google Scholar 

Download references

Acknowledgements

Funding for this work was provided by CONACYT Basic Science Research Project No. 178323 and Fronteras de la Ciencia FC-2015-2/944, TecNM (México) Research Projects 5414.14-P and 5621.15-P, and by the FP7 Marie Curie-IRSES 2013 European Commission program through project ACoBSEC with contract 439 No. 612689

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Leonardo Trujillo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Trujillo, L. et al. (2018). Local Search is Underused in Genetic Programming. In: Riolo, R., Worzel, B., Goldman, B., Tozier, B. (eds) Genetic Programming Theory and Practice XIV. Genetic and Evolutionary Computation. Springer, Cham. https://doi.org/10.1007/978-3-319-97088-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-97088-2_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-97087-5

  • Online ISBN: 978-3-319-97088-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics