Abstract
We introduce the use of high order automatic differentiation, implemented via the algebra of truncated Taylor polynomials, in genetic programming. Using the Cartesian Genetic Programming encoding we obtain a high-order Taylor representation of the program output that is then used to back-propagate errors during learning. The resulting machine learning framework is called differentiable Cartesian Genetic Programming (dCGP). In the context of symbolic regression, dCGP offers a new approach to the long unsolved problem of constant representation in GP expressions. On several problems of increasing complexity we find that dCGP is able to find the exact form of the symbolic expression as well as the constants values. We also demonstrate the use of dCGP to solve a large class of differential equations and to find prime integrals of dynamical systems, presenting, in both cases, results that confirm the efficacy of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The exact details on all these experiments can be found in two IPython notebooks available here: https://goo.gl/iH5GAR, https://goo.gl/0TFsSv.
- 2.
The exact details on all these experiments can be found in the IPython notebook available here: https://goo.gl/8fOzYM.
- 3.
The full details on these experiments can be found in an IPython notebook available here: https://goo.gl/wnCkO9.
- 4.
The full details on our experiments can be found in an IPython notebook available here: https://goo.gl/ATrQR5.
References
Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: 2005 IEEE Congress on Evolutionary Computation, vol. 2, pp. 1777–1784. IEEE (2005)
Bertz, M.: Modern Map Methods in Particle Beam Physics, vol. 108. Academic Press, Cambridge (1999)
Borisov, A.V., Kholmskaya, A., Mamaev, I.S.: S.V. Kovalevskaya top and generalizations of integrable systems. Regul. Chaotic Dyn. 6(1), 1–16 (2001)
Casasayas, J., Nunes, A., Tufillaro, N.: Swinging Atwood’s machine: integrability and dynamics. J. de Phys. 51(16), 1693–1702 (1990)
Cerny, B.M., Nelson, P.C., Zhou, C.: Using differential evolution for symbolic regression and numerical constant creation. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 1195–1202. ACM (2008)
Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S.G., et al.: Hybrid computing using a neural network with dynamic external memory. Nature 538(7626), 471–476 (2016)
Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a new evolutionary computation. SFSC, vol. 192, pp. 75–102. Springer, Heidelberg (2006). doi:10.1007/3-540-32494-1_4
Izzo, D.: dCGP: first release, November 2016. https://doi.org/10.5281/zenodo.164627
Izzo, D., Biscani, F.: AuDi: first release, November 2016. https://doi.org/10.5281/zenodo.164628
Khan, M.M., Ahmad, A.M., Khan, G.M., Miller, J.F.: Fast learning neural networks using cartesian genetic programming. Neurocomputing 121, 274–289 (2013)
Koza, J.: Tutorial on advanced genetic programming, at genetic programming 1997, Palo Alto, CA (1997)
Miller, J.F.: Cartesian genetic programming. In: Miller, J.F. (ed.) Cartesian Genetic Programming. NCS, pp. 17–34. Springer, Heidelberg (2011). doi:10.1007/978-3-642-17310-3_2
O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genet. Program. Evolvable Mach. 11(3–4), 339–363 (2010)
Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009)
Schmidt, M., Lipson, H.: Symbolic regression of implicit equations. In: Riolo, R., O’Reilly, U.-M., McConaghy, T. (eds.) Genetic Programming Theory and Practice VII. GEC, pp. 73–85. Springer, Heidelberg (2010)
Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 155–162 (2001)
Tsoulos, I.G., Lagaris, I.E.: Solving differential equations with genetic programming. Genet. Program. Evolvable Mach. 7(1), 33–54 (2006)
Turner, A.J., Miller, J.F.: Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 1005–1012. ACM (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Izzo, D., Biscani, F., Mereta, A. (2017). Differentiable Genetic Programming. In: McDermott, J., Castelli, M., Sekanina, L., Haasdijk, E., GarcÃa-Sánchez, P. (eds) Genetic Programming. EuroGP 2017. Lecture Notes in Computer Science(), vol 10196. Springer, Cham. https://doi.org/10.1007/978-3-319-55696-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-55696-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55695-6
Online ISBN: 978-3-319-55696-3
eBook Packages: Computer ScienceComputer Science (R0)