Skip to main content
Book cover

International Conference on Computer Aided Systems Theory

EUROCAST 2013: Computer Aided Systems Theory - EUROCAST 2013 pp 420–427Cite as

Nonlinear Least Squares Optimization of Constants in Symbolic Regression

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 8111)

Abstract

In this publication a constant optimization approach for symbolic regression by genetic programming is presented. The Levenberg-Marquardt algorithm, a nonlinear, least-squares method, tunes numerical values of constants in symbolic expression trees to improve their fit to observed data. The necessary gradient information for the algorithm is obtained by automatic programming, which efficiently calculates the partial derivatives of symbolic expression trees.

The performance of the methodology is tested for standard and offspring selection genetic programming on four well-known benchmark datasets. Although constant optimization includes an overhead regarding the algorithm runtime, the achievable quality increases significantly compared to the standard algorithms. For example, the average coefficient of determination on the Poly-10 problem changes from 0.537 without constant optimization to over 0.8 with constant optimization enabled. In addition to the experimental results, the effect of different parameter settings like the number of individuals to be optimized is detailed.

Keywords

  • Constant Optimization
  • Symbolic Regression
  • Genetic Programming
  • Levenberg-Marquard Algorithm
  • Automatic Differentiation

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-53856-8_53
  • Chapter length: 8 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-642-53856-8
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Affenzeller, M., Winkler, S., Wagner, S., Beham, A.: Genetic Algorithms and Genetic Programming - Modern Concepts and Practical Applications, Numerical Insights, vol. 6. CRC Press, Chapman & Hall (2009)

    Google Scholar 

  2. Alonso, C.L., Montaña, J.L., Borges, C.E.: Evolution strategies for constants optimization in genetic programming. In: ICTAI, pp. 703–707. IEEE Computer Society (2009), http://dblp.uni-trier.de/db/conf/ictai/ictai2009.html#AlonsoMB09

  3. Bochkanov, S., Bystritsky, V.: Alglib, http://www.alglib.net/

  4. Friedman, J.H.: Multivariate adaptive regression splines. The Annals of Statistics, 1–67 (1991)

    Google Scholar 

  5. Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 70–82. Springer, Heidelberg (2003)

    CrossRef  Google Scholar 

  6. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)

    MATH  Google Scholar 

  7. Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quarterly Journal of Applied Mathmatics II(2), 164–168 (1944)

    MathSciNet  Google Scholar 

  8. Mukherjee, S., Eppstein, M.J.: Differential evolution of constants in genetic programming improves efficacy and bloat. In: Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference Companion, GECCO Companion 2012, pp. 625–626. ACM, New York (2012), http://doi.acm.org/10.1145/2330784.2330891

    CrossRef  Google Scholar 

  9. Poli, R.: A simple but theoretically-motivated method to control bloat in genetic programming. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E.P.K., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 204–217. Springer, Heidelberg (2003), http://dl.acm.org/citation.cfm?id=1762668.1762688

    CrossRef  Google Scholar 

  10. Rall, L.B.: Automatic Differentiation: Techniques and Applications. LNCS, vol. 120. Springer, Heidelberg (1981)

    CrossRef  MATH  Google Scholar 

  11. Shtof, A.: Autodiff, http://autodiff.codeplex.com/

  12. Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), July 7-11, pp. 155–162. Morgan Kaufmann, San Francisco (2001), http://www.cs.bham.ac.uk/~wbl/biblio/gecco2001/d01.pdf

    Google Scholar 

  13. Wagner, S.: Heuristic Optimization Software Systems - Modeling of Heuristic Optimization Algorithms in the HeuristicLab Software Environment. Ph.D. thesis, Institute for Formal Models and Verification, Johannes Kepler University, Linz, Austria (2009)

    Google Scholar 

  14. White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B.W., Kronberger, G., Jaskowski, W., O’Reilly, U.M., Luke, S.: Better GP benchmarks: community survey results and proposals. Genetic Programming and Evolvable Machines 14(1), 3–29 (2013)

    CrossRef  Google Scholar 

  15. Zhang, Q., Zhou, C., Xiao, W., Nelson, P.C.: Improving gene expression programming performance by using differential evolution. In: Proceedings of the Sixth International Conference on Machine Learning and Applications, ICMLA 2007, pp. 31–37. IEEE Computer Society, Washington, DC (2007), http://dx.doi.org/10.1109/ICMLA.2007.55

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Heuristic and Evolutionary Algorithms Laboratory, School of Informatics, Communications and Media, University of Applied Sciences Upper Austria, Softwarepark 11, 4232, Hagenberg, Austria

    Michael Kommenda, Michael Affenzeller, Gabriel Kronberger & Stephan M. Winkler

Authors
  1. Michael Kommenda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Affenzeller
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gabriel Kronberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephan M. Winkler
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Universidad de Las Palmas de Gran Canaria, Instituto Universitario de Ciencias y Tecnologías Cibernéticas, Campus de Tafira, 35017, Las Palmas de Gran Canaria, Spain

    Roberto Moreno-Díaz & Alexis Quesada-Arencibia & 

  2. Johannes Kepler Universität Linz, Altenbergerstrasse 69, 4040, Linz, Austria

    Franz Pichler

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Kommenda, M., Affenzeller, M., Kronberger, G., Winkler, S.M. (2013). Nonlinear Least Squares Optimization of Constants in Symbolic Regression. In: Moreno-Díaz, R., Pichler, F., Quesada-Arencibia, A. (eds) Computer Aided Systems Theory - EUROCAST 2013. EUROCAST 2013. Lecture Notes in Computer Science, vol 8111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-53856-8_53

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-53856-8_53

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-53855-1

  • Online ISBN: 978-3-642-53856-8

  • eBook Packages: Computer ScienceComputer Science (R0)