Advertisement

Feature Standardisation in Symbolic Regression

  • Caitlin A.  Owen
  • Grant Dick
  • Peter A.  Whigham
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11320)

Abstract

While standardisation of variables is a common practice for many machine learning algorithms, it is rarely seen in the literature on genetic programming for symbolic regression. This paper compares the predictive performance of unscaled and standardised genetic programming, using artificial datasets and benchmark problems. Linear scaling is also applied to genetic programming for these problems. We show that unscaled genetic programming provides worse predictive performance than genetic programming augmented by linear scaling and/or standardisation as it is highly sensitive to the magnitude and range of explanatory or response variables. While linear scaling does provide better predictive performance on the simple artificial datasets, we attribute much of its success to an implicit standardisation within the predictive model. For benchmark problems, the combination of linear scaling and standardisation provides greater stability than only applying linear scaling to genetic programming. Also, for many of the simple artificial datasets, unscaled genetic programming produces larger individuals, which is undesirable in the search for parsimonious models.

Keywords

Genetic programming Standardisation Linear scaling 

References

  1. 1.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  2. 2.
    Breiman, L., Friedman, J., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. CRC Press, Boca Raton (1984)zbMATHGoogle Scholar
  3. 3.
    Dick, G.: Bloat and generalisation in symbolic regression. In: Dick, G., et al. (eds.) SEAL 2014. LNCS, vol. 8886, pp. 491–502. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-13563-2_42CrossRefGoogle Scholar
  4. 4.
    Dick, G.: Improving geometric semantic genetic programming with safe tree initialisation. In: Machado, P., et al. (eds.) EuroGP 2015. LNCS, vol. 9025, pp. 28–40. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16501-1_3CrossRefGoogle Scholar
  5. 5.
    Dick, G., Owen, C.A., Whigham, P.A.: Evolving bagging ensembles using a spatially-structured niching method. In: Proceedings of the 2018 Annual Conference on Genetic and Evolutionary Computation. ACM (2018)Google Scholar
  6. 6.
    Harrison, D., Rubinfeld, D.L.: Hedonic housing prices and the demand for clean air. J. Environ. Econ. Manage. 5(1), 81–102 (1978)CrossRefGoogle Scholar
  7. 7.
    Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 70–82. Springer, Heidelberg (2003).  https://doi.org/10.1007/3-540-36599-0_7CrossRefGoogle Scholar
  8. 8.
    Keijzer, M., Babovic, V.: Dimensionally aware genetic programming. In: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation, pp. 1069–1076. Morgan Kaufmann Publishers Inc., San Francisco (1999)Google Scholar
  9. 9.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  10. 10.
    LeCun, Y.A., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient backprop. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 9–48. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-35289-8_3CrossRefGoogle Scholar
  11. 11.
    Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 21–31. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32937-1_3CrossRefGoogle Scholar
  12. 12.
    Ni, J., Drieberg, R.H., Rockett, P.I.: The use of an analytic quotient operator in genetic programming. IEEE Trans. Evol. Comput. 17(1), 146–152 (2013)CrossRefGoogle Scholar
  13. 13.
    Nordin, P., Francone, F., Banzhaf, W.: Advances in genetic programming. In: Explicitly Defined Introns and Destructive Crossover in Genetic Programming, pp. 111–134. MIT Press, Cambridge (1996)Google Scholar
  14. 14.
    Quinlan, J.R.: Combining instance-based and model-based learning. In: Proceedings of the 10th International Conference on Machine Learning, pp. 236–243 (1993)CrossRefGoogle Scholar
  15. 15.
    Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, pp. 155–162. Morgan Kaufmann Publishers Inc., San Francisco (2001)Google Scholar
  16. 16.
    Uy, N.Q., Hoai, N.X., O’Neill, M., McKay, R.I., Galván-López, E.: Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Program. Evolvable Mach. 12(2), 91–119 (2011)CrossRefGoogle Scholar
  17. 17.
    White, D.R., et al.: Better GP benchmarks: community survey results and proposals. Genet. Program. Evolvable Mach. 14(1), 3–29 (2013)CrossRefGoogle Scholar
  18. 18.
    Yeh, I.C.: Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Information ScienceUniversity of OtagoDunedinNew Zealand

Personalised recommendations