Abstract
The subject of this paper is a new approach to symbolic regression. Other publications on symbolic regression use genetic programming. This paper describes an alternative method based on Pareto simulated annealing. Our method is based on linear regression for the estimation of constants. Interval arithmetic is applied to ensure the consistency of a model. To prevent overfitting, we merit a model not only on predictions in the data points, but also on the complexity of a model. For the complexity, we introduce a new measure. We compare our new method with the Kriging metamodel and against a symbolic regression metamodel based on genetic programming. We conclude that Pareto-simulated-annealing-based symbolic regression is very competitive compared to the other metamodel approaches.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Aarts E, Korst J (1989) Simulated annealing and boltzmann machines: a stochastic approach to combinatorial optimization and neural computing. Wiley, Chichester
Alexandrov NM, Dennis JE, Lewis RM, Torczon V (1998) A trust region framework for managing use of approximation models in optimization. J Struct Optim 15(1):16–23
Barthelemy JFM, Haftka RT (1993) Approximation concepts for optimum structural design—a review. J Struct Optim 5:129–144
Cuyt A, Verdonk B (1992) Multivariate rational data fitting: general data structure, maximal accuracy and object orientation. Numer Algorithms 3:159–172
Czyżak P, Jaszkiewicz A (1998) Pareto simulated annealing—a metaheuristic technique for multiple-objective combinatorial optimization. J Multi-Criteria Decis Anal 7(1):34–47
Gambling M, Jones RD, Toropov VV, Alvarez LF (2001) Application of optimization strategies to problems with highly non-linear response. In engineering design optimisation, proceedings of the 3rd ASMO UK/ISSMO conference on engineering design optimization. Harrogate, UK, pp 249–256 (9–10 July)
Garishina NV, Vladislavleva CJ (2004) On development of a complexity measure for symbolic regression via genetic programming. Technical report, Mathematics for industry program of the Stan Ackermans Institute, Eindhoven
Jin R, Chen W, Simpson T (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Multidiscipl Optim 23(1):1–13
Jin R, Du X, Chen W (2003) The use of metamodeling techniques for optimization under uncertainty. Struct Multidiscipl Optim 25:99–116
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492
Keijzer M (2003) Improving symbolic regression with interval arithmetic and linear scaling. In genetic programming, proceedings of EuroGP’2003, vol. 2610 of LNCS. Essex, UK, pp 71–83
Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT, Cambridge, MA, USA
Mathews JH, Fink KD (2004) Numerical methods using matlab, 4th edn. Prentice Hall Pub, Upper Saddle River, NJ, USA
Montgomery DC (1984) Design and analysis of experiments. Wiley, New York
Morris MD, Mitchell TJ (1995) Exploratory designs for computer experiments. J Stat Plan Inference 43:381–402
Powell MJD (1987) Radial basis functions for multivariable interpolation: a review. In: Algorithms for approximation of functions and data. Oxford University Press, Oxford, UK, pp 143–167
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4:409–435
Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscipl Optim 27(5):302–313
Smith KI, Everson RM, Fieldsend JE (2004) Dominance measures for multi-objective simulated annealing. In: Proceedings of the 2004 IEEE Congress on Evolutionary Computation. IEEE Press, New Jersey, pp 23–30
Smits G, Kotanchek M (2004) Pareto-front exploitation in symbolic regression. In: Genetic programming theory and practice II. Springer, Ann Arbor, USA, pp 283–299
Vapnik V, Golowich SE, Smola A (1997) Support vector method for function approximation, regression estimation, and signal processing. In: Advances in NeuralIAProcessings Systems. MIT, Cambridge, MA, USA, pp 281–287
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Stinstra, E., Rennen, G. & Teeuwen, G. Metamodeling by symbolic regression and Pareto simulated annealing. Struct Multidisc Optim 35, 315–326 (2008). https://doi.org/10.1007/s00158-007-0132-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-007-0132-4