Abstract
Due to the good performance in terms of accuracy, sparsity and flexibility, support vector regression (SVR) has become one of the most popular surrogate models and has been widely researched and applied in various fields. However, SVR only depends on a subset of the training data, because the 𝜖-insensitive loss function ignores any training data that is within the threshold 𝜖. Therefore, some extra information may be extracted from these training data to improve the accuracy of SVR. By using the least squares method, a new improved SVR (ISVR) is developed in this paper, which combines the characteristics of SVR and traditional regression methods. ISVR is based on a two-stage procedure. The principle of ISVR is to treat the response of SVR obtained in the first stage as feedback, and then add some highly nonlinear ingredients and extra linear ingredients accordingly in the second stage by utilizing a correction function. Particularly, three types of ISVR are constructed by selecting different correction functions. Additionally, the performance of ISVR is investigated through eight mathematical problems of varying dimensions and one structural mechanics problem. The results show that ISVR has some advantages in accuracy when compared with SVR, even though the number of training points varies.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Acar E (2010) Various approaches for constructing an ensemble of metamodels using local measures. Struct Multidiscip Optim 42(6):879–896. https://doi.org/10.1007/s00158-010-0520-z
Acar E (2014) Simultaneous optimization of shape parameters and weight factors in ensemble of radial basis functions. Struct Multidiscip Optim 49(6):969–978. https://doi.org/10.1007/s00158-013-1028-0
Basak D, Pal S, Patranabis DC (2007) Support vector regression. Neural Information Processing Letters and Reviews 11(10):203–224
Berci M, Toropov VV, Hewson RW, Gaskell PH (2014) Multidisciplinary multifidelity optimisation of a flexible wing aerofoil with reference to a small uav. Struct Multidiscip Optim 50(4):683–699. https://doi.org/10.1007/s00158-014-1066-2
Cevik A, Kurtoğlu AE, Bilgehan M, Gülşan ME, Albegmprli HM (2015) Support vector machines in structural engineering: a review. J Civ Eng Manag 21(3):261–281
Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):1–27. https://doi.org/10.1145/1961189.1961199
Chen R, Liang CY, Hong WC, Gu DX (2015) Forecasting holiday daily tourist flow based on seasonal support vector regression with adaptive genetic algorithm. Appl Soft Comput 26:435–443. https://doi.org/10.1016/j.asoc.2014.10.022
Cheng K, Lu Z, Wei Y, Shi Y, Zhou Y (2017) Mixed kernel function support vector regression for global sensitivity analysis. Mech Syst Sig Process 96:201–214. https://doi.org/10.1016/j.ymssp.2017.04.014
Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087. https://doi.org/10.1115/1.1897403
Crino S, Brown DE (2007) Global optimization with multivariate adaptive regression splines. IEEE Trans Syst Man Cybern Part B Cybern 37(2):333–40. https://doi.org/10.1109/tsmcb.2006.883430
Doherty J, Christensen S (2011) Use of paired simple and complex models to reduce predictive bias and quantify uncertainty. Water Resour Res 47(12):4154–4158. https://doi.org/10.1029/2011wr010763
Douha L, Benoudjit N, Douak F, Melgani F (2012) Support vector regression in spectrophotometry: an experimental study. Crit Rev Anal Chem 42(3):214–219. https://doi.org/10.1080/10408347.2011.651945
Eason J, Cremaschi S (2014) Adaptive sequential sampling for surrogate model generation with artificial neural networks. Comput Chem Eng 68(68):220–232. https://doi.org/10.1016/j.compchemeng.2014.05.021
Fang KT, Li R, Sudjianto A (2005) Design and modeling for computer experiments. CRC Press. https://doi.org/10.1201/9781420034899
Fletcher R (2013) Practical methods of optimization. Wiley
Forrester A, Sobester A, Keane A (2008) Engineering design via surrogate modelling: a practical guide. Wiley, New York. https://doi.org/10.2514/4.479557
Gunn SR et al (1998) Support vector machines for classification and regression. ISIS Technical Report 14:85–86
Huang J, Bo Y, Wang H (2011) Electromechanical equipment state forecasting based on genetic algorithm – support vector regression. Expert Syst Appl 38(7):8399–8402. https://doi.org/10.1016/j.eswa.2011.01.033
Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Multidiscip Optim 23(1):1–13. https://doi.org/10.1007/s00158-001-0160-4
Kavetski D, Kuczera G (2007) Model smoothing strategies to remove microscale discontinuities and spurious secondary optima in objective functions in hydrological calibration. Water Resour Res 43(3):399–407. https://doi.org/10.1029/2006wr005195
Keating EH, Doherty J, Vrugt JA, Kang Q (2010) Optimization and uncertainty assessment of strongly nonlinear groundwater models with high parameter dimensionality. Water Resour Res 46(10):5613–5618. https://doi.org/10.1029/2009wr008584
Kleijnen JPC (2009) Kriging metamodeling in simulation: a review. Eur J Oper Res 192(3):707–716. https://doi.org/10.2139/ssrn.980063
Li H, Liang Y, Xu Q (2009) Support vector machines and its applications in chemistry. Chemometr Intell Lab 95(2):188–198. https://doi.org/10.1016/j.chemolab.2008.10.007
Liem RP, Mader CA, Martins JR (2015) Surrogate models and mixtures of experts in aerodynamic performance prediction for aircraft mission analysis. Aerosp Sci Technol 43:126–151. https://doi.org/10.1016/j.ast.2015.02.019
Messac A, Mullur AA (2008) A computationally efficient metamodeling approach for expensive multiobjective optimization. Optim Eng 9:37–67
Min J, Lee Y (2005) Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters, vol 28. Elsevier, BV. https://doi.org/10.1016/j.eswa.2004.12.008
Myers RH, Montgomery DC, Anderson-Cook CM (2016) Response surface methodology: process and product optimization using designed experiments. Wiley, New York. https://doi.org/10.2307/1270613
Raghavendra NS, Deka PC (2014) Support vector machine applications in the field of hydrology: a review. Appl Soft Comput 19:372–386. https://doi.org/10.1016/j.asoc.2014.02.002
Razavi S, Tolson BA, Burn DH (2012) Review of surrogate modeling in water resources. Water Resour Res 48(7):54–62. https://doi.org/10.1029/2011wr011527
Rewienski M, White J (2006) Model order reduction for nonlinear dynamical systems based on trajectory piecewise-linear approximations. Linear Algebra Appl 415(2-3):426–454
Sanchez E, Pintos S, Queipo NV (2008) Toward an optimal ensemble of kernel-based approximations with engineering applications. Struct Multidiscip Optim 36(3):247–261. https://doi.org/10.1007/s00158-007-0159-6
Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang RJ (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27(5):302–313. https://doi.org/10.1007/s00158-004-0389-9
Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222. https://doi.org/10.1023/b:stco.0000035301.49549.88
Thokala P, Martins JRRA (2007) Variable-complexity optimization applied to airfoil design. Eng Optim 39(3):271–286
Umakant J, Sudhakar K, Mujumdar P, Rao C (2006) Customized regression model for improving low fidelity analysis tool. In: 11th AIAA/ISSMO multidisciplinary analysis and optimization conference. https://doi.org/10.2514/6.2006-7118
Vapnik V (1995) Support vector method for function approximation, regression estimation and signal processing. Springer, New York
Vapnik V, Golowich SE, Smola AJ (1997) Support vector method for function approximation, regression estimation and signal processing. In: Advances in neural information processing systems , pp 281–287
Viana FAC, Haftka RT, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39(4):439–457. https://doi.org/10.1007/s00158-008-0338-0
Viana FAC, Simpson TW, Balabanov V, Toropov V (2014) Special section on multidisciplinary design optimization: Metamodeling in multidisciplinary design optimization: how far have we really come? AIAA J 52 (4):670–690. https://doi.org/10.2514/1.j052375
Xiang H, Li Y, Liao H, Li C (2017) An adaptive surrogate model based on support vector regression and its application to the optimization of railway wind barriers. Struct Multidiscip Optim 55(2):701–713. https://doi.org/10.1007/s00158-016-1528-9. identifier: 1528
Zhou X, Jiang T (2016) Metamodel selection based on stepwise regression. Struct Multidiscip Optim. https://doi.org/10.1007/s00158-016-1442-1
Zhou XJ, Ma YZ, Li XF (2011) Ensemble of surrogates with recursive arithmetic average. Struct Multidiscip Optim 44(5):651–671. https://doi.org/10.1007/s00158-011-0655-6
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yan, C., Shen, X. & Guo, F. An improved support vector regression using least squares method. Struct Multidisc Optim 57, 2431–2445 (2018). https://doi.org/10.1007/s00158-017-1871-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-017-1871-5