Skip to main content
SpringerLink
Log in

A two-stage support vector regression assisted sequential sampling approach for global metamodeling

  • RESEARCH PAPER
  • Published:
Structural and Multidisciplinary Optimization Aims and scope Submit manuscript

Abstract

Support vector regression (SVR), as a promising surrogate model, has been widely used to approximate expensive simulations in engineering design problem. To build a SVR model accurately and efficiently, a two-stage support vector regression (TSSVR) assisted sequential sampling approach is proposed in this paper with the consideration of SVR’s two unique features. In each sampling iteration of TSSVR, two SVR models are constructed successively based on the same training data. As for the first feature that only support vectors (SVs) have impact on the construction of SVR, the first-stage SVR with lower ε precision is built to prescreen some important samples as current SVs. As for the second feature that SVR model does not completely go through the samples, the second-stage SVR with higher ε precision is built to calculate the prediction errors at the SVs without any other computational cost, and the prediction errors are used to approximately measure the accuracy of the local regions around the SVs. Moreover, to describe the local regions around the SVs, the design space is partitioned into a set of Voronoi cells according to the current samples before prescreening SVs from the sample points. Then a new sample can be exploited in the corresponding Voronoi cell with the largest prediction error. In the next sampling iteration, the Voronoi cells and SVs are redefined. As the change of the local cell with the largest prediction error, global exploration is achieved. Finally, the proposed approach is validated by seven numerical examples and an engineering example. An overall comparison between the proposed approach and some other methods demonstrates that the proposed approach is efficient and suitable for engineering design problems involving computational-expensive simulations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Aute V, Saleh K, Abdelaziz O, Azarm S, Radermacher R (2013) Cross-validation based single response adaptive design of experiments for kriging metamodeling of deterministic computer simulations. Struct Multidiscip Optim 48(3):581–605

    Article  Google Scholar 

  • Aurenhammer F (1991) Voronoi diagrams—a survey of a fundamental geometric data structure. ACM Comput Surv 23:345–405

    Article  Google Scholar 

  • Box GE, Draper NR (1987) Empirical model-building and response surfaces, vol 424. Wiley, New York

    MATH  Google Scholar 

  • Braconnier T, Ferrier M, Jouhaud J-C, Montagnac M, Sagaut P (2011) Towards an adaptive POD/SVD surrogate model for aeronautic design. Comput Fluids 40(1):195–209

    Article  MathSciNet  MATH  Google Scholar 

  • Busby D (2009) Hierarchical adaptive experimental design for Gaussian process emulators. Reliab Eng Syst Saf 94(7):1183–1193

    Article  Google Scholar 

  • Cai XW, Qiu HB, Gao L, Yang P, Shao XY (2016) An enhanced RBF-HDMR integrated with an adaptive sampling method for approximating high dimensional problems in engineering design. Struct Multidiscip Optim 53(6):1209–1229

    Article  Google Scholar 

  • Cai XW, Qiu HB, Gao L, Shao X (2017a) Metamodeling for high dimensional design problems by multi-fidelity simulations. Struct Multidiscip Optim 56(1):151–166

    Article  MathSciNet  Google Scholar 

  • Cai XW, Qiu HB, Gao L, Li W, Shao XY (2017b) Adaptive radial-basis-function-based multifidelity metamodeling for expensive black-box problems. AIAA J 55(7):2424–2436

    Article  Google Scholar 

  • Chang CC, Lin CJ (2011) LIBSVM: A library for support vector machines. ACM T Intel Syst Tec 2(3):1–27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  • Chen LM, Qiu HB, Jiang C, Cai XW, Gao L (2017a) Ensemble of surrogates with hybrid method using global and local measures for engineering design. Struct Multidiscip Optim 57(4):1711–1729. https://doi.org/10.1007/s00158-017-1841-y

  • Chen LM, Qiu HB, Jiang C, Xiao M, Gao L (2017b) Support vector enhanced kriging for metamodeling with noisy data. Struct Multidiscip Optim. https://doi.org/10.1007/s00158-017-1831-0

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087

    Article  Google Scholar 

  • Crombecq K, Gorissen D, Deschrijver D, Dhaene T (2011a) A novel hybrid sequential design strategy for global surrogate modeling of computer experiments. SIAM J Sci Comput 33(4):1948–1974

    Article  MathSciNet  MATH  Google Scholar 

  • Crombecq K, Laermans E, Dhaene T (2011b) Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling. Eur J Oper Res 214(3):683–696

    Article  Google Scholar 

  • Currin C, Mitchell T, Morris M, Ylvisaker D (1991) Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. J Am Stat Assoc 86(416):953–963

    Article  MathSciNet  Google Scholar 

  • Dyn N, Levin D, Rippa S (1986) Numerical procedures for surface fitting of scattered data by radial functions. SIAM J Sci Stat Comput 7(2):639–659

    Article  MathSciNet  MATH  Google Scholar 

  • Forrester AI, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1):50–79

    Article  Google Scholar 

  • Gramacy RB, Lee HK (2009) Adaptive design and analysis of supercomputer experiments. Technometrics 51(2):130–145

    Article  MathSciNet  Google Scholar 

  • Grosso A, Jamali A, Locatelli M (2009) Finding maximin latin hypercube designs by iterated local search heuristics. Eur J Oper Res 197(2):541–547

    Article  MATH  Google Scholar 

  • Haftka RT, Vilanueva D, Chaudhuri A (2016) Parallel surrogate-assisted global optimization with expensive functions-a survey. Struct Multidiscip Optim 54(1):3–13

    Article  MathSciNet  Google Scholar 

  • Hao P, Wang B, Li G (2012) Surrogate-based optimum Design for Stiffened Shells with adaptive sampling. AIAA J 50(11):2389–2407

    Article  Google Scholar 

  • Hao P, Wang B, Tian K, Li G, Sun Y, Zhou C (2017) Fast procedure for non-uniform optimum design of stiffened shells under buckling constraint. Struct Multidiscip Optim 55(4):1503–1516

    Article  MathSciNet  Google Scholar 

  • Huang ZY, Qiu HB, Zhao M, Cai XW, Gao L (2015) An adaptive SVR-HDMR model for approximating high dimensional problems. Eng Comput 32(3):643–667

    Article  Google Scholar 

  • Jin R, Chen W, Sudjianto A (2002) On sequential sampling for global metamodeling in engineering design. In: Proceedings of ASME Design Automation Conference, Montreal, September 29–October 2, 2002, ASME, 539–548

  • Jayaprakash G, Sivakumar K, Thilak M (2012) A numerical study on effect of temperature and inertia on tolerance design of mechanical assembly. Eng Comput 29(7):722–742

    Article  Google Scholar 

  • Joseph VR, Hung Y (2008) Orthogonal-maximin latin hypercube designs. Stat Sin 18(1):171–186

    MathSciNet  MATH  Google Scholar 

  • Kleijnen JPC (2017) Regression and kriging metamodels with their experimental designs in simulation: a review. Eur J Oper Res 256(1):1–16

    Article  MathSciNet  MATH  Google Scholar 

  • Kleijnen JPC, Van Beers W, Van Nieuwenhuyse I (2012) Expected improvement in efficient global optimization through bootstrapped kriging. J Glob Optim 54(1):59–73

    Article  MathSciNet  MATH  Google Scholar 

  • Li G, Aute V, Azarm S (2010) An accumulative error based adaptive design of experiments for offline metamodeling. Struct Multidiscip Optim 40:137–155

    Article  Google Scholar 

  • Liefvendahl M, Stocki R (2006) A study on algorithms for optimization of latin hypercubes. J Stat Plan Inference 136(9):3231–3247

    Article  MathSciNet  MATH  Google Scholar 

  • Liu H, Xu S, Wang X (2015) Sequential sampling designs based on space reduction. Eng Optim 47(7):867–884

    Article  Google Scholar 

  • Liu H, Ong YS, Cai JF (2018) A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design. Struct Multidiscip Optim 57(1):393–416

    Article  Google Scholar 

  • Mackman TJ, Allen CB (2010) Investigation of an adaptive sampling method for data interpolation using radial basis functions. Int J Numer Methods Eng 83(7):915–938

    MATH  Google Scholar 

  • Mackman TJ, Allen CB, Ghoreyshi M, Badcock KJ (2013) Comparison of adaptive sampling methods for generation of surrogate aerodynamic models. AIAA J 51(4):797–808

    Article  Google Scholar 

  • Mckay MD, Beckman RJ, Conover WJ (1979) Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245

    MathSciNet  MATH  Google Scholar 

  • Owen AB (1992) Orthogonal arrays for computer experiments, integration and visualization. Stat Sin 2(2):439–452

    MathSciNet  MATH  Google Scholar 

  • Pan G, Ye P, Wang P, Yang Z (2014) A sequential optimization sampling method for metamodels with radial basis functions. Sci World J 2014:192862

    Google Scholar 

  • Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci:409–423

  • Sasena MJ, Papalambros P, Goovaerts P (2002) Exploration of metamodeling sampling criteria for constrained global optimization. Eng Optim 34(3):263–278

    Article  Google Scholar 

  • Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222

    Article  MathSciNet  Google Scholar 

  • Van Dam ER, Husslage B, Den Hertog D, Melissen H (2007) Maximin latin hypercube designs in two dimensions. Oper Res 55(1):158–169

    Article  MathSciNet  MATH  Google Scholar 

  • Vapnik VN (1999) An overview of statistical learning theory. IEEE T Neural Net 10(5):988–999

    Article  Google Scholar 

  • Viana FA, Haftka RT, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39(4):439–457

    Article  Google Scholar 

  • Viana FA, Venter G, Balabanov V (2010) An algorithm for fast optimal latin hypercube design of experiments. Int J Numer Methods Eng 82(2):135–156

    MathSciNet  MATH  Google Scholar 

  • Wang GG (2003) Adaptive response surface method using inherited latin hypercube design points. ASME J Mech Des 125(2):210–220

    Article  Google Scholar 

  • Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. ASME J Mech Des 129(4):370–380

    Article  Google Scholar 

  • Wang H, Li E, Li GY (2010) Parallel boundary and best neighbor searching sampling algorithm for drawbead design optimization in sheet metal forming. Struct Multidiscip Optim 41(2):309–324

    Article  Google Scholar 

  • Xiong Y, Chen W, Apley D, Ding X (2007) A non-stationary covariance-based kriging method for metamodeling in engineering design. Int J Numer Methods Eng 71(6):733–756

    Article  MATH  Google Scholar 

  • Xiong F, Xiong Y, Chen W, Yang S (2009) Optimizing latin hypercube design for sequential sampling of computer experiments. Eng Optim 41(8):793–810

    Article  Google Scholar 

  • Xu S, Liu H, Wang X, Jiang X (2014) A robust error-pursuing sequential sampling approach for global metamodeling based on Voronoi diagram and cross validation. J Mech Des 136(7):69–74

    Article  Google Scholar 

  • Yao W, Chen X, Luo W (2009) A gradient-based sequential radial basis function neural network modeling method. Neural Comput & Applic 18(5):477–484

    Article  Google Scholar 

  • Zhou Q, Shao X, Jiang P, Gao Z, Zhou H, Shu L (2016) An active learning variable-fidelity metamodeling approach based on ensemble of metamodels and objective-oriented sequential sampling. J Eng Des 27(4–6):205–231

    Article  Google Scholar 

  • Zhou Q, Jiang P, Shao X, Hu J, Cao L, Wan L (2017) A variable fidelity information fusion method based on radial basis function. Adv Eng Inform 32:26–39

    Article  Google Scholar 

Download references

Acknowledgements

This study is supported by the National Natural Science Foundation of China under Grant No. 51675198, the 973 National Basic Research Program of China under Grant No. 2014CB046705, the National Natural Science Foundation of China under Grant No. 51721092, and the Program for HUST Academic Frontier Youth Team.

Author information

Authors and Affiliations

  1. The State Key Laboratory of Digital Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Chen Jiang, Xiwen Cai, Haobo Qiu, Liang Gao & Peigen Li

Authors
  1. Chen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiwen Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haobo Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peigen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haobo Qiu.

Additional information

Responsible Editor: Pingfeng Wang

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, C., Cai, X., Qiu, H. et al. A two-stage support vector regression assisted sequential sampling approach for global metamodeling. Struct Multidisc Optim 58, 1657–1672 (2018). https://doi.org/10.1007/s00158-018-1992-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00158-018-1992-5

Keywords

Search

Navigation