Structural and Multidisciplinary Optimization

, Volume 47, Issue 4, pp 479–491 | Cite as

Adaptive virtual support vector machine for reliability analysis of high-dimensional problems

  • Hyeongjin Song
  • K. K. Choi
  • Ikjin Lee
  • Liang Zhao
  • David Lamb
Research Paper


In this paper, an efficient classification methodology is developed for reliability analysis while maintaining an accuracy level similar to or better than existing response surface methods. The sampling-based reliability analysis requires only the classification information—a success or a failure—but the response surface methods provide function values on the domain as their output, which requires more computational effort. The problem is even more challenging when dealing with high-dimensional problems due to the curse of dimensionality. In the newly proposed virtual support vector machine (VSVM), virtual samples are generated near the limit state function by using an approximation method. The function values are used for approximations of virtual samples to improve accuracy of the resulting VSVM decision function. By introducing the virtual samples, VSVM can overcome the deficiency in existing classification methods where only classification values are used as their input. The universal Kriging method is used to obtain virtual samples to improve the accuracy of the decision function for highly nonlinear problems. A sequential sampling strategy that chooses new samples near the limit state function is integrated with VSVM to improve the accuracy. Examples show the proposed adaptive VSVM yields better efficiency in terms of modeling and response evaluation time and the number of required samples while maintaining similar level or better accuracy, especially for high-dimensional problems.


Surrogate model Support vector machine (SVM) Sequential sampling Virtual samples Virtual support vector machine (VSVM) High-dimensional problem 



The research is jointly supported by the ARO Project W911NF-09–1–0250 and the Automotive Research Center, which is sponsored by the U.S. Army TARDEC. The research is also partially supported by the World Class University Program through the National Research Foundation of Korea (NRF) grant funded by the Ministry of Education, Science and Technology (Grant Number R32–2008–000–10161–0 in 2009). These supports are greatly appreciated.


  1. Barton RR (1994) Metamodeling: a state of the art review. In: WSC ’94: Proceedings of the 26th conference on winter simulation, anonymous society for computer simulation international, San Diego, CA, USA, pp 237–244Google Scholar
  2. Basudhar A, Missoum S (2008) Adaptive explicit decision functions for probabilistic design and optimization using support vector machines. Comput Struct 86(19–20):1904–1917CrossRefGoogle Scholar
  3. Basudhar A, Missoum S (2010) An improved adaptive sampling scheme for the construction of explicit boundaries. Struct Multidisc Optim 42(4):517–529CrossRefGoogle Scholar
  4. Basudhar A, Dribusch C, Lacaze S (2012) Constrained efficient global optimization with support vector machines. Struct Multi-disc Optim 46(2):201–221CrossRefGoogle Scholar
  5. Bect J, Ginsbourger D, Li L, Picheny V, Vazquez E (2012) Sequential design of computer experiments for the estimation of a probability of failure. Stat Comput 22(3):773–793MathSciNetMATHCrossRefGoogle Scholar
  6. Bichon BJ, Eldred MS, Swiler LP, Mahadevan S, McFarland JM (2008) Efficient global reliability analysis for nonlinear implicit performance functions. AIAA J 46(10):2459–2468CrossRefGoogle Scholar
  7. Bichon BJ, McFarland JM, Mahadevan S (2011) Efficient surrogate models for reliability analysis of systems with multiple failure modes. Reliab Eng Syst Safety 96(10):1386–1395CrossRefGoogle Scholar
  8. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167CrossRefGoogle Scholar
  9. Byrd RH, Gilbert JC, Nocedal J (2000) A trust region method based on interior point techniques for nonlinear programming. Math Program 89(1):149–185MathSciNetMATHCrossRefGoogle Scholar
  10. Canu S, Grandvalet Y, Guigue V (2005) SVM and Kernel Methods Matlab Toolbox.
  11. Cherkassky V, Mulier F (1998) Learning from data: concepts, theory. Wiley, New YorkMATHGoogle Scholar
  12. Ching J (2011) Applications of Monte Carlo method in science and engineering. In Tech, Chap 31Google Scholar
  13. Coleman TF, Li Y (1994) On the convergence of reflective newton methods for large-scale nonlinear minimization subject to bounds. Math Program 67(2):189–224MathSciNetMATHCrossRefGoogle Scholar
  14. Coleman TF, Li Y (1996) An interior, trust region approach for nonlinear minimization subject to bounds. SIAM J Optim 6:418–445MathSciNetMATHCrossRefGoogle Scholar
  15. Cressie NAC (1991) Statistics for spatial data. Wiley, New YorkMATHGoogle Scholar
  16. Dixon LCW, Szego¨ GP (1978) Towards global optimization 2. North-Holland, AmsterdamGoogle Scholar
  17. Forrester A, Keane A (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79CrossRefGoogle Scholar
  18. Forrester A, Sobester A, Keane A (2008) Engineering design via surrogate modeling, a practical guide. Wiley, United KingdomCrossRefGoogle Scholar
  19. Haldar A, Mahadevan S (2000) Probability, reliability and statistical methods in engineering design. Wiley, New YorkGoogle Scholar
  20. Hsu CW, Chang CC, Lin CJ (2004) A practical guide to support vector classification. Technical Report.
  21. Hurtado JE, Alvarez DA (2003) Classification approach for reliability analysis with stochastic finite-element modeling. J Struct Eng 129(8):1141–1149CrossRefGoogle Scholar
  22. Jin R, ChenW, Simpson T (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Multidisc Optim 23(1):1–13CrossRefGoogle Scholar
  23. Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. MIT Press, CambridgeMATHGoogle Scholar
  24. Lee TH, Jung JJ (2008) A sampling technique enhancing accuracy and efficiency of metamodel-based RBDO: constraint boundary sampling. Comput Struct 86(13–14):1463–1476CrossRefGoogle Scholar
  25. Lee I, Choi K, Zhao L (2011) Sampling-based RBDO using the stochastic sensitivity analysis and dynamic Kriging method. Struct Multidisc Optim 44(3):299–317MathSciNetCrossRefGoogle Scholar
  26. Lewis RM, Torczon V (1999) Pattern search algorithms for bound constrained minimization. SIAM J Optim 9(4):1082–1099MathSciNetMATHCrossRefGoogle Scholar
  27. Martin JD (2009) Computational improvements to estimating Kriging metamodel parameters. J Mech Des 131(8). doi: 10.1115/1.3151807
  28. Powell MJD (1978a) A fast algorithm for nonlinearly constrained optimization calculations. In: Watson GA (ed) Numerical analysis, Lecture notes in mathematics, Springer Verlag, p 630Google Scholar
  29. Powell MJD (1978b) The convergence of variable metric methods for nonlinearly constrained optimization calculations. In: Mangasarian OL, Meyer RR, Robinson SM (eds) Nonlinear programming 3, Academic PressGoogle Scholar
  30. Ranjan P, Bingham D, Michailidis G (2008) Sequential experiment design for contour estimation from complex computer codes. Technometrics 50(4):527–541MathSciNetCrossRefGoogle Scholar
  31. Rubinstein RY (1981) Simulation and theMonte Carlo method.Wiley, New YorkMATHCrossRefGoogle Scholar
  32. Saka Y, Gunzburger M, Burkardt J (2007) Latinized, improved LHS, and CVT point sets in hypercubes. Int J Numer Anal Model 4(3–4):729–743MathSciNetMATHGoogle Scholar
  33. Schölkopf B (1999) Advances in Kernel methods support vector learning. MIT Press, CambridgeGoogle Scholar
  34. Schölkopf B, Smola AJ (2002) Learning with Kernels: support vector machines, regularization, optimization, and beyond. MIT Press, CambridgeGoogle Scholar
  35. Simpson T, Poplinski J, Koch P (2001) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17(2):129–150MATHCrossRefGoogle Scholar
  36. Tu J, Choi KK, Park YH (1999) A new study on reliability-based design optimization. J Mech Des 121(4):557–564CrossRefGoogle Scholar
  37. Vapnik VN (1998) Statistical learning theory. Wiley, New YorkMATHGoogle Scholar
  38. Vapnik VN (2000) The nature of statistical learning theory. Springer, New YorkMATHGoogle Scholar
  39. Viana FAC, Haftka R, Watson L (2012) Sequential sampling for contour estimation with concurrent function evaluations. Struct Multidisc Optim 45(4):615–618CrossRefGoogle Scholar
  40. Viana FAC (2010) SURROGATES Toolbox User’s Guide.
  41. Waltz RA, Morales JL, Nocedal J, Orban D (2006) An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math Program 107(3):391–408MathSciNetMATHCrossRefGoogle Scholar
  42. Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. JMech Des 129(4):11MathSciNetCrossRefGoogle Scholar
  43. Youn BD, Choi KK, Du L (2005) Enriched performance measure approach for reliability-based design optimization. AIAA J 43(4):874–884CrossRefGoogle Scholar
  44. Zhao L, Choi KK, Lee I (2011) Metamodeling method using dynamic Kriging for design optimization. AIAA J 49(9):2034–2046CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hyeongjin Song
    • 1
  • K. K. Choi
    • 1
  • Ikjin Lee
    • 2
  • Liang Zhao
    • 3
  • David Lamb
    • 4
  1. 1.Department of Mechanical & Industrial Engineering,College of EngineeringThe University of IowaIowa CityUSA
  2. 2.Department of Mechanical EngineeringThe University of ConnecticutStorrsUSA
  3. 3.SchlumbergerHoustonUSA

Personalised recommendations