Journal of Mechanical Science and Technology

, Volume 29, Issue 1, pp 279–289 | Cite as

Robust estimation of support vector regression via residual bootstrap adoption



As current system designs grow increasingly complex and expensive to analyze, the need for design optimization has also grown. In this study, a more stable approximation model is proposed via the application of a bootstrap to support vector regression (SVR). SVR expresses the nonlinearity of the system relatively well. However, using SVR does not always guarantee accurate results because it is sensitive to the input parameters. To overcome this drawback, we apply a bootstrap to SVR, using the residual from SVR as the bootstrap. The performance of the proposed method is evaluated via application to numerical examples and a real problem. We observed that the proposed method not only produced valuable results but also noticeably eliminated the negative effects of input parameters.


Support vector regression Bootstrap Residual Root median square error 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    R. L. Hardy, Multiquadric equations of topography and other irregular surfaces, Journal of Geophysical Research, 76 (8) (1971) 1905–1915.CrossRefGoogle Scholar
  2. [2]
    J. Sacks, W. J. Welch, T. J. Mitchell and H. P. Wynn, Design and analysis of computer experiments, Statistical Science, 4 (4) (1989) 409–435.CrossRefMATHMathSciNetGoogle Scholar
  3. [3]
    M. Smith, Neural networks for statistical modeling, Van Nostrand Reinhold (1993).MATHGoogle Scholar
  4. [4]
    J. Smola and B. Schölkopf, A tutorial on support vector regression, Statistics and Computing, 14 (2004) 199–222.CrossRefMathSciNetGoogle Scholar
  5. [5]
    V. Vapnik, The nature of statistical learning theory, Springer, New York (1995).CrossRefMATHGoogle Scholar
  6. [6]
    Y. Lee, S. Oh and D. H. Choi, Design optimization using support vector regression, Journal of Mechanical Science and Technology, 22 (2008) 213–220.CrossRefGoogle Scholar
  7. [7]
    N. Christianini and S. J. Taylor, An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press (2000).CrossRefGoogle Scholar
  8. [8]
    S. M. Clarke, J. H. Griebsch and T. W. Simpson, Analysis of support vector regression for approximation of complex engineering analysis, Journal of Mechanical Design, 127 (6) (2003) 1077–1087.CrossRefGoogle Scholar
  9. [9]
    D. B. Fairliea, P. Fletchera and C. K. Zachosb, Trigonometry structure constants for new infinite-dimensional algebras, Physics Letters B, 218 (2) (1989) 203–206.CrossRefMathSciNetGoogle Scholar
  10. [10]
    O. L. Mangasarian and S. Fromovitz, The Fritz John Necessary optimality conditions in the presence of equality and inequality constraints, Journal of Mathematical Analysis and Applications, 17 (1967) 37–47.CrossRefMATHMathSciNetGoogle Scholar
  11. [11]
    G. P. McCormick, Nonlinear programming: Theory, algorithms, and applications, John Wiley & Sons (1983).MATHGoogle Scholar
  12. [12]
    P. Fishburna, P. Schwanderb, L. Sheppa and J. V. Robert, The discrete Radon transform and its approximate inversion via linear programming, Discrete Applied Mathematics, 75 (1) (1997) 39–61.CrossRefMathSciNetGoogle Scholar
  13. [13]
    B. Efron and R. J. Tibshirani, An introduction to the bootstrap, Chapman & Hall (1993).CrossRefMATHGoogle Scholar
  14. [14]
    J. Shao and D. Tu, The jackknife and bootstrap, Springer (1995).CrossRefMATHGoogle Scholar
  15. [15]
    D. Anguita, A. Boni and S. Ridella, Evaluating the generalization ability of support vector machines through the bootstrap, Neural Processing Letters, 11 (1) (2000) 51–58.CrossRefGoogle Scholar
  16. [16]
    T. Wenhu, S. Almas and Q. H. Wu, Transformer dissolved gas analysis using least square support vector machine and bootstrap, Proceedings of the 26th Chinese Control Conference (2007) 482–486.Google Scholar
  17. [17]
    D. Kim and S. Cho, Bootstrap based pattern selection for support vector regression, Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, 5012 (2008) 608–615.Google Scholar
  18. [18]
    E. Acar, Various approaches for constructing an ensemble of metamodels using local measures, Structural and Multidisciplinary Optimization, 42 (6) (2010) 879–896.CrossRefGoogle Scholar
  19. [19]
    T. Hastie, R. Tibshirani and J. Friedman, The elements of statistical learning, Springer (2001).CrossRefMATHGoogle Scholar
  20. [20]
    P. Good, Permutation, Parametric, and Bootstrap Tests of Hypotheses, Third Edition, Springer (2004).Google Scholar
  21. [21]
    J. H. Kim, The bootstrap confidence interval in regression model with non-normal error, Thesis (M.A.), Hanyang University, Korea (1999).Google Scholar
  22. [22]
    B. Schölkopf, K. K. Sung, J. C. Burges, F. Girosi, P. Niyogi, T. Poggio and V. Vapnik, Comparing support vector machines with gaussian kernels to radial basis function classifiers, IEEE Transactions on signal processing, 45 (11) (1997) 2758–2765.CrossRefGoogle Scholar
  23. [23]
    B. E. Boser, I. M. Guyon and V. Vapnik, A training algorithm for optimum margin classifiers, In Fifth Annual Workshop on Computational Learning Theory (1992) 144–152.Google Scholar
  24. [24]
    PIAnO (Process Integration, Automation and Optimization) User’s Manual, Version 3.5, PIDOTECH Inc. (2013).Google Scholar
  25. [25]
    M. D. Mckay, R. J. Beckman and W. J. Conover, A comparison of three methods for selecting values of input variables in the analysis of output form a computer code, Technometrics, 42 (1) (2000) 55–61.CrossRefGoogle Scholar
  26. [26]
  27. [27]
    C. H. Park, J. H. Lee, J. H. Jeong and D. H. Choi, Design optimization of a laser printer cleaning blade for minimizing permanent set, Structural and Multidisciplinary Optimization, 49 (2014) 131–145.CrossRefGoogle Scholar

Copyright information

© The Korean Society of Mechanical Engineers and Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Won-Young Choi
    • 1
  • Dong-Hoon Choi
    • 2
  • Kyung-Joon Cha
    • 3
  1. 1.College of General EducationHanbat National UniversityDaejeonKorea
  2. 2.Department of Mechanical EngineeringHanyang UniversitySeoulKorea
  3. 3.Department of Mathematics and Research Institute for Natural SciencesHanyang UniversitySeoulKorea

Personalised recommendations