Skip to main content
SpringerLink
Log in

Robust estimation of support vector regression via residual bootstrap adoption

  • Published:
Journal of Mechanical Science and Technology Aims and scope Submit manuscript

Abstract

As current system designs grow increasingly complex and expensive to analyze, the need for design optimization has also grown. In this study, a more stable approximation model is proposed via the application of a bootstrap to support vector regression (SVR). SVR expresses the nonlinearity of the system relatively well. However, using SVR does not always guarantee accurate results because it is sensitive to the input parameters. To overcome this drawback, we apply a bootstrap to SVR, using the residual from SVR as the bootstrap. The performance of the proposed method is evaluated via application to numerical examples and a real problem. We observed that the proposed method not only produced valuable results but also noticeably eliminated the negative effects of input parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. R. L. Hardy, Multiquadric equations of topography and other irregular surfaces, Journal of Geophysical Research, 76 (8) (1971) 1905–1915.

    Article  Google Scholar 

  2. J. Sacks, W. J. Welch, T. J. Mitchell and H. P. Wynn, Design and analysis of computer experiments, Statistical Science, 4 (4) (1989) 409–435.

    Article  MATH  MathSciNet  Google Scholar 

  3. M. Smith, Neural networks for statistical modeling, Van Nostrand Reinhold (1993).

    MATH  Google Scholar 

  4. J. Smola and B. Schölkopf, A tutorial on support vector regression, Statistics and Computing, 14 (2004) 199–222.

    Article  MathSciNet  Google Scholar 

  5. V. Vapnik, The nature of statistical learning theory, Springer, New York (1995).

    Book  MATH  Google Scholar 

  6. Y. Lee, S. Oh and D. H. Choi, Design optimization using support vector regression, Journal of Mechanical Science and Technology, 22 (2008) 213–220.

    Article  Google Scholar 

  7. N. Christianini and S. J. Taylor, An introduction to support vector machines and other kernel-based learning methods, Cambridge University Press (2000).

    Book  Google Scholar 

  8. S. M. Clarke, J. H. Griebsch and T. W. Simpson, Analysis of support vector regression for approximation of complex engineering analysis, Journal of Mechanical Design, 127 (6) (2003) 1077–1087.

    Article  Google Scholar 

  9. D. B. Fairliea, P. Fletchera and C. K. Zachosb, Trigonometry structure constants for new infinite-dimensional algebras, Physics Letters B, 218 (2) (1989) 203–206.

    Article  MathSciNet  Google Scholar 

  10. O. L. Mangasarian and S. Fromovitz, The Fritz John Necessary optimality conditions in the presence of equality and inequality constraints, Journal of Mathematical Analysis and Applications, 17 (1967) 37–47.

    Article  MATH  MathSciNet  Google Scholar 

  11. G. P. McCormick, Nonlinear programming: Theory, algorithms, and applications, John Wiley & Sons (1983).

    MATH  Google Scholar 

  12. P. Fishburna, P. Schwanderb, L. Sheppa and J. V. Robert, The discrete Radon transform and its approximate inversion via linear programming, Discrete Applied Mathematics, 75 (1) (1997) 39–61.

    Article  MathSciNet  Google Scholar 

  13. B. Efron and R. J. Tibshirani, An introduction to the bootstrap, Chapman & Hall (1993).

    Book  MATH  Google Scholar 

  14. J. Shao and D. Tu, The jackknife and bootstrap, Springer (1995).

    Book  MATH  Google Scholar 

  15. D. Anguita, A. Boni and S. Ridella, Evaluating the generalization ability of support vector machines through the bootstrap, Neural Processing Letters, 11 (1) (2000) 51–58.

    Article  Google Scholar 

  16. T. Wenhu, S. Almas and Q. H. Wu, Transformer dissolved gas analysis using least square support vector machine and bootstrap, Proceedings of the 26th Chinese Control Conference (2007) 482–486.

    Google Scholar 

  17. D. Kim and S. Cho, Bootstrap based pattern selection for support vector regression, Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, 5012 (2008) 608–615.

    Google Scholar 

  18. E. Acar, Various approaches for constructing an ensemble of metamodels using local measures, Structural and Multidisciplinary Optimization, 42 (6) (2010) 879–896.

    Article  Google Scholar 

  19. T. Hastie, R. Tibshirani and J. Friedman, The elements of statistical learning, Springer (2001).

    Book  MATH  Google Scholar 

  20. P. Good, Permutation, Parametric, and Bootstrap Tests of Hypotheses, Third Edition, Springer (2004).

    Google Scholar 

  21. J. H. Kim, The bootstrap confidence interval in regression model with non-normal error, Thesis (M.A.), Hanyang University, Korea (1999).

    Google Scholar 

  22. B. Schölkopf, K. K. Sung, J. C. Burges, F. Girosi, P. Niyogi, T. Poggio and V. Vapnik, Comparing support vector machines with gaussian kernels to radial basis function classifiers, IEEE Transactions on signal processing, 45 (11) (1997) 2758–2765.

    Article  Google Scholar 

  23. B. E. Boser, I. M. Guyon and V. Vapnik, A training algorithm for optimum margin classifiers, In Fifth Annual Workshop on Computational Learning Theory (1992) 144–152.

    Google Scholar 

  24. PIAnO (Process Integration, Automation and Optimization) User’s Manual, Version 3.5, PIDOTECH Inc. (2013).

  25. M. D. Mckay, R. J. Beckman and W. J. Conover, A comparison of three methods for selecting values of input variables in the analysis of output form a computer code, Technometrics, 42 (1) (2000) 55–61.

    Article  Google Scholar 

  26. R 2.13.1, http://cran.r-project.org .

  27. C. H. Park, J. H. Lee, J. H. Jeong and D. H. Choi, Design optimization of a laser printer cleaning blade for minimizing permanent set, Structural and Multidisciplinary Optimization, 49 (2014) 131–145.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of General Education, Hanbat National University, Daejeon, 305-719, Korea

    Won-Young Choi

  2. Department of Mechanical Engineering, Hanyang University, Seoul, 133-791, Korea

    Dong-Hoon Choi

  3. Department of Mathematics and Research Institute for Natural Sciences, Hanyang University, Seoul, 133-791, Korea

    Kyung-Joon Cha

Authors
  1. Won-Young Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong-Hoon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyung-Joon Cha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung-Joon Cha.

Additional information

Recommended by Associate Editor Gang-Won Jang

Wonyoung Choi received her B.S., M.S., and Ph.D. degrees in Mathematics from Hanyang University, Korea in 2005, 2007, and 2011, respectively. She is currently a lecture professor of the College of General Education at Hanbat National University, Daejeon, Korea. Her research interests include computer statistics, metamodeling, and mathematics educations.

Dong-Hoon Choi is the Director of the Center of Innovative Design Optimization Technology, Hanyang University, Seoul, Korea. He received his B.S. degree from Seoul National University in 1975, M.S. degree from KAIST in 1977, and Ph.D. from the University of Wisconsin-Madison in 1985. He has been a professor of Mechanical Engineering at Hanyang University since 1986. His research interest include theory of engineering optimization and its application to engineering systems, multidisciplinary design optimization, sequential approximate optimization, and design under uncertainty.

Kyung-Joon Cha received his B.S. degree in Mathematics from Hanyang University, Korea in 1981, M.S. degree in Statistics from the University of Wisconsin-Madison in 1985, and Ph.D. in Statistics from Southern Methodist University in 1990. He is currently a professor in the Department of Mathematics at Hanyang University. His research interests include nonparametric statistics, computational statistics, and regression.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, WY., Choi, DH. & Cha, KJ. Robust estimation of support vector regression via residual bootstrap adoption. J Mech Sci Technol 29, 279–289 (2015). https://doi.org/10.1007/s12206-014-1234-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12206-014-1234-8

Keywords

Search

Navigation