Advertisement

Two smooth support vector machines for \(\varepsilon \)-insensitive regression

  • Weizhe Gu
  • Wei-Po Chen
  • Chun-Hsu Ko
  • Yuh-Jye Lee
  • Jein-Shan Chen
Article

Abstract

In this paper, we propose two new smooth support vector machines for \(\varepsilon \)-insensitive regression. According to these two smooth support vector machines, we construct two systems of smooth equations based on two novel families of smoothing functions, from which we seek the solution to \(\varepsilon \)-support vector regression (\(\varepsilon \)-SVR). More specifically, using the proposed smoothing functions, we employ the smoothing Newton method to solve the systems of smooth equations. The algorithm is shown to be globally and quadratically convergent without any additional conditions. Numerical comparisons among different values of parameter are also reported.

Keywords

Support vector machine \(\varepsilon \)-insensitive loss \(\varepsilon \)-smooth support vector regression Smoothing Newton algorithm 

References

  1. 1.
    Basak, D., Pal, S., Patranabts, D.C.: Support vector regression. Neural Inf. Process. Lett. Rev. 11, 203–224 (2007)Google Scholar
  2. 2.
    Clarke, F.H.: Opimization and Nonsmooth Analysis. Wiley, New York (1983)Google Scholar
  3. 3.
    Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Friedman, J.H.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–67 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Huang, C.-M., Lee, Y.-J.: Reduced support vector machines: a statistical theory. IEEE Trans. Neural Netw. 18, 1–13 (2007)CrossRefGoogle Scholar
  6. 6.
    Huang, Z.-H., Zhang, Y., Wu, W.: A smoothing-type algorithm for solving system of inequalities. J. Comput. Appl. Math. 220, 355–363 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Lee, Y.-J., Hsieh, W.-F., Huang, C.-M.: \(\varepsilon \)-SSVR: a smooth support vector machine for \(\varepsilon \)-insensitive regression. IEEE Trans. Knowl. Eng. 17, 678–685 (2005)CrossRefGoogle Scholar
  8. 8.
    Lee, Y.-J., Mangasarian, O.L.: SSVM: a smooth support vector machine for classification. Comput. Optim. Appl. 20, 5–22 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Lee, Y.-J., Mangasarian, O.L.: RSVM: Reduced support vector machines. In: Proceedings of the 2001 SIAM International Conference on Data Mining,  https://doi.org/10.1137/1.9781611972719.13, (2001)
  10. 10.
    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957–972 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Musicant, D.R., Feinberg, A.: Active set support vector regression. IEEE Trans. Neural Netw. 15, 268–275 (2004)CrossRefGoogle Scholar
  12. 12.
    Palais, R.S., Terng, C.-L.: Critical Point Theory and Submanifold Geometry, Lecture Notes in Mathematics, vol. 1353, Springer, Berlin (1988)Google Scholar
  13. 13.
    Platt, J.: Sequential minimal optimization: a fast algorithm for training support vector machines. In: Advances in Kernel Methods, Support Vector Learning, vol. 208, pp. 1–21, MIT Press, Boston. (1998)Google Scholar
  14. 14.
    Qi, L.-Q., Sun, J.: A nonsmooth version of Newton’s method. Math. Program. 58, 353–367 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Tseng, P., Yun, S.: A coordinate gradient descent method for linearly constrained smooth optimization and support vector machines training. Comput. Optim. Appl. 47, 179–206 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Vapnik, V.: Estimation of Dependences Based on Empirical Data. Springer, New York (1982)zbMATHGoogle Scholar
  17. 17.
    Vapnik, V.: The Natrure of Statistical Theory. Springer, New York (1995)Google Scholar
  18. 18.
    Vapnik, V., Golowith, S., Smola, A.: Support vector method for function approximation, regression estimation, and signal processing. Neural Inf. Process. Syst. 9, 281–287 (1997)Google Scholar
  19. 19.
    Yuan, Y.-B., Huang, T.-Z.: A polynomial smooth support vector machine for classification. Adv. Data Mining Appl. 3584, 157–164 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2017

Authors and Affiliations

  • Weizhe Gu
    • 1
  • Wei-Po Chen
    • 2
  • Chun-Hsu Ko
    • 3
  • Yuh-Jye Lee
    • 4
  • Jein-Shan Chen
    • 2
  1. 1.Department of Mathematics, School of ScienceTianjin UniversityTianjinPeople’s Republic of China
  2. 2.Department of MathematicsNational Taiwan Normal UniversityTaipeiTaiwan
  3. 3.Department of Electrical EngineeringI-Shou UniversityKaohsiungTaiwan
  4. 4.Department of Applied MathematicsNational Chiao Tung UniversityHsinchuTaiwan

Personalised recommendations