Skip to main content
SpringerLink
Log in

Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model

  • Original Article
  • Published:
Engineering with Computers Aims and scope Submit manuscript

Abstract

This paper offers a recurrent neural network to support vector machine (SVM) learning in stochastic support vector regression with probabilistic constraints. The SVM is first converted into an equivalent quadratic programming (QP) formulation in linear and nonlinear cases. An artificial neural network for SVM learning is then proposed. The presented neural network framework guarantees obtaining the optimal solution of the SVM problem. The existence and convergence of the trajectories of the network are studied. The Lyapunov stability for the considered neural network is also shown. The efficiency of the proposed method is shown by three illustrative examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abaszade M, Effati S (2018) Stochastic support vector regression with probabilistic constraints. Appl Intell 48(1):243–256

    Article  MATH  Google Scholar 

  2. Anguita D, Boni A (2002) Improved neural network for SVM learning. IEEE Trans Neural Netw 13(5):1243–1244

    Article  Google Scholar 

  3. Barnett NS, Dragomir SS, Agarwal R (2002) Some inequalities for probability, expectation, and variance of random variables defined over a finite interval. Comput Math Appl 43(10–11):1319–1357

    Article  MathSciNet  MATH  Google Scholar 

  4. Baser F, Apaydin A (2015) Hybrid fuzzy support vector regression analysis. J Intell Fuzzy Syst 28(5):2037–2045

    Article  MathSciNet  MATH  Google Scholar 

  5. Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. John Wiley & Sons

  6. Ben-Tal A, Bhadra S, Bhattacharyya C, Nath JS (2011) Chance constrained uncertain classification via robust optimization. Math Progr 127(1):145–173

    Article  MathSciNet  MATH  Google Scholar 

  7. Bennett KP, Mangasarian OL (1992) Robust linear programming discrimination of two linearly inseparable sets. Optim Methods Softw 1(1):23–34

    Article  Google Scholar 

  8. Bhattacharyya C, Shivaswamy PK, Smola AJ (2005) A second order cone programming formulation for classifying missing data. In: Advances in neural information processing systems. pp 153–160

  9. Boyd S, Vandenberghe L, Press CU (2004) Convex optimization. No. pt 1 in Berichte über verteilte messysteme. Cambridge University Press, Cambridge

    Google Scholar 

  10. Carrizosa E, Gordillo J, Plastria F (2008) Kernel support vector regression with imprecise output. Dept. MOSI, Vrije Univ. Brussel, Brussel, Belgium, Tech. Rep

  11. Chuang CC, Su SF, Jeng JT, Hsiao CC (2002) Robust support vector regression networks for function approximation with outliers. IEEE Trans Neural Netw 13(6):1322–1330

    Article  Google Scholar 

  12. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    Article  MATH  Google Scholar 

  13. Dierckx P (1995) Curve and surface fitting with splines. Oxford University Press

  14. Drucker H, Burges CJC, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. Advances in neural information processing systems 9. MIT Press, Cambridge, pp 155–161

    Google Scholar 

  15. Farag AA, Mohamed RM (2003) Classification of multispectral data using support vector machines approach for density estimation. In: International conference on intelligent engineering system, pp 6–8. Citeseer

  16. Gao X, Liao L (2009) A new projection-based neural network for constrained variational inequalities. IEEE Trans Neural Netw 20(3):373–388

    Article  Google Scholar 

  17. Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):60–73

    Article  MATH  Google Scholar 

  18. Hong DH, Hwang C (2003) Support vector fuzzy regression machines. Fuzzy Sets Syst 138(2):271–281

    Article  MathSciNet  MATH  Google Scholar 

  19. Hong DH, Hwang C (2004) Extended fuzzy regression models using regularization method. Inf Sci 164(1):31–46

    Article  MathSciNet  MATH  Google Scholar 

  20. Hopfield J, Tank D (1986) Computing with neural circuits: a model. Science 233(4764):625–633

    Article  Google Scholar 

  21. Hu X, Sun C, Zhang B (2010) Design of recurrent neural networks for solving constrained least absolute deviation problems. IEEE Trans Neural Netw 21(7):1073–1086

    Article  Google Scholar 

  22. Hu X, Zhang B (2009) An alternative recurrent neural network for solving variational inequalities and related optimization problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(6):1640–1645

    Article  Google Scholar 

  23. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  24. Kennedy MP, Chua LO (1988) Neural networks for nonlinear programming. IEEE Trans Circuits Syst 35(5):554–562

    Article  MathSciNet  Google Scholar 

  25. Lillo WE, Loh MH, Hui S, Zak SH (1993) On solving constrained optimization problems with neural networks: a penalty method approach. IEEE Trans Neural Netw 4(6):931–940

    Article  Google Scholar 

  26. Fu Lin C, de Wang S (2004) Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognit Lett 25(14):1647–1656

    Article  Google Scholar 

  27. Liu O, Zhao Y (2013) A continuous-time recurrent neural network for real-time support vector regression. In: 2013 IEEE symposium on computational intelligence in control and automation (CICA), pp 189–193. https://doi.org/10.1109/CICA.2013.6611683

  28. Lobo MS, Vandenberghe L, Boyd S, Lebret H (1998) Applications of second-order cone programming. Linear Algebra Appl 284(1):193–228

    Article  MathSciNet  MATH  Google Scholar 

  29. Mehrotra S (1992) On the implementation of a primal-dual interior point method. SIAM J Optim 2(4):575–601

    Article  MathSciNet  MATH  Google Scholar 

  30. Miller R, Michel A (1982) Ordinary differential equations. Academic press, New York

    MATH  Google Scholar 

  31. Nazemi A (2013) Solving general convex nonlinear optimization problems by an efficient neurodynamic model. Eng Appl Artif Intell 26(2):685–696

    Article  Google Scholar 

  32. Nazemi A (2014) A neural network model for solving convex quadratic programming problems with some applications. Eng Appl Artif Intell 32:54–62

    Article  Google Scholar 

  33. Nazemi A (2018) A capable neural network framework for solving degenerate quadratic optimization problems with an application in image fusion. Neural Process Lett 47(1):167–192

    Article  Google Scholar 

  34. Nazemi A, Nazemi M (2014) A gradient-based neural network method for solving strictly convex quadratic programming problems. Cognit Comput 6(3):484–495

    Article  MathSciNet  Google Scholar 

  35. Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372

    Article  MATH  Google Scholar 

  36. Perfetti R, Ricci E (2006) Analog neural network for support vector machine learning. IEEE Trans Neural Netw 17(4):1085–1091

    Article  Google Scholar 

  37. Sankowski D (2003) Signal processing systems. Theory and design. N. Kalouptsidis, a Wiley-interscience Publication, New York, 1997. Int J Adapt Control Signal Process 17(3):262–263

    Article  Google Scholar 

  38. Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245

    Article  Google Scholar 

  39. Shivaswamy PK, Bhattacharyya C, Smola AJ (2006) Second order cone programming approaches for handling missing and uncertain data. J Mach Learn Res 7(47):1283–1314

    MathSciNet  MATH  Google Scholar 

  40. Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222

    Article  MathSciNet  Google Scholar 

  41. Tank D, Hopfield J (1986) Simple ’neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans Circuits Syst 33(5):533–541

    Article  Google Scholar 

  42. Trafalis TB, Alwazzi SA (2007) Support vector regression with noisy data: a second order cone programming approach. Int J Gen Syst 36(2):237–250

    Article  MathSciNet  MATH  Google Scholar 

  43. Trafalis TB, Gilbert RC (2006) Robust classification and regression using support vector machines. Eur J Oper Res 173(3):893–909

    Article  MathSciNet  MATH  Google Scholar 

  44. Vapnik V (2013) The nature of statistical learning theory. Information science and statistics. Springer, New York

    Google Scholar 

  45. Vapnik V, Golowich SE, Smola AJ (1997) Support vector method for function approximation, regression estimation and signal processing. In: Advances in neural information processing systems, pp 281–287

  46. Vapnik V, Mukherjee S (2000) Support vector method for multivariate density estimation. In: Advances in neural information processing systems, pp 659–665

  47. Yang Y, Cao J (2008) A feedback neural network for solving convex constraint optimization problems. Appl Math Comput 201(1):340–350

    MathSciNet  MATH  Google Scholar 

  48. Yang Y, He Q, Hu X (2012) A compact neural network for training support vector machines. Neurocomputing 86:193–198

    Article  Google Scholar 

  49. Tan Y, Xia Y, Wang J (2000) Neural network realization of support vector methods for pattern classification. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks. IJCNN 2000. Neural computing: new challenges and perspectives for the New Millennium, vol 6, pp 411–416

  50. Yoshikawa T (1990) Foundations of robotics: analysis and control. East European Monographs, vol 279. MIT Press, Cambridge

    Google Scholar 

  51. Youshen X, Jun W (2004) A one-layer recurrent neural network for support vector machine learning. IEEE Trans Syst Man Cybern Part B (Cybern) 34(2):1261–1269

    Article  Google Scholar 

  52. Xia Y, Wang J (2005) A recurrent neural network for solving nonlinear convex programs subject to linear constraints. IEEE Trans Neural Netw 16(2):379–386

    Article  Google Scholar 

  53. Zhao Y, Liu Q (2012) Generalized recurrent neural network for \(\epsilon\)-insensitive support vector regression. Math Comput Simul 86:2–9

    Article  MathSciNet  MATH  Google Scholar 

  54. Zhao YP, Zhao J, Zhao M (2013) Twin least squares support vector regression. Neurocomputing 118:225–236

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Mathematical Sciences, Shahrood University of Technology, P.O. Box 3619995161-316, Shahrood, Iran

    Amir Feizi, Alireza Nazemi & Mohammad Reza Rabiei

Authors
  1. Amir Feizi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Nazemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Reza Rabiei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alireza Nazemi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Feizi, A., Nazemi, A. & Rabiei, M.R. Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model. Engineering with Computers 38 (Suppl 2), 1005–1020 (2022). https://doi.org/10.1007/s00366-020-01214-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00366-020-01214-5

Keywords

Search

Navigation