Skip to main content
SpringerLink
Log in

Stochastic subgradient descent method for large-scale robust chance-constrained support vector machines

Optimization Letters volume 11, pages 1013–1024 (2017)Cite this article

Abstract

Robust chance-constrained Support Vector Machines (SVM) with second-order moment information can be reformulated into equivalent and tractable Semidefinite Programming (SDP) and Second Order Cone Programming (SOCP) models. However, practical applications involve processing large-scale data sets. For the reformulated SDP and SOCP models, existed solvers by primal-dual interior method do not have enough computational efficiency. This paper studies the stochastic subgradient descent method and algorithms to solve robust chance-constrained SVM on large-scale data sets. Numerical experiments are performed to show the efficiency of the proposed approaches. The result of this paper breaks the computational limitation and expands the application of robust chance-constrained SVM.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Abe, S.: Support vector machines for pattern classification. Springer (2010)

  2. Ben-Hur, A., Weston, J.: A user’s guide to support vector machines. In: Data mining techniques for the life sciences, Springer, pp 223–239 (2010)

  3. Ben-Tal, A., Bhadra, S., Bhattacharyya, C., Nath, J.S.: Chance constrained uncertain classification via robust optimization. Math. Program. 127(1), 145–173 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bhattacharyya, C., Grate, L.R., Jordan, M.I., El Ghaoui, L., Mian, I.S.: Robust sparse hyperplane classifiers: application to uncertain molecular profiling data. J. Comput. Biol. 11(6), 1073–1089 (2004)

    Article  Google Scholar 

  5. Bordes, A., Bottou, L., Gallinari, P.: Sgd-qn: careful quasi-newton stochastic gradient descent. J. Mach. Learn. Res. 10, 1737–1754 (2009)

    MathSciNet  MATH  Google Scholar 

  6. Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Springer, pp. 177–186 (2010)

  7. Bousquet, O., Bottou, L.: The tradeoffs of large scale learning. In: Advances in neural information processing systems, pp. 161–168 (2008)

  8. Burges, C.J.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)

    Article  Google Scholar 

  9. Chang, C.C., Lin, C.J.: Libsvm: a library for support vector machines. ACM Trans. Intel. Syst. Technol. (TIST) 2(3), 27 (2011)

    Google Scholar 

  10. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  11. Dennis Jr, J.E., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations, vol. 16. Siam (1996)

  12. Hsieh, C.J., Chang, K.W., Lin, C.J., Keerthi, S.S., Sundararajan, S.: A dual coordinate descent method for large-scale linear svm. In: Proceedings of the 25th international conference on Machine learning. ACM, pp. 408–415 (2008)

  13. Murata, N.: A Statistical Study of On-Line Learning. Online Learning and Neural Networks. Cambridge University Press, Cambridge (1998)

    MATH  Google Scholar 

  14. Nesterov, Y., Nemirovskii, A., Ye, Y.: Interior-point polynomial algorithms in convex programming, vol. 13. SIAM (1994)

  15. Rajaraman, A., Ullman, J.D.: Mining of massive datasets. Cambridge University Press (2011)

  16. Shalev-Shwartz, S., Singer, Y., Srebro, N., Cotter, A.: Pegasos: primal estimated sub-gradient solver for svm. Math. Program. 127(1), 3–30 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Shivaswamy, P.K., Bhattacharyya, C., Smola, A.J.: Second order cone programming approaches for handling missing and uncertain data. J. Mach. Learn. Res. 7, 1283–1314 (2006)

    MathSciNet  MATH  Google Scholar 

  18. Sturm, J.F.: Using sedumi 1.02, a matlab toolbox for optimization over symmetric cones. Optim. Methods Softw. 11(1–4), 625–653 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  19. Sturm, J.F.: Implementation of interior point methods for mixed semidefinite and second order cone optimization problems. Optim. Methods Softw. 17(6), 1105–1154 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  20. Sturm, J.F., Zhang, S.: Symmetric primal-dual path-following algorithms for semidefinite programming. Appl. Num. Math. 29(3), 301–315 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  21. Tian, Y., Shi, Y., Liu, X.: Recent advances on support vector machines research. Technol. Econ. Develop. Econ. 18(1), 5–33 (2012)

    Article  Google Scholar 

  22. Vapnik, V.N.: Statistical Learning Theory. Wiley (1998)

  23. Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999)

    Article  Google Scholar 

  24. Wang, X., Fan, N., Pardalos, P.M.: Robust chance-constrained support vector machines with second-order moment information. Ann. Oper. Res. (2015). doi:10.1007/s10479-015-2039-6

  25. Wang, X., Pardalos, P.M.: A survey of support vector machines with uncertainties. Ann. Data Sci. 1(3–4), 293–309 (2014)

    Article  Google Scholar 

  26. Zhang, T.: Solving large scale linear prediction problems using stochastic gradient descent algorithms. In: Proceedings of the twenty-first international conference on Machine learning. ACM, pp. 116–123 (2004)

Download references

Acknowledgments

Research was partially supported by a DTRA grant and the Paul and Heidi Brown Preeminent Professorship in Industrial and Systems Engineering, University of Florida.

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, 32611, USA

    Ximing Wang & Panos M. Pardalos

  2. Department of Systems and Industrial Engineering, University of Arizona, Tucson, AZ, 85721, USA

    Neng Fan

Authors
  1. Ximing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Neng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Panos M. Pardalos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ximing Wang.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Fan, N. & Pardalos, P.M. Stochastic subgradient descent method for large-scale robust chance-constrained support vector machines. Optim Lett 11, 1013–1024 (2017). https://doi.org/10.1007/s11590-016-1026-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11590-016-1026-4

Keywords

search

Navigation