Sparse Support Vector Machines in Reproducing Kernel Banach Spaces

  • Zheng Li
  • Yuesheng XuEmail author
  • Qi Ye


We present a novel approach for support vector machines in reproducing kernel Banach spaces induced by a finite basis. In particular, we show that the support vector classification in the 1-norm reproducing kernel Banach space is mathematically equivalent to the sparse support vector machine. Finally, we develop fixed-point proximity algorithms for finding the solution of the non-smooth minimization problem that describes the sparse support vector machine. Numerical results are presented to demonstrate that the sparse support vector machine outperforms the classical support vector machine for the binary classification of simulation data.



The first author is supported in part by the Special Project on High-performance Computing under the National Key R&D Program (No. 2016YFB0200602), and by the Natural Science Foundation of China under grants 11471013 and 91530117. The third author would like to express his gratitude to the grant of the “Thousand Talents Program” for junior scholars of China, the grant of the Natural Science Foundation of China (11601162), and the grant of South China Normal University (671082, S80835, and S81031).


  1. 1.
    Alpaydin, E.: Introduction to Machine Learning. MIT, Cambridge, MA (2010)zbMATHGoogle Scholar
  2. 2.
    Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bruckstein, A., Donoho, D., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Cucker, F., Smale, S.: On the mathematical foundations of learning. Bull. Am. Math. Soc. (N.S.) 39(1), 1–49 (2002) (electronic)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fasshauer, G., Hickernell, F., Ye, Q.: Solving support vector machines in reproducing kernel Banach spaces with positive definite functions. Appl. Comput. Harmon. Anal. 38, 115–139 (2015)MathSciNetCrossRefGoogle Scholar
  7. 7.
    García, A., Portal, A.: Sampling in reproducing kernel Banach spaces. Mediterr. J. Math. 10, 1401–1417 (2013)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Li, Q., Micchelli, C.A., Shen, L., Xu, Y.: A proximity algorithm accelerated by Gauss-Seidel iterations for L1/TV denoising models. Inverse Prob. 28(9), 095003 (2012)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Li, Q., Shen, L., Xu, Y., Zhang, N.: Multi-step proximity algorithms for solving a class of convex optimization problems. Adv. Comput. Math. 41(2), 387–422 (2014)CrossRefGoogle Scholar
  10. 10.
    Li, Z., Song, G., Xu, Y.: Fixed-point proximity algorithms for solving sparse machine learning models. Int. J. Numer. Anal. Model. 15(1–2), 154–169 (2018)MathSciNetGoogle Scholar
  11. 11.
    Li, Q., Xu, Y., Zhang, N.: Two-step fixed-point proximity algorithms for multi-block separable convex problems. J. Sci. Comput. 70, 1204–1228 (2017)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Micchelli, C.A., Shen, L., Xu, Y.: Proximity algorithms for image models: denoising. Inverse Prob. 27(4), 045009, 30 (2011)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Micchelli, C.A., Shen, L., Xu, Y., Zeng, X.: Proximity algorithms for the L1/TV image denoising model. Adv. Comput. Math. 38(2), 401–426 (2013)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Schaback, R., Wendland, H.: Kernel techniques: from machine learning to meshless methods. Acta Numer. 15, 543–639 (2006)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Sriperumbudur, B., Fukumizu, K., Lanckriet, G.: Learning in Hilbert vs.Banach spaces: a measure embedding viewpoint. In: Advances in Neural Information Processing Systems, pp. 1773–1781. MIT, Cambridge (2011)Google Scholar
  16. 16.
    Steinwart, I.: Sparseness of support vector machines. J. Mach. Learn. Res. 4, 1071–1105 (2003)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Steinwart, I., Christmann, A.: Support Vector Machines. Springer, New York (2008)zbMATHGoogle Scholar
  18. 18.
    Villmann, T., Haase, S., Kästner, M.: Gradient based learning in vector quantization using differentiable kernels. In: Advances in Self-Organizing Maps, pp. 193–204. Springer, Santiago (2013)Google Scholar
  19. 19.
    Xu, Y., Ye, Q.: Generalized Mercer kernels and reproducing kernel Banach spaces. Mem. AMS (accepted). arXiv:1412.8663Google Scholar
  20. 20.
    Zhang, H., Xu, Y., Zhang, J.: Reproducing kernel Banach spaces for machine learning. J. Mach. Learn. Res. 10, 2741–2775 (2009)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Zhu, J., Rosset, S., Hastie, T., Tibshirani, R.: 1-norm support vector machines. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) The Annual Conference on Neural Information Processing Systems 16, pp. 1–8 (2004)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Guangdong Province Key Lab of Computational Science, School of MathematicsSun Yat-sen UniversityGuangzhouPeople’s Republic of China
  2. 2.School of Data and Computer Science, Guangdong Province Key Laboratory of Computational ScienceSun Yat-Sen UniversityGuangzhouPeople’s Republic of China
  3. 3.Department of Mathematics and StatisticsOld Dominion UniversityNorfolkUSA
  4. 4.School of Mathematical SciencesSouth China Normal UniversityGuangzhouPeople’s Republic of China

Personalised recommendations