Quadratic Kernel-Free Least Square Twin Support Vector Machine for Binary Classification Problems

Abstract

In this paper, a new quadratic kernel-free least square twin support vector machine (QLSTSVM) is proposed for binary classification problems. The advantage of QLSTSVM is that there is no need to select the kernel function and related parameters for nonlinear classification problems. After using consensus technique, we adopt alternating direction method of multipliers to solve the reformulated consensus QLSTSVM directly. To reduce CPU time, the Karush-Kuhn-Tucker (KKT) conditions is also used to solve the QLSTSVM. The performance of QLSTSVM is tested on two artificial datasets and several University of California Irvine (UCI) benchmark datasets. Numerical results indicate that the QLSTSVM may outperform several existing methods for solving twin support vector machine with Gaussian kernel in terms of the classification accuracy and operation time.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. 1.

    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  2. 2.

    Deng, N., Tian, Y., Zhang, C.: Support Vector Machines: Optimization Based Theory, Algorithms, and Extensions, pp. 1–28. Chapman and Hall/CRC, London (2012)

  3. 3.

    Mingheng, Z., Yaobao, Z., Ganglong, H., Gang, C.: Accurate multisteps traffic flow prediction based on SVM. Math. Probl. Eng. 2013, 1–8 (2013)

    Article  Google Scholar 

  4. 4.

    Wei, L., Wei, B., Wang, B.: Text classification using support vector machine with mixture of kernel. J. Softw. Eng. Appl. 05(12), 55–58 (2012)

    Article  Google Scholar 

  5. 5.

    Tian, Y., Ju, X.: Nonparallel support vector machine based on one optimization problem for pattern recognition. J. Oper. Res. Soc. China 3(4), 499–519 (2015)

    MathSciNet  Article  Google Scholar 

  6. 6.

    Narayana, K.V., Manoj, V.V.R., Swathi, K.: Enhanced face recognition based on PCA and SVM. Int. J. Comput. Appl. 117(2), 40–42 (2015)

    Google Scholar 

  7. 7.

    Cao, L., Tay, F.E.H.: Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans. Neural Netw. 14(6), 1506–1518 (2003)

    Article  Google Scholar 

  8. 8.

    Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  Google Scholar 

  9. 9.

    Mangasarian, O.L., Wild, E.W.: Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans. Patttern Anal. Mach. Intell. 28(1), 69–74 (2006)

    Article  Google Scholar 

  10. 10.

    Bai, Y., Shen, Y., Shen, K.: Consensus proximal support vector machine for classification problems with sparse solutions. J. Oper. Res. Soc. China 2(1), 57–74 (2014)

    MathSciNet  Article  Google Scholar 

  11. 11.

    Bai, Y., Zhu, Z., Yan, W.: Sparse proximal support vector machine with a specialized interior-point method. J. Oper. Res. Soc. China 3(1), 1–15 (2015)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Jayadeva, Khemchandani, R., Chandra, S: Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29(5), 905–910 (2007)

  13. 13.

    Arun Kumar, M., Gopal, M.: Least squares twin support vector machines for pattern classification. Exp. Syst. Appl. 36(4), 7535–7543 (2009)

    Article  Google Scholar 

  14. 14.

    Yang, Z.X., Shao, Y.H., Zhang, X.S.: Multiple birth support vector machine for multi-class classification. Neural Comput. Appl. 22(1), 153–161 (2013)

    Article  Google Scholar 

  15. 15.

    Chen, S.G., Wu, X.J.: Multiple birth least squares support vector machine for multi-class classification. Int. J. Mach. Learn. Cybern. 8(6), 1731–1742 (2017)

    Article  Google Scholar 

  16. 16.

    Shao, Y., Wang, Z., Chen, W., Deng, N.: Least squares twin parametric-margin support vector machine for classification. Appl. Intell. 39(3), 451–464 (2013)

    Article  Google Scholar 

  17. 17.

    Shao, Y., Deng, N.: A novel margin-based twin support vector machine with unity norm hyperplanes. Neural Comput. Appl. 22, 1627–1635 (2013)

    Article  Google Scholar 

  18. 18.

    Tomar, D., Agarwal, S.: Twin support vector machine: a review from 2007 to 2014. Egypt. Inform. J. 16(1), 55–69 (2015)

    Article  Google Scholar 

  19. 19.

    Tian, Y., Qi, Z., Ju, X., Shi, Y., Liu, X.: Nonparallel support vector machines for pattern classification. IEEE Trans. Syst. Man. Cybern. 44(7), 1067–1079 (2014)

    Google Scholar 

  20. 20.

    Shao, Y., Zhang, C., Wang, X., Deng, N.: Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22(6), 962–968 (2011)

    Article  Google Scholar 

  21. 21.

    Tian, Y., Ping, Y.: Large-scale linear nonparallel support vector machine solver. Neural Netw. 50, 166–174 (2014)

    Article  Google Scholar 

  22. 22.

    Dagher, I.: Quadratic kernel-free non-linear support vector machine. J. Glob. Optim. 41(1), 15–30 (2008)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Luo, J., Fang, S., Deng, Z., Guo, X.: Soft quadratic surface support vector machine for binary classification. Asia Pac. J. Oper. Res. 33(6), 1–22 (2016)

    MathSciNet  Article  Google Scholar 

  24. 24.

    Bai, Y., Han, X., Chen, T., Yu, H.: Quadratic kernel-free least squares support vector machine for target diseases classification. J. Comb. Optim. 30(4), 850–870 (2015)

    MathSciNet  Article  Google Scholar 

  25. 25.

    Yan, X., Bai, Y., Fang, S.C., Luo, J.: A kernel-free quadratic surface support vector machine for semi-supervised learning. J. Oper. Res. Soc. 67(7), 1001–1011 (2016)

    Article  Google Scholar 

  26. 26.

    Zhan, Y., Bai, Y., Zhang, W., Ying, S.: A P-ADMM for sparse quadratic kernel-free least squares semi-supervised support vector machine. Neurocomputing 306, 37–50 (2018)

    Article  Google Scholar 

  27. 27.

    Boyd, S.P., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

We are very grateful to the editor and the anonymous reviewers for their helpful and valuable comments of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yan-Qin Bai.

Additional information

This research was supported by the National Natural Science Foundation of China (No. 11771275).

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gao, Q., Bai, Y. & Zhan, Y. Quadratic Kernel-Free Least Square Twin Support Vector Machine for Binary Classification Problems. J. Oper. Res. Soc. China 7, 539–559 (2019). https://doi.org/10.1007/s40305-018-00239-4

Download citation

Keywords

  • Twin support vector machine
  • Quadratic kernel-free
  • Least square
  • Binary classification

Mathematics Subject Classification

  • 68T99
  • 90C20