Skip to main content

Advertisement

SpringerLink
Log in

A lagrangian-based approach for universum twin bounded support vector machine with its applications

  • S707: LION15
  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

The Universum provides prior knowledge about data in the mathematical problem to improve the generalization performance of the classifiers. Several works have shown that the Universum twin support vector machine (\( \mathfrak {U} \)-TSVM) is an efficient method for binary classification problems. In this paper, we improve the \( \mathfrak {U} \)-TSVM method and propose an improved Universum twin bounded support vector machine (named as IUTBSVM). Indeed, by introducing different Lagrangian functions for the primal problems, we obtain new dual formulations of \( \mathfrak {U} \)-TSVM so that we do not need to compute inverse matrices. To reduce the computational time of the proposed method, we suggest a smaller size of the rectangular kernel matrices than the other methods. Numerical experiments on gender classification of human faces, handwritten digits recognition, and several UCI benchmark data sets indicate that the IUTBSVM is more efficient than the other four algorithms, namely \(\mathfrak {U}\)-SVM, TSVM, \(\mathfrak {U}\)-TSVM, and IUTSVM in the sense of the classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Arabasadi, Z., Alizadehsani, R., Roshanzamir, M., Moosaei, H., Yarifard, A.A.: Computer aided decision making for heart disease detection using hybrid neural network-Genetic algorithm. Comput. Methods Programs Biomed. 141, 19–26 (2017)

    Article  Google Scholar 

  2. Bai, X., Cherkassky, V.: Gender classification of human faces using inference through contradictions. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 746?750. IEEE (2008)

  3. Bazikar, F., Ketabchi, S., Moosaei, H.: DC Programming and DCA for parametric-margin ν-support vector machine. Appl. Intell. 50(6), 1763–1774 (2020)

    Article  Google Scholar 

  4. Cai, Y.D., Ricardo, P.W., Jen, C.H., Chou, K.C.: Application of SVM to predict membrane protein types. J. Theor. Biol. 226(4), 373–376 (2004)

    Article  MathSciNet  Google Scholar 

  5. Chapelle, O., Agarwal, A., Sinz, F., Schölkopf, B.: An analysis of inference with the universum. Advances in Neural Information Processing Systems 20, 1369–1376 (2007)

    Google Scholar 

  6. Cheng, Y., Fu, L., Luo, P., Ye, Q., Liu, F., Zhu, W.: Multi-view generalized support vector machine via mining the inherent relationship between views with applications to face and fire smoke recognition. Knowl.-Based Syst. 210, 106488 (2020)

    Article  Google Scholar 

  7. Cherkassky, V., Mulier, F.M.: Learning from data: concepts, theory, and methods. Wiley (2007)

  8. Dua, D., Graff, C.: UCI machine learning repository. https://archive.ics.uci.edu/ml (2019)

  9. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11(1), 86–92 (1940)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hsu, C.W., Chang, C.C., Lin, C.J., et al.: A practical guide to support vector classification. https://www.csie.ntu.edu.tw/cjlin/papers/guide/guide.pdf (2003)

  11. Iman, R.L., Davenport, J.M.: Approximations of the critical region of the fbietkan statistic. Communications in Statistics - Theory and Methods 9 (6), 571–595 (1980)

    Article  MATH  Google Scholar 

  12. Javadi, S.H., Moosaei, H., Ciuonzo, D.: Learning wireless sensor networks for source localization. Sensors 19(3), 635 (2019)

    Article  Google Scholar 

  13. Jayadeva, Khemchandani, R., Chandra, S.: Twin support vector machines for pattern classification. IEEE Trans. Pattern Anal. Mach. Intell. 29(5), 905–910 (2007)

    Article  MATH  Google Scholar 

  14. Ketabchi, S., Moosaei, H., Razzaghi, M., Pardalos, P.M.: An improvement on parametric ν-support vector algorithm for classification. Ann. Oper. Res. 276(1-2), 155–168 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  15. Kumar, M.A., Gopal, M.: Least squares twin support vector machines for pattern classification. Expert Syst. Appl. 36(4), 7535–7543 (2009)

    Article  Google Scholar 

  16. LeCun, Y., Boser, B.E., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W.E., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2, pp. 396–404. Morgan Kaufmann Publishers Inc (1990)

  17. Liu, C.L., Hsaio, W.H., Lee, C.H., Chang, T.H., Kuo, T.H.: Semi-supervised text classification with universum learning. IEEE Trans. Cybern. 46(2), 462–473 (2015)

    Article  Google Scholar 

  18. Lu, S., Tong, L.: Weighted twin support vector machine with universum. Advances in Computer Science: an International Journal 3(2), 17–23 (2014)

    Google Scholar 

  19. Mangasarian, O.L., Wild, E.W.: Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 69–74 (2006)

    Article  Google Scholar 

  20. Moosaei, H., Bazikar, F., Ketabchi, S., Hladík, M.: Universum parametric-margin ν-support vector machine for classification using the difference of convex functions algorithm. Appl. Intell. https://doi.org/10.1007/s10489-021-02402-6 (2021)

  21. Moosaei, H., Hladík, M.: Least squares approach to k-SVCR multi-class classification with its applications. Ann. Math. Artif. Intell. https://doi.org/10.1007/s10472-021-09747-1 (2021)

  22. Moosaei, H., Ketabchi, S., Razzaghi, M., Tanveer, M.: Generalized twin support vector machines. Neural. Process. Lett. 53(2), 1545–1564 (2021)

    Article  Google Scholar 

  23. Noble, W.S., et al.: Support vector machine applications in computational biology. In: Schoelkopf, B.K.T., Vert, J.P. (eds.) Kernel Methods in Computational Biology, pp. 71?-92. Computational Molecular Biology, MIT Press (2004)

  24. Qi, Z., Tian, Y., Shi, Y.: Twin support vector machine with universum data. Neural Netw. 36, 112–119 (2012)

    Article  MATH  Google Scholar 

  25. Richhariya, B., Tanveer, M.: A fuzzy universum support vector machine based on information entropy. In: Tanveer, M., Pachori, R. (eds.) Machine Intelligence and Signal Analysis, pp 569–582. Springer, Berlin (2019)

  26. Richhariya, B., Gupta, D.: Facial expression recognition using iterative universum twin support vector machine. Appl. Soft Comput. 76, 53–67 (2019)

    Article  Google Scholar 

  27. Richhariya, B., Sharma, A., Tanveer, M.: Improved universum twin support vector machine. In: Sundaram, S. (ed.) 2018 IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2018), pp. 2045?2052. IEEE (2018)

  28. Richhariya, B., Tanveer, M.: An efficient angle-based universum least squares twin support vector machine for classification. ACM Transactions on Internet Technology (TOIT) 21(3), 1–24 (2021)

    Article  MATH  Google Scholar 

  29. Richhariya, B., Tanveer, M.: Eeg signal classification using universum support vector machine. Expert Syst. Appl. 106, 169–182 (2018)

    Article  Google Scholar 

  30. Richhariya, B., Tanveer, M.: A reduced universum twin support vector machine for class imbalance learning. Pattern Recogn. 102, 107150 (2020)

    Article  Google Scholar 

  31. Richhariya, B., Tanveer, M., Rashid, A.H.: Diagnosis of Alzheimer’s disease using universum support vector machine based recursive feature elimination (USVM-RFE). Biomedical Signal Processing and Control 59, 101903 (2020)

    Article  Google Scholar 

  32. Shao, Y.H., Zhang, C.H., Wang, X.B., Deng, N.Y.: Improvements on twin support vector machines. IEEE Trans. Neural Netw. 22(6), 962–968 (2011)

    Article  Google Scholar 

  33. Tang, L., Tian, Y., Li, W., Pardalos, P.M.: Structural improved regular simplex support vector machine for multiclass classification. Appl. Soft Comput. 91, 106235 (2020)

    Article  Google Scholar 

  34. Tang, L., Tian, Y., Li, W., Pardalos, P.M.: Valley-loss regular simplex support vector machine for robust multiclass classification. Knowl.-Based Syst. 216, 106801 (2021)

    Article  Google Scholar 

  35. Tang, L., Tian, Y., Pardalos, P.M.: A novel perspective on multiclass classification: Regular simplex support vector machine. Inform. Sci. 480, 324–338 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  36. Tang, L., Tian, Y., Yang, C., Pardalos, P.M.: Ramp-loss nonparallel support vector regression: robust, sparse and scalable approximation. Knowl.-Based Syst. 147, 55–67 (2018)

    Article  Google Scholar 

  37. Tanveer, M., Rajani, T., Rastogi, R., Shao, Y.: Comprehensive review on twin support vector machines. arXiv:2105.00336 (2021)

  38. Tanveer, M., Sharma, S., Muhammad, K.: Large-scale least squares twin svms. ACM Transactions on Internet Technology (TOIT) 21(2), 1–19 (2021)

    Article  Google Scholar 

  39. Tanveer, M., Tiwari, A., Choudhary, R., Ganaie, M.: Large-scale pinball twin support vector machines. Mach. Learn. https://doi.org/10.1007/s10994-021-06061-z(2021)

  40. Tanveer, M., Khan, M.A., Ho, S.S.: Robust energy-based least squares twin support vector machines. Appl. Intell. 45(1), 174–186 (2016)

    Article  Google Scholar 

  41. Tian, Y., Ju, X., Qi, Z., Shi, Y.: Improved twin support vector machine. Science China Mathematics 57(2), 417–432 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Tian, Y., Qi, Z.: Review on: twin support vector machines. Annals of Data Science 1(2), 253–277 (2014)

    Article  Google Scholar 

  43. Tong, S., Koller, D.: Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2(Nov), 45–66 (2001)

    MATH  Google Scholar 

  44. Vapnik, V.: The nature of statistical learning theory. Springer (2013)

  45. Vapnik, V.N., Chervonenkis, A.J.: Theory of Pattern Recognition. Moscow, Nauka (1974)

    MATH  Google Scholar 

  46. Wang, H., Zhou, Z., Xu, Y.: An improved ν-twin bounded support vector machine. Appl. Intell. 48(4), 1041–1053 (2018)

    Article  Google Scholar 

  47. Wang, X.Y., Wang, T., Bu, J.: Color image segmentation using pixel wise support vector machine classification. Pattern Recogn. 44(4), 777–787 (2011)

    Article  MATH  Google Scholar 

  48. Weston, J., Collobert, R., Sinz, F., Bottou, L., Vapnik, V.: Inference with the universum. In: Proceedings of the 23rd International Conference on Machine learning, pp. 1009–1016 (2006)

  49. Xu, Y., Chen, M., Li, G.: Least squares twin support vector machine with universum data for classification. Int. J. Syst. Sci. 47(15), 3637–3645 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  50. Xu, Y., Chen, M., Yang, Z., Li, G.: ν-twin support vector machine with universum data for classification. Appl. Intell. 44(4), 956–968 (2016)

    Article  Google Scholar 

  51. Yan, H., Ye, Q., Zhang, T., Yu, D.J., Yuan, X., Xu, Y., Fu, L.: Least squares twin bounded support vector machines based on l1-norm distance metric for classification. Pattern Recogn. 74, 434–447 (2018)

    Article  Google Scholar 

  52. Ye, Q., Huang, P., Zhang, Z., Zheng, Y., Fu, L., Yang, W.: Multiview learning with robust double-sided twin svm. IEEE Transactions on Cybernetics (2021)

  53. Ye, Q., Zhao, H., Li, Z., Yang, X., Gao, S., Yin, T., Ye, N.: L1-norm distance minimization-based fast robust twin support vector k-plane clustering. IEEE Trans. Neural Netw. Learn. Syst. 29(9), 4494–4503 (2017)

    Article  Google Scholar 

Download references

Acknowledgments

The work of H. Moosaei was supported by the Center for Foundations of Modern Computer Science (Charles Univ. project UNCE/SCI/004). The work of M. Hladík was supported by the Czech Science Foundation under project 22-19353S.

Author information

Authors and Affiliations

  1. Department of Mathematics, Faculty of Science, University of Bojnord, Bojnord, Iran

    Hossein Moosaei

  2. Department of Applied Mathematics, Charles University, Malostranské nám. 25, 118 00, Prague, Czech Republic

    Hossein Moosaei & Milan Hladík

  3. Department of Econometrics, Prague University of Economics and Business, nám. W. Churchilla 4, 130 67, Prague, Czech Republic

    Milan Hladík

Authors
  1. Hossein Moosaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Milan Hladík
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Moosaei.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moosaei, H., Hladík, M. A lagrangian-based approach for universum twin bounded support vector machine with its applications. Ann Math Artif Intell 91, 109–131 (2023). https://doi.org/10.1007/s10472-022-09783-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-022-09783-5

Keywords

Mathematics Subject Classification (2010)

Search

Navigation