Skip to main content
SpringerLink
Log in

Pinball loss-based multi-task twin support vector machine and its safe acceleration method

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Direct multi-task twin support vector machine (DMTSVM) performs well in handling multiple related tasks. But it is sensitive to noise points due to the use of hinge loss. In this paper, we propose a novel multi-task twin support vector machine with pinball loss (Pin-DMTSVM) to enhance the noise insensitivity of DMTSVM. Besides, in order to improve the computational speed of Pin-DMTSVM, we further construct a safe screening rule (SSR) for accelerating Pin-DMTSVM based on the optimality conditions. SSR could identify and pre assign most of the inactive instances before actually solving the optimization problem. So, the computational time will be reduced a lot by solving a smaller problem. More importantly, it can get an exactly same solution as solving the original larger optimization problem, so the classification accuracy will not be affected in theory. Numerical experiments on six benchmark datasets and seven image datasets have demonstrated the effectiveness and robustness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://people.ee.duke.edu/lcarin/LandmineData.zip.

References

  1. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  2. Xu Y, Wang L (2005) Fault diagnosis system based on rough set theory and support vector machine. Lect Notes Comput Sci 3614:980–988

    Article  Google Scholar 

  3. Zhang W, Yoshida T, Tang X (2008) Text classification based on multi-word with support vector machine. Knowl Based Syst 21(8):879–886

    Article  Google Scholar 

  4. Wang H, Xu D, Martinez A (2020) Parameter selection method for support vector machine based on adaptive fusion of multiple kernel functions and its application in fault diagnosis. Neural Comput Appl 32:183–193

    Article  Google Scholar 

  5. Jayadeva R, Khemchandani R, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  Google Scholar 

  6. Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  Google Scholar 

  7. Ding S, Zhang N, Zhang X, Wu F (2017) Twin support vector machine: theory, algorithm and applications. Neural Comput Appl 28(11):3119–3130

    Article  Google Scholar 

  8. Peng X (2010) A \(\nu\)-twin support vector machine (\(\nu\)-TSVM) classifier and its geometric algorithms. Inf Sci 180(20):3863–3875

  9. Xie F, Xu Y (2019) An efficient regularized K-nearest neighbor structural twin support vector machine. Appl Intell 49(12):4258–4275

    Article  Google Scholar 

  10. Tanveer M, Shubham K, Al-Dhaifallah M, Ho S (2016) An efficient regularized k-nearest neighbor-based weighted twin support vector regression. Knowl Based Syst 94:70–87

    Article  Google Scholar 

  11. Xu Y, Li X, Pan X, Yang Z (2018) Asymmetric \(\nu\)-twin support vector regression. Neural Comput Appl 30(12):3799–3814

  12. Gupta D, Richhariya B, Borah P (2019) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Appl 31:7153–7164

    Article  Google Scholar 

  13. Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  14. Zhang Y, Yang Q (2018) An overview of multi-task learning. Natl Sci Rev 5(01):34–47

    Google Scholar 

  15. Zhang Y, Yang Q (2017) A survey on multi-task learning. arXiv:1707.08114

  16. Leiva-Murillo J, Gomez-Chova L, Camps-Valls G (2013) Multitask remote sensing data classification. IEEE Trans Geosci Remote Sens 51(1):151–161

    Article  Google Scholar 

  17. Shiao H, Cherkassky V (2012) Implementation and comparison of SVM-based multi-task learning methods. In: WCCI 2012 IEEE word conference on computational intelligence, pp 10–15

  18. Yang H, King I, Lyu M (2010) Multi-task Learning for one-class classification. In: International joint conference on neural networks. WCCI 2010 IEEE word conference on computational intelligence, pp 18–23

  19. Xue Y, Beauseroy P (2017) Multi-Task learning for one-class SVM with additional new features. In: International conference on pattern recognition, pp 1571–1576

  20. Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining, pp 22–25

  21. He H, Mourot G, Maquin D et al (2014) Multi-task learning with one-class SVM. Neurocomputing 133:416–426

    Article  Google Scholar 

  22. Xu S, An X, Qiao X et al (2014) Multi-task least-squares support vector machines. Multimed Tools Appl 71(2):699–715

    Article  Google Scholar 

  23. Li Y, Tian X, Song M et al (2015) Multi-task proximal support vector machine. Pattern Recognit 48(10):3249–3257

    Article  Google Scholar 

  24. Lu L, Lin Q, Pei H, Zhong P (2018) The aLS-SVM based multi-task learning classifiers. Appl Intell 48(8):2393–2407

    Article  Google Scholar 

  25. Mei B, Xu Y (2020) Multi-task \(\nu\)-twin support vector machines. Neural Comput Appl 32(15):11329–11342

  26. Xie X, Sun S (2012) Multitask twin support vector machines. In: Proceedings of the 19th international coference on neural informachie processing ICONIP—volume Part II, pp 341–348

  27. Mei B, Xu Y (2019) Multi-task least squares twin support vector machine for classification. Neurocomputing 338:26–33

    Article  Google Scholar 

  28. Huang X, Shi L, Suykens J (2015) Support vector machine classifier with pinball loss. IEEE Trans Pattern Anal Mach Intell 36(5):984–997

    Article  Google Scholar 

  29. Shen X, Niu L, Qi Z, Tian Y (2017) Support vector machine classifier with truncated pinball loss. Pattern Recognit 68:199–210

    Article  Google Scholar 

  30. Xu Y, Yang Z, Pan X (2017) A novel twin support vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370

    Article  MathSciNet  Google Scholar 

  31. Tanveer M, Sharma A, Suganthan P (2019) General twin support vector machine with pinball loss function. Inf Sci 494:311–327

    Article  MathSciNet  Google Scholar 

  32. Ghaoui L, Viallon V, Rabbani T (2012) Safe feature elimination in sparse supervised learning. Pac J Optim 8(4):667–698

    MathSciNet  MATH  Google Scholar 

  33. Xiang Z, Xu H, Ramadge P (2011) Learning sparse representations of high dimensional data on large scale dictionaries. Conf Worksh Neural Inf Process Syst 24:900–908

    Google Scholar 

  34. Wang J, Wonka P, Ye J (2015) Lasso screening rules via dual polytope projection. J Mach Learn Res 16:1063–1101

    MathSciNet  MATH  Google Scholar 

  35. Yang T, Wang J, Sun Q (2015) Detecting genetic risk factors for Alzheimer’s disease in whole genome sequence data via Lasso screening. In: IEEE international symposium on biomedical imaging, pp 985–989

  36. Ogawa K, Suzuki Y, Takeuchi I et al (2013) Safe screening of non-support vectors in pathwise SVM computation. In: Proceeding of the 30th international conference on machine learning, pp 2419–2427

  37. Wang J, Wonka P, Ye J (2014) Scaling SVM and least absolute deviations via exact data reduction. In: Proceeding of the 31th international conference on machine learning, pp 289–297

  38. Güler O (2010) Foundations of optimizations. Springer

  39. Pan X, Yang Z, Xu Y, Wang L (2018) Safe screening rules for accelerating twin support vector machine classification. IEEE Trans Neural Netw Learn Syst 29(5):1876–1887

    Article  MathSciNet  Google Scholar 

  40. Mei B, Xu Y (2020) Safe sample screening for regularized multi-task learning. Knowl Based Syst 204:17

    Article  Google Scholar 

  41. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I, Mulan A (2011) A java library for multi-label learning. J Mach Learn Res 12:2411–2414

    MathSciNet  MATH  Google Scholar 

  42. Dheeru D, Taniskidou E K (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml

  43. Li F, Fergus R, Perona P (2006) One-shot learning of object categories. IEEE Trans Pattern Anal Mach Intell 28(4):594–611

    Article  Google Scholar 

  44. Griffin G, Holub A, Perona P (2006) The Caltech 256. Caltech Technical Report

  45. Ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(1):1–30

    MathSciNet  Google Scholar 

  46. Garc Salvador A, Ndez A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180(10):2044–2064

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. This work was supported in part by the National Natural Science Foundation of China (Nos. 12071475, 11671010, 61153003) and Beijing Natural Science Foundation (No. 4172035).

Author information

Authors and Affiliations

  1. College of Science, China Agricultural University, Beijing, 100083, China

    Fan Xie & Yitian Xu

  2. College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China

    Xinying Pang

Authors
  1. Fan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinying Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yitian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yitian Xu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, F., Pang, X. & Xu, Y. Pinball loss-based multi-task twin support vector machine and its safe acceleration method. Neural Comput & Applic 33, 15523–15539 (2021). https://doi.org/10.1007/s00521-021-06173-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06173-6

Keywords

Search

Navigation