Skip to main content
Log in

Abstract

The support vector ordinal regression (SVOR) method is derived from support vector machine and developed to tackle the ordinal regression problems. However, it ignores the distribution characteristics of the data. In this paper, we propose a novel method to handle the ordinal regression problems. This method is referred to as minimum class variance support vector ordinal regression (MCVSVOR). In contrast with SVOR, MCVSVOR explicitly takes into account the distribution of the categories and achieves better generalization performance. Moreover, the problem of MCVSVOR can be transformed into one of SVOR. Thus, the existing software of SVOR can be used to solve the problem of MCVSVOR. In the paper, we first discuss the linear case of MCVSVOR and then develop the nonlinear MCVSVOR through using the kernelization trick. The comprehensive experiment results show that the proposed method is effective and can achieve better generalization performance in contrast with SVOR.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. You ZH, Lei YK, Zhu L, Xia JF, Wang B (2013) Prediction of protein-protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis. BMC Bioinform 14(8):69–75

    Google Scholar 

  2. You ZH, Zhu L, Zheng CH, Yu HJ, Deng SP, Ji Z (2014) Prediction of protein-protein interactions from amino acid sequences using a novel multi-scale continuous and discontinuous feature set. BMC Bioinform 15(Suppl 15):S9–S9

    Article  Google Scholar 

  3. You ZH, Yu JZ, Zhu L, Li S, Wen ZK (2014) A mapreduce based parallel SVM for large-scale predicting protein-protein interactions. Neurocomputing 145(18):37–43

    Article  Google Scholar 

  4. Kundu MK, Chowdhury M, Banerjee M (2012) Interactive image retrieval using M-band wavelet, earth mover’s distance and fuzzy relevance feedback. J Mach Learn Cybern 3(4):285–296

    Article  Google Scholar 

  5. Yan J, Liu N, Yan SC, Yang Q, Fan WG, Wei W, Chen Z (2011) Trace-oriented feature analysis for large-scale text data dimension reduction. IEEE Trans Knowl Data Eng 23(7):1103–1117

    Article  Google Scholar 

  6. Pedregosa F, Gramfort A, Varoquaux G, Cauvet E, Pallier C, Thirion B (2012) Learning to rank from medical imaging data. In: Proceedings Int. Workshop Mach. Learn. Med. Imag, pp 234–241

  7. Li C, Liu Q, Liu J, Lu H (2015) Ordinal distance metric learning for image ranking. IEEE Trans Neural Netw Learn Syst 26(7):1551–1559

    Article  MathSciNet  Google Scholar 

  8. Han H, Otto C, Liu X, Jain AK (2015) Demographic estimation from face images: human vs. machine performance. IEEE Trans Pattern Anal Mach 37(7):1148–1161

    Article  Google Scholar 

  9. McCullagh P (1980) Regression models for ordinal data. J R Stat Soc B 42(2):109–142

    MATH  MathSciNet  Google Scholar 

  10. McCullagh P, Nelder A (1983) Generalized linear models. Chapman & Hall, London

    Book  MATH  Google Scholar 

  11. Johnson VE, Albert JH (1999) Ordinal data modeling (statistics for social science and public policy). Springer, New York

    Google Scholar 

  12. Chu W, Ghahramani Z (2005) Gaussian processes for ordinal regression. J Mach Learn Res 6:1019–1041

    MATH  MathSciNet  Google Scholar 

  13. Chu W, Keerthi SS (2007) Support vector ordinal regression. Neural Comput 19(3):792–815

    Article  MATH  MathSciNet  Google Scholar 

  14. Wang XZ, Ashfaq RAR, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196

    Article  MathSciNet  Google Scholar 

  15. Wang XZ (2015) Learning from big data with uncertainty-editorial. J Intell Fuzzy Syst 28(5):2329–2330

    Article  MathSciNet  Google Scholar 

  16. Mika S (2002) Kernel fisher discriminants (PhD thesis). University of Technology, Berlin

  17. Vapnik V (1995) The nature of statistical learning theory. Springer, New York

    Book  MATH  Google Scholar 

  18. Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564

    Article  MathSciNet  Google Scholar 

  19. Wang M, Chung FL, Wang ST (2010) On minimum class locality preserving variance support vector machine. Pattern Recogn 43(8):2753–2762

    Article  MATH  Google Scholar 

  20. Herbrich R, Graepel T, Obermayer K (1999) Support vector learning for ordinal regression. In: International conference on artificial neural networks, pp 97–102

  21. Kramer S, Widmer G, Pfahringer B, DeGroeve M (2001) Prediction of ordinal classes using regression trees. Fundamenta Informaticae 47:1–13

    MATH  MathSciNet  Google Scholar 

  22. Sun BY, Li J, Wu DD, Zhang XM, Li WB (2010) Kernel discriminant learning for ordinal regression. IEEE Trans Knowl Data Eng 22(6):906–910

    Article  Google Scholar 

  23. Seah CW, Tsang IW, Ong YS (2012) Transductive ordinal regression. IEEE Trans Neural Netw Learn Syst 23(7):1074–1086

    Article  Google Scholar 

  24. Scholkopf B, Smola A (2002) Learning with kernels. MIT, Cambridge

    MATH  Google Scholar 

  25. Shashua A, Levin A (2003) Ranking with large margin principle: two approaches. Adv Neural Inf Process Syst 15:961–968

    Google Scholar 

  26. Shevade SK, Chu W (2006) Minimum enclosing spheres formulations for support vector ordinal regression. In: Sixth international conference on data mining, pp 1054–1058

  27. Zhao B, Wang F, Zhang CS (2009) Block-quantized support vector ordinal regression. IEEE Trans Neural Netw 20(5):882–890

    Article  Google Scholar 

  28. Fletcher R (1987) Practical methods of optimization, 2nd edn. Wiley, New York

    MATH  Google Scholar 

  29. Guo Y, Hastie T, Tibshirani R (2007) Regularized linear discriminant analysis and its application in microarrays. Biostatistics 8(1):86–100

    Article  MATH  Google Scholar 

  30. He XF, Yan SC, Hu YX, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340

    Article  Google Scholar 

  31. Alpaydin E (2004) Introduction to machine learning. The MIT, Cambridge

    MATH  Google Scholar 

  32. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  33. Graham DB, Allinson NM (1998) Characterizing virtual eigensignatures for general purpose face recognition. In: Face recognition: from theory to applications, pp 446–456

  34. Tom SM, Fox CR, Trepel C, Poldrack RA (2007) The neuralbasis of loss aversion in decision-making under risk. Science 315(5811):515–518

    Article  Google Scholar 

  35. Cootes T, Edwards G, Taylor C (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):68–685

    Article  Google Scholar 

  36. Pedregosa F, Gramfort A, Varoquaux G, Cauvet E, Pallier C, Thirion B (2012) Learning to rank from medical imaging data. In: Proc. Int. Workshop Mach. Learn. Med. Imag, pp 234–241

Download references

Acknowledgments

This work is supported in part by the Scientific Research Project “Chunhui Plan” of Ministry of Education of China (Grant Nos. Z2015102, Z2015108), the Key Scientific Research Foundation of Sichuan Provincial Department of Education (No. 11ZA004), the Sichuan Province Science and Technology Support Program (No. 2016RZ0051), the Open Research Fund from Province Key Laboratory of Xihua University (No. szjj2013-022) and the National Science Foundation of China (Grant Nos. 61303126, 61103168).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoming Wang.

Ethics declarations

Conflict of interest

The authors have declared that there is no conflict of interests regarding the publication of this article.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Hu, J. & Huang, Z. Minimum class variance support vector ordinal regression. Int. J. Mach. Learn. & Cyber. 8, 2025–2034 (2017). https://doi.org/10.1007/s13042-016-0582-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-016-0582-3

Keywords

Navigation