Abstract
The support vector ordinal regression (SVOR) method is derived from support vector machine and developed to tackle the ordinal regression problems. However, it ignores the distribution characteristics of the data. In this paper, we propose a novel method to handle the ordinal regression problems. This method is referred to as minimum class variance support vector ordinal regression (MCVSVOR). In contrast with SVOR, MCVSVOR explicitly takes into account the distribution of the categories and achieves better generalization performance. Moreover, the problem of MCVSVOR can be transformed into one of SVOR. Thus, the existing software of SVOR can be used to solve the problem of MCVSVOR. In the paper, we first discuss the linear case of MCVSVOR and then develop the nonlinear MCVSVOR through using the kernelization trick. The comprehensive experiment results show that the proposed method is effective and can achieve better generalization performance in contrast with SVOR.
Similar content being viewed by others
References
You ZH, Lei YK, Zhu L, Xia JF, Wang B (2013) Prediction of protein-protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis. BMC Bioinform 14(8):69–75
You ZH, Zhu L, Zheng CH, Yu HJ, Deng SP, Ji Z (2014) Prediction of protein-protein interactions from amino acid sequences using a novel multi-scale continuous and discontinuous feature set. BMC Bioinform 15(Suppl 15):S9–S9
You ZH, Yu JZ, Zhu L, Li S, Wen ZK (2014) A mapreduce based parallel SVM for large-scale predicting protein-protein interactions. Neurocomputing 145(18):37–43
Kundu MK, Chowdhury M, Banerjee M (2012) Interactive image retrieval using M-band wavelet, earth mover’s distance and fuzzy relevance feedback. J Mach Learn Cybern 3(4):285–296
Yan J, Liu N, Yan SC, Yang Q, Fan WG, Wei W, Chen Z (2011) Trace-oriented feature analysis for large-scale text data dimension reduction. IEEE Trans Knowl Data Eng 23(7):1103–1117
Pedregosa F, Gramfort A, Varoquaux G, Cauvet E, Pallier C, Thirion B (2012) Learning to rank from medical imaging data. In: Proceedings Int. Workshop Mach. Learn. Med. Imag, pp 234–241
Li C, Liu Q, Liu J, Lu H (2015) Ordinal distance metric learning for image ranking. IEEE Trans Neural Netw Learn Syst 26(7):1551–1559
Han H, Otto C, Liu X, Jain AK (2015) Demographic estimation from face images: human vs. machine performance. IEEE Trans Pattern Anal Mach 37(7):1148–1161
McCullagh P (1980) Regression models for ordinal data. J R Stat Soc B 42(2):109–142
McCullagh P, Nelder A (1983) Generalized linear models. Chapman & Hall, London
Johnson VE, Albert JH (1999) Ordinal data modeling (statistics for social science and public policy). Springer, New York
Chu W, Ghahramani Z (2005) Gaussian processes for ordinal regression. J Mach Learn Res 6:1019–1041
Chu W, Keerthi SS (2007) Support vector ordinal regression. Neural Comput 19(3):792–815
Wang XZ, Ashfaq RAR, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196
Wang XZ (2015) Learning from big data with uncertainty-editorial. J Intell Fuzzy Syst 28(5):2329–2330
Mika S (2002) Kernel fisher discriminants (PhD thesis). University of Technology, Berlin
Vapnik V (1995) The nature of statistical learning theory. Springer, New York
Zafeiriou S, Tefas A, Pitas I (2007) Minimum class variance support vector machines. IEEE Trans Image Process 16(10):2551–2564
Wang M, Chung FL, Wang ST (2010) On minimum class locality preserving variance support vector machine. Pattern Recogn 43(8):2753–2762
Herbrich R, Graepel T, Obermayer K (1999) Support vector learning for ordinal regression. In: International conference on artificial neural networks, pp 97–102
Kramer S, Widmer G, Pfahringer B, DeGroeve M (2001) Prediction of ordinal classes using regression trees. Fundamenta Informaticae 47:1–13
Sun BY, Li J, Wu DD, Zhang XM, Li WB (2010) Kernel discriminant learning for ordinal regression. IEEE Trans Knowl Data Eng 22(6):906–910
Seah CW, Tsang IW, Ong YS (2012) Transductive ordinal regression. IEEE Trans Neural Netw Learn Syst 23(7):1074–1086
Scholkopf B, Smola A (2002) Learning with kernels. MIT, Cambridge
Shashua A, Levin A (2003) Ranking with large margin principle: two approaches. Adv Neural Inf Process Syst 15:961–968
Shevade SK, Chu W (2006) Minimum enclosing spheres formulations for support vector ordinal regression. In: Sixth international conference on data mining, pp 1054–1058
Zhao B, Wang F, Zhang CS (2009) Block-quantized support vector ordinal regression. IEEE Trans Neural Netw 20(5):882–890
Fletcher R (1987) Practical methods of optimization, 2nd edn. Wiley, New York
Guo Y, Hastie T, Tibshirani R (2007) Regularized linear discriminant analysis and its application in microarrays. Biostatistics 8(1):86–100
He XF, Yan SC, Hu YX, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340
Alpaydin E (2004) Introduction to machine learning. The MIT, Cambridge
Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554
Graham DB, Allinson NM (1998) Characterizing virtual eigensignatures for general purpose face recognition. In: Face recognition: from theory to applications, pp 446–456
Tom SM, Fox CR, Trepel C, Poldrack RA (2007) The neuralbasis of loss aversion in decision-making under risk. Science 315(5811):515–518
Cootes T, Edwards G, Taylor C (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):68–685
Pedregosa F, Gramfort A, Varoquaux G, Cauvet E, Pallier C, Thirion B (2012) Learning to rank from medical imaging data. In: Proc. Int. Workshop Mach. Learn. Med. Imag, pp 234–241
Acknowledgments
This work is supported in part by the Scientific Research Project “Chunhui Plan” of Ministry of Education of China (Grant Nos. Z2015102, Z2015108), the Key Scientific Research Foundation of Sichuan Provincial Department of Education (No. 11ZA004), the Sichuan Province Science and Technology Support Program (No. 2016RZ0051), the Open Research Fund from Province Key Laboratory of Xihua University (No. szjj2013-022) and the National Science Foundation of China (Grant Nos. 61303126, 61103168).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have declared that there is no conflict of interests regarding the publication of this article.
Rights and permissions
About this article
Cite this article
Wang, X., Hu, J. & Huang, Z. Minimum class variance support vector ordinal regression. Int. J. Mach. Learn. & Cyber. 8, 2025–2034 (2017). https://doi.org/10.1007/s13042-016-0582-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-016-0582-3