Abstract
Error Correcting Output Codes (ECOC) is an effective method to handle multi-class classification problems, whose performance is heavily affected by encoding strategies. However, traditional encoding strategies are usually data-independent which more likely leads to a less representation coding matrix. Therefore, recent researches emphasize more on the data-dependent construction of coding matrix. However, these methods usually can not guarantee that the binary problems in coding matrix are linearly separable. When the problems are linearly non-separable, the difficulty of designing base classifiers will increase. Even the non-linear base classifiers can not ensure that they can handle each binary problem well, which will decrease the performance of ECOC. In this paper, we propose subclass maximum margin tree error correcting output codes (SM\(^2\)ECOC) which aims to make each binary problem simple and linearly separable. Concretely, SM\(^2\)ECOC firstly uses hierarchical clustering techniques to split original classes into a series of linearly separable subclasses. Then it takes the margin as a criterion to evaluate the separability among subclasses and guide the construction of coding matrix. As a result, the binary problems in coding matrix more likely tend to be linearly separable which can reduce the difficulty in base classifiers effectively. Experimental results show the superiority of SM\(^2\)ECOC.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1(Dec), 113–141 (2000). https://doi.org/10.1162/15324430152733133
An-rong, Y., Xiang, X., Jing-ming, K.: Application of hadamard ECOC in multi-class problems based on SVM. Chin. J. Electron. 36(1), 122–126 (2008). https://doi.org/10.3321/j.issn:0372-2112.2008.01.022
Asuncion, A., Newman, D.: UCI machine learning repository (2007)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). https://doi.org/10.1007/BF00994018
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995). https://doi.org/10.1613/jair.105
Escalera, S., Pujol, O., Radeva, P.: Loss-weighted decoding for error-correcting output coding. In: VISAPP, no. 2, pp. 117–122 (2008)
Escalera, S., Tax, D.M., Pujol, O., Radeva, P., Duin, R.P.: Subclass problem-dependent design for error-correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1041–1054 (2008). https://doi.org/10.1109/TPAMI.2008.38
Hastie, T., Tibshirani, R., et al.: Classification by pairwise coupling. Ann. Stat. 26(2), 451–471 (1998). https://doi.org/10.1214/aos/1028144844
Kaufman, L., Rousseeuw, P.J.: Finding Groups in Data: An Introduction to Cluster Analysis, vol. 344. Wiley, Hoboken (2009)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, Hoboken (2004)
Lei, L., Xiao-Dan, W., Xi, L., Ya-Fei, S.: Hierarchical error-correcting output codes based on SVDD. Pattern Anal. Appl. 19(1), 163–171 (2016)
Liu, M., Zhang, D., Chen, S., Xue, H.: Joint binary classifier learning for ECOC-based multi-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 38, 2335–2341 (2016). https://doi.org/10.1109/TPAMI.2015.2430325
Nilsson, N.J.: Learning Machines: Foundations of Trainable Pattern-Classifying Systems. McGraw-Hill, New York (1965)
Pujol, O., Escalera, S., Radeva, P.: An incremental node embedding technique for error correcting output codes. Pattern Recogn. 41(2), 713–725 (2008). https://doi.org/10.1016/j.patcog.2007.04.008
Pujol, O., Radeva, P., Vitria, J.: Discriminant ECOC: a heuristic method for application dependent design of error correcting output codes. IEEE Trans. Pattern Anal. Mach. Intell. 28(6), 1007–1012 (2006). https://doi.org/10.1109/TPAMI.2006.116
Rocha, A., Goldenstein, S.K.: Multiclass from binary: expanding one-versus-all, one-versus-one and ECOC-based approaches. IEEE Trans. Neural Netw. Learn. Syst. 25(2), 289–302 (2014). https://doi.org/10.1109/TNNLS.2013.2274735
Zheng, F., Xue, H., Chen, X., Wang, Y.: Maximum margin tree error correcting output codes. In: Booth, R., Zhang, M.-L. (eds.) PRICAI 2016. LNCS (LNAI), vol. 9810, pp. 681–691. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42911-3_57
Acknowledgements
This work is supported by the National Key R&D Program of China (No. 2017YFB1002801), the National Natural Science Foundations of China (Grant Nos. 61375057, 61300165 and 61403193), the Natural Science Foundation of Jiangsu Province of China (Grant No. BK20131298) and Fundamental Research Funds for the Central Universities (SJLX_160053) and Research Innovation Program for College Graduates of Jiangsu Province of China (SJLX_160053). It is also supported by Collaborative Innovation Center of Wireless Communications Technology.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zheng, F., Xue, H. (2018). Subclass Maximum Margin Tree Error Correcting Output Codes. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11012. Springer, Cham. https://doi.org/10.1007/978-3-319-97304-3_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-97304-3_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97303-6
Online ISBN: 978-3-319-97304-3
eBook Packages: Computer ScienceComputer Science (R0)