Minimally-Sized Balanced Decomposition Schemes for Multi-class Classification
- 1.4k Downloads
Error-Correcting Output Coding (ECOC) is a well-known class of decomposition schemes for multi-class classification. It allows representing any multiclass classification problem as a set of binary classification problems. Due to code redundancy ECOC schemes can significantly improve generalization performance on multi-class classification problems. However, they can face a computational complexity problem when the number of classes is large.
In this paper we address the computational-complexity problem of the decomposition schemes. We study a particular class of minimally-sized ECOC decomposition schemes, namely the class of minimally-sized balanced decomposition schemes (MBDSs) .We show thatMBDSs do not face a computational-complexity problem for large number of classes. However we also show that MBDSs cannot correct the classification errors of the binary classifiers in MBDS ensembles. Therefore we propose voting with MBDS ensembles (VMBDSs).We show that the generalization performance of the VMBDSs ensembles improves with the number of MBDS classifiers. However this number can become large and thus the VMBDSs ensembles can have a computational-complexity problem as well. Fortunately our experiments show that VMBDSs are comparable with ECOC ensembles and can outperform one-against-all ensembles using only a small number of MBDS ensembles.
KeywordsSupport Vector Machine Generalization Performance Code Word Decomposition Scheme Decomposition Matrix
Unable to display preview. Download preview PDF.
- 2.Asuncion, A., Newman, D.J.: UCI machine learning repository, http://www.ics.uci.edu/~mlearn/MLRepository.html
- 3.Cohen, W.: Fast effective rule induction. In: Prieditis, A., Russell, R. (eds.) Proc. the 12th Int. Conf. Machine Learning, Tahoe City, CA, pp. 115–123. Morgan Kaufmann, San Francisco (1995)Google Scholar
- 5.Domingos, P.: A unified bias-variance decomposition for zero-one and squared loss. In: Kautz, H., Porter, B. (eds.) Proc. the 17th National Conf. Artif. Intell. and 12th Conf. Innovative Applications Artif. Intell., pp. 564–569. AAAI Press, Menlo Park (2000)Google Scholar
- 9.Kong, E.B., Dietterich, T.G.: Error-correcting output coding corrects bias and variance. In: Prieditis, A., Russell, S.J. (eds.) Proc. the 12th Int. Conf. Machine Learning, Tahoe City, CA, pp. 313–321. Morgan Kaufmann, San Francisco (1995)Google Scholar
- 11.Libor, S.: Face recognition database (2011), http://cswww.essex.ac.uk/mv/allfaces/index.html
- 12.Lissoni, F., Llerena, P., Sanditov, B.: Inventors’ small worlds: academic and CNRS researchers in networks of inventors in France. In: Proc. the DIME Final Conf., Maastricht, The Netherlands (2011)Google Scholar
- 14.Mayoraz, E., Moreira, M.: On the decomposition of polychotomies into dichotomies. In: Fisher, D.H. (ed.) Proc. the 14th Int. Conf. Machine Learning, Nashville, TN, pp. 219–226. Morgan Kaufmann, San Francisco (1997)Google Scholar
- 19.Weston, J., Watkins, C.: Support vector machines for multi-class pattern recognition. In: Verleysen, M. (ed.) Proc. the 7th European Symp. Artif. Neural Networks, Bruges, Belgium, pp. 219–224 (1999)Google Scholar
- 20.Witten, I., Frank, E., Hall, M.: Data mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2011)Google Scholar