Abstract
In this chapter, we discuss different strategies of extending an ensemble approach based on local binary classifiers to solve multi-class problems. The ensembles of binary classifiers were developed with the objective of providing interpretable submodels s for use in safety-related application domains. The ensembles assume highly imbalanced misclassification costs between the two classes. The extension to multi-class problems is not straightforward because common multi-class extensions might induce inconsistent decisions. We propose a solution of this problem that avoids such inconsistencies by introducing a hierarchy of misclassification costs. We show that by following such a hierarchy it becomes feasible to extend the binary ensemble, to maintain the desirable properties (that is, the good interpretability) of the binary ensemble, and to achieve a good predictive performance.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences, http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L., Friedman, J.H., Stone, C.J., Olshen, R.A.: Classification and Regression Trees. CRC Press, Boca Raton (1984)
Cardoso, J.S., da Costa, P.J.F., Cardoso, M.J.: Modelling ordinal relations with SVMs: an application to objective aesthetic evaluation of breast cancer conservative treatment. Neural Networks 18(5–6), 808–817 (2005)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines, http://www.csie.ntu.edu.tw/~cjlin/libsvm
Ferri, C., Hernández-Orallo, J., Salido, M.A.: Volume under the ROC surface for multi-class problems. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) ECML 2003. LNCS (LNAI), vol. 2837, pp. 108–120. Springer, Heidelberg (2003)
Friedman, J.H.: Another approach to polychotomous classification. Department of Statistics, Stanford University (1996)
Friedman, J.H., Tukey, J.W.: A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comp. 23(9), 881–890 (1974)
Fürnkranz, J.: Separate-and-conquer rule learning. Artif. Intell. Review 13(1), 3–54 (1999)
Guyon, I., Elisseeff, A.: An introduction to feature extraction. In: Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.) Feature Extraction: Foundations and Applications, pp. 1–25. Springer, Heidelberg (2006)
Güvenir, H.A., Demiröz, G., Ilter, N.: Learning differential diagnosis of erythemato-squamous diseases using voting feature intervals. Artif. Intell. in Medicine 13(3), 147–165 (1998)
Hastie, T., Tibshirani, R.: Generalized Additive Models. Chapman & Hall, Boca Raton (1990)
Hsu, C.-W., Lin, C.-J.: A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Networks 13(2), 415–425 (2002)
Huber, P.J.: Projection pursuit. The Annals of Statistics 13(2), 435–475 (1985)
Kohavi, R.: A Study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proc. 14th Int. Joint Conf. Artif. Intell., Montréal, QB, pp. 1137–1143. Morgan Kaufmann, San Francisco (1995)
Kohavi, R., John, G.H.: Wrappers for feature subset selection. Artif. Intell. 97(1–2), 273–324 (1997)
Nusser, S., Otte, C., Hauptmann, W.: Learning binary classifiers for applications in safety-related domains. In: Mikut, R., Reischl, M. (eds.) Proc. 17th Workshop Computational Intell., Dortmund, Germany, pp. 139–151. Universitätsverlag Karlsruhe (2007)
Stone, C.J.: Additive regression and other nonparametric models. The Annals of Statistics 13(2), 689–705 (1985)
Szepannek, G., Weihs, C.: Local modelling in classification on different feature subspaces. In: Perner, P. (ed.) ICDM 2006. LNCS (LNAI), vol. 4065, pp. 226–238. Springer, Heidelberg (2006)
Tax, D.M.J., Duin, R.P.W.: Using two-class classifiers for multiclass classification. In: Proc. 16th Int. Conf. Patt. Recogn., Québec, QB, pp. 124–127 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Nusser, S., Otte, C., Hauptmann, W. (2009). Verifiable Ensembles of Low-Dimensional Submodels for Multi-class Problems with Imbalanced Misclassification Costs. In: Okun, O., Valentini, G. (eds) Applications of Supervised and Unsupervised Ensemble Methods. Studies in Computational Intelligence, vol 245. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03999-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-03999-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03998-0
Online ISBN: 978-3-642-03999-7
eBook Packages: EngineeringEngineering (R0)