Abstract
Because of the lack of a clear guideline or technique for selecting classifiers which maximise diversity and accuracy, the development of techniques for analysing classifier relationships and methods for generating good constituent classifiers remains an important research direction. In this paper we propose a framework based on the Bayesian Belief Networks (BBN) approach to classification. In the proposed approach the multiple-classifier system is conceived at a meta-level and the relationships between individual classifiers are abstracted using Bayesian structural learning methods. We show that relationships revealed by the BBN structures are supported by standard correlation and diversity measures. We use the dependency properties obtained by the learned Bayesian structure to illustrate that BBNs can be used to explore classifier relationships, and for classifier selection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kuncheva, L.I., Whitaker, C.J.: Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy. Machine Learning 51(2), 181–207 (2003)
Roli, F., Giacinto, G., Vernazza, G.: Methods for designing multiple classifier systems. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, Springer, Heidelberg (2001)
Yang, J., Honavar, V.: Feature subset selection using a genetic algorithm. In: Genetic Programming 1997: Proceedings of the Second Annual Conference, San Mateo, CA (1997)
Stefano, C.D., Marcelli, A.: Exploiting Reliability for Dynamic Selection of Classifiers by Means of Genetic Algorithms. In: Proceedings of the Seventh International Conference on Document Analysis and Recognition (ICDAR 2003) (2003)
Chindaro, S., Sirlantzis, K., Fairhurst, M.C.: Analysis and Modelling of Diversity Contribution to Ensemble-Based Texture Recognition Performance. In: Oza, N.C., et al. (eds.) MCS 2005. LNCS, vol. 3541, Springer, Heidelberg (2005)
Kuncheva, L., Kountch, R.: Generating classifier outputs of fixed accuracy and diversity. Pattern Recognition Letters 23, 593–600 (2002)
Ho, T.K.: Multiple Classifier Combination: Lessons and Next Steps. In: Hybrid Methods in Pattern Recognition, pp. 171–198. World Scientific, Singapore (2002)
Pavlovic, V., Garg, A., Kasif, S.: A Bayesian Framework for Combining Gene Predictions. Bioinformatics 1, 19–27 (2002)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learning 29, 131–163 (1997)
Jordan, M.I.: Learning Graphical Models. Kluwer Academic Publishers, Dordrecht (1996)
Cooper, G.F., Herskovits, E.: A Bayesian Method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)
Herskovits, E.H.: Computer-based probabilistic network construction. Medical information sciences, Doctoral dissertation, Stanford University, Stanford, CA (1991)
Suzuki, J.: Learning Bayesian belief networks based on the MDL principle: An efficient algorithm using the branch and bound technique. In: Proceedings of the international conference on machine learning, Bally, Italy (1996)
Fung, R.M.,, R.M., Crawford, S.L.: Constructor: a system for the induction of probabilistic models. In: Proceedings of AAAI, Boston, MA, MIT Press, Cambridge
Srinivas, S., Russell, S., Agogino, A.: Automated construction of sparse Bayesian networks from unstructured probabilistic models and domain information. In: Uncertainty in artificial intelligence vol. 5, North-Holland, Amsterdam (1990)
Heckerman, D., Geiger, D., Chickering, D.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Duin, R.P.W., et al.: PRTools 4, “A Matlab Toolbox for Pattern Recognition”. Delft University of Technology (2004)
UCI: Repository of machine learning databases. University of California, Irvine, Dept. of Inform. and Comp. Sc. (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Sirlantzis, K., Hoque, S., Fairhurst, M.C.: Classifier Diversity Estimation in a Multiclassifier Face Recognition System based on Binary Feature Quantisation. In: Proceedings of the 4th International Conference on Recent Advances in Soft Computing (RASC 2002), Nottingham, UK (2002)
Yule, G.: On the association of attributes in statistics. Phil. Transaction A(194), 257–319 (1900)
Sneath, P., Sokal, R.: Numerical Taxonomy. W.H. Freeman and Company, New York (1973)
Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Trans. on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Giacinto, G., Roli, F.: Design of effective neural network ensembles for image classification processes. Image and Vision Computing Journal 19(9-10), 699–707 (2001)
Ruta, D., Gabrys, B.: New Measure of Classifier Dependency in Multiple Classifier Systems. In: Roli, F., Kittler, J. (eds.) MCS 2002. LNCS, vol. 2364, Springer, Heidelberg (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Chindaro, S., Sirlantzis, K., Fairhurst, M. (2007). Modelling Multiple-Classifier Relationships Using Bayesian Belief Networks. In: Haindl, M., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2007. Lecture Notes in Computer Science, vol 4472. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72523-7_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-72523-7_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72481-0
Online ISBN: 978-3-540-72523-7
eBook Packages: Computer ScienceComputer Science (R0)