MCS 2013: Multiple Classifier Systems pp 319-330 | Cite as
Randomized Bayesian Network Classifiers
Abstract
In this paper, we propose Randomized Bayesian Network Classifiers (RBNC). It borrows the idea of ensemble learning by constructing a collection of semi-naive Bayesian network classifiers and then combines their predictions as the final output. Specifically, the structure learning of each component Bayesian network classifier is performed by just randomly choosing the parent of each attribute in addition to class attribute, and parameter learning is performed by using maximum likelihood method. RBNC retains many of naive Bayes’ desirable property, such as scaling linearly with respect to both the number of instances and attributes, needing a single pass through the training data and robust to noise, etc. On the 60 widely used benchmark UCI datasets, RBNC outperforms state-of-the-art Bayesian classifiers.
Keywords
Bayesian Network Random Forest Class Attribute Structure Learning Bayesian Network ModelPreview
Unable to display preview. Download preview PDF.
References
- 1.Pearl, J.: Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
- 2.Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learning 29, 131–163 (1997)MATHCrossRefGoogle Scholar
- 3.Carvalho, A.M., Roos, T., Oliveira, A., Myllymaki, P.: Discriminative learning of bayesian networks via factorized conditional log-likelihood. Journal of Machine Learning Research 12, 2181–2210 (2011)MathSciNetGoogle Scholar
- 4.Domingos, P., Pazzani, M.J.: On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 29(2), 103–130 (1997)MATHCrossRefGoogle Scholar
- 5.Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive bayes: Aggregating one-dependence estimators. Machine Learning 58(1), 5–24 (2005)MATHCrossRefGoogle Scholar
- 6.Webb, G.I., Boughton, J.R., Zheng, F., Ting, K., Salem, H.: Learning by extrapolation from marginal to full-multivariate probability distributions: Decreasingly naive bayesian classification. Machine Learning 86(2), 233–272 (2012)MathSciNetMATHCrossRefGoogle Scholar
- 7.Salem, H., Suraweera, P., Webb, G.I., Boughton, J.R.: Techniques for efficient learning without search. In: Tan, P.-N., Chawla, S., Ho, C.K., Bailey, J. (eds.) PAKDD 2012, Part I. LNCS, vol. 7301, pp. 50–61. Springer, Heidelberg (2012)CrossRefGoogle Scholar
- 8.Wu, X.D., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z.H., Steinbach, M., Hand, D.J., Steinberg, D.: Top 10 algorithms in data mining. Knowledge and Information Systems 14(1) (2008) Google Scholar
- 9.Grossman, D., Domingos, P.: Learning bayesian network classifiers by maximizing conditional likelihood. In: Proceedings of the 21st International Conference on Machine Learning, pp. 46–53 (2004)Google Scholar
- 10.Jing, Y.S., Pavlovi, V., Rehg, J.M.: Efficient discriminative learning of bayesian network classifier via boosted augmented naive bayes. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 369–376 (2005)Google Scholar
- 11.Jiang, L., Zhang, H., Cai, Z.: A novel bayes model: hidden naive bayes. IEEE Transations on Knowledge and Data Engineering 21(10), 1361–1371 (2009)CrossRefGoogle Scholar
- 12.Langley, P., Sage, S.: Induction of selective bayesian classifiers. In: Proceedings of the Uncertainty in Artificial Intelligence, pp. 399–406 (1994)Google Scholar
- 13.Zhang, H., Jiang, L.X., Su, J.: Hidden naive bayes. In: The Twentieth National Conference on Artificial Intelligence (AAAI 2005), pp. 919–924 (2005)Google Scholar
- 14.Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (2000)Google Scholar
- 15.Blake, C., Merz, C.J.: UCI repository of machine learning databases. Department of ICS, University of California, Irvine, http://www.ics.uci.edu/~mlearn/MLRepository.html
- 16.Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)MATHCrossRefGoogle Scholar