Abstract
Class binarizations are effective methods for improving weak learners by decomposing multi-class problems into several two-class problems. This paper analyzes how these methods can be applied to a Naive Bayes learner. The key result is that the pairwise variant of Naive Bayes is equivalent to a regular Naive Bayes. This result holds for several aggregation techniques for combining the predictions of the individual classifiers, including the commonly used voting and weighted voting techniques. On the other hand, Naive Bayes with one-against-all binarization is not equivalent to a regular Naive Bayes. Apart from the theoretical results themselves, the paper offers a discussion of their implications.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
References
Allwein, E.L., Schapire, R.E., Singer, Y.: Reducing multiclass to binary: A unifying approach for margin classifiers. Journal of Machine Learning Research 1, 113–141 (2000)
Anand, R., Mehrotra, K.G., Mohan, C.K., Ranka, S.: Efficient classification for multiclass problems using modular networks. IEEE Transactions on Neural Networks 6, 117–124 (1995)
Clark, P., Boswell, R.: Rule induction with CN2: Some recent improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991)
Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995)
Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 263–286 (1995)
Domingos, P., Pazzani, M.J.: On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning 29(2-3), 103–130 (1997)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Wiley, New York (1972)
Friedman, J.H.: Another approach to polychotomous classification. Technical report, Department of Statistics, Stanford University, Stanford, CA (1996)
Fürnkranz, J.: Round robin classification. Journal of Machine Learning Research (JMLR) 2, 721–747 (2002)
Fürnkranz, J.: Round robin ensembles. Intelligent Data Analysis 7(5), 385–403 (2003)
Hastie, T., Tibshirani, R.: Classification by pairwise coupling. In: Jordan, M.I., Kearns, M.J., Solla, S.A. (eds.) Advances in Neural Information Processing Systems 10 (NIPS-97), pp. 507–513. MIT Press, Cambridge (1998)
Hüllermeier, E., Fürnkranz, J.: Ranking by pairwise comparison: A note on risk minimization. In: Proceedings of the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE-2004), Budapest, Hungary (2004)
Kononenko, I.: Inductive and bayesian learning in medical diagonsis. Applied Artificial Intelligence 7(4), 331–337 (1993)
Knerr, S., Personnaz, L., Dreyfus, G.: Handwritten digit recognition by neural networks with single-layer training. IEEE Transactions on Neural Networks 3(6), 962–968 (1992)
Rifkin, R., Klautau, A.: In defense of one-vs-all classification. Journal of Machine Learning Research 5, 101–141 (2004)
Sulzmann, J.-N.: Pairwise Naive Bayes classifier. In: Althoff, K.-D., Schaaf, M. (eds.) Proceedings of the LWA 2006, Lernen Wissensentdeckung Adaptivität, Hildesheim, Germany, pp. 356–363. Gesellschaft für Informatik e. V (GI) (2006)
Wu, T.-F., Lin, C.-J., Weng, R.: Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research (JMLR) 5, 975–1005 (2004)
Zadrozny, B., Elkan, C.: Transforming classifier scores into accurate multiclass probability estimates. In: Proceedings of the 8th International Conference on Knowledge Discovery and Data Mining (KDD-2002), pp. 694–699 (2002)
Zhang, H., Su, J.: Naive Bayesian Classifiers for Ranking. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 501–512. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sulzmann, JN., Fürnkranz, J., Hüllermeier, E. (2007). On Pairwise Naive Bayes Classifiers. In: Kok, J.N., Koronacki, J., Mantaras, R.L.d., Matwin, S., Mladenič, D., Skowron, A. (eds) Machine Learning: ECML 2007. ECML 2007. Lecture Notes in Computer Science(), vol 4701. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74958-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-540-74958-5_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74957-8
Online ISBN: 978-3-540-74958-5
eBook Packages: Computer ScienceComputer Science (R0)