Definition
Semi-naive Bayesian learning refers to a field of Supervised Classification that seeks to enhance the classification and conditional probability estimation accuracy of naive Bayes by relaxing its attribute independence assumption.
Motivation and Background
The assumption underlying naive Bayes is that attributes are independent of each other, given the class. This is an unrealistic assumption for many applications. Violations of this assumption can render naive Bayes’ classification suboptimal. There have been many attempts to improve the classification accuracy and probability estimation of naive Bayes by relaxing the attribute independence assumption while at the same time retaining much of its simplicity and efficiency.
Taxonomy of Semi-Naive Bayesian Techniques
Semi-naive Bayesian methods can be roughly subdivided into five high-level strategies for relaxing the independence assumption.
The first strategy forms an attribute subset by deleting attributes to remove harmful...
This is a preview of subscription content, log in via an institution.
Recommended Reading
Ayer, M., Brunk, H. D., Ewing, G. M., Reid, W. T., & Silverman, E. (1955). An empirical distribution function for sampling with incomplete information. The Annals of Mathematical Statistics, 26(4), 641–647.
Brain, D., & Webb, G. I. (2002). The need for low bias algorithms in classification learning from large data sets. In Proceedings of the Sixteenth European Conference on Principles of Data Mining and Knowledge Discovery (pp. 62–73). Berlin: Springer-Verlag.
Cerquides, J., & Mántaras, R. L. D. (2005). Robust Bayesian linear classifier ensembles. In Proceedings of the Sixteenth European Conference on Machine Learning, pp. 70–81.
Frank, E., Hall, M., & Pfahringer, B. (2003). Locally weighted naive Bayes. In Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, Acapulco, Mexico (pp. 249–256). San Francisco, CA: Morgan Kaufmann.
Friedman, N., Geiger, D., & Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning, 29(2), 131–163.
Gama, J. (2003). Iterative Bayes. Theoretical Computer Science, 292(2), 417–430.
Keogh, E. J., & Pazzani, M. J. (1999). Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, pp. 225–230.
Kittler, J., (1986). Feature selection and extraction. In T. Y. Young & K. S. Fu (Eds.), Handbook of Pattern Recognition and Image Processing. New York: Academic Press.
Kohavi, R. (1996). Scaling up the accuracy of naive-Bayes classifiers: A decisiontree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 202–207.
Pazzani, M. J. (1996). Constructive induction of Cartesian product attributes. In ISIS: Information. Statistics and Induction in Science, Melbourne, Australia, (pp. 66–77). Singapore: World Scientific.
Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 465–471.
Sahami, M. (1996). Learning limited dependence Bayesian classifiers. In Proceedings of the Second International Conference on Knowledge Discovery in Databases (pp. 334–338) Menlo Park: AAAI Press.
Webb, G. I., & Pazzani, M. J. (1998). Adjusted probability naive Bayesian induction. In Proceedings of the Eleventh Australian Joint Conference on Artificial Intelligence, Sydney, Australia (pp. 285–295). Berlin: Springer.
Webb, G. I., Boughton, J., & Wang, Z. (2005). Not so naive Bayes: Aggregating onedependence estimators. Machine Learning, 58(1), 5–24.
Zadrozny, B., & Elkan, C. (2002). Transforming classifier scores into accurate multiclass probability estimates. In Proceedings of the Eighth International Conference on Knowledge Discovery and Data Mining, Edmonton, Alberta, Canada (pp. 694–699). New York: ACM Press.
Zhang, N. L., Nielsen, T. D., & Jensen, F. V. (2004). Latent variable discovery in classification models. Artificial Intelligence in Medicine, 30(3), 283–299.
Zheng, Z., & Webb, G. I. (2000). Lazy learning of Bayesian rules. Machine Learning, 41(1), 53–84.
Zheng, F., & Webb, G. I. (2005). A comparative study of semi-naive Bayes methods in classification learning. In Proceedings of the Fourth Australasian Data Mining Conference, Sydney, pp. 141–156.
Zheng, F., & Webb, G. I. (2006). Efficient lazy elimination for averaged-one dependence estimators. In Proceedings of the Twenty-third International Conference on Machine Learning (pp. 1113–1120). New York: ACM Press.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this entry
Cite this entry
Zheng, F., Webb, G.I. (2011). Semi-Naive Bayesian Learning. In: Sammut, C., Webb, G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-30164-8_748
Download citation
DOI: https://doi.org/10.1007/978-0-387-30164-8_748
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-30768-8
Online ISBN: 978-0-387-30164-8
eBook Packages: Computer ScienceReference Module Computer Science and Engineering