Abstract
Naïve Bayes learners are widely used, efficient, and effective supervised learning methods for labeled datasets in noisy environments. It has been shown that naïve Bayes learners produce reasonable performance compared with other machine learning algorithms. However, the conditional independence assumption of naïve Bayes learning imposes restrictions on the handling of real-world data. To relax the independence assumption, we propose a smooth kernel to augment weights for the likelihood estimation. We then select an attribute weighting method that uses the mutual information metric to cooperate with the proposed framework. A series of experiments are conducted on 17 UCI benchmark datasets to compare the accuracy of the proposed learner against that of other methods that employ a relaxed conditional independence assumption. The results demonstrate the effectiveness and efficiency of our proposed learning algorithm. The overall results also indicate the superiority of attribute-weighting methods over those that attempt to determine the structure of the network.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aitchison J, Aitken CGG (1976) Multivariate binary discrimination by the kernel method. Biometrika 63(3):413–420
Bache K, Lichman M (2013) UCI Machine Learning Repository. http://archive.ics.uci.edu/ml
Chen L, Wang S (2012a) Automated feature weighting in naïve Bayes for high-dimensional data classification. In: Proceedings of the 21st acm international conference on information and knowledge management. Cikm ’12. ISBN 978-1-4503-1156-4. ACM, New York, NY, USA, pp 1243–1252
Chen L, Wang S (2012b) Semi-naïve Bayesian classification by weighted kernel density estimation. In: Proceedings of the 8th international conference on advanced data mining and applications (adma 2012). Springer, Nanjing, China, pp 260–270
Cooper G, Herskovits E (1992) A bayesian method for the induction of probabilistic networks from data. Mach Learn 9(4):309–347
Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29(2-3):131–163
Jiang L, Zhang H, Cai Z (2009) A novel Bayes model: Hidden naive Bayes. IEEE Trans Knowl Data Eng 21(10):1361– 1371
Kang D-K, Kim M-J (2011) Propositionalized attribute taxonomies from data for data-driven construction of concise classifiers. Expert Systems with Applications 38(10):12739– 12746
Kang D-K, Sohn K (2009) Learning decision trees with taxonomy of propositionalized attributes. Pattern Recogn 42(1):84–92
Lee C-H, Gutierrez F, Dou D (2011) Calculating feature weights in naïve Bayes with Kullback-Leibler measure. In: Proceedings of the 2011 ieee 11th international conference on data mining. Icdm ’11. ISBN 978-0-7695-4408-3. IEEE Computer Society, Washington, pp 1146–1151
Lewis D D (1998) Naïve (Bayes) at forty: The independence assumption in information retrieval. In: Proceedings of the 10th european conference on machine learning. Ecml ’98. ISBN 3-540-64417-2. Springer, London, pp 4–15
Li Q, Racine JS (2006) Nonparametric econometrics: Theory and practice. Princeton University Press
Omura K, Kudo M, Endo T, Murai T (2012) Weighted naïve Bayes classifier on categorical features. In: Abraham A, Zomaya AY, Ventura S, Yager RR, Snásel V, Muda AK, Samuel P (eds) Proceedings of the 12th international conference on intelligent systems design and applications (isda 2012). ISBN 978-1-4673-5117-1. IEEE, India, pp 865–870
Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc, San Francisco
Webb GI, Boughton JR, Wang Z (2005) Not so naïve Bayes: Aggregating one-dependence estimators. Mach Learn 58(1):5– 24
Wong T-T (2012) A hybrid discretization method for naïve Bayesian classifiers. Pattern Recogn 45 (6):2321–2325
Wu J, Cai Z (2011) Learning averaged one-dependence estimators by attribute weighting. Journal of Information & Computational Science 8(7):1063–1073
Zaidi NA, Cerquides J, Carman MJ, Webb GI (2013) Alleviating naïve Bayes attribute independence assumption by attribute weighting. J Mach Learn Res 14:1947–1988
Acknowledgments
This work was supported by Dongseo University, “Dongseo Frontier Project” Research Fund of 2012.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xiang, ZL., Yu, XR. & Kang, DK. Experimental analysis of naïve Bayes classifier based on an attribute weighting framework with smooth kernel density estimations. Appl Intell 44, 611–620 (2016). https://doi.org/10.1007/s10489-015-0719-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-015-0719-1