Abstract
Naïve Bayes, the simplest Bayesian classifier, has shown excellent performance given its unrealistic independence assumption. This paper studies the selective neighborhood-based naïve Bayes (SNNB) for lazy classification, and develops three variant algorithms, SNNB-G, SNNB-L, and SNNB-LV, all with linear computational complexity. The SNNB algorithms use local learning strategy for alleviating the independence assumption. The underlying idea is, for a test example, first to construct multiple classifiers on its multiple neighborhoods with different radius, and then to select out the classifier with the highest estimated accuracy to make decision. Empirical results show that both SNNB-L and SNNB-LV generate more accurate classifiers than naïve Bayes and several other state-of-the-art classification algorithms including C4.5, Naïve Bayes Tree, and Lazy Bayesian Rule. The SNNB-L and SNNB-LV algorithms are also computationally more efficient than the Lazy Bayesian Rule algorithm, especially on the domains with high dimensionality.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aha, D.W.: Lazy learning. Kluwer Academic Publishers, Dordrecht (1997)
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36, 105–142 (1999)
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases. University of California, Irvine, CA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)
Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10, 1895–1924 (1998)
Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. John Wiley, New York (1973)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pp. 1022–1027. Morgan Kaufmann, San Francisco (1993)
Frank, E., Hall, M., Pfahringer, B.: Locally weighted naive Bayes. In: Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence, pp. 249–256. Morgan Kaufmann, San Francisco (2003)
Friedman, N., Goldszmidt, M.: Building classifiers using Bayesian networks. In: Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 1277–1284. AAAI Press/ MIT Press (1996)
Kohavi, R.: Scaling up the accuracy of naïve-Bayes classifiers: a decision-tree hybrid. In: Proceedings of the Second International Conference on Knowledge Discovery & Data Mining, pp. 202–207. AAAI Press/MIT press, Cambridge/Menlo Park (1996)
Kononenko, I.: Semi-naïve Bayesian classifier. In: Proceedings of the Sixth European Working Session on Learning, pp. 206–219. Springer, Berlin (1991)
Langley, P., Sage, S.: Induction of selective Bayesian classifiers. In: Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pp. 339–406. Morgan Kaufmann, San Francisco (1994)
Pazzani, M.: Constructive induction of Cartesian product attributes. In: Proceedings of the Conference ISIS 1996: Information, Statistics and Induction in Science, pp. 66–77. World Scientific, Singapore (1996)
Quinlan, J.R.: C4.5: Programs for machine learning. Morgan Kaufmann, San Francisco (1993)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)
Xie, Z., Hsu, W., Liu, Z., Lee, M.-L.: SNNB: a selective neighborhood-based naïve Bayes for lazy classification. In: Chen, M.-S., Yu, P.S., Liu, B. (eds.) PAKDD 2002. LNCS (LNAI), vol. 2336, pp. 104–114. Springer, Heidelberg (2002)
Zheng, Z.: Naïve Bayesian classifier committees. In: Proceedings of the Tenth European Conference on Machine Learning, pp. 196–207. Springer, Berlin (1998)
Zheng, Z., Webb, G.I.: Lazy learning of Bayesian rules. Machine Learning 41, 53–84 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Xie, Z., Zhang, Q., Hsu, W., Lee, M.L. (2005). Enhancing SNNB with Local Accuracy Estimation and Ensemble Techniques. In: Zhou, L., Ooi, B.C., Meng, X. (eds) Database Systems for Advanced Applications. DASFAA 2005. Lecture Notes in Computer Science, vol 3453. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11408079_46
Download citation
DOI: https://doi.org/10.1007/11408079_46
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25334-1
Online ISBN: 978-3-540-32005-0
eBook Packages: Computer ScienceComputer Science (R0)