A Bayesian Network Based Approach for Data Classification Using Structural Learning
This paper describes the theory and implementation of Bayesian networks in the context of data classification. Bayesian networks provide a very general and yet effective graphical language for factoring joint probability distributions which in turn make them very popular for classification. Finding the optimal structure of Bayesian networks from data has been shown to be NP-hard. In this paper score-based algorithms such as K2, Hill Climbing, Iterative Hill Climbing and simulated annealing have been developed to provide more efficient structure learning through more investigation on MDL, BIC and AIC scores borrowed from information theory. Our experimental results show that the BIC score is the best one though it is very time consuming. Bayesian naive classifier is the simplest Bayesian network with known structure for data classification. For the purpose of comparison, we considered several cases and applied general Bayesian networks along with this classifier to these cases. The simulation results approved that using structural learning in order to find Bayesian networks structure improves the classification accuracy. Indeed it was shown that the Iterative Hill Climbing is the most appropriate search algorithm and K2 is the simplest one with the least time complexity.
KeywordsBayesian Networks Data Classification Machine learning Structural learning
Unable to display preview. Download preview PDF.
- 2.Heckerman, D.: A tutorial on learning with Bayesian networks, Microsoft Technical Report 95-06 (1996)Google Scholar
- 3.Gyftodimos, E., Flach, P.: Hierarchical Bayesian networks: an approach to classification and learning for structured data, Methods and Applications of Artificial Intelligence. In: Third Hellenic Conference on AI, SETN 2004, Samos, Greece (2004)Google Scholar
- 4.Friedman, N., Murphy, K., Russell, S.: Learning the Structure of Dynamic Probabilistic Networks. In: Proceedings of the 14th Conference on Uncertainty in AI, pp. 139–147 (1998)Google Scholar
- 5.Murphy, K.: Dynamic bayesian networks: representation, inference and learning Ph. D. thesis, University of California, Berkeley (2002)Google Scholar
- 9.Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pp. 252–262 (1996)Google Scholar
- 10.Heckerman, D.: A tutorial on learning with Bayesian networks. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 301–354. MIT Press, Cambridge (1999)Google Scholar
- 13.Cooper, G., Herskovitz, E.: A Bayesian Method for Constructing Bayesian Belief Networks from Databases. In: Proceedings of the 7th Conference on Uncertainty in AI, pp. 86–94 (1991)Google Scholar