A Bayesian Network Based Approach for Data Classification Using Structural Learning

  • A. R. Khanteymoori
  • M. M. Homayounpour
  • M. B. Menhaj
Part of the Communications in Computer and Information Science book series (CCIS, volume 6)

Abstract

This paper describes the theory and implementation of Bayesian networks in the context of data classification. Bayesian networks provide a very general and yet effective graphical language for factoring joint probability distributions which in turn make them very popular for classification. Finding the optimal structure of Bayesian networks from data has been shown to be NP-hard. In this paper score-based algorithms such as K2, Hill Climbing, Iterative Hill Climbing and simulated annealing have been developed to provide more efficient structure learning through more investigation on MDL, BIC and AIC scores borrowed from information theory. Our experimental results show that the BIC score is the best one though it is very time consuming. Bayesian naive classifier is the simplest Bayesian network with known structure for data classification. For the purpose of comparison, we considered several cases and applied general Bayesian networks along with this classifier to these cases. The simulation results approved that using structural learning in order to find Bayesian networks structure improves the classification accuracy. Indeed it was shown that the Iterative Hill Climbing is the most appropriate search algorithm and K2 is the simplest one with the least time complexity.

Keywords

Bayesian Networks Data Classification Machine learning Structural learning 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, LosAlios (1988)MATHGoogle Scholar
  2. 2.
    Heckerman, D.: A tutorial on learning with Bayesian networks, Microsoft Technical Report 95-06 (1996)Google Scholar
  3. 3.
    Gyftodimos, E., Flach, P.: Hierarchical Bayesian networks: an approach to classification and learning for structured data, Methods and Applications of Artificial Intelligence. In: Third Hellenic Conference on AI, SETN 2004, Samos, Greece (2004)Google Scholar
  4. 4.
    Friedman, N., Murphy, K., Russell, S.: Learning the Structure of Dynamic Probabilistic Networks. In: Proceedings of the 14th Conference on Uncertainty in AI, pp. 139–147 (1998)Google Scholar
  5. 5.
    Murphy, K.: Dynamic bayesian networks: representation, inference and learning Ph. D. thesis, University of California, Berkeley (2002)Google Scholar
  6. 6.
    Russell, S., Norvig, P.: Artificial intelligence, a modern approach. Prentice Hall, New York (2003)MATHGoogle Scholar
  7. 7.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)MATHGoogle Scholar
  8. 8.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 2, 131–163 (1997)CrossRefMATHGoogle Scholar
  9. 9.
    Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, pp. 252–262 (1996)Google Scholar
  10. 10.
    Heckerman, D.: A tutorial on learning with Bayesian networks. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 301–354. MIT Press, Cambridge (1999)Google Scholar
  11. 11.
    Lam, W., Bachus, F.: Learning Bayesian Networks. An approach based on the MDL principal. Computational Intelligence 10(3), 269–293 (1994)CrossRefGoogle Scholar
  12. 12.
    Cooper, G.: Computational complexity of probabilistic inference using Bayesian belief networks (Research Note). Artificial Intelligence 42, 393–405 (1990)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Cooper, G., Herskovitz, E.: A Bayesian Method for Constructing Bayesian Belief Networks from Databases. In: Proceedings of the 7th Conference on Uncertainty in AI, pp. 86–94 (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • A. R. Khanteymoori
    • 1
  • M. M. Homayounpour
    • 2
  • M. B. Menhaj
    • 3
  1. 1.PhD Candidate, Computer Engineering DepartmentAmirKabir UniversityTehranIran
  2. 2.Assistant Professors, Computer Engineering DepartmentAmirKabir UniversityTehranIran
  3. 3.Professor, Electrical Engineering DepartmentAmirKabir UniversityTehranIran

Personalised recommendations