Progress in Artificial Intelligence

, Volume 5, Issue 1, pp 15–26 | Cite as

Learning Bayesian networks with low inference complexity

Regular Paper

Abstract

One of the main research topics in machine learning nowadays is the improvement of the inference and learning processes in probabilistic graphical models. Traditionally, inference and learning have been treated separately, but given that the structure of the model conditions the inference complexity, most learning methods will sometimes produce inefficient inference models. In this paper we propose a framework for learning low inference complexity Bayesian networks. For that, we use a representation of the network factorization that allows efficiently evaluating an upper bound in the inference complexity of each model during the learning process. Experimental results show that the proposed methods obtain tractable models that improve the accuracy of the predictions provided by approximate inference in models obtained with a well-known Bayesian network learner.

Keywords

Probabilistic graphical models Bayesian networks Arithmetic circuits Network polynomials Structure learning  Thin models 

References

  1. 1.
    Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974)MATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    Andreassen, S., Rosenfalck, A., Falck, B., Olesen, K.G., Andersen, S.K.: Evaluation of the diagnostic performance of the expert EMG assistant MUNIN. Electromyogr. Mot. Control 101(2), 129–144 (1996)CrossRefGoogle Scholar
  3. 3.
    Arnborg, S., Corneil, D.G., Proskurowski, A.: Complexity of finding embeddings in a \(k\)-tree. SIAM J. Algebraic Discret. 8(2), 277–284 (1987)MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Bach, F.R., Jordan, M.I.: Thin junction trees. In: Adv. Neural Inf., pp. 569–576 (2001)Google Scholar
  5. 5.
    Beygelzimer, A., Rish, I.: Approximability of probability distributions. In: Adv. Neural Inf. pp. 377–384 (2004)Google Scholar
  6. 6.
    Bielza, C., Li, G., Larranaga, P.: Multi-dimensional classification with Bayesian networks. Int. J. Approx. Reason. 52(6), 705–727 (2011)MATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    Bielza, C., Larranaga, P.: Discrete Bayesian network classifiers: a survey. ACM Comput. Surv. 47(1), 5 (2014)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Bodlaender, H.L.: A linear time algorithm for finding tree-decompositions of small treewidth. In: Proceedings of the Twenty-Fifth Annual ACM Symposium on Theory of Computing, pp. 226–234 (1993)Google Scholar
  9. 9.
    Bodlaender, H.L., Koster, A.M.: Treewidth computations I. Upper bounds. Inf. Comput. 208(3), 259–275 (2010)MATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    Bouckaert, R.R.: Probabilistic network construction using the minimum description length principle. In: Lect. Notes Artif. Int., pp. 41–48 (1993)Google Scholar
  11. 11.
    Chechetka, A., Guestrin, C.: Efficient principled learning of thin junction trees. In: Adv. Neural Inf., pp. 273–280 (2008)Google Scholar
  12. 12.
    Cooper, G.F.: The computational complexity of probabilistic inference using Bayesian belief networks. Artif. Intell. 42(2), 393–405 (1990)MATHCrossRefGoogle Scholar
  13. 13.
    Cooper, G.F., Herskovits, E.: A Bayesian method for constructing Bayesian belief networks from databases. In: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, pp. 86–94 (1991)Google Scholar
  14. 14.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)MATHGoogle Scholar
  15. 15.
    Dagum, P., Luby, M.: Approximating probabilistic inference in Bayesian belief networks is NP-hard. Artif. Intell. 60(1), 141–153 (1993)MATHMathSciNetCrossRefGoogle Scholar
  16. 16.
    Darwiche, A.: A differential approach to inference in Bayesian networks. J. Assoc. Comput. Mach. 50(3), 280–305 (2003)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Elidan, G., Gould, S.: Learning bounded treewidth Bayesian networks. In: Adv. Neural Inf., pp. 417–424 (2009)Google Scholar
  18. 18.
    Fung, R.M., Chang, K.C.: Weighing and integrating evidence for stochastic simulation in Bayesian networks. In: Uncertainty in Artificial Intelligence, pp. 209–220 (1989)Google Scholar
  19. 19.
    Gámez, J.A., Mateo, J.L., Puerta, J.M.: Learning Bayesian networks by hill climbing: effficient methods based on progressive restriction of the neighborhood. Data Min. Knowl. Discov. 22(1–2), 106–148 (2011)MATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)MATHGoogle Scholar
  21. 21.
    Heckerman, D., Horwitz, E., Nathwani, B.: Towards normative expert systems: part I. The pathfinder project. Methods Inf. Med. 31, 90–105 (1992)Google Scholar
  22. 22.
    Kim, J., Pearl, J.: A computational model for causal and diagnostic reasoning in inference systems. In: Proceedings of the Eighth International Joint Conference on Artificial Intelligence, pp. 190–193 (1983)Google Scholar
  23. 23.
    Kwisthout, J.: Most probable explanations in Bayesian networks: complexity and tractability. Int. J. Approx. Reason. 52(9), 1452–1469 (2011)MATHMathSciNetCrossRefGoogle Scholar
  24. 24.
    Lam, W., Bacchus, F.: Learning Bayesian belief networks: an approach based on the MDL principle. Comput. Intell. 10(3), 269–293 (1994)CrossRefGoogle Scholar
  25. 25.
    Larranaga, P., Kuijpers, C.M., Murga, R.H., Yurramendi, Y.: Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Trans. Syst. Man Cybern. 26(4), 487–493 (1996)CrossRefGoogle Scholar
  26. 26.
    Larrañaga, P., Poza, M., Yurramendi, Y., Murga, R.H., Kuijpers, C.M.: Structure learning of Bayesian networks by genetic algorithms: a performance analysis of control parameters. IEEE Trans. Pattern Anal. 18(9), 912–926 (1996)CrossRefGoogle Scholar
  27. 27.
    Lowd, D., Domingos, P.: Learning arithmetic circuits. In: Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence, pp. 383–392 (2008)Google Scholar
  28. 28.
    Pham, D.T., Ruz, G.A.: Unsupervised training of Bayesian networks for data clustering. Proc. Roy. Soc. Lond. A Mat., pp. 2927–2948 (2009)Google Scholar
  29. 29.
    Shachter, R.D., Peot, M.A.: Simulation approaches to general probabilistic inference on belief networks. In: Uncertainty in Artificial Intelligence, pp. 221–234 (1989)Google Scholar
  30. 30.
    Shahaf, D., Guestrin, C.: Learning thin junction trees via graph cuts. In: International Conference on Artificial Intelligence and Statistics, pp. 113–120 (2009)Google Scholar
  31. 31.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Marco Benjumeda
    • 1
  • Pedro Larrañaga
    • 1
  • Concha Bielza
    • 1
  1. 1.Computational Intelligence Group, Departamento de Inteligencia ArtificialUniversidad Politécnica de MadridMadridSpain

Personalised recommendations