MCMC Learning of Bayesian Network Models by Markov Blanket Decomposition

  • Carsten Riggelsen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


We propose a Bayesian method for learning Bayesian network models using Markov chain Monte Carlo (MCMC). In contrast to most existing MCMC approaches that define components in term of single edges, our approach is to decompose a Bayesian network model in larger dependence components defined by Markov blankets. The idea is based on the fact that MCMC performs significantly better when choosing the right decomposition, and that edges in the Markov blanket of the vertices form a natural dependence relationship. Using the ALARM and Insurance networks, we show that this decomposition allows MCMC to mix more rapidly, and is less prone to getting stuck in local maxima compared to the single edge approach.


  1. 1.
    Beinlich, I.A., Suermondt, H.J., Chavez, R.M., Cooper, G.F.: The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. In: Proc. of the European Conf. on AI in Medicine (1989)Google Scholar
  2. 2.
    Binder, J., Koller, D., Russell, S.J., Kanazawa, K.: Adaptive probabilistic networks with hidden variables. Machine Learning 29, 213–244 (1997)zbMATHCrossRefGoogle Scholar
  3. 3.
    Castelo, R., Kocka, T.: On inclusion-driven learning of Bayesian networks. J. of Machine Learning Research 4, 527–574 (2003)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Friedman, N., Koller, D.: Being Bayesian about network structure. A Bayesian approach to structure discovery in Bayesian networks. Machine Learning 50(1–2), 95–125 (2003)zbMATHCrossRefGoogle Scholar
  5. 5.
    Giudici, P., Castelo, R.: Improving Markov chain Monte Carlo model search for data mining. Machine Learning 50(1), 127–158 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Giudici, P., Green, P.: Decomposable graphical gaussian model determination. Biometrika 86(4), 785–801 (1999)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Green, P.: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732 (1998)CrossRefGoogle Scholar
  8. 8.
    Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)zbMATHGoogle Scholar
  9. 9.
    Kocka, T., Castelo, R.: Improved learning of Bayesian networks. In: Koller, D., Breese, J. (eds.) Proc. of the Conf. on Uncertainty in AI, pp. 269–276 (2001)Google Scholar
  10. 10.
    Madigan, D., Raftery, A.: Model selection and accounting for model uncertainty in graphical models using Occam’s window. J. of the Am. Stat. Assoc. 89, 1535–1546 (1994)zbMATHCrossRefGoogle Scholar
  11. 11.
    Madigan, D., York, J.: Bayesian graphical models for discrete data. Intl. Statistical Review 63, 215–232 (1995)zbMATHCrossRefGoogle Scholar
  12. 12.
    Robert, C.P., Casella, G.: Monte Carlo statistical methods, 3rd edn. Springer, Heidelberg (2002)Google Scholar
  13. 13.
    Spiegelhalter, D.J., Lauritzen, S.L.: Sequential updating of conditional probabilities on directed graphical structures. Networks 20, 579–605 (1990)zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Carsten Riggelsen
    • 1
  1. 1.Institute of Information & Computing SciencesUtrecht UniversityUtrechtThe Netherlands

Personalised recommendations