Learning Hybrid Bayesian Networks by MML

  • Rodney T. O’Donnell
  • Lloyd Allison
  • Kevin B. Korb
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4304)


We use a Markov Chain Monte Carlo (MCMC) MML algorithm to learn hybrid Bayesian networks from observational data. Hybrid networks represent local structure, using conditional probability tables (CPT), logit models, decision trees or hybrid models, i.e., combinations of the three. We compare this method with alternative local structure learning algorithms using the MDL and BDe metrics. Results are presented for both real and artificial data sets. Hybrid models compare favourably to other local structure learners, allowing simple representations given limited data combined with richer representations given massive data.


Logit Model Markov Chain Monte Carlo Bayesian Network Local Structure Hybrid Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Friedman, N., Goldszmidt, M.: Learning Bayesian networks with local structure. In: Uncertainty in Artificial Intelligence (1996)Google Scholar
  2. 2.
    Neil, J.R., Wallace, C.S., Korb, K.B.: Learning Bayesian networks with restricted causal interactions. In: Uncertainty in Artificial Intelligence (1999)Google Scholar
  3. 3.
    Wallace, C.S., Korb, K.B.: Learning linear causal models by MML samplling. In: Gammerman, A. (ed.) Causal Models and Intelligent Data Management, Springer, Heidelberg (1999)Google Scholar
  4. 4.
    Korb, K., Nicholson, A.: Bayesian Artificial Intelligence. CRC Press, Boca Raton (2003)CrossRefGoogle Scholar
  5. 5.
    O’Donnell, R.T., Nicholson, A.E., Han, B., Korb, K.B., Alam, M.J., Hope, L.R.: Causal discovery with prior information. In: 19th Australian Joint Conf. on AI (2006)Google Scholar
  6. 6.
    Heckerman, D., Geiger, D., Chickering, D.: Learning bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)MATHGoogle Scholar
  7. 7.
    Lam, W., Bacchus, F.: Learning Bayesian belief networks. Computational Intelligence 10 (1994)Google Scholar
  8. 8.
    Wallace, C., Patrick, J.: Coding decision trees. Machine Learning 11, 7 (1993)MATHCrossRefGoogle Scholar
  9. 9.
    Bayes, T.: An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Soc. of London (1764/1958) reprinted in Biometrika 45(3/4), 293–315 (1958)Google Scholar
  10. 10.
    Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27(3), 379–423 (1948)MATHMathSciNetGoogle Scholar
  11. 11.
    Farr, G.E., Wallace, C.S.: The complexity of strict minimum message length inference. Computer Journal 45(3), 285–292 (2002)MATHCrossRefGoogle Scholar
  12. 12.
    Wallace, C.S., Boulton, D.M.: An information measure for classification. The Computer Journal 11, 185–194 (1968)MATHGoogle Scholar
  13. 13.
    Wallace, C.S.: Statistical and Inductive Inference by Minimum Message Length. Springer, Berlin (2005)MATHGoogle Scholar
  14. 14.
    Allison, L.: Models for machine learning and data mining in functional programming. Journal of Functional Programming (2005)Google Scholar
  15. 15.
    Rissanen, J.: Modeling by shortest data description. Automatica 14 (1978)Google Scholar
  16. 16.
    Baxter, R., Oliver, J.: MDL and MML: similarities and differences. Technical Report 207, Dept of Computer Science, Monash University (1994)Google Scholar
  17. 17.
    Chickering, D.: A transformational characterization of equivalent Bayesian network structures. In: Uncertainty in Artificial Intelligence (1995)Google Scholar
  18. 18.
    Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 309–347 (1992)MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Rodney T. O’Donnell
    • 1
  • Lloyd Allison
    • 1
  • Kevin B. Korb
    • 1
  1. 1.School of Information TechnologyMonash UniversityClaytonAustralia

Personalised recommendations