Skip to main content
SpringerLink
Log in

Empirical Behavior of Bayesian Network Structure Learning Algorithms

  • Conference paper
  • First Online:
Advanced Methodologies for Bayesian Networks (AMBN 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9505))

Included in the following conference series:

Abstract

Bayesian network structure learning (BNSL) is the problem of finding a BN structure which best explains a dataset. Score-based learning assigns a score to each network structure. The goal is to find the structure which optimizes the score. We review two recent studies of empirical behavior of BNSL algorithms.

The score typically reflects fit to a training dataset; however, models which fit training data well may generalize poorly. Thus, it is not clear that finding an optimal network is worthwhile. We review a comparison of exact and approximate search techniques. Sometimes, approximate algorithms suffice; for complex datasets, the optimal algorithms produce better networks.

BNSL is known to be NP-hard, so exact solvers prune the search space using heuristics. We next review problem-dependent characteristics which affect their efficacy. Empirical results show that machine learning techniques based on these characteristics can often be used to accurately predict the algorithms’ running times.

B. Malone—This paper is based on Malone et al. (2014, 2015), with co-authors Matti Järvisalo, Petri Myllymäki, Kusta Kangas and Mikko Koiviso from HIIT and the Department of Computer Science at the University of Helsinki.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    This work assumes s is score-equivalent (Heckerman et al. 1995).

  2. 2.

    The datasets are available at http://bnportfolio.cs.helsinki.fi/.

  3. 3.

    Our results were not very sensitive to the scoring function, except its effect on the number of CPSs, so our results generalize to other decomposable scores.

References

  • Bartlett, M., Cussens, J.: Integer linear programming for the Bayesian network structure learning problem. In: Artificial Intelligence (2015)

    Google Scholar 

  • Chickering, D.M.: Learning Bayesian networks is NP-complete. In: Fisher, D., Lenz, H.-J. (eds.) Learning from Data: Artificial Intelligence and Statistics V. Lecture Notes in Statistics, vol. 112, pp. 121–130. Springer, New York (1996)

    Chapter  Google Scholar 

  • Chickering, D.M.: Learning equivalence classes of Bayesian-network structures. J. Mach. Learn. Res. 2, 445–498 (2002)

    MathSciNet  MATH  Google Scholar 

  • Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory 14(3), 462–467 (1968)

    Article  MATH  Google Scholar 

  • Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9, 309–347 (1992)

    Article  MATH  Google Scholar 

  • Cussens, J., Bartlett, M., Jones, E.M., Sheehan, N.A.: Maximum likelihood pedigree reconstruction using integer linear programming. Genet. Epidemiol. 37(1), 69–83 (2013)

    Article  Google Scholar 

  • de Campos, C.P., Ji, Q.: Efficient learning of Bayesian networks using constraints. J. Mach. Learn. Res. 12, 663–689 (2011)

    MathSciNet  MATH  Google Scholar 

  • de Campos, L.M., Huete, J.F.: A new approach for learning belief networks using independence criteria. Int. J. Approximate Reasoning 24(1), 11–37 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Fan, X., Yuan, C., Malone, B.: Tightening bounds for Bayesian network structure learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence (2014)

    Google Scholar 

  • Friedman, N., Nachman, I., Peer, D.: Learning Bayesian network structure from massive datasets: the “sparse candidate” algorithm. In: Proceedings 13th Conference on Uncertainty in Artificial Intelligence (1999)

    Google Scholar 

  • Glover, F.: Tabu search: a tutorial. Interfaces 20(4), 74–94 (1990)

    Article  Google Scholar 

  • Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20, 197–243 (1995)

    Article  MATH  Google Scholar 

  • Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5, 549–573 (2004)

    MathSciNet  MATH  Google Scholar 

  • Leyton-Brown, K., Nudelman, E., Shoham, Y.: Learning the empirical hardness of optimization problems: the case of combinatorial auctions. In: Hentenryck, P. (ed.) CP 2002. LNCS, vol. 2470, pp. 556–572. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  • Malone, B., Järvisalo, M., Myllymäki, P.: Impact of learning strategies on the qualpacking Bayesian networks: an empirical evaluation. In: Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence (2015)

    Google Scholar 

  • Malone, B., Kangas, K., Järvisalo, M., Koivisto, M., Myllymäki, P.: Predicting the hardness of learning Bayesian networks. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence (2014)

    Google Scholar 

  • Malone, B., Yuan, C.: Evaluating anytime algorithms for learning optimal Bayesian networks. In: Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence (2013)

    Google Scholar 

  • Mitchell, T.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  • Moore, A., Wong, W.-K.: Optimal reinsertion: a new search operator for accelerated and more accurate Bayesian network structure learning. In: Proceedings of the 20th International Conference on Machine Learning, pp. 552–559 (2003)

    Google Scholar 

  • Ott, S., Imoto, S., Miyano, S.: Finding optimal models for small gene networks. In: Proceedings of the Pacific Symposium on Biocomputing (2004)

    Google Scholar 

  • Parviainen, P., Koivisto, M.: Exact structure discovery in Bayesian networks with less space. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, Quebec, Canada. AUAI Press (2009)

    Google Scholar 

  • Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Mateo (1988)

    MATH  Google Scholar 

  • Quinlan, J.R.: Simplifying decision trees. Int. J. Man Mach. Stud. 27, 221–234 (1987)

    Article  Google Scholar 

  • Rice, J.R.: The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)

    Article  Google Scholar 

  • Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education, Upper Saddle River (2003)

    MATH  Google Scholar 

  • Silander, T., Myllymäki, P.: A simple approach for finding the globally optimal Bayesian network structure. In: Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (2006)

    Google Scholar 

  • Silander, T., Roos, T., Kontkanen, P., Myllymäki, P.: Factorized normalized maximum likelihood criterion for learning Bayesian network structures. In: Proceedings of the 4th European Workshop on Probabilistic Graphical Models (2008)

    Google Scholar 

  • Spirtes, P., Glymour, C., Schemes, R.: Causation, Prediction, and Search, 2nd edn. MIT Press, Cambridge (2000)

    MATH  Google Scholar 

  • Suzuki, J.: Learning Bayesian belief networks based on the MDL principle: an efficient algorithm using the branch and bound technique. IEICE Trans. Inf. Syst. E82–D(2), 356–367 (1999)

    Google Scholar 

  • Teyssier, M., Koller, D.: Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (2005)

    Google Scholar 

  • Tian, J.: A branch-and-bound algorithm for MDL learning Bayesian networks. In: Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (2000)

    Google Scholar 

  • Tsamardinos, I., Brown, L., Aliferis, C.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65, 31–78 (2006)

    Article  MATH  Google Scholar 

  • van Beek, P., Hoffmann, H.-F.: Machine learning of Bayesian networks using constraint programming. In: Pesant, G. (ed.) CP 2015. LNCS, vol. 9255, pp. 429–445. Springer, Heidelberg (2015)

    Chapter  Google Scholar 

  • Yuan, C., Malone, B.: Learning optimal Bayesian networks: a shortest path perspective. J. Artif. Intell. Res. 48, 23–65 (2013)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Max Planck Institute for the Biology of Ageing, Cologne, Germany

    Brandon Malone

Authors
  1. Brandon Malone
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brandon Malone .

Editor information

Editors and Affiliations

  1. Osaka University, Osaka, Japan

    Joe Suzuki

  2. The University of Electro-Communications, Tokyo, Japan

    Maomi Ueno

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Malone, B. (2015). Empirical Behavior of Bayesian Network Structure Learning Algorithms. In: Suzuki, J., Ueno, M. (eds) Advanced Methodologies for Bayesian Networks. AMBN 2015. Lecture Notes in Computer Science(), vol 9505. Springer, Cham. https://doi.org/10.1007/978-3-319-28379-1_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28379-1_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28378-4

  • Online ISBN: 978-3-319-28379-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation