Advertisement

Machine Learning

, Volume 107, Issue 8–10, pp 1209–1227 | Cite as

Approximate structure learning for large Bayesian networks

  • Mauro Scanagatta
  • Giorgio Corani
  • Cassio Polpo de Campos
  • Marco Zaffalon
Article
  • 426 Downloads
Part of the following topical collections:
  1. Special Issue of the ECML PKDD 2018 Journal Track

Abstract

We present approximate structure learning algorithms for Bayesian networks. We discuss the two main phases of the task: the preparation of the cache of the scores and structure optimization, both with bounded and unbounded treewidth. We improve on state-of-the-art methods that rely on an ordering-based search by sampling more effectively the space of the orders. This allows for a remarkable improvement in learning Bayesian networks from thousands of variables. We also present a thorough study of the accuracy and the running time of inference, comparing bounded-treewidth and unbounded-treewidth models.

Keywords

Bayesian networks Structural learning Treewidth 

Notes

Acknowledgements

Work partially supported by the Swiss NSF Grant Nos. 200021_146606 / 1 and IZKSZ2_162188.

References

  1. Bartlett, M., & Cussens, J. (2017). Integer linear programming for the Bayesian network structure learning problem. Artificial Intelligence, 244, 258–271.MathSciNetCrossRefzbMATHGoogle Scholar
  2. Berg, J., Järvisalo, M., & Malone, B. (2014). Learning optimal bounded treewidth Bayesian networks via maximum satisfiability. In Proceedings of the 17th international conference on artificial intelligence and statistics (pp. 86–95).Google Scholar
  3. Bodlaender, H. L., Koster, A. M. C. A., van den Eijkhof, F., & van der Gaag, L. C. (2001). Pre-processing for triangulation of probabilistic networks. In Proceedings of the 17th conference on uncertainty in artificial intelligence (pp. 32–39).Google Scholar
  4. Chickering, D. M., Heckerman, D., & Meek, C. (2014). Large-sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research, 5, 1287–1330.MathSciNetzbMATHGoogle Scholar
  5. Cooper, G. F., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9, 309–347.zbMATHGoogle Scholar
  6. Cussens, J. (2011). Bayesian network learning with cutting planes. In Proceedings of the 27th conference on uncertainty in artificial intelligence (pp. 153–160).Google Scholar
  7. Cussens, J., Järvisalo, M., Korhonen, J. H., & Bartlett, M. (2017). Bayesian network structure learning with integer programming: Polytopes, facets and complexity. Journal of Artificial Intelligence Research, 58, 185–229.MathSciNetCrossRefzbMATHGoogle Scholar
  8. Cussens, J., Malone, B., & Yuan, C. (2013). IJCAI 2013 tutorial on optimal algorithms for learning Bayesian networks. https://sites.google.com/site/ijcai2013bns/slides. Accessed Jan 2017.
  9. Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge: Cambridge University Press.CrossRefzbMATHGoogle Scholar
  10. de Campos, C. P., & Ji, Q. (2011). Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research, 12, 663–689.MathSciNetzbMATHGoogle Scholar
  11. de Campos, C. P., Zeng, Z., & Ji, Q. (2009). Structure learning of Bayesian networks using constraints. In Proceedings of the 26th international conference on machine learning (pp. 113–120).Google Scholar
  12. Elidan, G., & Gould, S. (2008). Learning bounded treewidth Bayesian networks. Journal of Machine Learning Research, 9, 2699–2731.MathSciNetzbMATHGoogle Scholar
  13. Jaakkola, T., Sontag, D., Globerson, A., & Meila, M. (2010). Learning Bayesian network structure using LP relaxations. In Proceedings of the 13th international conference on artificial intelligence and statistics (pp. 358–365).Google Scholar
  14. Koivisto, M., & Sood, K. (2004). Exact Bayesian structure discovery in Bayesian networks. Journal of Machine Learning Research, 5, 549–573.MathSciNetzbMATHGoogle Scholar
  15. Korhonen, J., & Parviainen, P. (2013). Exact learning of bounded treewidth Bayesian networks. In Proceedings of the 16th international conference on artificial intelligence and statistics (pp. 370–378).Google Scholar
  16. Liu, Z., Malone, B., & Yuan, C. (2012). Empirical evaluation of scoring functions for Bayesian network model selection. BMC Bioinformatics, 13(15), 1–16.Google Scholar
  17. Lowd, D., & Domingos, P. (2008). Learning arithmetic circuits. In Proceedings of the 24th conference on uncertainty in artificial intelligence (pp. 383–392).Google Scholar
  18. Mateescu, R., Kask, K., Gogate, V., & Dechter, R. (2010). Join-graph propagation algorithms. Journal of Artificial Intelligence Research, 37, 279–328.MathSciNetCrossRefzbMATHGoogle Scholar
  19. Nie, S., de Campos, C. P., & Ji, Q. (2015). Learning bounded treewidth Bayesian networks via sampling. In Proceedings of the 13th European conference on symbolic and quantitative approaches to reasoning with uncertainty (pp. 387–396).Google Scholar
  20. Nie, S., de Campos, C. P., & Ji, Q. (2016). Learning Bayesian networks with bounded treewidth via guided search. In Proceedings of the 30th AAAI conference on artificial intelligence (pp. 3294–3300).Google Scholar
  21. Nie, S., Mauá, D. D., de Campos, C. P., & Ji, Q. (2014). Advances in learning Bayesian networks of bounded treewidth. Advances in Neural Information Processing Systems, 27, 2285–2293.Google Scholar
  22. Parviainen, P., Farahani, H. S., & Lagergren, J. (2014). Learning bounded treewidth Bayesian networks using integer linear programming. In Proceedings of the 17th international conference on artificial intelligence and statistics (pp. 751–759).Google Scholar
  23. Patil, H. P. (1986). On the structure of k-trees. Journal of Combinatorics, Information and System Sciences, 11(2–4), 57–64.MathSciNetzbMATHGoogle Scholar
  24. Poon, H., & Domingos, P. (2011). Sum-product networks: A new deep architecture. In Proceedings of the 27th conference on uncertainty in artificial intelligence (pp. 689–690).Google Scholar
  25. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–164.CrossRefGoogle Scholar
  26. Rooshenas, A., & Lowd, D. (2014). Learning sum-product networks with direct and indirect variable interactions. In Proceedings of the 31st international conference on machine learning (pp. 710–718).Google Scholar
  27. Scanagatta, M., Corani, G., de Campos, C. P., & Zaffalon, M. (2016). Learning treewidth-bounded Bayesian networks with thousands of variables. Advances in Neural Information Processing Systems, 29, 1462–1470.Google Scholar
  28. Scanagatta, M., Corani, G., Zaffalon, M., Yoo, J., & Kang, U. (2018). Efficient learning of bounded-treewidth Bayesian networks from complete and incomplete data sets. International Journal of Approximate Reasoning, 95, 152–166.MathSciNetCrossRefzbMATHGoogle Scholar
  29. Scanagatta, M., de Campos, C. P., Corani, G., & Zaffalon, M. (2015). Learning Bayesian networks with thousands of variables. Advances in Neural Information Processing Systems, 28, 1855–1863.Google Scholar
  30. Silander, T., & Myllymaki, P. A. (2006). A simple approach for finding the globally optimal Bayesian network structure. In Proceedings of the 22nd conference on uncertainty in artificial intelligence (pp. 445–452).Google Scholar
  31. Teyssier, M., & Koller, D. (2005). Ordering-based search: A simple and effective algorithm for learning Bayesian networks. In Proceedings of the 21st conference on uncertainty in artificial intelligence (pp. 584–590).Google Scholar
  32. Yuan, C., & Malone, B. (2012). An improved admissible heuristic for learning optimal Bayesian networks. In Proceedings of the 28th conference on uncertainty in artificial intelligence (pp. 924–933).Google Scholar
  33. Yuan, C., & Malone, B. (2013). Learning optimal Bayesian networks: A shortest path perspective. Journal of Artificial Intelligence Research, 48, 23–65.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Istituto Dalle Molle di studi sull’Intelligenza Artificiale (IDSIA)MannoSwitzerland
  2. 2.Queen’s University BelfastBelfastNorthern Ireland, UK
  3. 3.Utrecht UniversityUtrechtThe Netherlands

Personalised recommendations