Skip to main content

Bayesian Optimization of the PC Algorithm for Learning Gaussian Bayesian Networks

  • Conference paper
  • First Online:
Advances in Artificial Intelligence (CAEPIA 2018)

Abstract

The PC algorithm is a popular method for learning the structure of Gaussian Bayesian networks. It carries out statistical tests to determine absent edges in the network. It is hence governed by two parameters: (i) The type of test, and (ii) its significance level. These parameters are usually set to values recommended by an expert. Nevertheless, such an approach can suffer from human bias, leading to suboptimal reconstruction results. In this paper we consider a more principled approach for choosing these parameters in an automatic way. For this we optimize a reconstruction score evaluated on a set of different Gaussian Bayesian networks. This objective is expensive to evaluate and lacks a closed-form expression, which means that Bayesian optimization (BO) is a natural choice. BO methods use a model to guide the search and are hence able to exploit smoothness properties of the objective surface. We show that the parameters found by a BO method outperform those found by a random search strategy and the expert recommendation. Importantly, we have found that an often overlooked statistical test provides the best over-all reconstruction results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/irenecrsn/bopc.

References

  1. Bielza, C., Larrañaga, P.: Bayesian networks in neuroscience: a survey. Front. Comput. Neurosci. 8, 131 (2014)

    Article  Google Scholar 

  2. Colombo, D., Maathuis, M.H.: Order-independent constraint-based causal structure learning. J. Mach. Learn. Res. 15(1), 3741–3782 (2014)

    MathSciNet  MATH  Google Scholar 

  3. Garrido-Merchán, E.C., Hernández-Lobato, D.: Dealing with categorical and integer-valued variables in Bayesian optimization with Gaussian processes (2018). arXiv:1805.03463

  4. Hausser, J., Strimmer, K.: Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks. J. Mach. Learn. Res. 10, 1469–1484 (2009)

    MathSciNet  MATH  Google Scholar 

  5. Hernández-Lobato, J.M., Hoffman, M.W., Ghahramani, Z.: Predictive entropy search for efficient global optimization of black-box functions. In: Advances in Neural Information Processing Systems, pp. 918–926 (2014)

    Google Scholar 

  6. Kalisch, M., Bühlmann, P.: Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J. Mach. Learn. Res. 8, 613–636 (2007)

    MATH  Google Scholar 

  7. Lauritzen, S.L., Dawid, A.P., Larsen, B.N., Leimer, H.G.: Independence properties of directed Markov fields. Networks 20(5), 491–505 (1990)

    Article  MathSciNet  Google Scholar 

  8. Malone, B., Järvisalo, M., Myllymäki, P.: Impact of learning strategies on the quality of Bayesian networks: an empirical evaluation. In: Proceedings of the Thirty-First the Conference on Uncertainty in Artificial Intelligence, pp. 562–571 (2015)

    Google Scholar 

  9. Malone, B., Kangas, K., Järvisalo, M., Koivisto, M., Myllymäki, P.: Empirical hardness of finding optimal Bayesian network structures: algorithm selection and runtime prediction. Mach. Learn. 107(1), 247–283 (2018)

    Article  MathSciNet  Google Scholar 

  10. Minka, T.P.: Expectation propagation for approximate Bayesian inference. In: Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, pp. 362–369 (2001)

    Google Scholar 

  11. Ness, R.O., Sachs, K., Vitek, O.: From correlation to causality: statistical approaches to learning regulatory relationships in large-scale biomolecular investigations. J. Proteome Res. 15(3), 683–690 (2016)

    Article  Google Scholar 

  12. Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) ML -2003. LNCS (LNAI), vol. 3176, pp. 63–71. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28650-9_4

    Chapter  Google Scholar 

  13. Scutari, M.: Learning Bayesian networks with the bnlearn R package. J. Stat. Softw. 35(3), 1–22 (2010)

    Article  MathSciNet  Google Scholar 

  14. Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: a review of Bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)

    Article  Google Scholar 

  15. Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, pp. 2951–2959 (2012)

    Google Scholar 

  16. Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65(1), 31–78 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

We acknowledge the use of the facilities of Centro de Computación Científica (CCC) at Universidad Autónoma de Madrid, and financial support from Comunidad de Madrid, grant S2013/ICE-2845; from the Spanish Ministerio de Economía, Industria y Competitividad, grants TIN2016-79684-P, TIN2016-76406-P, TEC2016-81900-REDT; from the Cajal Blue Brain project (C080020-09, the Spanish partner of the EPFL Blue Brain initiative); and from Fundación BBVA (Scientific Research Teams in Big Data 2016). Irene Córdoba is supported by grant FPU15/03797 from the Spanish Ministerio de Educación, Cultura y Deporte.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Irene Córdoba .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Córdoba, I., Garrido-Merchán, E.C., Hernández-Lobato, D., Bielza, C., Larrañaga, P. (2018). Bayesian Optimization of the PC Algorithm for Learning Gaussian Bayesian Networks. In: Herrera, F., et al. Advances in Artificial Intelligence. CAEPIA 2018. Lecture Notes in Computer Science(), vol 11160. Springer, Cham. https://doi.org/10.1007/978-3-030-00374-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-00374-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-00373-9

  • Online ISBN: 978-3-030-00374-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics