Bayesian Markov Logic Networks

Bayesian Inference for Statistical Relational Learning
  • Radim NedbalEmail author
  • Luciano SerafiniEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11298)


One of the most important foundational challenge of Statistical relational learning is the development of a uniform framework in which learning and logical reasoning are seamlessly integrated. State of the art approaches propose to modify well known machine learning methods based on parameter optimization (e.g., neural networks and graphical models) in order to take into account structural knowledge expressed by logical constraints. In this paper, we follow an alternative direction, considering the Bayesian approach to machine learning. In particular, given a partial knowledge in hybrid domains (i.e., domains that contains relational structure and continuous features) as a set \(\mathcal {T}\) of axioms and a stochastic (in)dependence hypothesis \(\mathscr {F}\) encoded in a first order language \(\mathcal {L}\), we propose to model it by a probability distribution function (PDF) \(p(\mathbf {x}\mid \mathcal {T},\mathscr {F})\) over the \(\mathcal {L}\)-interpretations \(\mathbf {x}\). The stochastic (in)dependence \(\mathscr {F}\) is represented as a Bayesian Markov Logic Network w.r.t. a parametric undirected graph, interpreted as the PDF. We propose to approximate \(P(\mathbf {x}\mid \mathcal {T},\mathscr {F})\) by variational inference and show that such approximation is possible if and only if \(\mathscr {F}\) satisfies a property called orthogonality. This property can be achieved also by extending \(\mathcal {L}\), and adjusting \(\mathcal {T}\) and \(\mathscr {F}\).


  1. 1.
    Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, New York (2006)zbMATHGoogle Scholar
  2. 2.
    Bishop, C.M.: A new framework for machine learning. In: Zurada, J.M., Yen, G.G., Wang, J. (eds.) WCCI 2008. LNCS, vol. 5050, pp. 1–24. Springer, Heidelberg (2008). Scholar
  3. 3.
    Bishop, C.M., Spiegelhalter, D., Winn, J.: VIBES: a variational inference engine for Bayesian networks. In: Becker, S., Thrun, S., Obermayer, K., (eds.) Advances in Neural Information Processing Systems 15, pp. 777–784. MIT Press (2002). [Neural Information Processing Systems, NIPS 2002, 9–14 December 2002, Vancouver, British Columbia, Canada]Google Scholar
  4. 4.
    Clifford, P.: Markov random fields in statistics. In: Grimmett, G., Welsh, D. (eds.) Disorder in Physical Systems: A Volume in Honour of John M. Hammersley, pp. 19–32. Oxford University Press, Oxford (1990)Google Scholar
  5. 5.
    Domingos, P., Lowd, D.: Markov Logic: An Interface Layer for Artificial Intelligence. Morgan and Claypool Publishers, San Rafael/California (2009)zbMATHGoogle Scholar
  6. 6.
    Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)CrossRefGoogle Scholar
  7. 7.
    Kindermann, R., Laurie Snell, J.: Markov Random Fields and Their Applications. AMS, Providence (1980)CrossRefGoogle Scholar
  8. 8.
    Kok, S., et al.: The alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington (2007)Google Scholar
  9. 9.
    Ranganath, R., Gerrish, S., Blei, D.M.: Black box variational inference. In: Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, AISTATS 2014, Reykjavik, Iceland, 22–25 April 2014, Volume 33 of JMLR Proceedings, pp. 814–822. (2014)Google Scholar
  10. 10.
    Richardson, M., Domingos, P.M.: Markov logic networks. Mach. Learn. 62(1–2), 107–136 (2006)CrossRefGoogle Scholar
  11. 11.
    Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach. Pearson Education, London (2010). (3. internat. ed.)zbMATHGoogle Scholar
  12. 12.
    Shao, J.: Mathematical Statistics. Springer, New York (1999). Springer texts in statisticsCrossRefzbMATHGoogle Scholar
  13. 13.
    Singla, P., Domingos, P.M.: Markov logic in infinite domains. CoRR, abs/1206.5292 (2012)Google Scholar
  14. 14.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1–2), 1–305 (2008)CrossRefGoogle Scholar
  15. 15.
    Wang, J., Domingos, P.M.: Hybrid Markov logic networks. In: Fox, D., Gomes, C.P. (eds.) Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, 13–17 July 2008, pp. 1106–1111. AAAI Press (2008)Google Scholar
  16. 16.
    Wingate, D., Weber, T.: Automated variational inference in probabilistic programming. CoRR, abs/1301.1299 (2013)Google Scholar
  17. 17.
    Winn, J.M., Bishop, C.M.: Variational message passing. J. Mach. Learn. Res. 6, 661–694 (2005)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Fondazione Bruno KesslerTrentoItaly

Personalised recommendations