Expectation Maximization in Deep Probabilistic Logic Programming

  • Arnaud Nguembang FadjaEmail author
  • Fabrizio RiguzziEmail author
  • Evelina LammaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11298)


Probabilistic Logic Programming (PLP) combines logic and probability for representing and reasoning over domains with uncertainty. Hierarchical probability Logic Programming (HPLP) is a recent language of PLP whose clauses are hierarchically organized forming a deep neural network or arithmetic circuit. Inference in HPLP is done by circuit evaluation and learning is therefore cheaper than any generic PLP language. We present in this paper an Expectation Maximization algorithm, called Expectation Maximization Parameter learning for HIerarchical Probabilistic Logic programs (EMPHIL), for learning HPLP parameters. The algorithm converts an arithmetic circuit into a Bayesian network and performs the belief propagation algorithm over the corresponding factor graph.


Hierarchical probabilistic logic programming Arithmetic circuits Expectation Maximization Factor graph Belief propagation 


  1. 1.
    Alberti, M., Bellodi, E., Cota, G., Riguzzi, F., Zese, R.: cplint on SWISH: probabilistic logical inference with a web browser. Intell. Artif. 11(1), 47–64 (2017)Google Scholar
  2. 2.
    Alberti, M., Cota, G., Riguzzi, F., Zese, R.: Probabilistic logical inference on the web. In: Adorni, G., Cagnoni, S., Gori, M., Maratea, M. (eds.) AI*IA 2016. LNCS (LNAI), vol. 10037, pp. 351–363. Springer, Cham (2016). Scholar
  3. 3.
    De Raedt, L., Kimmig, A., Toivonen, H.: ProbLog: a probabilistic Prolog and its application in link discovery. In: Veloso, M.M. (ed.) IJCAI 2007, vol. 7, pp. 2462–2467. AAAI Press/IJCAI (2007)Google Scholar
  4. 4.
    Kersting, K., De Raedt, L.: Basic principles of learning Bayesian logic programs. Institute for Computer Science, University of Freiburg. Citeseer (2002)Google Scholar
  5. 5.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  6. 6.
    Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: ICML 2005, pp. 441–448. ACM (2005)Google Scholar
  7. 7.
    Koller, D., Pfeffer, A.: Learning probabilities for noisy first-order rules. In: IJCAI, pp. 1316–1323 (1997)Google Scholar
  8. 8.
    Meert, W., Struyf, J., Blockeel, H.: CP-logic theory inference with contextual variable elimination and comparison to BDD based inference methods. In: De Raedt, L. (ed.) ILP 2009. LNCS (LNAI), vol. 5989, pp. 96–109. Springer, Heidelberg (2010). Scholar
  9. 9.
    Mørk, S., Holmes, I.: Evaluating bacterial gene-finding HMM structures as probabilistic logic programs. Bioinformatics 28(5), 636–642 (2012)CrossRefGoogle Scholar
  10. 10.
    Natarajan, S., Tadepalli, P., Kunapuli, G., Shavlik, J.: Learning parameters for relational probabilistic models with noisy-or combining rule. In: 2009 International Conference on Machine Learning and Applications, ICMLA 2009, pp. 141–146. IEEE (2009)Google Scholar
  11. 11.
    Nguembang Fadja, A., Lamma, E., Riguzzi, F.: Deep probabilistic logic programming. In: Theil Have, C., Zese, R. (eds.) Proceedings of the 4th International Workshop on Probabilistic Logic Programming (PLP 2017), CEUR Workshop Proceedings, vol. 1916, pp. 3–14. Sun SITE Central Europe, Aachen (2017)Google Scholar
  12. 12.
    Holzinger, A., Goebel, R., Palade, V., Ferri, M.: Towards integrative machine learning and knowledge extraction. In: Holzinger, A., Goebel, R., Ferri, M., Palade, V. (eds.) Towards Integrative Machine Learning and Knowledge Extraction. LNCS (LNAI), vol. 10344, pp. 1–12. Springer, Cham (2017). Scholar
  13. 13.
    Nguembang Fadja, A., Riguzzi, F., Lamma, E.: Deep parameter learning for probabilistic logic programming (2018, submitted)Google Scholar
  14. 14.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, Burlington (1988)zbMATHGoogle Scholar
  15. 15.
    Poole, D.: The independent choice logic for modelling multiple agents under uncertainty. Artif. Intell. 94, 7–56 (1997)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Riguzzi, F., Bellodi, E., Lamma, E., Zese, R., Cota, G.: Probabilistic logic programming on the web. Softw.-Pract. Exper. 46(10), 1381–1396 (2016)CrossRefGoogle Scholar
  17. 17.
    Riguzzi, F., Lamma, E., Alberti, M., Bellodi, E., Zese, R., Cota, G.: Probabilistic logic programming for natural language processing. In: Chesani, F., Mello, P., Milano, M. (eds.) Workshop on Deep Understanding and Reasoning, URANIA 2016. CEUR Workshop Proceedings, vol. 1802, pp. 30–37. Sun SITE Central Europe, Aachen, Germany (2017)Google Scholar
  18. 18.
    Sato, T.: A statistical learning method for logic programs with distribution semantics. In: Sterling, L. (ed.) ICLP 1995, pp. 715–729. MIT Press (1995)Google Scholar
  19. 19.
    Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic programs with annotated disjunctions. In: Demoen, B., Lifschitz, V. (eds.) ICLP 2004. LNCS, vol. 3132, pp. 431–445. Springer, Heidelberg (2004). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Dipartimento di IngegneriaUniversity of FerraraFerraraItaly
  2. 2.Dipartimento di Matematica e InformaticaUniversity of FerraraFerraraItaly

Personalised recommendations