Efficient EM Learning with Tabulation for Parameterized Logic Programs

  • Yoshitaka Kameya
  • Taisuke Sato
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1861)


We have been developing a general symbolic-statistical modeling language [6,19,20] based on the logic programming framework that semantically unifies (and extends) major symbolic-statistical frameworks such as hidden Markov models (HMMs) [18], probabilistic context-free grammars (PCFGs) [23] and Bayesian networks [16]. The language, PRISM, is intended to model complex symbolic phenomena governed by rules and probabilities based on the distributional semantics [19]. Programs contain statistical parameters and they are automatically learned from randomly sampled data by a specially derived EM algorithm, the graphical EM algorithm. It works on support graphs representing the shared structure of explanations for an observed goal. In this paper, we propose the use of tabulation technique to build support graphs, and show that as a result, the graphical EM algorithm attains the same time complexity as specilized EM algorithms for HMMs (the Baum-Welch algorithm [18]) and PCFGs (the Inside-Outside algorithm [1]).


Bayesian Network Logic Program Logic Programming Predicate Symbol Ground Atom 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, J. K., Trainable grammars for speech recognition, Proc. of Spring Conf. of the Acoustical Society of America, pp. 547–550, 1979.Google Scholar
  2. 2.
    Breese, J. S., Construction of belief and decision networks, Computational Intelligence, Vol. 8, No. 4, pp. 624–647, 1992.CrossRefGoogle Scholar
  3. 3.
    Cussens, J., Loglinear models for first-order probabilistic reasoning, Proc. of UAI’99, pp. 126–133, 1999.Google Scholar
  4. 4.
    Dekhtyar, A. and Subrahmanian, V. S., Hybrid probabilistic programs, Proc. of ICLP’97, pp. 391–405, 1997.Google Scholar
  5. 5.
    Doets, K., From Logic to Logic Programming, The MIT Press, 1994.Google Scholar
  6. 6.
    Kameya, Y., Ueda, N. and Sato, T., A graphical method for parameter learning of symbolic-statistical models, Proc. of DS’99, LNAI 1721, pp. 264–276, 1999.Google Scholar
  7. 7.
    Kita, K., Morimoto, T., Ohkura, K., Sagayama, S. and Yano, Y., Spoken sentence recognition based on HMM-LR with hybrid language modeling, IEICE Trans. on Information & Systems, Vol. E77-D, No. 2, 1994.Google Scholar
  8. 8.
    Koller, D. and Pfeffer, A., Learning probabilities for noisy first-order rules, Proc. of IJCAI’97, pp. 1316–1321, 1997.Google Scholar
  9. 9.
    Koller, D., McAllester, D. and Pfeffer, A., Effective Bayesian Inference for Stochastic Programs, Proc. of AAAI’97, pp. 740–747, 1997.Google Scholar
  10. 10.
    Lakshmanan, L. V. S. and Sadri, F., Probabilistic deductive databases, Proc. of ILPS’94, pp. 254–268, 1994.Google Scholar
  11. 11.
    Manning, C. D. and Schütze, H., Foundations of Statistical Natural Language Processing, The MIT Press, 1999.Google Scholar
  12. 12.
    Muggleton, S., Stochastic logic programs, In Advances in Inductive Logic Programming (Raedt, L.Deed.), OSP Press, pp. 254–264, 1996.Google Scholar
  13. 13.
    Ng, R. and Subrahmanian, V. S., Probabilistic logic programming, Information and Computation, Vol. 101, pp. 150–201, 1992.zbMATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Ngo, L. and Haddawy, P., Answering queries from context-sensitive probabilistic knowledge bases, Theoretical Computer Science, Vol. 171, pp. 147–177, 1997.zbMATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Nilsson, N. J., Probabilistic logic, Artificial Intelligence, Vol. 28, pp. 71–87, 1986.zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Pearl, J., Probabilistic Reasoning in Intelligent Systems, Morgan Kaufmann, 1988.Google Scholar
  17. 17.
    Poole, D., Probabilistic Horn abduction and Bayesian networks, Artificial Intelligence, Vol. 64, pp. 81–129, 1993.zbMATHCrossRefGoogle Scholar
  18. 18.
    Rabiner, L. and Juang, B., Foundations of Speech Recognition, Prentice-Hall, 1993.Google Scholar
  19. 19.
    Sato, T., A statistical learning method for logic programs with distribution semantics, Proc. of ICLP’95, pp. 715–729, 1995.Google Scholar
  20. 20.
    Sato, T. and Kameya, Y., PRISM: a language for symbolic-statistical modeling, Proc. of IJCAI’97, pp. 1330–1335, 1997.Google Scholar
  21. 21.
    Tamaki, H. and Sato, T., OLD resolution with tabulation, Proc. of ICLP’86, LNCS 225, pp. 84–98, 1986.Google Scholar
  22. 22.
    Tanner, M., Tools for Statistical Inference (2nd ed.), Springer-Verlag, 1986.Google Scholar
  23. 23.
    Wetherell, C.S., Probabilistic languages: a review and some open questions, Computing Surveys, Vol. 12, No. 4, pp. 361–379, 1980.zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Yoshitaka Kameya
    • 1
  • Taisuke Sato
    • 1
  1. 1.Dept. of Computer Science, Graduate School of Information Science and EngineeringTokyo Institute of TechnologyTokyoJapan

Personalised recommendations