New Advances in Logic-Based Probabilistic Modeling by PRISM

  • Taisuke Sato
  • Yoshitaka Kameya
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4911)

Abstract

We review a logic-based modeling language PRISM and report recent developments including belief propagation by the generalized inside-outside algorithm and generative modeling with constraints. The former implies PRISM subsumes belief propagation at the algorithmic level. We also compare the performance of PRISM with state-of-the-art systems in statistical natural language processing and probabilistic inference in Bayesian networks respectively, and show that PRISM is reasonably competitive.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Sato, T., Kameya, Y.: PRISM: A language for symbolic-statistical modeling. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 1330–1335 (1997)Google Scholar
  2. 2.
    Sato, T.: A statistical learning method for logic programs with distribution semantics. In: Proceedings of the 12th International Conference on Logic Programming (ICLP 1995), pp. 715–729 (1995)Google Scholar
  3. 3.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Royal Statistical Society B39(1), 1–38 (1977)MathSciNetGoogle Scholar
  4. 4.
    Tamaki, H., Sato, T.: OLD resolution with tabulation. In: Shapiro, E. (ed.) ICLP 1986. LNCS, vol. 225, pp. 84–98. Springer, Heidelberg (1986)Google Scholar
  5. 5.
    Zhou, N.F., Sato, T.: Efficient fixpoint computation in linear tabling. In: Proceedings of the 5th ACM-SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP 2003), pp. 275–283 (2003)Google Scholar
  6. 6.
    Sato, T., Kameya, Y., Abe, S., Shirai, K.: Fast EM learning of a family of PCFGs. Technical Report (Dept. of CS) TR01-0006, Tokyo Institute of Technology (2001)Google Scholar
  7. 7.
    Cussens, J.: Parameter estimation in stochastic logic programs. Machine Learning 44(3), 245–271 (2001)MATHCrossRefGoogle Scholar
  8. 8.
    Sato, T., Kameya, Y., Zhou, N.F.: Generative modeling with failure in PRISM. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 847–852 (2005)Google Scholar
  9. 9.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)Google Scholar
  10. 10.
    Sato, T.: Inside-Outside probability computation for belief propagation. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2605–2610 (2007)Google Scholar
  11. 11.
    Lauritzen, S., Spiegelhalter, D.: Local computations with probabilities on graphical structures and their applications to expert systems. Journal of the Royal Statistical Society, B 50, 157–224 (1988)MATHMathSciNetGoogle Scholar
  12. 12.
    Jensen, F.V.: An Introduction to Bayesian Networks. UCL Press (1996)Google Scholar
  13. 13.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  14. 14.
    Baker, J.K.: Trainable grammars for speech recognition. In: Proceedings of Spring Conference of the Acoustical Society of America, pp. 547–550 (1979)Google Scholar
  15. 15.
    Eisner, J., Goldlust, E., Smith, N.: Compiling Comp Ling: Weighted dynamic programming and the Dyna language. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT-EMNLP)., pp. 281–290 (2005)Google Scholar
  16. 16.
    Darwiche, A.: A compiler for deterministic, decomposable negation normal form. In: Proceedings of the 18th national conference on Artificial intelligence (AAAI 2002), pp. 627–634 (2002)Google Scholar
  17. 17.
    Chavira, M., Darwiche, A.: Compiling Bayesian networks with local structure. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 1306–1312 (2005)Google Scholar
  18. 18.
    Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational bayesian networks for exact inference. International Journal of Approximate Reasoning 42, 4–20 (2006)MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Doets, K.: From Logic to Logic Programming. The MIT Press, Cambridge (1994)MATHGoogle Scholar
  20. 20.
    Manning, C.D., Schütze, H.: Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge (1999)MATHGoogle Scholar
  21. 21.
    Castillo, E., Gutierrez, J.M., Hadi, A.S.: Expert Systems and Probabilistic Network Models. Springer, Heidelberg (1997)Google Scholar
  22. 22.
    Chow, Y., Teicher, H.: Probability Theory, 3rd edn. Springer, Heidelberg (1997)MATHGoogle Scholar
  23. 23.
    Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence 64(1), 81–129 (1993)MATHCrossRefGoogle Scholar
  24. 24.
    Poole, D.: The independent choice logic for modeling multiple agents under uncertainty. Artificial Intelligence 94(1-2), 7–56 (1997)MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Clark, K.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Databases, pp. 293–322. Plenum Press, New York (1978)Google Scholar
  26. 26.
    Mateescu, R., Dechter, R.: The relationship between AND/OR search spaces and variable elimination. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI 2005), pp. 380–387 (2005)Google Scholar
  27. 27.
    Sato, T.: Modeling scientific theories as PRISM programs. In: Proceedings of ECAI 1998 Workshop on Machine Discovery, pp. 37–45 (1998)Google Scholar
  28. 28.
    Mitomi, H., Fujiwara, F., Yamamoto, M., Sato, T.: Bayesian classification of human custom based on stochastic context-free grammar (in Japanese). IEICE Transaction on Information and Systems J88-D-II(4), 716–726 (2005)Google Scholar
  29. 29.
    Wang, S., Wang, S., Greiner, R., Schuurmans, D., Cheng, L.: Exploiting syntactic, semantic and lexical regularities in language modeling via directed Markov random fields. In: Proceedings of the 22th International Conference on Machine Learning (ICML 2005), pp. 948–955 (2005)Google Scholar
  30. 30.
    Sato, T., Kameya, Y.: Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research 15, 391–454 (2001)MATHMathSciNetGoogle Scholar
  31. 31.
    Smyth, P., Heckerman, D., Jordan, M.: Probabilistic independence networks for hidden Markov probability models. Neural Computation 9(2), 227–269 (1997)MATHCrossRefGoogle Scholar
  32. 32.
    Kask, K., Dechter, R., Larrosa, J., Cozman, F.: Bucket-tree elimination for automated reasoning. ICS Technical Report Technical Report No.R92, UC Irvine (2001)Google Scholar
  33. 33.
    Shafer, G., Shenoy, P.: Probability propagation. Annals of Mathematics and Artificial Intelligence 2, 327–352 (1990)MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    McAllester, D., Collins, M., Pereira, F.: Case-factor diagrams for structured probabilistic modeling. In: Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI2004), Arlington, Virginia, pp. 382–391. AUAI Press (2004)Google Scholar
  35. 35.
    Minato, S., Satoh, K., Sato, T.: Compiling bayesian networks by symbolic probability calculation based on zero-suppressed bdds. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007), pp. 2550–2555 (2007)Google Scholar
  36. 36.
    Charniak, E.: Tree-bank grammars. In: Proceedings of the 13th National Conference on Artificial Intelligence(AAAI 1996), pp. 1031–1036 (1996)Google Scholar
  37. 37.
    Marcus, M., Santorini, B., Marcinkiewicz, M.: Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19, 313–330 (1993)Google Scholar
  38. 38.
    Boutilier, C., Friedman, N., Goldszmidt, M., Koller, D.: Context-specific independence in Bayesian networks. In: Procceding of the 12th Conference on Uncertainty in Artificial Intelligence (UAI 1996), pp. 115–123 (1996)Google Scholar
  39. 39.
    Chi, Z., Geman, S.: Estimation of probabilistic context-free grammars. Computational Linguistics 24(2), 299–305 (1998)MathSciNetGoogle Scholar
  40. 40.
    Wetherell, C.S.: Probabilistic languages: A review and some open questions. Computing Surveys 12(4), 361–379 (1980)MATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Abney, S.: Stochastic attribute-value grammars. Computational Linguistics 23(4), 597–618 (1997)MathSciNetGoogle Scholar
  42. 42.
    Schmid, H.: A generative probability model for unification-based grammars. In: Proceedings of the 21st International Conference on Computational Linguistics (COLING 2002, pp. 884–896 (2002)Google Scholar
  43. 43.
    Sag, I., Wasow, T.: Syntactic Theory: A Formal Introduction. CSLI Publications, Stanford (1999)MATHGoogle Scholar
  44. 44.
    Sato, T.: First Order Compiler: A deterministic logic program synthesis algorithm. Journal of Symbolic Computation 8, 605–627 (1989)MATHMathSciNetGoogle Scholar
  45. 45.
    Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001, pp. 282–289 (2001)Google Scholar
  46. 46.
    Sato, T., Kameya, Y.: Negation elimination for finite PCFGs. In: Etalle, S. (ed.) LOPSTR 2004. LNCS, vol. 3573, pp. 119–134. Springer, Heidelberg (2005)Google Scholar
  47. 47.
    Kameya, Y., Sato, T.: Efficient EM learning for parameterized logic programs. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 269–294. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  48. 48.
    Nilsson, N.J.: Probabilistic logic. Artificial Intelligence 28, 71–87 (1986)MATHCrossRefMathSciNetGoogle Scholar
  49. 49.
    Frish, A., Haddawy, P.: Anytime deduction for probabilistic logic. Journal of Artificial Intelligence 69, 93–122 (1994)CrossRefGoogle Scholar
  50. 50.
    Lukasiewicz, T.: Probabilistic deduction with conditional constraints over basic events. Journal of Artificial Intelligence Research 10, 199–241 (1999)MATHMathSciNetGoogle Scholar
  51. 51.
    Ng, R., Subrahmanian, V.S.: Probabilistic logic programming. Information and Computation 101, 150–201 (1992)MATHCrossRefMathSciNetGoogle Scholar
  52. 52.
    Lakshmanan, L.V.S., Sadri, F.: Probabilistic deductive databases. In: Proceedings of the 1994 International Symposium on Logic Programming (ILPS 1994), pp. 254–268 (1994)Google Scholar
  53. 53.
    Dekhtyar, A., Subrahmanian, V.S.: Hybrid probabilistic programs. In: Proceedings of the 14th International Conference on Logic Programming (ICLP 1997), pp. 391–405 (1997)Google Scholar
  54. 54.
    Saad, E., Pontelli, E.: Toward a more practical hybrid probabilistic logic programming framework. In: Hermenegildo, M.V., Cabeza, D. (eds.) PADL 2004. LNCS, vol. 3350, pp. 67–82. Springer, Heidelberg (2005)Google Scholar
  55. 55.
    Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI 2002), pp. 485–492 (2002)Google Scholar
  56. 56.
    Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)CrossRefGoogle Scholar
  57. 57.
    Breese, J.S.: Construction of belief and decision networks. Computational Intelligence 8(4), 624–647 (1992)CrossRefGoogle Scholar
  58. 58.
    Wellman, M., Breese, J., Goldman, R.: From knowledge bases to decision models. Knowledge Engineering Review 7(1), 35–53 (1992)CrossRefGoogle Scholar
  59. 59.
    Koller, D., Pfeffer, A.: Learning probabilities for noisy first-order rules. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 1316–1321 (1997)Google Scholar
  60. 60.
    Ngo, L., Haddawy, P.: Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science 171, 147–177 (1997)MATHCrossRefMathSciNetGoogle Scholar
  61. 61.
    Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1300–1309 (1999)Google Scholar
  62. 62.
    Kristian Kersting, K., De Raedt, L.: Bayesian logic programs. In: Cussens, J., Frisch, A.M. (eds.) ILP 2000. LNCS (LNAI), vol. 1866, pp. 138–155. Springer, Heidelberg (2000)Google Scholar
  63. 63.
    Jaeger, J.: Complex probabilistic modeling with recursive relational Bayesian networks. Annals of Mathematics and Artificial Intelligence 32(1-4), 179–220 (2001)CrossRefMathSciNetGoogle Scholar
  64. 64.
    Getoor, L., Friedman, N., Koller, D.: Learning probabilistic models of relational structure. In: Proceedings of the 18th International Conference on Machine Learning (ICML 2001), pp. 170–177 (2001)Google Scholar
  65. 65.
    Kersting, K., De Raedt, L.: Basic principles of learning bayesian logic programs. Technical Report Technical Report No. 174, Institute for Computer Science, University of Freiburg (2002)Google Scholar
  66. 66.
    Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational bayesian networks for exact inference. In: Proceedings of the Second European Workshop on Probabilistic Graphical Models (PGM 2004), pp. 49–56 (2004)Google Scholar
  67. 67.
    Fierens, D., Blockeel, H., Bruynooghe, M., Ramon, J.: Logical Bayesian networks and their relation to other probabilistic logical models. In: Kramer, S., Pfahringer, B. (eds.) ILP 2005. LNCS (LNAI), vol. 3625, pp. 121–135. Springer, Heidelberg (2005)Google Scholar
  68. 68.
    Muggleton, S.: Stochastic logic programs. In: de Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)Google Scholar
  69. 69.
    Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic programs with annotated disjunctions. In: Demoen, B., Lifschitz, V. (eds.) ICLP 2004. LNCS, vol. 3132, pp. 431–445. Springer, Heidelberg (2004)Google Scholar
  70. 70.
    Baral, C., Gelfond, M., Rushton, N.: Probabilistic reasoning with answer sets. In: Lifschitz, V., Niemelä, I. (eds.) LPNMR 2004. LNCS (LNAI), vol. 2923, pp. 21–33. Springer, Heidelberg (2003)Google Scholar
  71. 71.
    Kersting, K., De Raedt, L., Raiko, T.: Logical hidden Markov models. Journal of Artificial Intelligence Research 25, 425–456 (2006)MathSciNetMATHGoogle Scholar
  72. 72.
    De Raedt, L., Angelika, K., Toivonen, H.: ProbLog: A probabilistic Prolog and its application in link discoverry. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI 2007) (2007)Google Scholar
  73. 73.
    Pfeffer, A.: IBAL: A probabilistic rational programming language. In: Proceedings of the 17th International Conference on Artificial Intelligence (IJCAI 2001), pp. 733–740 (2001)Google Scholar
  74. 74.
    Laskey, K.: MEBN: A logic for open-world probabilistic reasoning. C4I Center Technical Report C4I06-01, George Mason University Department of Systems Engineering and Operations Research (2006)Google Scholar
  75. 75.
    Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D., Kolobov, A.: BLOG: Probabilistic models with unknown objects. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), pp. 1352–1359 (2005)Google Scholar
  76. 76.
    Milch, B., Marthi, B., Sontag, D., Russell, S., Ong, D., Kolobov, A.: Approximate Inference for Infinite Contingent Bayesian Networks. In: Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS 2005), pp. 1352–1359 (2005)Google Scholar
  77. 77.
    Pynadath, D.V., Wellman, M.P.: Generalized queries on probabilistic context-free grammars. IEEE Transaction on Pattern Analysis and Machine Intelligence 20(1), 65–77 (1998)CrossRefGoogle Scholar
  78. 78.
    Rauzy, A., Chatelet, E., Dutuit, Y., Berenguer, C.: A practical comparison of methods to assess sum-of-products. Reliability Engineering and System Safety 79, 33–42 (2003)CrossRefGoogle Scholar
  79. 79.
    Stolcke, A.: An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics 21(2), 165–201 (1995)MathSciNetGoogle Scholar
  80. 80.
    Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6(2), 461–464 (1978)MATHCrossRefMathSciNetGoogle Scholar
  81. 81.
    Cheeseman, P., Stutz, J.: Bayesian classification (AutoClass): Theory and results. In: Fayyad, U., Piatesky, G., Smyth, P., Uthurusamy, R. (eds.) Advances in Knowledge Discovery and Data Mining, pp. 153–180. The MIT Press, Cambridge (1995)Google Scholar
  82. 82.
    Izumi, Y., Kameya, Y., Sato, T.: Parallel EM learning for symbolic-statistical models. In: Proceedings of the International Workshop on Data-Mining and Statistical Science (DMSS 2006), pp. 133–140 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Taisuke Sato
    • 1
  • Yoshitaka Kameya
    • 1
  1. 1.Tokyo Institute of Technology, Ookayama Meguro TokyoJapan

Personalised recommendations