Journal of Intelligent Information Systems

, Volume 31, Issue 2, pp 161–176 | Cite as

A glimpse of symbolic-statistical modeling by PRISM

Article

Abstract

We give a brief overview of a logic-based symbolic modeling language PRISM which provides a unified approach to generative probabilistic models including Bayesian networks, hidden Markov models and probabilistic context free grammars. We include some experimental result with a probabilistic context free grammar extracted from the Penn Treebank. We also show EM learning of a probabilistic context free graph grammar as an example of exploring a new area.

Keywords

Symbolic-statistical modeling PRISM Probabilistic context free grammar 

References

  1. Baker, J. K. (1979). Trainable grammars for speech recognition. In Proceedings of Spring conference of the acoustical society of America (pp. 547–550).Google Scholar
  2. Boutilier, C., Friedman, N., Goldszmidt, M., & Koller, D. (1996). Context-specific independence in Bayesian networks. In Proceeding of the 12th conference on uncertainty in artificial intelligence (UAI’96) (pp. 115–123).Google Scholar
  3. Breese, J. S. (1992). Construction of belief and decision networks. Computational Intelligence, 8, 624–647.CrossRefGoogle Scholar
  4. Chavira, M., & Darwiche, A. (2005). Compiling Bayesian networks with local structure. In Proceedings of the 19th international joint conference on artificial intelligence (IJCAI’05) (pp. 1306–1312)Google Scholar
  5. Chavira, M., Darwiche, A., & Jaeger, M. (2004). Compiling relational bayesian networks for exact inference. In Proceedings of the second European workshop on probabilistic graphical models (PGM’04) (pp. 49–56).Google Scholar
  6. Chow, Y., & Teicher, H. (1997). Probability theory (3rd ed.). New York: SpringerMATHGoogle Scholar
  7. Cussens, J. (2001). Parameter estimation in stochastic logic programs. Machine Learning, 44, 245–271.MATHCrossRefGoogle Scholar
  8. Darwiche, A. (2002). A logical approach to factoring belief networks. In Proceedings of the eighth international conference on principles of knowledge representation and reasoning (KR’02) (pp. 409–420).Google Scholar
  9. De Raedt, L., & Kersting, K. (2003). Probabilistic logic learning. ACM-SIGKDD Explorations, Special Issue on Multi-Relational Data Mining, 5, 31–48.Google Scholar
  10. Dean, T., & Kanazawa, K. (1989). A model for reasoning about persistence and causation. Computational Intelligence, 5, 142–150.CrossRefGoogle Scholar
  11. Dechter, R. (1996). Bucket elimination: A unifying framework for probabilistic inference. In Proceedings of the 12th conference on uncertainty in artificial intelligence (UAI’96) (pp. 211–219).Google Scholar
  12. Dehaspe, L., Toivonen, H., & King, R. D. (1998). Finding frequent substructures in chemical compounds. In R. Agrawal, P. Stolorz, & G. Piatetsky-Shapiro (Eds.), In Proceedings of the 4th international conference on knowledge discovery and data mining (pp. 30–36). AAAI Press.Google Scholar
  13. Dempster, A. P., Laird, N. M., Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Royal Statistical Society, B39, 1–38.MathSciNetGoogle Scholar
  14. Doets, K. (1994). From logic to logic programming. Cambridge: MIT.MATHGoogle Scholar
  15. Domingos, P., Richardson, M. (2004). Markov logic: A unifying framework for statistical relational learning. In Proceedings of the ICML-2004 workshop on statistical relational learning and its connections to other fields (SRL’04).Google Scholar
  16. Fierens, D., Blockeel, H., Bruynooghe, M., & Ramon, J. (2005). Logical Bayesian networks and their relation to other probabilistic logical models. In Proceedings of the 15th international conference on inductive logic programming (ILP’05). Lecture Notes in Computer Science, (Vol. 3625, pp. 121–135).Google Scholar
  17. Friedman, N., Getoor, L., Koller, D., & Pfeffer, A. (1999). Learning probabilistic relational models. In Proceedings of the 16th international joint conference on artificial intelligence (IJCAI’99) (pp. 1300–1309).Google Scholar
  18. Getoor, L., Friedman, N., & Koller, D. (2001). Learning probabilistic models of relational structure. In Proceedings of the 18th international conference on machine learning (ICML’01) (pp. 170–177).Google Scholar
  19. Hinton, A., Kwiatkowska, M., Norman, G., & Parker, D. (2006). PRISM: A tool for automatic verification of probabilistic systems. In Proceedings of the 12th international conference on tools and algorithms for the construction and analysis of systems (TACAS’06). LNCS (Vol. 3920, pp. 441–444). New York: Springer.Google Scholar
  20. Jaeger, J. (2001). Complex probabilistic modeling with recursive relational Bayesian networks. Annals of Mathematics and Artificial Intelligence, 32, 179–220.CrossRefMathSciNetGoogle Scholar
  21. Kameya, Y., & Sato, T. (2000). Efficient EM learning for parameterized logic programs. In Proceedings of the 1st conference on computational logic (CL’00). Lecture Notes in Artificial Intelligence (Vol. 1861, pp. 269–294). New York: Springer.Google Scholar
  22. Kersting, K., & De Raedt, L. (2002). Basic principles of learning bayesian logic programs. Technical Report Technical Report No. 174, Institute for Computer Science, University of Freiburg.Google Scholar
  23. Kersting, K., De Raedt, L., & Raiko, T. (2006). Logical hidden Markov models. Journal of Artificial Intelligence Research, 25, 425–456.MathSciNetGoogle Scholar
  24. Koller, D., & Pfeffer, A. (1997). Learning probabilities for noisy first-order rules. In Proceedings of the 15th international joint conference on artificial intelligence (IJCAI’97) (pp. 1316–1321).Google Scholar
  25. Kristian Kersting, K., De Raedt, L. (2000). Bayesian logic programs. In Proceedings of the work- in-progress track at the 10th international conference on inductive logic programming (ILP ’00) (pp. 138–155).Google Scholar
  26. Lakshmanan, L. V. S., & Sadri, F. (1994). Probabilistic deductive databases. In Proceedings of the 1994 international symposium on logic programming (ILPS’94) (pp. 254–268).Google Scholar
  27. Lari, K., & Young, S. J. (1990). The estimation of stochastic context-free grammars using the Inside-Outside algorithm. Computer Speech and Language, 4, 35–56.CrossRefGoogle Scholar
  28. Laskey, K. (2006). MEBN: A logic for open-world probabilistic reasoning. C4I Center Technical Report C4I06-01. George Mason University Department of Systems Engineering and Operations Research.Google Scholar
  29. Lauritzen, S., & Spiegelhalter, D. (1988). Local computations with probabilities on graphical structures and their applications to expert systems. Journal of the Royal Statistical Society, B50, 157–224.MATHMathSciNetGoogle Scholar
  30. Lukasiewicz, T. (1999). Probabilistic deduction with conditional constraints over basic events. Journal of Artificial Intelligence Research, 10, 199–241.MATHMathSciNetGoogle Scholar
  31. Marcus, M., Santorini, B., & Marcinkiewicz, M. (1993). Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19, 313–330.Google Scholar
  32. McLachlan, G. J., & Krishnan, T. (1997). The EM algorithm and extensions. New York: Wiley Interscience.MATHGoogle Scholar
  33. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D., & Kolobov, A. (2005). BLOG: Probabilistic models with unknown objects. In Proceedings of the 19th international joint conference on artificial intelligence (IJCAI’05) (pp. 1352–1359).Google Scholar
  34. Muggleton, S. (1996). Stochastic logic programs. In L. de Raedt (Ed.), Advances in inductive logic programming (pp. 254–264). Amsterdam: IOS.Google Scholar
  35. Ng, R., & Subrahmanian, V. S. (1992). Probabilistic logic programming. Information and Computation, 101, 150–201.MATHCrossRefMathSciNetGoogle Scholar
  36. Ngo, L., & Haddawy, P. (1997). Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science, 171, 147–177.MATHCrossRefMathSciNetGoogle Scholar
  37. Oates, T., Doshi, S., & Huang, F. (2003). Estimating maximum likelihood parameters for stochastic context-free graph grammars. In Proceedings of the 13th international conference on inductive logic programming (ILP’03) (pp. 281–298)Google Scholar
  38. Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Francisco: Morgan KaufmannGoogle Scholar
  39. Poole, D. (1993). Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence, 64, 81–129.MATHCrossRefGoogle Scholar
  40. Pynadath, D. V., & Wellman, M. P. (1996). Generalized queries on probabilistic context-free grammars. In Proceedings of the 14th national conference on artificial intelligence (AAAI’96) (pp. 1285–1290).Google Scholar
  41. Rozenberg, G. (Ed.) (1997). Handbook of graph grammars and computing by graph transformations. (Vol. 1). Hackensack: Foundations World Scientific.MATHGoogle Scholar
  42. Sato, T. (1995). A statistical learning method for logic programs with distribution semantics. In Proceedings of the 12th international conference on logic programming (ICLP’95) (pp. 715–729).Google Scholar
  43. Sato, T. (2005). A generic approach to EM learning for symbolic-statistical models. In Proceedings of the 4th learning language in logic workshop (LLL’05).Google Scholar
  44. Sato, T. (2007). Inside-Outside probability computation for belief propagation. In Proceedings of the 20th international joint conference on artificial intelligence (IJCAI’07) (pp. 00–00).Google Scholar
  45. Sato, T., & Kameya, Y. (2001). Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research, 15, 391–454.MATHMathSciNetGoogle Scholar
  46. Sato, T., & Kameya, Y. (2004a). A dynamic programming approach to parameter learning of generative models with failure. In Proceedings of ICML 2004 workshop on learning statistical models from relational data (SRL ’04).Google Scholar
  47. Sato, T., & Kameya, Y. (2004b). Negation elimination for finite PCFGs. In Proceedings of the international symposium on logic-based program synthesis and transformation 2004 (LOPSTR’04) (pp. 119–134).Google Scholar
  48. Sato, T., Kameya, Y., & Zhou, N. F. (2005). Generative modeling with failure in PRISM. In Proceedings of the 19th international joint conference on artificial intelligence (IJCAI’05) (pp. 847–852).Google Scholar
  49. Shan, B. (2005). Stochastic context-free graph grammars for glycoprotein modelling. In Implementation and Application of Automata, Lecture Notes in Computer Science (Vol. 3317, pp. 247–258).Google Scholar
  50. Van Gelder, A., Ross, K., & Schlipf, J. (1991). The well-founded semantics for general logic programs. The Journal of ACM (JACM), 38(3), 620–650.MATHGoogle Scholar
  51. Vennekens, J., Verbaeten, S., & Bruynooghe, M. (2004). Logic programs with annotated disjunctions. In Proceedings of the 20th international conference on logic programming (ICLP’04). Lecture Notes in Computer Science (Vol. 3132, pp. 431–445).Google Scholar
  52. Washio, T., & Motoda, H. (2003). State of the art of graph-based data mining. SIGKDD Explorations, 5, 59–68.CrossRefGoogle Scholar
  53. Zhang, N., & Poole, D. (1996). Exploiting causal independence in Bayesian network inference. Journal of Artificial Intelligence Research, 5, 301–328.MATHMathSciNetGoogle Scholar
  54. Zhou, N. F., & Sato, T. (2003). Efficient fixpoint computation in linear tabling. In Proceedings of the 5th ACM-SIGPLAN international conference on principles and practice of declarative programming (PPDP’03) (pp. 275–283).Google Scholar
  55. Zhou, N. F., Shen, Y., & Sato, T. (2004). Semi-naive evaluation in linear tabling. In Proceedings of the 6th ACM-SIGPLAN international conference on principles and practice of declarative programming (PPDP ’04) (pp. 90–97).Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Tokyo Institute of TechnologyTokyoJapan

Personalised recommendations