Skip to main content
SpringerLink
Log in

Probabilistic Inductive Logic Programming

  • Chapter
Probabilistic Inductive Logic Programming

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4911))

Abstract

Probabilistic inductive logic programming aka. statistical relational learning addresses one of the central questions of artificial intelligence: the integration of probabilistic reasoning with machine learning and first order and relational logic representations. A rich variety of different formalisms and learning techniques have been developed. A unifying characterization of the underlying learning settings, however, is missing so far.

In this chapter, we start from inductive logic programming and sketch how the inductive logic programming formalisms, settings and techniques can be extended to the statistical case. More precisely, we outline three classical settings for inductive logic programming, namely learning from entailment, learning from interpretations, and learning from proofs or traces, and show how they can be adapted to cover state-of-the-art statistical relational learning approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abney, S.P.: Stochastic Attribute-Value Grammars. Computational Linguistics 23(4), 597–618 (1997)

    MathSciNet  Google Scholar 

  2. Anderson, C.R., Domingos, P., Weld, D.S.: Relational Markov Models and their Application to Adaptive Web Navigation. In: Hand, D., Keim, D., Ng, R. (eds.) Proceedings of the Eighth International Conference on Knowledge Discovery and Data Mining (KDD 2002), Edmonton, Canada, July 2002, pp. 143–152. ACM Press, New York (2002)

    Chapter  Google Scholar 

  3. Bangsø, O., Langseth, H., Nielsen, T.D.: Structural learning in object oriented domains. In: Russell, I., Kolen, J. (eds.) Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2001), Key West, Florida, USA, pp. 340–344. AAAI Press, Menlo Park (2001)

    Google Scholar 

  4. Cussens, J.: Loglinear models for first-order probabilistic reasoning. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the Fifteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI 1999), Stockholm, Sweden, pp. 126–133. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  5. Cussens, J.: Parameter estimation in stochastic logic programs. Machine Learning Journal 44(3), 245–271 (2001)

    Article  MATH  Google Scholar 

  6. Cussens, J.: Integrating by separating: Combining probability and logic with ICL, PRISM and SLPs. Technical report, APrIL Projetc (January 2005)

    Google Scholar 

  7. De Raedt, L.: Logical settings for concept-learning. Artificial Intelligence Journal 95(1), 197–201 (1997)

    Article  Google Scholar 

  8. De Raedt, L., Džeroski, S.: First-Order jk-Clausal Theories are PAC-Learnable. Artificial Intelligence Journal 70(1-2), 375–392 (1994)

    Article  MATH  Google Scholar 

  9. De Raedt, L., Kersting, K.: Probabilistic Logic Learning. ACM-SIGKDD Explorations: Special issue on Multi-Relational Data Mining 5(1), 31–48 (2003)

    Google Scholar 

  10. De Raedt, L., Kersting, K., Torge, S.: Towards learning stochastic logic programs from proof-banks. In: Veloso, M., Kambhampati, S. (eds.) Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA, July 9–13, 2005, pp. 752–757. AAAI (2005)

    Google Scholar 

  11. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39, 1–39 (1977)

    MathSciNet  Google Scholar 

  12. Domingos, P., Richardson, M.: Markov Logic: A Unifying Framework for Statistical Relational Learning. In: Dietterich, T.G., Getoor, L., Murphy, K. (eds.) Proceedings of the ICML-2004 Workshop on Statistical Relational Learning and its Connections to Other Fields (SRL 2004), Banff, Alberta, Canada, July 8, 2004, pp. 49–54 (2004)

    Google Scholar 

  13. Džeroski, S., Lavrač, N. (eds.): Relational data mining. Springer, Berlin (2001)

    Google Scholar 

  14. Eisele, A.: Towards Probabilistic Extensions of Contraint-based Grammars. In: Dörne, J. (ed.) Computational Aspects of Constraint-Based Linguistics Decription-II, DYNA-2 deliverable R1.2.B (1994)

    Google Scholar 

  15. Flach, P.: Simply Logical: Intelligent Reasoning by Example. John Wiley, Chichester (1994)

    MATH  Google Scholar 

  16. Flach, P.A., Lachiche, N.: Naive Bayesian classification of structured data. Machine Learning Journal 57(3), 233–269 (2004)

    Article  MATH  Google Scholar 

  17. Getoor, L.: Learning Statistical Models from Relational Data. PhD thesis, Stanford University, USA (June 2001)

    Google Scholar 

  18. Getoor, L., Friedman, N., Koller, D., Taskar, B.: Learning Probabilistic Models of Link Structure. Journal of Machine Leaning Research (JMLR) 3, 679–707 (2002)

    Article  MathSciNet  Google Scholar 

  19. Heckerman, D.: A Tutorial on Learning with Bayesian Networks. Technical Report MSR-TR-95-06, Microsoft Research (1995)

    Google Scholar 

  20. Helft, N.: Induction as nonmonotonic inference. In: Brachman, R.J., Levesque, H.J. (eds.) Proceedings of the First International Conference on Principles of Knowledge Representation and Reasoning(KR 1989), Toronto, Canada, May 15-18, 1989, pp. 149–156. Morgan Kaufmann, San Francisco (1989)

    Google Scholar 

  21. Jäger, M.: Relational Bayesian Networks. In: Laskey, K.B., Prade, H. (eds.) Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence (UAI 1997), Stockholm, Sweden, July 30–August 1, 1997, pp. 266–273. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  22. Jensen, F.V.: Bayesian networks and decision graphs. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  23. Kameya, Y., Sato, T., Zhou, N.-G.: Yet more efficient EM learning for parameterized logic programs by inter goal sharing. In: de Mantaras, R.L., Saitta, L. (eds.) Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004), Valencia, Spain, August 22-27, 2004, pp. 490–494. IOS Press, Amsterdam (2004)

    Google Scholar 

  24. Kersting, K.: Bayes’sche-logische Programme. Master’s thesis, Institute for Computer Science, University of Freiburg (2000)

    Google Scholar 

  25. Kersting, K., De Raedt, L.: Bayesian logic programs. Technical Report 151, Institute for Computer Science, University of Freiburg, Freiburg, Germany (April 2001)

    Google Scholar 

  26. Kersting, K., De Raedt, L.: Towards Combining Inductive Logic Programming with Bayesian Networks. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, pp. 118–131. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  27. Kersting, K., De Raedt, L.: Bayesian Logic Programming: Theory and Tool. In: Getoor, L., Taskar, B. (eds.) An Introduction to Statistical Relational Learning, pp. 291–321. MIT Press, Cambridge (2007)

    Google Scholar 

  28. Kersting, K., De Raedt, L., Raiko, T.: Logial Hidden Markov Models. Journal of Artificial Intelligence Research (JAIR) 25, 425–456 (2006)

    Google Scholar 

  29. Kersting, K., Raiko, T.: ’Say EM’ for Selecting Probabilistic Models for Logical Sequences. In: Bacchus, F., Jaakkola, T. (eds.) Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI 2005), Edinburgh, Scotland, July 26-29, 2005, pp. 300–307 (2005)

    Google Scholar 

  30. Koller, D., Pfeffer, A.: Learning probabilities for noisy first order rules. In: Pollack, M. (ed.) Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI 1997), Nagoya, Japan, pp. 1316–1321. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  31. Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: Integrating Naïve Bayes and Foil. In: Veloso, M., Kambhampati, S. (eds.) Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI 2005), Pittsburgh, Pennsylvania, USA, July 9–13, 2005, pp. 795–800. AAAI Press, Menlo Park (2005)

    Google Scholar 

  32. Lavrač, N., Džeroski, S.: Inductive Logic Programming. Ellis Horwood (1994)

    Google Scholar 

  33. Lloyd, J.W.: Foundations of Logic Programming, 2nd edn. Springer, Berlin (1989)

    Google Scholar 

  34. Manning, C.H., Schütze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  35. Marcus, M., Kim, G., Marcinkiewicz, M.A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., Schasberger, B.: The Penn treebank: Annotating predicate argument structure. In: Weinstein, C.J. (ed.) In ARPA Human Language Technology Workshop, Plainsboro, NJ, USA, March 8–11, 1994, pp. 114–119 (1994)

    Google Scholar 

  36. McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (1997)

    MATH  Google Scholar 

  37. Muggleton, S.H.: Inverse Entailment and Progol. New Generation Computing Journal, 245–286 (1995)

    Google Scholar 

  38. Muggleton, S.H.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)

    Google Scholar 

  39. Muggleton, S.H.: Learning Stochastic Logic Programs. Electronic Transactions in Artificial Intelligence 4(041) (2000)

    Google Scholar 

  40. Muggleton, S.H.: Learning stochastic logic programs. In: Getoor, L., Jensen, D. (eds.) Working Notes of the AAAI-2000 Workshop on Learning Statistical Models from Relational Data (SRL 2000), Austin, Texas, July 31, 2000, pp. 36–41. AAAI Press, Menlo Park (2000)

    Google Scholar 

  41. Muggleton, S.H.: Learning Structure and Parameters of Stochastic Logic Programs. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  42. Muggleton, S.H., Feng, C.: Efficient Induction of Logic Programs. In: Muggleton, S.H. (ed.) Inductive Logic Programming, Acadamic Press (1992)

    Google Scholar 

  43. Muggleton, S.H., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19(20), 629–679 (1994)

    Article  MathSciNet  Google Scholar 

  44. Neville, J., Jensen, D.: Dependency Networks for Relational Data. In: Rastogi, R., Morik, K., Bramer, M., Wu, X. (eds.) Proceedings of The Fourth IEEE International Conference on Data Mining (ICDM 2004), Brighton, UK, November 1–4, 2004, pp. 170–177. IEEE Computer Society Press, Los Alamitos (2004)

    Chapter  Google Scholar 

  45. Ngo, L., Haddawy, P.: Answering Queries from Context-Sensitive Probabilistic Knowledge Bases. Theoretical Computer Science 171, 147–177 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  46. Pearl, J.: Reasoning in Intelligent Systems: Networks of Plausible Inference, 2nd edn. Morgan Kaufmann, San Francisco (1991)

    Google Scholar 

  47. Pfeffer, A.J.: Probabilistic Reasoning for Complex Systems. PhD thesis, Computer Science Department, Stanford University (December 2000)

    Google Scholar 

  48. Plotkin, G.D.: A note on inductive generalization. In: Machine Intelligence, vol. 5, pp. 153–163. Edinburgh University Press (1970)

    Google Scholar 

  49. Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence Journal 64, 81–129 (1993)

    Article  MATH  Google Scholar 

  50. Quinlan, J.R., Cameron-Jones, R.M.: Induction of logic programs: FOIL and related systems. New Generation Computing, 287–312 (1995)

    Google Scholar 

  51. Riguzzi, F.: Learning logic programs with annotated disjunctions. In: Camacho, R., King, R., Srinivasan, A. (eds.) ILP 2004. LNCS (LNAI), vol. 3194, pp. 270–287. Springer, Heidelberg (2004)

    Google Scholar 

  52. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice-Hall, Inc., Englewood Cliffs (2002)

    Google Scholar 

  53. Sato, T.: A Statistical Learning Method for Logic Programs with Distribution Semantics. In: Sterling, L. (ed.) Proceedings of the Twelfth International Conference on Logic Programming (ICLP 1995), Tokyo, Japan, pp. 715–729. MIT Press, Cambridge (1995)

    Google Scholar 

  54. Sato, T., Kameya, Y.: Parameter Learning of Logic Programs for Symbolic-Statistical Modeling. Journal of Artificial Intelligence Research (JAIR) 15, 391–454 (2001)

    MATH  MathSciNet  Google Scholar 

  55. Shapiro, E.: Algorithmic Program Debugging. MIT Press, Cambridge (1983)

    Google Scholar 

  56. Srinivasan, A.: The Aleph Manual (1999), Available at: http://www.comlab.ox.ac.uk/oucl/~research/areas/machlearn/Aleph/

  57. Srinivasan, A., Muggleton, S.H., King, R.D., Sternberg, M.J.E.: Theories for Mutagenicity: A Study of First-Order and Feature based Induction. Artificial Intelligence Journal 85, 277–299 (1996)

    Article  Google Scholar 

  58. Sterling, L., Shapiro, E.: The Art of Prolog: Advanced Programming Techniques. MIT Press, Cambridge (1986)

    MATH  Google Scholar 

  59. Stolcke, A., Omohundro, S.: Hidden Markov model induction by Bayesian model merging. In: Hanson, S.J., Cowan, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 11–18. Morgan Kaufmann, San Francisco (1993); (Proceedings of NIPS-92, Denver, Colorado, USA, November 30–December 3 (1992)

    Google Scholar 

  60. Taskar, B., Abbeel, P., Koller, D.: Discriminative Probabilistic Models for Relational Data. In: Darwiche, A., Friedman, N. (eds.) Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI 2002), Edmonton, Alberta, Canada, August 1-4, 2002, pp. 485–492 (2002)

    Google Scholar 

  61. Valiant, L.G.: A theory of the Learnable. Communications of the ACM 27(11), 1134–1142 (1984)

    Article  MATH  Google Scholar 

  62. Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic Programs with Annotated Disjunctions. In: Demoen, B., Lifschitz, V. (eds.) Proceedings of 20th International Conference on Logic Programming (ICLP 2004), Saint-Malo, France, September 6-10, 2004, pp. 431–445 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departement Computerwetenschappen, K.U. Leuven, Celestijnenlaan 200A - bus 2402, B-3001, Heverlee, Belgium

    Luc De Raedt

  2. CSAIL, Massachusetts Institute of Technologie, 32 Vassar Street, Cambridge, MA, 02139-4307, USA

    Kristian Kersting

Authors
  1. Luc De Raedt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristian Kersting
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Luc De Raedt Paolo Frasconi Kristian Kersting Stephen Muggleton

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

De Raedt, L., Kersting, K. (2008). Probabilistic Inductive Logic Programming. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds) Probabilistic Inductive Logic Programming. Lecture Notes in Computer Science(), vol 4911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78652-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78652-8_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78651-1

  • Online ISBN: 978-3-540-78652-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation