Encyclopedia of Machine Learning

2010 Edition
| Editors: Claude Sammut, Geoffrey I. Webb

Statistical Relational Learning

  • Luc De Raedt
  • Kristian Kersting
Reference work entry
DOI: https://doi.org/10.1007/978-0-387-30164-8_786

Definition

Statistical relational learning a.k.a. probabilistic inductive logic programming deals with machine learning and data mining in relational domains where observations may be missing, partially observed, or noisy. In doing so, it addresses one of the central questions of artificial intelligence – the integration of probabilistic reasoning with machine learning and first-order and relational representations – and deals with all related aspects such as reasoning, parameter estimation, and structure learning.

Motivation and Background

One of the central questions of artificial intelligence is concerned with combining expressive knowledge representation formalisms such as relational and first-order logic with principled probabilistic and statistical approaches to inference and learning. While traditionally relational and logical representations, probabilistic and statistical reasoning, and machine learning have been studied independently of one another, statistical relational...
This is a preview of subscription content, log in to check access

Recommended Reading

  1. In addition to the references embedded in the text above, we also recommend De Raedt et al. (2008), Getoor (2007), De Raedt (2008) and the SRL tutorials at major artificial intelligence and machine learning conferences.Google Scholar
  2. Anguelov, D., Taskar, B., Chatalbashev, V., Koller, D., Gupta, D., Heitz, G., et al. (2005). Discriminative learning of Markov random fields for segmentation of 3D scan data. In C. Schmid, S. Soatto, & C. Tomasi (Eds.), IEEE Computer Society international conference on computer vision and pattern recognition (CVPR-05), San Diego, CA, USA (Vol. 2, pp. 169–176).Google Scholar
  3. Chen, J., Muggleton, S., & Santos, J. (2008). Learning probabilistic logic models from probabilistic examples. Machine Learning, 73(1), 55–85.CrossRefGoogle Scholar
  4. Chu, W., Sindhwani, V., Ghahramani, Z., & Keerthi, S. (2006). Relational learning with Gaussian processes. In Advances in Neural information processing systems19 (NIPS-2006). Cambridge, MA: MIT Press.Google Scholar
  5. Cussens, J. (1999). Loglinear models for first-order probabilistic reasoning. In K. Blackmond Laskey & H. Prade (Eds.), Proceedings of the fifteenth annual conference on uncertainty in artificial intelligence (UAI-99) (pp. 126–133), Stockholm, Sweden. San Francisco: Morgan Kaufmann.Google Scholar
  6. Cussens, J. (2001). Parameter estimation in stochastic logic programs. Machine Learning Journal, 44(3), 245–271.MATHCrossRefGoogle Scholar
  7. De Raedt, L. (2008). Logical and relational learning. Springer.Google Scholar
  8. De Raedt, L., Frasconi, P., Kersting, K., & Muggleton, S. (Eds.). (2008). Probabilistic inductive logic programming. Lecture notes in computer science (Vol. 4911). Berlin/Heidelberg: Springer.Google Scholar
  9. De Raedt, L., Kimmig, A., & Toivonen, H. (2007). Problog: A probabilistic Prolog and its application in link discovery. In M. Veloso (Ed.), Proceedings of the 20th international joint conference on artificial intelligence (pp. 2462–2467). Hyderabad, India.Google Scholar
  10. de Salvo Braz, R., Amir, E., & Roth, D. (2005). Lifted first order probabilistic inference. In Proceedings of the 19th international joint conference on artificial intelligence (IJCAI-05) (pp. 1319–1325). Edinburgh, Scotland.Google Scholar
  11. Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39, 1–39.MathSciNetMATHGoogle Scholar
  12. Džeroski, S., & Lavrač, N. (Eds.). (2001). Relational data mining. Berlin: Springer.MATHGoogle Scholar
  13. Eisele, A. (1994). Towards probabilistic extensions of contraint-based grammars. In J. Dörne (Ed.), Computational aspects of constraint-based linguistics description-II. Stuttgart: Institute for Computational Linguistics (IMS-CL). DYNA-2 deliverable R1.2.B.Google Scholar
  14. Getoor, L. (2001). Learning statistical models from relational data. PhD thesis, Stanford University, USA.Google Scholar
  15. Getoor, L., Rhee, J., Koller, D., & Small, P. (2004). Understanding tuberculosis epidemiology using probabilistic relational models. Journal of Artificial Intelligence in Medicine, 30, 233–256.CrossRefGoogle Scholar
  16. Getoor, L., & Taskar, B. (Eds.). (2007). Introduction to statistical relational learning. Cambridge, MA, USA: The MIT Press.Google Scholar
  17. Getoor, L., Taskar, B., & Koller, D. (2001). Using probabilistic models for selectivity estimation. In Proceedings of ACM SIGMOD international conference on management of data (pp. 461–472). Santa Barbara, CA, USA: ACM Press.Google Scholar
  18. Gutmann, B., & Kersting, K. (2006). TildeCRF: Conditional random fields for logical sequences. In J. Fuernkranz, T. Scheffer, & M. Spiliopoulou (Eds.), Proceedings of the 17th European conference on machine learning (ECML-2006), Berlin, Germany (pp. 174–185).Google Scholar
  19. Jäger, M. (1997). Relational Bayesian networks. In K. Laskey & H. Prade (Eds.), Proceedings of the thirteenth conference on uncertainty in artificial intelligence (UAI-97), Stockholm, Sweden (pp. 266–273). San Franciso, CA, USA: Morgan Kaufmann.Google Scholar
  20. Jaimovich, A., Meshi, O., & Friedman, N. (2007). Template-based inference in symmetric relational Markov random fields. In Proceedings of the conference on uncertainty in artificial intelligence (UAI-07) (pp. 191–199).Google Scholar
  21. Kemp, C., Tenenbaum, J., Griffiths, T., Yamada, T., & Ueda, N. (2006). Learning systems of concepts with an infinite relational model. In Proceedings of 21st AAAI.Google Scholar
  22. Kersting, K., Ahmadi, B., & Natarajan, S. (2009). Counting belief propagation. In Proceedings of the 25th conference on uncertainty in artificial intelligence (UAI-09). Montreal, Canada.Google Scholar
  23. Kersting, K., & De Raedt, L. (2007). Bayesian logic programming: Theory and tool. In L. Getoor & B. Taskar (Eds.), An introduction to statistical relational learning (pp. 291–321). Cambridge, MA, USA: MIT Press.Google Scholar
  24. Kersting, K., De Raedt, L., & Raiko, T. (2006). Logial Hidden Markov Models. Journal of Artificial Intelligence Research (JAIR), 25, 425–456.MATHGoogle Scholar
  25. Kersting, K., & Xu, Z. (2009). Learning preferences with hidden common cause relations. In Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD 09). LNAI, Bled, Slovenia, Springer.Google Scholar
  26. Kisynski, J., & Poole, D. (2009). Lifted aggregation in directed first-order probabilistic models. In C. Boutilier (Ed.), Proceedings of the international joint conference on artificial intelligence (IJCAI-09). Pasadena, CA, USA.Google Scholar
  27. Kok, S., & Domingos, P. (2007). Statistical predicate invention. In Proceedings of the twenty-fourth international conference on machine learning (ICML-07), Corvallis, OR, USA (pp. 433–440). ACM Press.CrossRefGoogle Scholar
  28. Limketkai, B., Liao, L., & Fox, D. (2005). Relational object maps for mobile robots. In F. Giunchiglia & L. P. Kaelbling (Eds.), Proceedings of the nineteenth international joint conference on artificial intelligence (IJCAI-05), Edinburgh, Scotland (pp. 1471–1476). AAAI Press.Google Scholar
  29. McGovern, A., Friedland, L., Hay, M., Gallagher, B., Fast, A., Neville, J., et al. (2003). Exploiting relational structure to understand publication patterns in high-energy physics. SIGKDD Explorations, 5(2), 165–173.CrossRefGoogle Scholar
  30. McLachlan, G., & Krishnan, T. (1997). The EM algorithm and extensions. New York: Wiley.MATHGoogle Scholar
  31. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D., & Kolobov, A. (2005). BLOG: Probabilistic models with unknown objects. In F. Giunchiglia, L. P. Kaelbling (Eds.), Proceedings of the nineteenth international joint conference on artificial intelligence (IJCAI-05), Edinburgh, Scotland (pp. 1352–1359). Edinburgh, Scotland: AAAI Press.Google Scholar
  32. Milch, B., Zettlemoyer, L., Kersting, K., Haimes, M., & Pack Kaelbling, L. (2008). Lifted probabilistic inference with counting formulas. In Proceedings of the 23rd AAAI conference on artificial intelligence (AAAI-08).Google Scholar
  33. Muggleton, S. (1996). Stochastic logic programs. In L. De Raedt (Ed.), Advances in inductive logic programming (pp. 254–264). Amsterdam: IOS Press.Google Scholar
  34. Muggleton, S., & De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19(20), 629–679.MathSciNetCrossRefGoogle Scholar
  35. Neville, J., & Jensen, D. (2004). Dependency networks for relational data. In R. Rastogi, K. Morik, M. Bramer, & X. Wu (Eds.), Proceedings of the fourth IEEE international conference on data mining (ICDM-04), Brighton, UK (pp. 170–177). IEEE Computer Society Press.Google Scholar
  36. Neville, J., Simsek, Ö., Jensen, D., Komoroske, J., Palmer, K., & Goldberg, H. (2005). Using relational knowledge discovery to prevent securities fraud. In Proceedings of the 11th ACM SIGKDD international conference on knowledge discovery and data mining. Chicago, IL, USA: ACM Press.Google Scholar
  37. Ngo, L., & Haddawy, P. (1997). Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science, 171, 147–177.MathSciNetMATHCrossRefGoogle Scholar
  38. Passerini, A., Frasconi, P., & De Raedt, L. (2006). Kernels on prolog proof trees: Statistical learning in the ILP setting. Journal of Machine Learning Research, 7, 307–342.Google Scholar
  39. Pfeffer, A. (2000). Probabilistic reasoning for complex systems. PhD thesis, Computer Science Department, Stanford University.Google Scholar
  40. Poole, D. (1993). Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence Journal, 64, 81–129.MATHCrossRefGoogle Scholar
  41. Poole, D. (1997). The independent choice logic for modelling multiple agents under uncertainty. Artificial Intelligence, 94(1–2), 7–56.MathSciNetMATHCrossRefGoogle Scholar
  42. Poole, D. (2003). First-order probabilistic inference. In G. Gottlob & T. Walsh (Eds.), Proceedings of the eighteenth international joint conference on artificial intelligence (IJCAI-03), Acapulco, Mexico (pp. 985–991). San Francisco: Morgan Kaufmann.Google Scholar
  43. Poon, H., & Domingos, P. (2008). Joint unsupervised coreference resolution with markov logic. In: Proceedings of the 2008 conference on empirical methods in natural language processing (EMNLP), Honolulu, HI, USA.Google Scholar
  44. Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 conference on empirical methods in natural language processing (EMNLP), Singapore.Google Scholar
  45. Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning, 62, 107–136.CrossRefGoogle Scholar
  46. Sato, T. (1995). A statistical learning method for logic programs with distribution semantics. In L. Sterling (Ed.), Proceedings of the twelfth international conference on logic programming (ICLP-95), Tokyo, Japan (pp. 715–729). MIT Press.Google Scholar
  47. Segal, E., Battle, A., & Koller, D. (2003). Decomposing gene expression into cellular processes. In Proceedings of Pacific symposium on biocomputing (PSB) (pp. 89–100). World Scientific.Google Scholar
  48. Segal, E., Taskar, B., Gasch, A., Friedman, N., & Koller, D. (2001). Rich probabilistic models for gene expression. Bioinformatics, 17(Suppl. 1), S243–252 (Proceedings of ISMB 2001).Google Scholar
  49. Sen, P., Deshpande, A., & Getoor, L. (2008). Exploiting shared correlations in probabilistic databases. In Proceedings of the international conference on very large data bases (VLDB-08). Auckland, New Zealand.Google Scholar
  50. Silva, R., Chu, W., & Ghahramani, Z. (2007). Hidden common cause relations in relational learning. In Advances in Neural information processing systems20 (NIPS-2007). Cambridge, MA: MIT Press.Google Scholar
  51. Singh, A., & Gordon, G. (2008). Relational learning via collective matrix factorization. In Proceedings of 14th international conference on knowledge discovery and data mining. Las Vegas, US.Google Scholar
  52. Singla, P., & Domingos, P. (2008). Lifted first-order belief propagation. In Proceedings of the 23rd AAAI conference on artificial intelligence (AAAI-08), Chicago, IL, USA (pp. 1094–1099).Google Scholar
  53. Taskar, B., Abbeel, P., & Koller, D. (2002). Discriminative probabilistic models for relational data. In A. Darwiche & N. Friedman (Eds.), Proceedings of the eighteenth conference on uncertainty in artificial intelligence (UAI-02), Edmonton, Alberta, Canada (pp. 485–492).Google Scholar
  54. Wang, J., & Domingos, P. (2008). Hybrid markov logic networks. In Proceedings of the 23rd AAAI conference on artificial intelligence (AAAI-08), Chicago, IL, USA (pp. 1106–1111).Google Scholar
  55. Xu, Z., Kersting, K., & Tresp, V. (2009). Multi-relational learning with Gaussian processes. In C. Boutilier (Ed.) Proceedings of the international joint conference on artificial intelligence (IJCAI-09). Pasadena, CA.Google Scholar
  56. Xu, Z., Tresp, V., Rettinger, A., & Kersting, K. (2010). Social network mining with nonparametric relational models. In Advances in social network mining and analysis. Lecture Notes in Computer Science (Vol. 5498), Berlin/Heidelberg: Springer.Google Scholar
  57. Xu, Z., Tresp, V., Yu, K., & Kriegel, H. P. (2006). Infinite hidden relational models. In Proceedings of 22nd UAI.Google Scholar
  58. Yu, K., & Chu, W. (2007). Gaussian process models for link analysis and transfer learning. In Advances in Neural information processing systems20 (NIPS-2007). Cambridge, MA: MIT Press.Google Scholar
  59. Yu, K., Chu, W., Yu, S., Tresp, V., & Xu, Z. (2006). Stochastic relational models for discriminative link prediction. In Advances in Neural information processing systems19 (NIPS-2006). Cambridge, MA: MIT Press.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Luc De Raedt
  • Kristian Kersting

There are no affiliations available