First-Order Probabilistic Languages: Into the Unknown

  • Brian Milch
  • Stuart Russell
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4455)


This paper surveys first-order probabilistic languages (FOPLs), which combine the expressive power of first-order logic with a probabilistic treatment of uncertainty. We provide a taxonomy that helps make sense of the profusion of FOPLs that have been proposed over the past fifteen years. We also emphasize the importance of representing uncertainty not just about the attributes and relations of a fixed set of objects, but also about what objects exist. This leads us to Bayesian logic, or BLOG, a language for defining probabilistic models with unknown objects. We give a brief overview of BLOG syntax and semantics, and emphasize some of the design decisions that distinguish it from other languages. Finally, we consider the challenge of constructing FOPL models automatically from data.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Elidan, G., Friedman, N.: Learning hidden variable networks: The information bottleneck approach. JMLR 6, 81–127 (2005)MathSciNetGoogle Scholar
  2. 2.
    Fierens, D., Blockeel, H., Bruynooghe, M., Ramon, J.: Logical Bayesian networks and their relation to other probabilistic logical models. In: Kramer, S., Pfahringer, B. (eds.) ILP 2005. LNCS (LNAI), vol. 3625, Springer, Heidelberg (2005)Google Scholar
  3. 3.
    Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proc. 16th IJCAI, pp. 1300–1307 (1999)Google Scholar
  4. 4.
    Getoor, L., Friedman, N., Koller, D., Taskar, B.: Learning probabilistic models of relational structure. In: Proc. 18th ICML, pp. 170–177 (2001)Google Scholar
  5. 5.
    Halpern, J.Y.: An analysis of first-order logics of probability. Artificial Intelligence 46, 311–350 (1990)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Heckerman, D., Meek, C., Koller, D.: Probabilistic models for relational data. Technical Report MSR-TR-2004-30, Microsoft Research (2004)Google Scholar
  7. 7.
    Jaeger, M.: Relational Bayesian networks. In: Proc. 13th UAI, pp. 266–273 (1997)Google Scholar
  8. 8.
    Kersting, K., Raedt, L.D.: Adaptive Bayesian logic programs. In: Rouveirol, C., Sebag, M. (eds.) ILP 2001. LNCS (LNAI), vol. 2157, Springer, Heidelberg (2001)Google Scholar
  9. 9.
    Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: Proc. 22nd ICML, pp. 441–448 (2005)Google Scholar
  10. 10.
    Koller, D., Pfeffer, A.: Probabilistic frame-based systems. In: Proc. 15th AAAI, pp. 580–587 (1998)Google Scholar
  11. 11.
    Laskey, K.B., da Costa, P.C.G.: Of starships and Klingons: First-order Bayesian logic for the 23rd century. In: Proc. 21st UAI, pp. 346–353 (2005)Google Scholar
  12. 12.
    Lukasiewicz, T., Kern–Isberner, G.: Probabilistic logic programming under maximum entropy. In: Hunter, A., Parsons, S. (eds.) ECSQARU 1999. LNCS (LNAI), vol. 1638, pp. 279–292. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  13. 13.
    Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: BLOG: Probabilistic models with unknown objects. In: Proc. 19th IJCAI (2005)Google Scholar
  14. 14.
    Milch, B., Marthi, B., Sontag, D., Russell, S., Ong, D.L., Kolobov, A.: Approximate inference for infinite contingent Bayesian networks. In: Proc. 10th AISTATS (2005)Google Scholar
  15. 15.
    Milch, B., Russell, S.: General-purpose MCMC inference over relational structures. In: Proc. 22nd UAI, pp. 349–358 (2006)Google Scholar
  16. 16.
    Muggleton, S., Buntine, W.: Machine invention of first-order predicates by inverting resolution. In: Proc. 5th ICML, pp. 339–352 (1988)Google Scholar
  17. 17.
    Muggleton, S.H.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)Google Scholar
  18. 18.
    Muggleton, S.H.: Learning structure and parameters of stochastic logic programs. Electronic Trans. on AI 6 (2002)Google Scholar
  19. 19.
    J.,, Neville, D.J.: Dependency networks for relational data. In: Proc. 4th IEEE Int’l Conf. on Data Mining, IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  20. 20.
    Neville, J., Jensen, D., Friedland, L., Hay, M.: Learning relational probability trees. In: Proc. 9th KDD (2003)Google Scholar
  21. 21.
    Ng, R.T., Subrahmanian, V.S.: Probabilistic logic programming. Information and Computation 101(2), 150–201 (1992)MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Ngo, L., Haddawy, P.: Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Comp. Sci. 171(1–2), 147–177 (1997)MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Otero, R., Muggleton, S.: On McCarthy’s appearance and reality problem. In: ILP 2006: Short Papers (2006)Google Scholar
  24. 24.
    Pasula, H., Marthi, B., Milch, B., Russell, S., Shpitser, I.: Identity uncertainty and citation matching. In: NIPS 15, MIT Press, Cambridge, MA (2003)Google Scholar
  25. 25.
    Perlich, C., Provost, F.: Aggregation-based feature invention and relational concept classes. In: Proc. 9th KDD (2003)Google Scholar
  26. 26.
    Pfeffer, A.: IBAL: A probabilistic rational programming language. In: Proc. 17th IJCAI (2001)Google Scholar
  27. 27.
    Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence 64(1), 81–129 (1993)MATHCrossRefGoogle Scholar
  28. 28.
    Poole, D.: The Independent Choice Logic for modelling multiple agents under uncertainty. Artificial Intelligence 94(1–2), 5–56 (1997)MathSciNetGoogle Scholar
  29. 29.
    Popescul, A., Ungar, L.H.: Cluster-based concept invention for statistical relational learning. In: Proc. 10th KDD (2004)Google Scholar
  30. 30.
    Popescul, A., Ungar, L.H., Lawrence, S., Pennock, D.M.: Statistical relational learning for document mining. In: Proc. 3rd IEEE Int’l Conf. on Data Mining, pp. 275–282. IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  31. 31.
    Puech, A., Muggleton, S.: A comparison of stochastic logic programs and Bayesian logic programs. In: IJCAI Workshop on Learning Statistical Models from Relational Data (2003)Google Scholar
  32. 32.
    Revoredo, V.K., Paes, A., Zaverucha, G., Costa, S.: Combining predicate invention and revision of probabilistic FOL theories. In: ILP 2006: Short Papers (2006)Google Scholar
  33. 33.
    Richardson, M., Domingos, P.: Markov logic networks. MLJ 62, 107–136 (2006)Google Scholar
  34. 34.
    Sato, T., Kameya, Y.: PRISM: A symbolic–statistical modeling language. In: Proc. 15th IJCAI, pp. 1330–1335 (1997)Google Scholar
  35. 35.
    Sato, T., Kameya, Y.: Parameter learning of logic programs for symbolic–statistical modeling. JAIR 15, 391–454 (2001)MATHMathSciNetGoogle Scholar
  36. 36.
    Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proc. 18th UAI, pp. 485–492 (2002)Google Scholar
  37. 37.
    Thomas, A., Spiegelhalter, D., Gilks, W.: BUGS: A program to perform Bayesian inference using Gibbs sampling. In: Bernardo, J., Berger, J., Dawid, A., Smith, A. (eds.) Bayesian Statistics 4, Oxford Univ. Press, Oxford (1992)Google Scholar
  38. 38.
    Van Assche, A., Vens, C., Blockeel, H., Džeroski, S.: First order random forests: Learning relational classifiers with complex aggregates. MLJ 64, 149–182 (2006)MATHGoogle Scholar
  39. 39.
    Vennekens, J., Verbaeten, S., Bruynooghe, M.: Logic programs with annotated disjunctions. In: Demoen, B., Lifschitz, V. (eds.) ICLP 2004. LNCS, vol. 3132, pp. 431–445. Springer, Heidelberg (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Brian Milch
    • 1
  • Stuart Russell
    • 2
  1. 1.Computer Science and AI Laboratory, Massachusetts Institute of Technology, 32 Vassar St. Room 32-G480, Cambridge, MA 02139USA
  2. 2.Computer Science Division, University of California at Berkeley, Berkeley, CA 94720-1776USA

Personalised recommendations