Comparing and Evaluating Approaches to Probabilistic Reasoning: Theory, Implementation, and Applications

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7600)


The handling of uncertain information is of crucial importance for the success of expert systems. This paper gives an overview on logic-based approaches to probabilistic reasoning and goes into more details about recent developments for relational, respectively first-order, probabilistic methods like Markov logic networks, and Bayesian logic programs. In particular, we feature the maximum entropy approach as a powerful and elegant method that combines convenience with respect to knowledge representation with excellent inference properties. While comparing the different approaches is a difficult task due to the variety of the available concepts and to the absence of a common interface, we address this problem from both a conceptual and practical point of view. On a conceptual layer we propose and discuss several criteria by which first-order probabilistic methods can be distinguished, and apply these criteria to a series of approaches. On the practical layer, we briefly describe some systems for probabilistic reasoning, and go into more details on the KReator system as a versatile toolbox for various approaches to first-order probabilistic relational learning, modelling, and reasoning. Moreover, we illustrate applications of probabilistic logics in various scenarios.


Bayesian Network Maximum Entropy Knowledge Representation Inductive Logic Programming Conditional Probability Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baral, C., Gelfond, M., Rushton, N.: Probabilistic Reasoning with Answer Sets. Theory and Practice of Logic Programming 9, 57–144 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Baumbach, J., Bunkowski, A., Lange, S., Oberwahrenbrock, T., Kleinbölting, N., Rahmen, S., Baumbach, J.I.: IMS2 – An integrated medical software system for early lung cancer detection using ion mobility spectometry data of human breath. J. of Integrative Bioinformatics 4(3) (2007)Google Scholar
  3. 3.
    Baumbach, J.I., Westhoff, M.: Ion mobility spectometry to detect lung cancer and airway infections. Spectroscopy Europe 18(6), 22–27 (2006)Google Scholar
  4. 4.
    Beierle, C., Finthammer, M., Kern-Isberner, G., Thimm, M.: Evaluation and Comparison Criteria for Approaches to Probabilistic Relational Knowledge Representation. In: Bach, J., Edelkamp, S. (eds.) KI 2011. LNCS, vol. 7006, pp. 63–74. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  5. 5.
    Beierle, C., Kern-Isberner, G.: The Relationship of the Logic of Big-Stepped Probabilities to Standard Probabilistic Logics. In: Link, S., Prade, H. (eds.) FoIKS 2010. LNCS, vol. 5956, pp. 191–210. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Bödeker, B., Vautz, W., Baumbach, J.I.: Peak finding and referencing in MCC/IMS-data. International Journal for Ion Mobility Spectrometry 11(1-4), 83–87 (2008)CrossRefGoogle Scholar
  7. 7.
    Breese, J.S.: Construction of Belief and Decision Networks. Computational Intelligence 8(4), 624–647 (1992)CrossRefGoogle Scholar
  8. 8.
    Broecheler, M., Simari, G.I., Subrahmanian, V.S.: Using Histograms to Better Answer Queries to Probabilistic Logic Programs. In: Hill, P.M., Warren, D.S. (eds.) ICLP 2009. LNCS, vol. 5649, pp. 40–54. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Bruynooghe, M., De Cat, B., Drijkoningen, J., Fierens, D., Goos, J., Gutmann, B., Kimmig, A., Labeeuw, W., Langenaken, S., Landwehr, N., Meert, W., Nuyts, E., Pellegrims, R., Rymenants, R., Segers, S., Thon, I., Van Eyck, J., Van den Broeck, G., Vangansewinkel, T., Van Hove, L., Vennekens, J., Weytjens, T., De Raedt, L.: An Exercise with Statistical Relational Learning Systems. In: Domingos, P., Kersting, K. (eds.) International Workshop on Statistical Relational Learning (SRL 2009), Leuven, Belgium (2009)Google Scholar
  10. 10.
    Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational Bayesian networks for exact inference. International Journal of Approximate Reasoning 42(1-2), 4–20 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Cussens, J.: Logic-based formalisms for statistical relational learning. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)Google Scholar
  12. 12.
    Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. In: Annals of Mathematical Statistics, vol. 43, pp. 1470–1480. Institute of Mathematical Statistics (1972)Google Scholar
  13. 13.
    De Raedt, L., Kersting, K.: Probabilistic Inductive Logic Programming. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds.) Probabilistic ILP 2007. LNCS (LNAI), vol. 4911, pp. 1–27. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Dehaspe, L.: Maximum Entropy Modeling with Clausal Constraints. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS, vol. 1297, pp. 109–125. Springer, Heidelberg (1997)Google Scholar
  15. 15.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum-likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Domingos, P., Lowd, D.: Markov Logic: An Interface Layer for Artificial Intelligence. In: Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan and Claypool, San Rafael (2009)Google Scholar
  17. 17.
    Fagin, R., Halpern, J.Y.: Reasoning about knowledge and probability. J. ACM 41(2), 340–367 (1994)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Fierens, D.: Learning Directed Probabilistic Logical Models from Relational Data. PhD thesis, Katholieke Universiteit Leuven (2008)Google Scholar
  19. 19.
    Finthammer, M.: An Iterative Scaling Algorithm for Maximum Entropy Reasoning in Relational Probabilistic Conditional Logic. In: Hüllermeier, E. (ed.) SUM 2012. LNCS (LNAI), vol. 7520, pp. 351–364. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  20. 20.
    Finthammer, M., Beierle, C., Berger, B., Kern-Isberner, G.: Probabilistic reasoning at optimum entropy with the MEcore system. In: Lane, H.C., Guesgen, H.W. (eds.) Proceedings 22nd International FLAIRS Conference, FLAIRS 2009. AAAI Press, Menlo Park (2009)Google Scholar
  21. 21.
    Finthammer, M., Beierle, C., Fisseler, J., Kern-Isberner, G., Baumbach, J.I.: Using probabilistic relational learning to support bronchial carcinoma diagnosis based on ion mobility spectrometry. International Journal for Ion Mobility Spectrometry 13, 83–93 (2010)CrossRefGoogle Scholar
  22. 22.
    Finthammer, M., Thimm, M.: An Integrated Development Environment for Probabilistic Relational Reasoning. International Journal of the IGPL (2011) (to appear)Google Scholar
  23. 23.
    Fisseler, J.: Toward Markov Logic with Conditional Probabilities. In: Wilson, D.C., Lane, H.C. (eds.) Proceedings of the 21st International FLAIRS Conference, FLAIRS 2008, pp. 643–648. AAAI Press (2008)Google Scholar
  24. 24.
    Fisseler, J.: Learning and Modeling with Probabilistic Conditional Logic. Dissertations in Artificial Intelligence, vol. 328. IOS Press, Amsterdam (2010)zbMATHGoogle Scholar
  25. 25.
    Fisseler, J., Kern-Isberner, G., Beierle, C., Koch, A., Müller, C.: Algebraic Knowledge Discovery Using Haskell. In: Hanus, M. (ed.) PADL 2007. LNCS, vol. 4354, pp. 80–93. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  26. 26.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley (1994)Google Scholar
  27. 27.
    Getoor, L., Friedman, N., Koller, D., Tasker, B.: Learning Probabilistic Models of Relational Structure. In: Brodley, C.E., Danyluk, A.P. (eds.) Proceedings of the 18th International Conference on Machine Learning, ICML 2001, pp. 170–177. Morgan Kaufmann (2001)Google Scholar
  28. 28.
    Getoor, L., Grant, J.: PRL: A probabilistic relational language. Machine Learning 62(1), 7–31 (2006)CrossRefGoogle Scholar
  29. 29.
    Getoor, L., Taskar, B. (eds.): Introduction to Statistical Relational Learning. MIT Press (2007)Google Scholar
  30. 30.
    Gosling, J., Joy, B., Steele, G., Bracha, G.: The Java Language Specification, 3rd edn. Addison-Wesley (2005)Google Scholar
  31. 31.
    Jaeger, M.: Relational Bayesian Networks: A Survey. Electronic Transactions in Artificial Intelligence 6 (2002)Google Scholar
  32. 32.
    Jaeger, M.: Model-Theoretic Expressivity Analysis. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds.) Probabilistic ILP 2007. LNCS (LNAI), vol. 4911, pp. 325–339. Springer, Heidelberg (2008)Google Scholar
  33. 33.
    Jain, D., Mösenlechner, L., Beetz, M.: Equipping Robot Control Programs with First-Order Probabilistic Reasoning Capabilities. In: International Conference on Robotics and Automation (ICRA), pp. 3130–3135 (2009)Google Scholar
  34. 34.
    Jensen, F.V., Nielsen, T.D.: Bayesian Networks and Decision Graphs. Springer (2007)Google Scholar
  35. 35.
    Kern-Isberner, G.: Characterizing the principle of minimum cross-entropy within a conditional-logical framework. Artificial Intelligence 98, 169–208 (1998)MathSciNetzbMATHCrossRefGoogle Scholar
  36. 36.
    Kern-Isberner, G.: Conditionals in Nonmonotonic Reasoning and Belief Revision. LNCS (LNAI), vol. 2087. Springer, Heidelberg (2001)zbMATHCrossRefGoogle Scholar
  37. 37.
    Kern-Isberner, G.: Linking iterated belief change operations to nonmonotonic reasoning. In: Brewka, G., Lang, J. (eds.) Proceedings 11th International Conference on Knowledge Representation and Reasoning, KR 2008, pp. 166–176. AAAI Press, Menlo Park (2008)Google Scholar
  38. 38.
    Kern-Isberner, G., Beierle, C., Finthammer, M., Thimm, M.: Probabilistic Logics in Expert Systems: Approaches, Implementations, and Applications. In: Hameurlain, A., Liddle, S.W., Schewe, K.-D., Zhou, X. (eds.) DEXA 2011, Part I. LNCS, vol. 6860, pp. 27–46. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  39. 39.
    Kern-Isberner, G., Fisseler, J.: Knowledge Discovery by Reversing Inductive Knowledge Representation. In: Proceedings of the Ninth International Conference on the Principles of Knowledge Representation and Reasoning, KR 2004, pp. 34–44. AAAI Press (2004)Google Scholar
  40. 40.
    Kern-Isberner, G., Lukasiewicz, T.: Combining probabilistic logic programming with the power of maximum entropy. Artificial Intelligence, Special Issue on Nonmonotonic Reasoning 157(1-2), 139–202 (2004)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Kern-Isberner, G., Thimm, M.: Novel Semantical Approaches to Relational Probabilistic Conditionals. In: Proceedings of the Twelfth International Conference on the Principles of Knowledge Representation and Reasoning, KR 2010, pp. 382–392 (May 2010)Google Scholar
  42. 42.
    Kern-Isberner, G., Thimm, M.: A ranking semantics for first-order conditionals. In: Proceedings 20th European Conference on Artificial Intelligence, ECAI 2012 (to appear, 2012)Google Scholar
  43. 43.
    Kersting, K., De Raedt, L.: Bayesian Logic Programming: Theory and Tool. In: Getoor, L., Taskar, B. (eds.) An Introduction to Statistical Relational Learning. MIT Press (2007)Google Scholar
  44. 44.
    Ketkar, N.S., Holder, L.B., Cook, D.J.: Comparison of Graph-based and Logic-based Multi-relational Data Mining. SIGKDD Explor. Newsl. 7(2), 64–71 (2005)CrossRefGoogle Scholar
  45. 45.
    Kok, S., Singla, P., Richardson, M., Domingos, P., Sumner, M., Poon, H., Lowd, D., Wang, J.: The Alchemy System for Statistical Relational AI: User Manual. Department of Computer Science and Engineering. University of Washington (2008)Google Scholar
  46. 46.
    Krämer, A., Beierle, C.: On Lifted Inference for a Relational Probabilistic Conditional Logic with Maximum Entropy Semantics. In: Lukasiewicz, T., Sali, A. (eds.) FoIKS 2012. LNCS, vol. 7153, pp. 224–243. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  47. 47.
    Loh, S., Thimm, M., Kern-Isberner, G.: On the problem of grounding a relational probabilistic conditional knowledge base. In: Proceedings of the 14th International Workshop on Non-Monotonic Reasoning, NMR 2010, Toronto, Canada (May 2010)Google Scholar
  48. 48.
    Loh, S., Thimm, M., Kern-Isberner, G.: On the problem of grounding a relational probabilistic conditional knowledge base. In: Meyer, T., Ternovska, E. (eds.) Proceedings 13th International Workshop on Nonmonotonic Reasoning, NMR 2010. Subworkshop on NMR and Uncertainty (2010)Google Scholar
  49. 49.
    Muggleton, S.H., Chen, J.: A Behavioral Comparison of Some Probabilistic Logic Models. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds.) Probabilistic ILP 2007. LNCS (LNAI), vol. 4911, pp. 305–324. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  50. 50.
    Muggleton, S.H.: Stochastic Logic Programs. In: de Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam (1996)Google Scholar
  51. 51.
    Nilsson, N.: Probabilistic logic. Artificial Intelligence 28, 71–87 (1986)MathSciNetzbMATHCrossRefGoogle Scholar
  52. 52.
    Nute, D., Cross, C.: Conditional Logic. In: Gabbay, D., Guenther, F. (eds.) Handbook of Philosophical Logic, vol. 4, pp. 1–98. Kluwer Academic Publishers (2002)Google Scholar
  53. 53.
    Paris, J.: The uncertain reasoner’s companion – A mathematical perspective. Cambridge University Press (1994)Google Scholar
  54. 54.
    Pearl, J.: Fusion, propagation and structuring in belief networks. Artificial Intelligence 29, 241–288 (1986)MathSciNetzbMATHCrossRefGoogle Scholar
  55. 55.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann (1988)Google Scholar
  56. 56.
    Raedt, L.D., Dehaspe, L.: Clausal Discovery. Machine Learning 26, 99–146 (1997)zbMATHCrossRefGoogle Scholar
  57. 57.
    Raedt, L.D., Kimmig, A., Gutmann, B., Kersting, K., Costa, V.S., Toivonen, H.: Probabilistic Inductive Querying Using ProbLog. Technical Report CW 552, Department of Computer Science. Katholieke Universiteit Leuven, Belgium (June 2009)Google Scholar
  58. 58.
    Richardson, M., Domingos, P.: Markov Logic Networks. Machine Learning 62(1-2), 107–136 (2006)CrossRefGoogle Scholar
  59. 59.
    Robert Koch-Institut: Public Use File KiGGS, Kinder- und Jugendgesundheitssurvey 2003-2006, Berlin (2008)Google Scholar
  60. 60.
    Rödder, W.: Conditional Logic and the Principle of Entropy. Artificial Intelligence 117, 83–106 (2000)MathSciNetzbMATHCrossRefGoogle Scholar
  61. 61.
    Rödder, W., Meyer, C.-H.: Coherent Knowledge Processing at Maximum Entropy by SPIRIT. In: Proceedings UAI 1996, pp. 470–476 (1996)Google Scholar
  62. 62.
    Rödder, W., Reucher, E., Kulmann, F.: Features of the expert-system-shell SPIRIT. Logic Journal of the IGPL 14(3), 483–500 (2006)zbMATHCrossRefGoogle Scholar
  63. 63.
    Schmaußer-Hechfellner, E.: Probabilistic logic knowledge modelling of statistical medical data by applying learning- and inference-techniques of Markov logic networks. Bachelor Thesis, Dept. of Computer Science, FernUniversität in Hagen (2011) (In German)Google Scholar
  64. 64.
    Srinivasan, A.: The Aleph Manual (2007),
  65. 65.
    Thimm, M., Finthammer, M., Loh, S., Kern-Isberner, G., Beierle, C.: A system for relational probabilistic reasoning on maximum entropy. In: Guesgen, H.W., Murray, R.C. (eds.) Proceedings 23rd International FLAIRS Conference, FLAIRS 2010, pp. 116–121. AAAI Press, Menlo Park (2010)Google Scholar
  66. 66.
    Thimm, M., Kern-Isberner, G., Fisseler, J.: Relational Probabilistic Conditional Reasoning at Maximum Entropy. In: Liu, W. (ed.) ECSQARU 2011. LNCS, vol. 6717, pp. 447–458. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  67. 67.
    Wellman, M.P., Breese, J.S., Goldman, R.P.: From Knowledge Bases to Decision Models. The Knowledge Engineering Review 7(1), 35–53 (1992)CrossRefGoogle Scholar
  68. 68.
    Yue, A., Liu, W., Hunter, A.: Measuring the Ignorance and Degree of Satisfaction for Answering Queries in Imprecise Probabilistic Logic Programs. In: Greco, S., Lukasiewicz, T. (eds.) SUM 2008. LNCS (LNAI), vol. 5291, pp. 386–400. Springer, Heidelberg (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Dept. of Computer ScienceTU DortmundDortmundGermany
  2. 2.Dept. of Computer ScienceFernUniversität in HagenHagenGermany
  3. 3.Dept. of Computer ScienceUniversität KoblenzKoblenzGermany

Personalised recommendations