Applied Intelligence

, Volume 36, Issue 2, pp 472–497 | Cite as

On the combination of logical and probabilistic models for information analysis

  • Jingsong Wang
  • John Byrnes
  • Marco ValtortaEmail author
  • Michael Huhns


Formal logical tools are able to provide some amount of reasoning support for information analysis, but are unable to represent uncertainty. Bayesian network tools represent probabilistic and causal information, but in the worst case scale as poorly as some formal logical systems and require specialized expertise to use effectively. We describe a framework for systems that incorporate the advantages of both Bayesian and logical systems. We define a formalism for the conversion of automatically generated natural deduction proof trees into Bayesian networks. We then demonstrate that the merging of such networks with domain-specific causal models forms a consistent Bayesian network with correct values for the formulas derived in the proof. In particular, we show that hard evidential updates in which the premises of a proof are found to be true force the conclusions of the proof to be true with probability one, regardless of any dependencies and prior probability values assumed for the causal model. We provide several examples that demonstrate the generality of the natural deduction system by using inference schemes not supportable directly in Horn clause logic. We compare our approach to other ones, including some that use non-standard logics.


Reasoning Uncertainty Probabilistic reasoning Bayesian networks Natural deduction proofs Logic for knowledge representation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abe J, Akama S (2001) On some aspects of decidability of annotated systems. In: Proceedings of the international conference on artificial intelligence, pp 789–795 Google Scholar
  2. 2.
    Akama S, Abe J (1998) Many-valued and annotated modal logics. In: Proceedings of the 28th international symposium on multiple-valued logic. IEEE Computer Society, Washington, pp 114–119 Google Scholar
  3. 3.
    Bacchus F (1990) Representing and reasoning with probabilistic knowledge: a logical approach to probabilities. MIT Press, Cambridge Google Scholar
  4. 4.
    Benferhat S, Dubois D, Prade H (2001) Towards a possibilistic logic handling of preferences. Appl Intell 14:303–317 zbMATHCrossRefGoogle Scholar
  5. 5.
    Bertelè U, Brioschi F (1972) Nonserial dynamic programming. Academic Press, New York zbMATHGoogle Scholar
  6. 6.
    Biba M, Ferilli S, Esposito F (2008) Discriminative structure learning of Markov logic networks. In: Proceedings of the 18th international conference on inductive logic programming, ILP ’08. Springer, Berlin, pp 59–76 Google Scholar
  7. 7.
    Bundy A (1985) Incidence calculus: a mechanism for probabilistic reasoning. J Autom Reason 1(3):263–283 MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Byrnes J (1999) Proof search and normal forms in natural deduction. PhD thesis, Department of Philosophy, Carnegie Mellon University Google Scholar
  9. 9.
    Carbogim DV, da Silva FSC (1998) Annotated logic applications for imperfect information. Appl Intell 9:163–172 CrossRefGoogle Scholar
  10. 10.
    Chachoua M, Pacholczyk D (2000) A symbolic approach to uncertainty management. Appl Intell 13:265–283 CrossRefGoogle Scholar
  11. 11.
    Cheng J, Emami R, Kerschberg L, Santos JE, Zhao Q, Nguyen H, Wang H, Huhns M, Valtorta M, Dang J, Goradia H, Huang J, Xi S (2005) Omniseer: a cognitive framework for user modeling, reuse of prior and tacit knowledge, and collaborative knowledge services. In: Proceedings of the 38th Hawaii international conference on system sciences (HICSS38), Big Island, HI Google Scholar
  12. 12.
    Cooper GF (1987) Probabilistic inference using belief networks is np-hard, memo KSL-87-27 (revised July 1988). Tech rep, Medical Computer Science Group, Knowledge Systems Laboratory, Stanford University Google Scholar
  13. 13.
    Cooper GF (1990) The computational complexity of probabilistic inference using Bayesian belief networks. Artif Intell 42:393–405 zbMATHCrossRefGoogle Scholar
  14. 14.
    Darwiche A (2009) Modeling and reasoning with Bayesian networks. Cambridge University Press, Cambridge zbMATHCrossRefGoogle Scholar
  15. 15.
    Dekhtyar A, Subrahmanian VS (2000) Hybrid probabilistic programs. J Log Program 43(3):187–250 MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Dietterich TG, Domingos P, Getoor L, Muggleton S, Tadepalli P (2008) Structured machine learning: the next ten years. Mach Learn 73:3–23 CrossRefGoogle Scholar
  17. 17.
    Dubois D, Prade H (1987) Necessity measures and the resolution principle. IEEE Trans Syst Man Cybern 17:474–478 MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Dubois D, Prade H (1988) Possibility theory. Plenum Press, New York zbMATHGoogle Scholar
  19. 19.
    Dubois D, Prade H (1990) An introduction to possibilistic and fuzzy logics. In: Readings in uncertain reasoning. Morgan Kaufmann Publishers, San Francisco, pp 742–761 Google Scholar
  20. 20.
    Dubois D, Prade H (1994) Can we enforce full compositionality in uncertainty calculi? In: Proc of the 11th nat conf on artificial intelligence (AAAI-94). AAAI Press/MIT Press, Menlo Park/Cambridge, pp 149–154 Google Scholar
  21. 21.
    Dubois D, Prade H (2001) Possibility theory, probability theory and multiple-valued logics: a clarification. Ann Math Artif Intell 32(1–4):35–66 MathSciNetCrossRefGoogle Scholar
  22. 22.
    Dubois D, Lang J, Prade H (1987) Theorem proving under uncertainty: a possibility theory-based approach. In: IJCAI’87: proceedings of the 10th international joint conference on artificial intelligence. Morgan Kaufmann Publishers Inc, San Francisco, pp 984–986 Google Scholar
  23. 23.
    Dubois D, Lang J, Prade H (1990) Poslog, an inference system based on possibilistic logic. In: Proc of the North American fuzzy information processing society conference (NAFIPS’90): quarter century of fuzzyness, Toronto, Canada, 06/06/90–08/06/90, pp 177–180 Google Scholar
  24. 24.
    Dubois D, Lang J, Prade H (1994) Automated reasoning using possibilistic logic: semantics, belief revision and variable certainty weights. IEEE Trans Knowl Data Eng 6(1):64–71 CrossRefGoogle Scholar
  25. 25.
    Dubois D, Lang J, Prade H (1994) Possibilistic logic. In: Gabbay DM, Hogger CJ, JA Robinson (eds) Handbook of logic in artificial intelligence and logic programming. Nonmonotonic reasoning and uncertain reasoning, vol 3. Oxford University Press, New York, pp 439–513 Google Scholar
  26. 26.
    Fagin R, Halpern JY, Megiddo N (1990) A logic for reasoning about probabilities. Inf Comput 87:78–128 MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Fierens D, Blockeel H, Bruynooghe M, Ramon J (2005) Logical Bayesian networks and their relation to other probabilistic logical models. In: Proceedings of the 15th international conference on inductive logic programming. Springer, Berlin, pp 121–135 Google Scholar
  28. 28.
    Fierens D, Ramon J, Bruynooghe M, Blockeel H (2008) Learning directed probabilistic logical models: ordering-search versus structure-search. Ann Math Artif Intell 54:99–133 MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Friedman N, Getoor L, Koller D, Pfeffer A (1999) Learning probabilistic relational models. In: IJCAI. Springer, Berlin, pp 1300–1309 Google Scholar
  30. 30.
    Getoor L, Taskar B (2007) Introduction to statistical relational learning (adaptive computation and machine learning). MIT Press, Cambridge Google Scholar
  31. 31.
    Getoor L, Friedman N, Koller D, Taskar B (2001) Learning probabilistic models of relational structure. In: Proceedings of the eighteenth international conference on machine learning. Morgan Kaufmann, San Mateo, pp 170–177 Google Scholar
  32. 32.
    Getoor L, Friedman N, Koller D, Taskar B (2002) Learning probabilistic models of link structure. J Mach Learn Res 3:679–707 MathSciNetGoogle Scholar
  33. 33.
    Haarslev Pai HI V, Shiri N (2009) A formal framework for description logics with uncertainty. Int J Approx Reason 50(9):1399–1415 MathSciNetzbMATHCrossRefGoogle Scholar
  34. 34.
    Halpern JY (1990) An analysis of first-order logics of probability. Artif Intell 46:311–350 zbMATHCrossRefGoogle Scholar
  35. 35.
    Halpern JY (2003) Reasoning about uncertainty. MIT Press, Cambridge zbMATHGoogle Scholar
  36. 36.
    Delugach H (ed) (2005) Common logic—a framework for a family of logic-based languages. Tech rep, International Standards Organization: iSO/IEC JTC 1/SC 32N1377, International Standards Organization Final Committee Draft, 2005-12-13, available at
  37. 37.
    Hayes PJ (2006) IKL guide. Tech rep, Florida Institute for Human and Machine Cognition, unpublished memorandum available at
  38. 38.
    Hayes PJ, Menzel C (2006) IKL specification document. Tech rep, Florida Institute for Human and Machine Cognition, unpublished memorandum available at
  39. 39.
    Hollunder B (1995) An alternative proof method for possibilistic logic and its application to terminological logics. Int J Approx Reason 12(2):85–109 MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Huhns M, Valtorta M, Wang J (2010) Design principles for ontological support of Bayesian evidence management. In: Obrst L, Janssen T, Ceusters W (eds) Semantic technologies, ontologies, and information sharing for intelligence analysis. IOS Press, Amsterdam, pp 163–178 Google Scholar
  41. 41.
    Huynh TN, Mooney RJ (2008) Discriminative structure and parameter learning for Markov logic networks. In: Proceedings of the 25th international conference on machine learning, ICML ’08. ACM, New York, pp 416–423 CrossRefGoogle Scholar
  42. 42.
    Huynh TN, Mooney RJ (2009) Max-margin weight learning for Markov logic networks. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases (ECML/PKDD-09). Bled, pp 248–263 Google Scholar
  43. 43.
    Jaeger M (1997) Relational Bayesian networks. In: Proceedings of the 13th conference of uncertainty in artificial intelligence (UAI-13). Morgan Kaufmann, San Mateo, pp 266–273 Google Scholar
  44. 44.
    Jaeger M (2002) Relational Bayesian networks: a survey. Electron Trans Artif Intell 6 Google Scholar
  45. 45.
    Jaeger M (2007) Parameter learning for relational Bayesian networks. In: Proceedings of the international conference in machine learning Google Scholar
  46. 46.
    Jensen FV, Nielsen TD (2007) Bayesian networks and decision graphs, 2nd edn. Springer, New York zbMATHCrossRefGoogle Scholar
  47. 47.
    Johnson DS (1990) A catalog of complexity classes. In: van Leeuwen J (ed) Handbook of theoretical computer science, vol. A: algorithms and complexity. MIT Press, Cambridge, pp 67–161 Google Scholar
  48. 48.
    Kersting K, De Raedt L (2008) Basic principles of learning Bayesian logic programs. In: De Raedt L, Frasconi P, Kersting K, Muggleton S (eds) Probabilistic inductive logic programming. Springer, Berlin, pp 189–221 CrossRefGoogle Scholar
  49. 49.
    Kersting K, Raedt LD (2001) Bayesian logic programs. CoRR cs.AI/0111058
  50. 50.
    Kifer M, Subrahmanian VS (1989) On the expressive power of annotated logic programs. In: Proceedings of the North American conference on logic programming, pp 1069–1089 Google Scholar
  51. 51.
    Kifer M, Subrahmanian VS (1992) Theory of generalized annotated logic programming and its applications. J Log Program 12:335–367 MathSciNetCrossRefGoogle Scholar
  52. 52.
    Kim YG, Valtorta M, Vomlel J (2004) A prototypical system for soft evidential update. Appl Intell 21(1):81–97 zbMATHCrossRefGoogle Scholar
  53. 53.
    Kok S, Domingos P (2009) Learning Markov logic network structure via hypergraph lifting. In: Proceedings of the 26th international conference on machine learning (ICML-09) Google Scholar
  54. 54.
    Koller D (1998) Pfeffer a probabilistic frame-based systems. In: Proc AAAI. AAAI Press, Menlo Park, pp 580–587 Google Scholar
  55. 55.
    Laskey KB (2006) First-order Bayesian logic. Technical report C4I06-01. Tech rep, SEOR Department, George Mason University Google Scholar
  56. 56.
    Laskey KB (2008) MEBN: a language for first-order knowledge bases. Artif Intell 172:140–178 MathSciNetzbMATHCrossRefGoogle Scholar
  57. 57.
    Laskey KB, Mahoney SM (1997) Network fragments: representing knowledge for constructing probabilistic models. In: Proceedings of the thirteenth annual conference on uncertainty in artificial intelligence (UAI-97), Providence, pp 334–341 Google Scholar
  58. 58.
    Liu W, Bundy A (1994) A comprehensive comparison between generalized incidence calculus and the Dempster-Shafer theory of evidence. Int J Hum-Comput Stud 40:1009–1032 CrossRefGoogle Scholar
  59. 59.
    Liu W, McBryan D, Bundy A (1998) The method of assigning incidences. Appl Intell 9:139–161 CrossRefGoogle Scholar
  60. 60.
    Loveland DW, Stickel M (1976) A hole in goal trees: Some guidance from resolution theory. IEEE Trans Comput 25:335–341 zbMATHCrossRefGoogle Scholar
  61. 61.
    Lowd D, Domingos P (2007) Efficient weight learning for Markov logic networks. In: Proceedings of the eleventh European conference on principles and practice of knowledge discovery in databases, pp 200–211 Google Scholar
  62. 62.
    Loyer Y, Straccia U (2009) Approximate well-founded semantics, query answering and generalized normal logic programs over lattices. Ann Math Artif Intell 55:389–417 MathSciNetzbMATHCrossRefGoogle Scholar
  63. 63.
    Lu JJ, Murray NV, Rosenthal E (1993) Signed formulas and annotated logics. In: Proceedings of int symposium on multiple-valued logic, pp 48–53 Google Scholar
  64. 64.
    Lukasiewicz T (2007) Probabilistic description logic programs. Int J Approx Reason 45(2):288–307 MathSciNetzbMATHCrossRefGoogle Scholar
  65. 65.
    Lukasiewicz T (2008) Probabilistic description logic programs under inheritance with overriding for the semantic web. Int J Approx Reason 49(1):18–34 MathSciNetzbMATHCrossRefGoogle Scholar
  66. 66.
    Milch B, Russell S (2007) First-order probabilistic languages: Into the unknown. In: Proceedings of the 16th international conference on inductive logic programming, pp 10–24 Google Scholar
  67. 67.
    Muggleton S (1996) Stochastic logic programs. In: Advances in inductive logic programming. IOS Press, Amsterdam, pp 254–264 Google Scholar
  68. 68.
    Muggleton S (2000) Learning stochastic logic programs. In: Getoor L, Jensen D (eds) Proceedings of the AAAI2000 workshop on learning statistical models from relational data, URL: Google Scholar
  69. 69.
    Neapolitan RE (1990) Probabilistic reasoning in expert systems: theory and algorithms. Wiley, New York Google Scholar
  70. 70.
    Ng R, Subrahmanian VS (1992) Probabilistic logic programming. Inf Comput 101:150–201 MathSciNetzbMATHCrossRefGoogle Scholar
  71. 71.
    Ngo L, Haddawy P (1996) Answering queries from context-sensitive probabilistic knowledge bases. Theor Comput Sci 171:147–177 MathSciNetCrossRefGoogle Scholar
  72. 72.
    Niles I, Pease A (2001) Towards a standard upper ontology. In: Welty C, Smith B (eds) Proceedings of the 2nd international conference on formal ontology in information systems (FOIS-2001), Ogunquit, ME, USA, pp 2–9 CrossRefGoogle Scholar
  73. 73.
    Nilsson NJ (1986) Probabilistic logic. Artif Intell 28(1):71–87 MathSciNetzbMATHCrossRefGoogle Scholar
  74. 74.
    Obeid N (2005) A formalism for representing and reasoning with temporal information, event and change. Appl Intell 23:109–119 CrossRefGoogle Scholar
  75. 75.
    Orponen P (1990) Dempster’s rule of combination is #p-complete. Artif Intell 44:245–253 MathSciNetzbMATHCrossRefGoogle Scholar
  76. 76.
    Paris J (1994) The uncertain reasoner’s companion: a mathematical perspective. Cambridge tracts in theoretical computer science, vol 39. Cambridge University Press, Cambridge zbMATHGoogle Scholar
  77. 77.
    Park J (2002) Map complexity results and approximation methods. In: Proceedings of the 18th annual conference on uncertainty in artificial intelligence (UAI-02). Morgan Kaufmann, San Francisco, pp 388–439 Google Scholar
  78. 78.
    Pearl J (2000) Causality: modeling, reasoning, and inference. Cambridge University Press, Cambridge Google Scholar
  79. 79.
    Peng Y, Reggia JA (1990) Abductive inference models for diagnostic problem solving. Springer, New York zbMATHCrossRefGoogle Scholar
  80. 80.
    Poole D (2008) The independent choice logic and beyond. In: Probabilistic inductive logic programming: theory and applications. Springer, Berlin, pp 222–243 CrossRefGoogle Scholar
  81. 81.
    Qi G, Pan JZ, Ji Q (2007) A possibilistic extension of description logics. In: Proceedings of the international workshop on description logics (DL’07), pp 435–442 Google Scholar
  82. 82.
    Riazanov A, Voronkov A (2002) The design and implementation of Vampire. AI Commun. 15:91–110 zbMATHGoogle Scholar
  83. 83.
    Richardson M, Domingos P (2006) Markov logic networks. Mach Learn 62(1–2):107–136 CrossRefGoogle Scholar
  84. 84.
    Rose DJ (1972) A graph-theoretic study of the numerical solution of sparse positive definite systems of linear equations. In: Read R (ed) Graph theory and computing. Academic Press, New York, pp 183–217 Google Scholar
  85. 85.
    Roth D (1996) On the hardness of approximate reasoning. Artif Intell 82:273–302 CrossRefGoogle Scholar
  86. 86.
    de Salvo Braz R, Amir E, Roth D (2008) A survey of first-order probabilistic models. In: Holmes D, Jain L (eds) Innovations in Bayesian networks. Springer, Berlin, pp 289–317. URL: Google Scholar
  87. 87.
    Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton zbMATHGoogle Scholar
  88. 88.
    Singla P, Domingos P (2005) Discriminative training of Markov logic networks. In: Proc of the natl conf on artificial intelligence Google Scholar
  89. 89.
    Smets P, Mamdani E, Dubois D, Prade H (eds) (1988) Non-standard logics for automated reasoning. Academic Press, San Diego zbMATHGoogle Scholar
  90. 90.
    Stoilos G, Stamou G, Pan JZ, Tzouvaras V, Horrocks I (2007) Reasoning with very expressive fuzzy description logics. J Artif Intell Res 273–320 Google Scholar
  91. 91.
    Straccia U (2001) Reasoning within fuzzy description logics. J Artif Intell Res 14:137–166 MathSciNetzbMATHGoogle Scholar
  92. 92.
    Straccia U (2006) A fuzzy description logic for the semantic web. In: Sanchez E (ed) Fuzzy logic and the semantic web, capturing intelligence. Elsevier, Amsterdam, pp 73–90. Chap 4 CrossRefGoogle Scholar
  93. 93.
    Straccia U (2008) Managing uncertainty and vagueness in description logics, logic programs and description logic programs. In: Baroglio C, Bonatti PA, Maluszyński J, Marchiori M, Polleres A, Schaffert S (eds) Reasoning web. Springer, Berlin, pp 54–103 CrossRefGoogle Scholar
  94. 94.
    Straccia U, Bobillo F (2007) Mixed integer programming, general concept inclusions and fuzzy description logics. In: Proceedings of the 5th conference of the European society for fuzzy logic and technology (EUSFLAT-07), vol 2, University of Ostrava, Ostrava, Czech Republic, pp 213–220 Google Scholar
  95. 95.
    Subrahmanian V (2007) Uncertainty in logic programming: some recollections. Assoc Log Program Newslett 20(2) Google Scholar
  96. 96.
    Subrahmanian VS (1987) On the semantics of quantitative logic programs. In: Proceedings of the 4th IEEE symposium on logic programming, pp 173–182 Google Scholar
  97. 97.
    Valtorta M, Kim YG, Vomlel J (2002) Soft evidential update for probabilistic multiagent systems. Int J Approx Reason 29(1):71–106 MathSciNetzbMATHCrossRefGoogle Scholar
  98. 98.
    Valtorta M, Dang J, Goradia H, Huang J, Huhns M (2005) Extending Heuer’s analysis of competing hypotheses method to support complex decision analysis. In: Proceedings of the 2005 international conference on intelligence analysis (IA-05) (CD-ROM), extended version available at Google Scholar
  99. 99.
    Zadeh LA (1978) Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst 1:3–28 MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Jingsong Wang
    • 1
  • John Byrnes
    • 2
  • Marco Valtorta
    • 1
    Email author
  • Michael Huhns
    • 1
  1. 1.Department of Computer Science and EngineeringUniversity of South CarolinaColumbiaUSA
  2. 2.SRI InternationalSan DiegoUSA

Personalised recommendations