A Generalized Iterative Scaling Algorithm for Maximum Entropy Model Computations Respecting Probabilistic Independencies

  • Marco WilhelmEmail author
  • Gabriele Kern-Isberner
  • Marc Finthammer
  • Christoph Beierle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10833)


Maximum entropy distributions serve as favorable models for commonsense reasoning based on probabilistic conditional knowledge bases. Computing these distributions requires solving high-dimensional convex optimization problems, especially if the conditionals are composed of first-order formulas. In this paper, we propose a highly optimized variant of generalized iterative scaling for computing maximum entropy distributions. As a novel feature, our improved algorithm is able to take probabilistic independencies into account that are established by the principle of maximum entropy. This allows for exploiting the logical information given by the knowledge base, represented as weighted conditional impact systems, in a very condensed way.



This research was supported by the German National Science Foundation (DFG), Research Unit FOR 1513 on Hybrid Reasoning for Intelligent Systems.


  1. 1.
    Getoor, L., Taskar, B. (eds.): Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)zbMATHGoogle Scholar
  2. 2.
    Raedt, L.D., Frasconi, P., Kersting, K., Muggleton, S.H. (eds.): Probabilistic Inductive Logic Programming. Springer, Heidelberg (2008). Scholar
  3. 3.
    Van Den Broeck, G.: First-order model counting in a nutshell. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), pp. 4086–4089. AAAI Press (2016)Google Scholar
  4. 4.
    Paris, J.B.: The Uncertain Reasoner’s Companion - A Mathematical Perspective. Cambridge University Press, Cambridge (1994)zbMATHGoogle Scholar
  5. 5.
    Kern-Isberner, G.: Conditionals in Nonmonotonic Reasoning and Belief Revision. Springer, Heidelberg (2001). Scholar
  6. 6.
    Finthammer, M., Beierle, C.: A two-level approach to maximum entropy model computation for relational probabilistic logic based on weighted conditional impacts. In: Straccia, U., Calì, A. (eds.) SUM 2014. LNCS (LNAI), vol. 8720, pp. 162–175. Springer, Cham (2014). Scholar
  7. 7.
    Thimm, M., Kern-Isberner, G.: On probabilistic inference in relational conditional logics. Logic J. IGPL 20(5), 872–908 (2012)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Halpern, J.Y.: An analysis of first-order logics of probability. Artif. Intell. 46(3), 311–350 (1990)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Paris, J.B.: Common sense and maximum entropy. Synthese 117(1), 75–93 (1999)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Darroch, J.N., Ratcliff, D.: Generalized iterative scaling for log-linear models. Ann. Math. Stat. 43(5), 1470–1480 (1972)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Koller, D., Friedman, N.: Probabilistic Graphical Models. MIT Press, Cambridge (2009)zbMATHGoogle Scholar
  12. 12.
    Kern-Isberner, G., Thimm, M.: A ranking semantics for first-order conditionals. In: Proceedings of the 20th European Conference on Artificial Intelligence (ECAI). FAIA, vol. 242, pp. 456–461. IOS Press (2012)Google Scholar
  13. 13.
    Finthammer, M., Beierle, C.: Using equivalences of worlds for aggregation semantics of relational conditionals. In: Glimm, B., Krüger, A. (eds.) KI 2012. LNCS (LNAI), vol. 7526, pp. 49–60. Springer, Heidelberg (2012). Scholar
  14. 14.
    Wilhelm, M., Kern-Isberner, G., Ecke, A.: Basic independence results for maximum entropy reasoning based on relational conditionals. In: Proceedings of the 3rd Global Conference on Artificial Intelligence (GCAI). EPiC Series in Computing, vol. 50, pp. 36–50 (2017)Google Scholar
  15. 15.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)CrossRefGoogle Scholar
  16. 16.
    Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Marco Wilhelm
    • 1
    Email author
  • Gabriele Kern-Isberner
    • 1
  • Marc Finthammer
    • 2
  • Christoph Beierle
    • 2
  1. 1.Department of Computer ScienceTU DortmundDortmundGermany
  2. 2.Department of Computer ScienceUniversity of HagenHagenGermany

Personalised recommendations