The Factorized Distribution Algorithm and the Minimum Relative Entropy Principle

  • Heinz Mühlenbein
  • Robin Höns
Part of the Studies in Computational Intelligence book series (SCI, volume 33)


Marginal Distribution Relative Entropy Boltzmann Distribution Interaction Graph Exact Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. M. Aji and R. J. McEliece. The generalized distributive law. IEEE Transactions of Information Theory, 46(2):325-343, March 2000zbMATHCrossRefMathSciNetGoogle Scholar
  2. [2]
    S. M. Aji and R. J. McEliece. The generalized distributive law and free energy minimization. In Proceedings of the 39th Annual Allerton Conference on Communication, Control, and Computing, pages 672-681, 2001Google Scholar
  3. [3]
    R. G. Almond. Graphical Belief Modelling. Chapman and Hall, London, 1995Google Scholar
  4. [4]
    T. M. Cover and J. Thomas. Elements of Information Theory. Wiley, New York, 1989Google Scholar
  5. [5]
    J. Darroch and D. Ratcliff. Generalized iterative scaling for log-linear models. Ann.Math. Statistics, 43:1470-1480, 1972zbMATHCrossRefMathSciNetGoogle Scholar
  6. [6]
    Y. Gao and J. Culberson. Space complexity of estimation of distribution algorithms. Evolutionary Computation, 13(1):125-143, 2005CrossRefMathSciNetGoogle Scholar
  7. [7]
    R. Höns. Estimation of Distribution Algorithms and Minimum Relative Entropy. PhD thesis, University of Bonn, 2005Google Scholar
  8. [8]
    E. T. Jaynes. Information theory and statistical mechanics. Physical Review, 6:620-643, 1957CrossRefMathSciNetGoogle Scholar
  9. [9]
    E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press, Cambridge, 1978Google Scholar
  10. [10]
    F. V. Jensen and F. Jensen. Optimal junction trees. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, pages 360-366, Seattle, 1994Google Scholar
  11. [11]
    M. I. Jordan, editor. Learning in Graphical Models. MIT Press, Cambrigde, 1999Google Scholar
  12. [12]
    B. W. Kernighan and S. Lin. An efficient heuristic for partitioning graphs. Bells Systems Technical Journal, 2:291-307, 1970Google Scholar
  13. [13]
    S. L. Lauritzen. Graphical Models. Clarendon Press, Oxford, 1996Google Scholar
  14. [14]
    R. J. McEliece and M. Yildirim. Belief propagation on partially ordered sets. In Proceedings of the 15th Internatonal Symposium on Mathematical Theory of Networks and Systems (MTNS 2002), 2002Google Scholar
  15. [15]
    H. Mühlenbein and R. Höns. The estimation of distributions and the minimum relative entropy principle. Evolutionary Computation, 13(1): 1-27, 2005CrossRefGoogle Scholar
  16. [16]
    H. Mühlenbein and T. Mahnig. FDA - a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolutionary Computation, 7(4):353-376, 1999CrossRefGoogle Scholar
  17. [17]
    H. Mühlenbein and T. Mahnig. Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. Journal of Approximate Reasoning, 31(3):157-192, 2002zbMATHCrossRefMathSciNetGoogle Scholar
  18. [18]
    H. Mühlenbein and T. Mahnig. Mathematical analysis of evolutionary algorithms. In C. C. Ribeiro and P. Hansen, editors, Essays and Surveys in Metaheuristics, Operations Research/Computer Science Interface Series, pages 525-556. Kluwer, Norwell, 2002Google Scholar
  19. [19]
    H. Mühlenbein and T. Mahnig. Evolutionary algorithms and the Boltzmann distribution. In K. D. Jong, R. Poli, and J. C. Rowe, editors, Foundations of Genetic Algorithms 7, pages 525-556. Morgan Kaufmann Publishers, San Francisco, 2003Google Scholar
  20. [20]
    H. Mühlenbein, T. Mahnig, and A. Ochoa. Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics, 5(2):213-247, 1999CrossRefGoogle Scholar
  21. [21]
    H. Mühlenbein and G. Paaß. From recombination of genes to the estimation of distributions I. binary parameters. In H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, editors, Lecture Notes in Computer Science 1141: Parallel Problem Solving from Nature - PPSN IV), pages 178-187, Berlin, 1996. Springer-VerlagCrossRefGoogle Scholar
  22. [22]
    M. Pelikan and D. Goldberg. Hierarchical BOA solves Ising spin glasses and MAXSAT. In Genetic and Evolutionary Computation Conference 2003, volume 2724 of Lecture Notes in Computer Science, pages 1271-1282. Springer, Berlin Heidelberg New York 2003. Also IlliGAL Report No. 2003001Google Scholar
  23. [23]
    R. Santana. Estimation of distribution algorithms with Kikuchi approximations. Evolutionary Computation, 13(1):67-97, 2005CrossRefGoogle Scholar
  24. [24]
    Y. W. Teh and M. Welling. On improving the efficiency of the iterative proportional fitting procedure. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, volume 9, 2003Google Scholar
  25. [25]
    C. Yanover and Y. Weiss. Finding the M most probable configurations using loopy belief propagation. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004Google Scholar
  26. [26]
    J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. Technical Report 2004-040, Mitsubishi Electric Research Laboratories, May 2004Google Scholar
  27. [27]
    A. L. Yuille. CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. Neural Computation, 14(7):1691-1722, 2002zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Heinz Mühlenbein
    • 1
  • Robin Höns
    • 2
  1. 1.Fraunhofer Institute for Autonomous Intelligent SystemsSankt AugustinGermany
  2. 2.Fraunhofer Institute for Autonomous Intelligent SystemsSankt AugustinGermany

Personalised recommendations