Journal of Heuristics

, Volume 5, Issue 2, pp 215–247 | Cite as

Schemata, Distributions and Graphical Models in Evolutionary Optimization

  • Heinz Mühlenbein
  • Thilo Mahnig
  • Alberto Ochoa Rodriguez


In this paper the optimization of additively decomposed discrete functions is investigated. For these functions genetic algorithms have exhibited a poor performance. First the schema theory of genetic algorithms is reformulated in probability theory terms. A schema defines the structure of a marginal distribution. Then the conceptual algorithm BEDA is introduced. BEDA uses a Boltzmann distribution to generate search points. From BEDA a new algorithm, FDA, is derived. FDA uses a factorization of the distribution. The factorization captures the structure of the given function. The factorization problem is closely connected to the theory of conditional independence graphs. For the test functions considered, the performance of FDA—in number of generations till convergence—is similar to that of a genetic algorithm for the OneMax function. This result is theoretically explained.

graphical models conditional independence graphs additively decomposed functions estimation of distributions population based search genetic algorithm 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aarts, E.H., H.M. Korst, and P.J. van Laarhoven. (1997). “Simulated Annealing.” In E. Aarts and J.K. Lenstra (eds.), Local Search in Combinatorial Optimization. Chichester: Wiley, pp 121–136.Google Scholar
  2. Baluja, S. and S. Davies. (1997). “Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space,” Carnegie Mellon Report CMU-CS-97-107.Google Scholar
  3. De Bonet, J.S., Ch.L. Isbell, and P. Viola. (1997). “MIMIC: Finding Optima by Estimating Probability Densities.” In M. Mozer, M. Jordan, and Th. Petsche (eds.), Advances in Neural Information Processing Systems, vol. 9.Google Scholar
  4. Dechter, R., A. Dechter, and J. Pearl. (1990). “Optimization in Constraint Networks.” In Oliver R.M. and J.Q. Smith (eds.), Influence Diagrams, Belief Nets and Decision Analysis. New York: Wiley, pp. 411–426.Google Scholar
  5. de la Maza, M. and B. Tidor. (1993). “An Analysis of Selection Procedures with Particular Attention Paid to Proportional and Boltzmann Selection.” In S. Forrest (ed.), Proc. of the Fifth Int. Conf. on Genetic Algorithms. San Mateo, CA: Morgan Kaufman, pp. 124–131.Google Scholar
  6. Forrest, S. and M. Mitchell. (1993). “What Makes a Problem Hard for a Genetic Algorithm? Some Anomalous Results and Their Explanation,” Machine Learning 13, 285–319.Google Scholar
  7. Frey, B.J. (1998). Graphical Models for Machine Learning and Digital Communication. Cambridge: MIT Press.Google Scholar
  8. Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Reading: Addison-Wesley.Google Scholar
  9. Goldberg, D.E., K. Deb, H. Kargupta, and G. Harik. (1993). “Rapid, Accurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms. In S. Forrest (ed.), Proc. of the Fifth Int. Conf. on Genetic Algorithms. San Mateo, CA: Morgan Kaufman, pp. 56–64.Google Scholar
  10. Holland, J. (1992). Adaptation in Natural And Artificial Systems. Cambridge: MIT Press.Google Scholar
  11. Kargupta, H. and D.E. Goldberg. (1997). “SEARCH, Blackbox Optimization, and Sample Complexity.” In R.K. Belew and M. Vose (eds.), Foundations of Genetic Algorithms 4. San Mateo, CA: Morgan Kaufman.Google Scholar
  12. Kargupta, H. (1997). Revisiting the GEMGA: Scalable Evolutionary Optimization Through Linkage Learning. Personal Communication.Google Scholar
  13. Lauritzen, S.L. (1996). Graphical Models. Oxford: Clarendon Press.Google Scholar
  14. Lawler, E.L. (1976). Combinatorial Optimization: Networks and Matroids. New York: Holt, Rinehart and Winston.Google Scholar
  15. Mühlenbein, H. and D. Schlierkamp-Voosen. (1993a). “Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization,” Evolutionary Computation 1, 25–49.Google Scholar
  16. Mühlenbein, H. and D. Schlierkamp-Voosen. (1993b). “The Science of Breeding and its Application to the Breeder Genetic Algorithm,” Evolutionary Computation 1, 335–360.Google Scholar
  17. Mühlenbein, H. (1998). “The Equation for Response to Selection and its Use for Prediction, Evolutionary Computation 5, 303–346.Google Scholar
  18. Mühenbein, H. and Th. Mahnig. (1999a). “Convergence Theory and Applications of the Factorized Distribution Algorithm, Journal of Computing and Information Technology, to appear.Google Scholar
  19. Mühlenbein, H. and Th. Mahnig. (1999b). FDA—A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Discrete Functions, Evolutionary Compilation, to appear.Google Scholar
  20. Mühlenbein, H. and G. Paaß. (1996). “From Recombination of Genes to the Estimation of Distributions I. Binary Parameters.” In Voigt, H.-M. et al. (eds.), Lecture Notes in Computer Science 1141: Parallel Problem Solving from Nature—PPSN IV. Berlin: Springer, pp. 178–187.Google Scholar
  21. Pelikan, M. and H. Mühlenbein. (1999). “The Bivariate Marginal Distribution Algorithm.” In R. Roy, T. Furuhashi, and P.K. Chawdhry (eds.), Advances in Soft Computing—Engineering Design and Manufacturing. Berlin: Springer-Verlag, pp. 521–535.Google Scholar
  22. Tanese, R. (1989). “Distributed Genetic Algorithms for Function Optimization.” Unpublished doctoral dissertation, University of Michigan, Ann Arbor.Google Scholar
  23. Toulouse, G. (1977). “Optimal Graph Matching for 2D Ising Models,” Commun. Phys. 2, 115–119.Google Scholar
  24. Whitley, D., R. Beveridge, C. Graves, and K. Mathias. (1995). “Test Driving Three 1995 Genetic Algorithms: New Test Functions and Geometric Matching,” Journal of Heuristics 1, 77–104.Google Scholar

Copyright information

© Kluwer Academic Publishers 1999

Authors and Affiliations

  • Heinz Mühlenbein
    • 1
  • Thilo Mahnig
    • 1
  • Alberto Ochoa Rodriguez
    • 1
  1. 1.Real World Computing Partnership Theoretical Foundation GMD Laboratory, GMD—Forschungszentrum InformationstechnikSt. Augustin

Personalised recommendations