Advertisement

Journal of Heuristics

, Volume 18, Issue 5, pp 795–819 | Cite as

A review on probabilistic graphical models in evolutionary computation

  • Pedro Larrañaga
  • Hossein Karshenas
  • Concha Bielza
  • Roberto Santana
Article

Abstract

Thanks to their inherent properties, probabilistic graphical models are one of the prime candidates for machine learning and decision making tasks especially in uncertain domains. Their capabilities, like representation, inference and learning, if used effectively, can greatly help to build intelligent systems that are able to act accordingly in different problem domains. Evolutionary algorithms is one such discipline that has employed probabilistic graphical models to improve the search for optimal solutions in complex problems. This paper shows how probabilistic graphical models have been used in evolutionary algorithms to improve their performance in solving complex problems. Specifically, we give a survey of probabilistic model building-based evolutionary algorithms, called estimation of distribution algorithms, and compare different methods for probabilistic modeling in these algorithms.

Keywords

Probabilistic graphical model Bayesian network Evolutionary computation Estimation of distribution algorithm 

Notes

Acknowledgements

This work has been partially supported by TIN2010-20900-C04-04, Consolider Ingenio 2010-CSD2007-00018 and Cajal Blue Brain projects (Spanish Ministry of Science and Innovation).

References

  1. Ahn, C.W., An, J., Yoo, J.C.: Estimation of particle swarm distribution algorithms: combining the benefits of PSO and EDAs. Inf. Sci. 192, 109–119 (2012) CrossRefGoogle Scholar
  2. Ahn, C., Ramakrishna, R., Goldberg, D.: Real-coded Bayesian optimization algorithm: bringing the strength of BOA into the continuous world. In: 6th Annual Conference on Genetic and Evolutionary Computation (GECCO’04), pp. 840–851. Springer, Berlin (2004) Google Scholar
  3. Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974) MathSciNetzbMATHCrossRefGoogle Scholar
  4. Alden, M.E.: MARLEDA: effective distribution estimation through Markov random fields. Ph.D. Thesis, The University of Texas at Austin (2007) Google Scholar
  5. Baluja, S.: Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. Tech. Rep. CMU-CS-94-163, Carnegie-Mellon University (1994) Google Scholar
  6. Baluja, S., Davies, S.: Using optimal dependency-trees for combinational optimization. In: 14th International Conference on Machine Learning, pp. 30–38. Morgan Kaufmann, San Mateo (1997) Google Scholar
  7. Bengoetxea, E., Larrañaga, P.: EDA-PSO: a hybrid paradigm combining estimation of distribution algorithms and particle swarm optimization. In: Swarm Intelligence. Lecture Notes in Computer Science, vol. 6234, pp. 416–423. Springer, Berlin (2010) CrossRefGoogle Scholar
  8. Bosman, P.A.N., Grahl, J.: Matching inductive search bias and problem structure in continuous estimation of distribution algorithms. Eur. J. Oper. Res. 185(3), 1246–1264 (2008) zbMATHCrossRefGoogle Scholar
  9. Bosman, P.A.N., Grahl, J., Thierens, D.: Enhancing the performance of maximum-likelihood Gaussian EDAs using anticipated mean shift. In: 10th International Conference on Parallel Problem Solving from Nature (PPSN X), pp. 133–143. Springer, Berlin (2008) CrossRefGoogle Scholar
  10. Bosman, P.A.N., Thierens, D.: Advancing continuous IDEAs with mixture distributions and factorization selection metrics. In: Optimization by building and using probabilistic models (OBUPM) Workshop at the Genetic and Evolutionary Computation Conference (GECCO’01), pp. 208–212. ACM, New York (2001) Google Scholar
  11. Bosman, P.A.N., de Jong, E.: Adaptation of a success story in GAs: Estimation-of-distribution algorithms for tree-based optimization problems. In: Success in Evolutionary Computation. Studies in Computational Intelligence, vol. 92, pp. 3–18. Springer, Berlin (2008) CrossRefGoogle Scholar
  12. Bosman, P.A.N., Thierens, D.: Linkage information processing in distribution estimation algorithms. In: Genetic and Evolutionary Computation Conference (GECCO’99), pp. 60–67. Morgan Kaufmann, San Mateo (1999) Google Scholar
  13. Bosman, P.A.N., Thierens, D.: Continuous iterated density estimation evolutionary algorithms within the IDEA framework. In: Genetic and Evolutionary Computation Conference (GECCO’00) Workshop, pp. 197–200 (2000a) Google Scholar
  14. Bosman, P.A.N., Thierens, D.: Expanding from discrete to continuous estimation of distribution algorithms: the IDEA. In: 6th International Conference on Parallel Problem Solving from Nature (PPSN VI), pp. 767–776. Springer, Berlin (2000b) CrossRefGoogle Scholar
  15. Bouckaert, R.R.: Bayesian belief networks: from construction to inference. Ph.D. Thesis, Universiteit Utrecht, Faculteit Wiskunde en Informatica (1995) Google Scholar
  16. Brownlee, A., McCall, J., Zhang, Q., Brown, D.: Approaches to selection and their effect on fitness modelling in an estimation of distribution algorithm. In: IEEE Congress on Evolutionary Computation (CEC 2008)—IEEE World Congress on Computational Intelligence, pp. 2621–2628. IEEE Comput. Soc., Los Alamitos (2008) CrossRefGoogle Scholar
  17. Brownlee, A.E.I.: Multivariate Markov networks for fitness modelling in an estimation of distribution algorithm. Ph.D. Thesis, The Robert Gordon University. School of Computing (2009) Google Scholar
  18. Buntine, W.: Theory refinement on Bayesian networks. In: 7th Conference on Uncertainty in Artificial Intelligence (UAI’91), vol. 91, pp. 52–60. Morgan Kaufmann, San Mateo (1991) Google Scholar
  19. Chickering, D.: Learning Bayesian networks is NP-complete. In: Learning from Data: Artificial Intelligence and Statistics V. Lecture Notes in Statistics, vol. 112, pp. 121–130. Springer, Berlin (1996) Google Scholar
  20. Chickering, D., Geiger, D., Heckerman, D.: Learning Bayesian networks is NP-hard. Tech. Rep. MSR-TR-94-17, Microsoft Research (1994) Google Scholar
  21. Chickering, D., Heckerman, D., Meek, C.: Large-sample learning of Bayesian networks is NP-hard. J. Mach. Learn. Res. 5, 1287–1330 (2004) MathSciNetzbMATHGoogle Scholar
  22. Cho, D.Y., Zhang, B.T.: Evolutionary optimization by distribution estimation with mixtures of factor analyzers. In: IEEE Congress on Evolutionary Computation (CEC’02), vol. 2, pp. 1396–1401. IEEE Comput. Soc., Los Alamitos (2002) Google Scholar
  23. Cho, D.Y., Zhang, B.T.: Evolutionary continuous optimization by distribution estimation with variational Bayesian independent component analyzers mixture model. In: Parallel Problem Solving from Nature (PPSN VIII). Lecture Notes in Computer Science, vol. 3242, pp. 212–221. Springer, Berlin (2004) CrossRefGoogle Scholar
  24. Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992) zbMATHGoogle Scholar
  25. Costa, M., Minisci, E.: MOPED: a multi-objective Parzen-based estimation of distribution algorithm for continuous problems. In: Evolutionary Multi-Criterion Optimization. Lecture Notes in Computer Science, vol. 2632, p. 71. Springer, Berlin (2003) CrossRefGoogle Scholar
  26. Cramer, N.L.: A representation for the adaptive generation of simple sequential programs. In: First International Conference on Genetic Algorithms, pp. 183–187. Erlbaum, Hillsdale (1985) Google Scholar
  27. Cuesta-Infante, A., Santana, R., Hidalgo, J.I., Bielza, C., Larrañaga, P.: Bivariate empirical and n-variate Archimedean copulas in estimation of distribution algorithms. In: IEEE Congress on Evolutionary Computation (CEC’10) (2010) Google Scholar
  28. Dawid, A.P.: Conditional independence in statistical theory. J. R. Stat. Soc. B 41(1), 1–31 (1979) MathSciNetzbMATHGoogle Scholar
  29. De Bonet, J., Isbell, C., Viola, P.M.: Finding optima by estimating probability densities. Adv. Neural Inf. Process. Syst. 9, 424–430 (1997) Google Scholar
  30. de Castro, P.A.D., Zuben, F.J.V.: BAIS: a Bayesian artificial immune system for the effective handling of building blocks. Inf. Sci. 179(10), 1426–1440 (2009) CrossRefGoogle Scholar
  31. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39(1), 1–38 (1977) MathSciNetzbMATHGoogle Scholar
  32. Ding, N., Zhou, S., Sun, Z.: Histogram-based estimation of distribution algorithm: a competent method for continuous optimization. J. Comput. Sci. Technol. 23(1), 35–43 (2008) CrossRefGoogle Scholar
  33. Echegoyen, C., Mendiburu, A., Santana, R., Lozano, J.: Analyzing the probability of the optimum in EDAs based on Bayesian networks. In: IEEE Congress on Evolutionary Computation (CEC’09), pp. 1652–1659 (2009) CrossRefGoogle Scholar
  34. Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Second Symposium on Artificial Intelligence (CIMAF-99), pp. 332–339 (1999) Google Scholar
  35. Fogel, L.J.: Artificial Intelligence Through Simulated Evolution. Wiley, New York (1966) zbMATHGoogle Scholar
  36. Frey, B.J., Dueck, D.: Mixture modeling by affinity propagation. In: Advances in Neural Information Processing Systems, vol. 18, pp. 379–386. MIT Press, Cambridge (2006) Google Scholar
  37. Frydenberg, M.: The chain graph Markov property. Scand. J. Stat. 17(4), 333–353 (1990) MathSciNetzbMATHGoogle Scholar
  38. Gámez, J., Mateo, J., Puerta, J.E.: Estimation of dependency networks algorithm. In: Bio-inspired Modeling of Cognitive Tasks. Lecture Notes in Computer Science, vol. 4527, pp. 427–436. Springer, Berlin (2007) CrossRefGoogle Scholar
  39. Geiger, D., Heckerman, D.: Learning Gaussian networks. In: 10th Conference on Uncertainty in Artificial Intelligence (UAI’94), pp. 235–243 (1994) Google Scholar
  40. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984) zbMATHCrossRefGoogle Scholar
  41. Goldberg, D.E.: The Design of Innovation: Lessons from and for Competent Genetic Algorithms. Kluwer Academic, Norwell (2002) zbMATHGoogle Scholar
  42. González, C., Lozano, J., Larrañaga, P.: Mathematical modelling of UMDAc algorithm with tournament selection. Behaviour on linear and quadratic functions. Int. J. Approx. Reason. 31(3), 313–340 (2002) zbMATHCrossRefGoogle Scholar
  43. Grahl, J., Bosman, P.A.N., Rothlauf, F.: The correlation-triggered adaptive variance scaling IDEA. In: 8th Annual Conference on Genetic and Evolutionary Computation (GECCO’06), pp. 397–404. ACM, New York (2006) CrossRefGoogle Scholar
  44. Grünwald, P.: The minimum description length principle and reasoning under uncertainty. Ph.D. Thesis, University of Amsterdam (1998) Google Scholar
  45. Hansen, N.: The CMA evolution strategy: a comparing review. In: (Lozano et al. 2006), pp. 75–102 (2006) Google Scholar
  46. Harik, G., Cantú-Paz, E., Goldberg, D., Miller, B.: The gambler’s ruin problem, genetic algorithms, and the sizing of populations. Evol. Comput. 7(3), 231–253 (1999) CrossRefGoogle Scholar
  47. Harik, G.R., Lobo, F.G., Sastry, K.: Linkage learning via probabilistic modeling in the Extended Compact Genetic Algorithm (ECGA). In: (Pelikan et al. 2006), pp. 39–61 (2006). Chap. 3 Google Scholar
  48. Harik, G., Lobo, F., Goldberg, D.: The compact genetic algorithm. IEEE Trans. Evol. Comput. 3(4), 287–297 (1999) CrossRefGoogle Scholar
  49. Hasegawa, Y., Iba, H.: A Bayesian network approach to program generation. IEEE Trans. Evol. Comput. 12(6), 750–764 (2008) CrossRefGoogle Scholar
  50. Heckerman, D., Geiger, D., Chickering, D.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995) zbMATHGoogle Scholar
  51. Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2001) zbMATHGoogle Scholar
  52. Holland, J.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975) Google Scholar
  53. Hong, Y., Zhu, G., Kwong, S., Ren, Q.: Estimation of distribution algorithms making use of both high quality and low quality individuals. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE’09), pp. 1806–1813. IEEE Comput. Soc., Los Alamitos (2009) Google Scholar
  54. Karshenas, H., Nikanjam, A., Helmi, B.H., Rahmani, A.T.: Combinatorial effects of local structures and scoring metrics in Bayesian optimization algorithm. In: First ACM/SIGEVO Summit on Genetic and Evolutionary Computation (GEC’09), pp. 263–270. ACM, New York (2009) CrossRefGoogle Scholar
  55. Karshenas, H., Santana, R., Bielza, C., Larrañaga, P.: Multi-objective optimization with joint probabilistic modeling of objectives and variables. In: Evolutionary Multi-Criterion Optimization. Lecture Notes in Computer Science, vol. 6576, pp. 298–312. Springer, Berlin (2011) CrossRefGoogle Scholar
  56. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009) Google Scholar
  57. Koza, J.R.: Genetic Programming: on the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992) zbMATHGoogle Scholar
  58. Larrañaga, P., Etxeberria, R., Lozano, J., Pena, J.: Optimization by learning and simulation of Bayesian and Gaussian networks. Tech. Rep. EHU-KZAAIK-IK-4/99, Intelligent Systems Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country (1999) Google Scholar
  59. Larrañaga, P., Etxeberria, R., Lozano, J., Peña, J.: Combinatonal optimization by learning and simulation of Bayesian networks. In: 16th Conference on Uncertainty in Artificial Intelligence (UAI’00), pp. 343–352. Morgan Kaufmann, San Mateo (2000a) Google Scholar
  60. Larrañaga, P., Etxeberria, R., Lozano, J., Peña, J.: Optimization in continuous domains by learning and simulation of Gaussian networks. In: Conference on Genetic and Evolutionary Computation (GECCO’00) Workshop Program, pp. 201–204. Morgan Kaufmann, San Mateo (2000b) Google Scholar
  61. Larrañaga, P., Lozano, J. (eds.): Estimation of Distribution Algorithms: a New Tool for Evolutionary Computation. Kluwer Academic, Norwell (2001) Google Scholar
  62. Larrañaga, P., Moral, S.: Probabilistic graphical models in artificial intelligence. Appl. Soft Comput. 11(2), 1511–1528 (2011) CrossRefGoogle Scholar
  63. Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structures and their application to expert systems. J. R. Stat. Soc. B 50(2), 157–224 (1988) MathSciNetzbMATHGoogle Scholar
  64. Li, B., Zhong, R.T., Wang, X.J., Zhuang, Z.Q.: Continuous optimization based-on boosting Gaussian mixture model. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 1192–1195 (2006) Google Scholar
  65. Lima, C., Pelikan, M., Goldberg, D., Lobo, F., Sastry, K., Hauschild, M.: Influence of selection and replacement strategies on linkage learning in BOA. In: CEC 2007, IEEE Congress on Evolutionary Computation, pp. 1083–1090 (2007) CrossRefGoogle Scholar
  66. Lima, C., Pelikan, M., Lobo, F., Goldberg, D.: Loopy substructural local search for the Bayesian optimization algorithm. In: Engineering Stochastic Local Search Algorithms. Designing, Implementing and Analyzing Effective Heuristics. Lecture Notes in Computer Science, vol. 5752, pp. 61–75. Springer, Berlin (2009) CrossRefGoogle Scholar
  67. Lozano, J., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.): Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms. Studies in Fuzziness and Soft Computing, vol. 192. Springer, Berlin (2006) zbMATHGoogle Scholar
  68. Luo, N., Qian, F.: Evolutionary algorithm using kernel density estimation model in continuous domain. In: 7th Asian Control Conference (ASCC’09), pp. 1526–1531 (2009) Google Scholar
  69. Martí, L., García, J., Berlanga, A., Coello, C.A.C., Molina, J.M.: MB-GNG: addressing drawbacks in multi-objective optimization estimation of distribution algorithms. Oper. Res. Lett. 39(2), 150–154 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  70. McKay, R., Hoai, N., Whigham, P., Shan, Y., O’Neill, M.: Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11, 365–396 (2010) CrossRefGoogle Scholar
  71. Mendiburu, A., Santana, R., Lozano, J.A.: Introducing belief propagation in estimation of distribution algorithms: a parallel framework. Tech. Rep. EHU-KAT-IK-11-07, Intelligent Systems Group, University of the Basque Country (2007) Google Scholar
  72. Michalski, R.S.: Learnable evolution model: evolutionary processes guided by machine learning. Mach. Learn. 38, 9–40 (2000) zbMATHCrossRefGoogle Scholar
  73. Miquélez, T., Bengoetxea, E., Larrañaga, P.: Evolutionary computation based on Bayesian classifiers. Int. J. Appl. Math. Comput. Sci. 14(3), 335–350 (2004) MathSciNetzbMATHGoogle Scholar
  74. Miquélez, T., Bengoetxea, E., Larrañaga, P.: Evolutionary Bayesian classifier-based optimization in continuous domains. In: 6th International Conference on Simulated Evolution and Learning (SEAL’06). Lecture Notes in Computer Science, vol. 4247, pp. 529–536. Springer, Berlin (2006) CrossRefGoogle Scholar
  75. Mühlenbein, H., Mahnig, T.: FDA–A scalable evolutionary algorithm for the optimization of additively decomposed functions. Evol. Comput. 7(4), 353–376 (1999) CrossRefGoogle Scholar
  76. Mühlenbein, H., Mahnig, T., Ochoa Rodríguez, A.: Schemata, distributions and graphical models in evolutionary optimization. J. Heuristics 5(2), 215–247 (1999) zbMATHCrossRefGoogle Scholar
  77. Mühlenbein, H., Paaß, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In: 4th International Conference on Parallel Problem Solving from Nature (PPSN IV). Lecture Notes in Computer Science, vol. 1141, pp. 178–187. Springer, Berlin (1996) CrossRefGoogle Scholar
  78. Očenášek, J., Schwarz, J.: Estimation distribution algorithm for mixed continuous-discrete optimization problems. In: Intelligent Technologies: Theory and Applications: New Trends in Intelligent Technologies, pp. 227–232. IOS Press, Amsterdam (2002) Google Scholar
  79. Očenášek, J., Kern, S., Hansen, N., Koumoutsakos, P.: A mixed Bayesian optimization algorithm with variance adaptation. In: Parallel Problem Solving from Nature (PPSN VIII). Lecture Notes in Computer Science, vol. 3242, pp. 352–361. Springer, Berlin (2004) CrossRefGoogle Scholar
  80. Pearl, J.: Bayesian networks: a model of self-activated memory for evidential reasoning. In: 7th Conference of the Cognitive Science Society, pp. 329–334 (1985) Google Scholar
  81. Pelikan, M., Goldberg, D., Lobo, F.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002) MathSciNetzbMATHCrossRefGoogle Scholar
  82. Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Advances in Soft Computing-Engineering Design and Manufacturing, pp. 521–535 (1999) Google Scholar
  83. Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.): Scalable Optimization via Probabilistic Modeling: from Algorithms to Applications. Springer, Berlin (2006) zbMATHGoogle Scholar
  84. Pelikan, M., Sastry, K., Goldberg, D.: Sporadic model building for efficiency enhancement of the hierarchical BOA. Genet. Program. Evol. Mach. 9(1), 53–84 (2008) CrossRefGoogle Scholar
  85. Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms, 1st edn. Studies in Fuzziness and Soft Computing, vol. 170. Springer, Berlin (2005) zbMATHGoogle Scholar
  86. Pelikan, M., Goldberg, D.: Genetic algorithms, clustering, and the breaking of symmetry. In: Parallel Problem Solving from Nature (PPSN VI). Lecture Notes in Computer Science, vol. 1917, pp. 385–394. Springer, Berlin (2000) CrossRefGoogle Scholar
  87. Pelikan, M., Goldberg, D.E., Cantú-Paz, E.B.: The Bayesian optimization algorithm. In: Conference on Genetic and Evolutionary Computation (GECCO’99), vol. 1, pp. 525–532. Morgan Kaufmann, San Mateo (1999) Google Scholar
  88. Pelikan, M., Hartmann, A.: Searching for ground states of Ising spin glasses with hierarchical BOA and cluster exact approximation. In: (Pelikan et al. 2006), pp. 333–349 (2006) Google Scholar
  89. Pelikan, M., Sastry, K.: Fitness inheritance in the Bayesian optimization algorithm. In: Conference on Genetic and Evolutionary Computation (GECCO’04). Lecture Notes in Computer Science, vol. 3103, pp. 48–59. Springer, Berlin (2004) Google Scholar
  90. Peña, J.M., Lozano, J.A., Larrañaga, P.: Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks. Evol. Comput. 13(1), 43–66 (2005) CrossRefGoogle Scholar
  91. Pošík, P.: Preventing premature convergence in a simple EDA via global step size setting. In: 10th International Conference on Parallel Problem Solving from Nature (PPSN X). Lecture Notes in Computer Science, vol. 5199, pp. 549–558. Springer, Berlin (2008) CrossRefGoogle Scholar
  92. Pošík, P.: BBOB-benchmarking a simple estimation of distribution algorithm with cauchy distribution. In: 11th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’09), pp. 2309–2314. ACM, New York (2009a) Google Scholar
  93. Pošík, P.: Stochastic local search techniques with unimodal continuous distributions: a survey. In: EvoWorkshops on Applications of Evolutionary Computing (EvoWorkshops’09), pp. 685–694. Springer, Berlin (2009b) Google Scholar
  94. Rechenberg, I.: Evolutionsstrategie-Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Ph.D. Thesis, reprinted by Fromman-Holzboog (1973) Google Scholar
  95. Rissanen, J.: Modeling by shortest data description. Automatica 14(5), 465–471 (1978) zbMATHCrossRefGoogle Scholar
  96. Robinson, R.: Counting unlabeled acyclic digraphs. In: Combinatorial Mathematics V. Lecture Notes in Mathematics, vol. 622, pp. 28–43. Springer, Berlin (1977) CrossRefGoogle Scholar
  97. Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.: Using copulas in estimation of distribution algorithms. In: Advances in Artificial Intelligence (MICAI’09). Lecture Notes in Computer Science, vol. 5845, pp. 658–668. Springer, Berlin (2009) Google Scholar
  98. Sałustowicz, R.P., Schmidhuber, J.: Probabilistic incremental program evolution: stochastic search through program space. In: 9th European Conference on Machine Learning (ECML’97). Lecture Notes in Computer Science, vol. 1224, pp. 213–220. Springer, Berlin (1997) Google Scholar
  99. Santana, R., Bielza, C., Lozano, J., Larrañaga, P.: Mining probabilistic models learned by EDAs in the optimization of multi-objective problems. In: 11th Annual Conference on Genetic and Evolutionary Computation (GECCO’09), pp. 445–452. ACM, New York (2009a) CrossRefGoogle Scholar
  100. Santana, R., Larrañaga, P., Lozano, J.: Research topics in discrete estimation of distribution algorithms based on factorizations. Memet. Comput. 1(1), 35–54 (2009b) CrossRefGoogle Scholar
  101. Santana, R., Larrañaga, P., Lozano, J.: Learning factorizations in estimation of distribution algorithms using affinity propagation. Evol. Comput. 18(4), 515–546 (2010) CrossRefGoogle Scholar
  102. Santana, R.: A Markov network based factorized distribution algorithm for optimization. In: 14th European Conference on Machine Learning (ECML’03). Lecture Notes in Computer Science, vol. 2837, pp. 337–348. Springer, Berlin (2003) Google Scholar
  103. Santana, R.: Estimation of distribution algorithms with Kikuchi approximations. Evol. Comput. 13, 67–97 (2005) CrossRefGoogle Scholar
  104. Santana, R.: Estimation of distribution algorithms: from available implementations to potential developments. In: 13th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’11), pp. 679–686. ACM, New York (2011) Google Scholar
  105. Santana, R., Bielza, C., Larrañaga, P., Lozano, J.A., Echegoyen, C., Mendiburu, A., Armañanzas, R., Shakya, S.: Mateda-2.0: estimation of distribution algorithms in MATLAB. J. Stat. Softw. 35(7), 1–30 (2010) Google Scholar
  106. Sastry, K., Goldberg, D.E.: Probabilistic model building and competent genetic programming. In: Genetic Programming Theory and Practice, pp. 205–220. Kluwer Academic, Norwell (2003). Chap. 13 CrossRefGoogle Scholar
  107. Sastry, K., Pelikan, M., Goldberg, D.: Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. In: IEEE Congress on Evolutionary Computation (CEC’04), vol. 1, pp. 720–727 (2004) Google Scholar
  108. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978) zbMATHCrossRefGoogle Scholar
  109. Sebag, M., Ducoulombier, A.: Extending population-based incremental learning to continuous search spaces. In: 5th International Conference on Parallel Problem Solving from Nature (PPSN V). Lecture Notes in Computer Science, vol. 1498, pp. 418–427. Springer, Berlin (1998) CrossRefGoogle Scholar
  110. Shakya, S.: DEUM: a framework for an estimation of distribution algorithm based on Markov random fields. Ph.D. Thesis, The Robert Gordon University (2006) Google Scholar
  111. Shakya, S., Santana, R. (eds.): Markov Networks in Evolutionary Computation. Adaptation, Learning, and Optimization, vol. 14. Springer, Berlin (2012) zbMATHGoogle Scholar
  112. Shan, Y., McKay, R., Essam, D., Abbass, H.: a survey of probabilistic model building genetic programming. In: (Pelikan et al. 2006), pp. 121–160 (2006) Google Scholar
  113. Spirtes, P., Glymour, C.: An algorithm for fast recovery of sparse causal graphs. Soc. Sci. Comput. Rev. 9(1), 62–72 (1991) CrossRefGoogle Scholar
  114. Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search, 2nd edn. MIT Press, Cambridge (2001) zbMATHGoogle Scholar
  115. Sun, J., Zhang, Q., Tsang, E.: DE/EDA: a new evolutionary algorithm for global optimization. Inf. Sci. 169(3–4), 249–262 (2005) MathSciNetCrossRefGoogle Scholar
  116. Thierens, D.: The linkage tree genetic algorithm. In: Parallel Problem Solving from Nature (PPSN XI). Lecture Notes in Computer Science, vol. 6238, pp. 264–273. Springer, Berlin (2011) Google Scholar
  117. Tsutsui, S., Pelikan, M., Goldberg, D.: Node histogram vs. edge histogram: a comparison of pmbgas in permutation domains. Tech. Rep. 2006009, Missouri Estimation of Distribution Algorithms Laboratory (MEDAL), Department of Mathematics and Computer Science, University of Missouri–St. Louis (2006) Google Scholar
  118. Tsutsui, S.: Probabilistic model-building genetic algorithms in permutation representation domain using edge histogram. In: Parallel Problem Solving from Nature (PPSN VII). Lecture Notes in Computer Science, vol. 2439, pp. 224–233. Springer, Berlin (2002) Google Scholar
  119. Tsutsui, S., Pelikan, M., Goldberg, D.E.: Evolutionary algorithm using marginal histogram in continuous domain. In: Optimization by Building and Using Probabilistic Models (OBUPM) Workshop—Conference on Genetic and Evolutionary Computation (GECCO’01), pp. 230–233 (2001) Google Scholar
  120. Valdez-Peña, S.I., Hernández-Aguirre, A., Botello-Rionda, S.: Approximating the search distribution to the selection distribution in EDAs. In: 11th Annual Conference on Genetic and Evolutionary Computation (GECCO’09), pp. 461–468. ACM, New York (2009) CrossRefGoogle Scholar
  121. Wang, L.F., Zeng, J.C.: Estimation of distribution algorithm based on copula theory. In: Exploitation of Linkage Learning in Evolutionary Algorithms. Evolutionary Learning and Optimization, vol. 3, pp. 139–162. Springer, Berlin (2010) CrossRefGoogle Scholar
  122. Wang, L.F., Zeng, J.C., Hong, Y.: Estimation of distribution algorithm based on Archimedean copulas. In: First ACM/SIGEVO Summit on Genetic and Evolutionary Computation (GEC’09), pp. 993–996. ACM, New York (2009) CrossRefGoogle Scholar
  123. Wang, X., Wang, H.: Evolutionary optimization with Markov random field prior. IEEE Trans. Evol. Comput. 8(6), 567–579 (2004) CrossRefGoogle Scholar
  124. Weise, T., Niemczyk, S., Chiong, R., Wan, M.: A framework for multi-model EDAs with model recombination. In: Applications of Evolutionary Computation. Lecture Notes in Computer Science, vol. 6624, pp. 304–313. Springer, Berlin (2011) CrossRefGoogle Scholar
  125. Xiao, J., Yan, Y., Zhang, J.: HPBILc: a histogram-based EDA for continuous optimization. Appl. Math. Comput. 215(3), 973–982 (2009) zbMATHCrossRefGoogle Scholar
  126. Yanai, K., Iba, H.: Estimation of distribution programming based on Bayesian network. In: IEEE Congress on Evolutionary Computation (CEC’03), vol. 3, pp. 1618–1625 (2003) Google Scholar
  127. Zhang, Q.: On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm. IEEE Trans. Evol. Comput. 8(1), 80–93 (2004) CrossRefGoogle Scholar
  128. Zhang, Q., Zhou, A., Jin, Y.: RM-MEDA: a regularity model based multiobjective estimation of distribution algorithm. IEEE Trans. Evol. Comput. 12(1), 41–63 (2008) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Pedro Larrañaga
    • 1
  • Hossein Karshenas
    • 1
  • Concha Bielza
    • 1
  • Roberto Santana
    • 2
  1. 1.Computational Intelligence Group, Facultad de InformáticaUniversidad Politécnica de MadridBoadilla del MonteSpain
  2. 2.Intelligent System Group, Department of Computer Science and Artificial IntelligenceUniversity of the Basque CountrySan SebastinSpain

Personalised recommendations