Advertisement

Natural Computing

, Volume 15, Issue 1, pp 165–180 | Cite as

A review of message passing algorithms in estimation of distribution algorithms

  • Roberto Santana
  • Alexander Mendiburu
  • Jose A. Lozano
Article

Abstract

Message passing algorithms (MPAs) have been traditionally used as an inference method in probabilistic graphical models. Some MPA variants have recently been introduced in the field of estimation of distribution algorithms (EDAs) as a way to improve the efficiency of these algorithms. Multiple developments on MPAs point to an increasing potential of these methods for their application as part of hybrid EDAs. In this paper we review recent work on EDAs that apply MPAs and propose ways to further extend the useful synergies between MPAs and EDAs. Furthermore, we analyze some of the implications that MPA developments can have in their future application to EDAs and other evolutionary algorithms.

Keywords

Message passing algorithms Propagation Estimation of distribution algorithms Hybridization 

Notes

Acknowledgments

This work has been partially supported by the Saiotek and Research Groups 2013–2018 (IT-609-13) programs (Basque Government), TIN2013-41272P (Ministry of Science and Technology of Spain), COMBIOMED network in computational bio-medicine (Carlos III Health Institute), and by the NICaiA Project PIRSES-GA-2009-247619 (European Commission).

References

  1. Abbeel P, Koller D, Ng AY (2006) Learning factor graphs in polynomial time and sample complexity. J Mach Learn Res 7:1743–1788MathSciNetzbMATHGoogle Scholar
  2. Baluja S, Davies S (1997) Using optimal dependency-trees for combinatorial optimization: learning the structure of the search space. In: Fisher DH (ed) Proceedings of the 14th international conference on machine learning. Morgan Kaufmann, San Francisco, pp 30–38Google Scholar
  3. Batra D, Gallagher A, Parikh D, Chen T (2010) Beyond trees: MRF inference via outer-planar decomposition. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, San Francisco, pp 2496–2503Google Scholar
  4. Bickson D (2008) Gaussian belief propagation: theory and application. arXiv preprint arXiv:0811.2518. Accessed 18 Dec 2014
  5. Braunstein A, Mézard M, Zecchina R (2005) Survey propagation: an algorithm for satisfiability. Random Struct Algorithms 27(2):201–226CrossRefzbMATHGoogle Scholar
  6. Braunstein A, Mézard M, Zecchina R (2006) Constraint satisfaction by survey propagation. In: Percus A, Istrate G, Moore C (eds) Computational complexity and statistical physics. Oxford University Press, Oxford, pp 107–124Google Scholar
  7. Brownlee AE, McCall JA, Shakya SK, Zhang Q (2010) Structure learning and optimisation in a Markov network based estimation of distribution algorithm. In: Exploitation of linkage learning in evolutionary algorithms. Springer, Berlin, pp 45–69Google Scholar
  8. Brownlee AEI (2009) Multivariate Markov networks for fitness modelling in an estimation of distribution algorithm. PhD Thesis, The Robert Gordon University, School of Computing, AberdeenGoogle Scholar
  9. Brownlee AEI, McCall J, Pelikan M (2012) Influence of selection on structure learning in Markov network EDAs: an empirical study. MEDAL Report No. 2012006. Missouri Estimation of Distribution Algorithms Laboratory (MEDAL)Google Scholar
  10. Brownlee AEI, McCall J, Zhang Q, Brown D (2008) Approaches to selection and their effect on fitness modelling in an estimation of distribution algorithm. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 2621–2628Google Scholar
  11. Ceberio J, Irurozki E, Mendiburu A, Lozano JA (2012) A review on estimation of distribution algorithms in permutation-based combinatorial optimization problems. Prog Artif Intell 1(1):103–117CrossRefGoogle Scholar
  12. Ceberio J, Mendiburu A, Lozano JA (2013) The Plackett–Luce ranking model on permutation-based optimization problems. In: 2013 IEEE congress on evolutionary computation (CEC). IEEE, Cancún, pp 494–501Google Scholar
  13. Chen B, Hu J (2010a) An adaptive niching EDA based on clustering analysis. In: 2010 IEEE congress on evolutionary computation (CEC). IEEE, Barcelona, pp 1–7Google Scholar
  14. Chen B, Hu J (2010b) A novel clustering based niching EDA for protein folding. In: Proceedings of the world congress on nature and biologically inspired computing, 2009. NaBIC 2009. IEEE, Coimbatore, pp 748–753Google Scholar
  15. Crick C, Pfeffer A (2003) Loopy belief propagation as a basis for communication in sensor networks. In: Proceedings of the 19th annual conference on uncertainty in artificial intelligence (UAI-2003). Morgan Kaufmann Publishers, San Francisco, pp 159–166Google Scholar
  16. Deming WE, Stephan FF (1940) On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann Math Stat 11(4):427–444CrossRefMathSciNetGoogle Scholar
  17. Dolev D, Bickson D, Johnson J (2009) Fixing convergence of Gaussian belief propagation. In: IEEE international symposium on information theory, 2009. ISIT 2009. IEEE, Seoul, pp 1674–1678Google Scholar
  18. Domínguez E, Lage-Castellanos A, Mulet R, Ricci-Tersenghi F, Rizzo T (2011) Characterizing and improving generalized belief propagation algorithms on the 2D Edwards–Anderson model. J Stat Mech Theory Exp 2011(12):P12007CrossRefGoogle Scholar
  19. Dong W, Yao X (2008) NichingEDA: utilizing the diversity inside a population of EDAs for continuous optimization. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 1260–1267Google Scholar
  20. Duchi J, Tarlow D, Elidan G, Koller D (2007) Using combinatorial optimization within max-product belief propagation. In: Advances in neural information processing systems 19: proceedings of the 2006 conference, vol 19. The MIT Press, Cambridge, p 369Google Scholar
  21. Echegoyen C (2012) Contributions to the analysis and understanding of estimation of distribution algorithms. PhD Thesis, Department of Computer Science and Artificial Intelligence, University of the Basque CountryGoogle Scholar
  22. Echegoyen C, Lozano JA, Santana R, Larrañaga P (2007) Exact Bayesian network learning in estimation of distribution algorithms. In: Proceedings of the 2007 congress on evolutionary computation CEC-2007. IEEE Press, Los Alamitos, pp 1051–1058. doi: 10.1109/CEC.2007.4424586
  23. Echegoyen C, Santana R, Lozano JA, Larrañaga P (2008) The impact of probabilistic learning algorithms in EDAs based on Bayesian networks. In: Linkage in evolutionary computation, studies in computational intelligence. Springer, Berlin, pp 109–139. doi: 10.1007/978-3-540-85068-7_6
  24. Echegoyen C, Mendiburu A, Santana R, Lozano JA (2009) Analyzing the probability of the optimum in EDAs based on Bayesian networks. In: Proceedings of the 2009 congress on evolutionary computation CEC-2009. IEEE Press, Trondheim, pp 1652–1659. doi: 10.1109/CEC.2009.4983140
  25. Echegoyen C, Mendiburu A, Santana R, Lozano JA (2010a) Analyzing the k most probable solutions in EDAs based on Bayesian networks. In: Exploitation of linkage learning in evolutionary algorithms, evolutionary learning and optimization. Springer, pp 163–189. doi: 10.1007/978-3-642-12834-9_8
  26. Echegoyen C, Mendiburu A, Santana R, Lozano JA (2010b) Estimation of Bayesian networks algorithms in a class of complex networks. In: Proceedings of the 2010 congress on evolutionary computation CEC-2010. IEEE Press, Barcelona. doi: 10.1109/CEC.2010.5586511
  27. Echegoyen C, Zhang Q, Mendiburu A, Santana R, Lozano JA (2011) On the limits of effectiveness in estimation of distribution algorithms. In: Proceedings of the 2011 congress on evolutionary computation CEC-2007. IEEE Press, New Orleans, pp 1573–1580. doi: 10.1109/CEC.2011.5949803
  28. Echegoyen C, Mendiburu A, Santana R, Lozano JA (2012) Toward understanding EDAs based on Bayesian networks through a quantitative analysis. IEEE Trans Evol Comput 16(2):173–189. doi: 10.1109/TEVC.2010.2102037 CrossRefGoogle Scholar
  29. Etxeberria R, Larrañaga P (1999) Global optimization using Bayesian networks. In: Ochoa A, Soto MR, Santana R (eds) Proceedings of the second symposium on artificial intelligence (CIMAF-99). Editorial Academia, Havana, pp 332–339Google Scholar
  30. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315:972–976CrossRefMathSciNetzbMATHGoogle Scholar
  31. Furtlehner C, Schoenauer M (2010) Multi-objective 3-SAT with survey-propagation. In: Proceedings of the NIPS 2010 workshop on discrete optimization in machine learning: structures, algorithms and applications (DISCML), Whistler, Canada. http://hal.inria.fr/inria-00533149 Accessed 18 Dec 2014
  32. Gao Y, Culberson JC (2005) Space complexity of estimation of distribution algorithms. Evol Comput 13(1):125–143CrossRefMathSciNetGoogle Scholar
  33. Givoni IE, Frey BJ (2009) A binary variable model for affinity propagation. Neural Comput 21(6):1589–1600CrossRefMathSciNetzbMATHGoogle Scholar
  34. Givoni IE, Chung C, Frey BJ (2011) Hierarchical affinity propagation. In: Proceedings of the 27th annual conference on uncertainty in artificial intelligence (UAI-2011). Morgan Kaufmann, BarcelonaGoogle Scholar
  35. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, ReadingzbMATHGoogle Scholar
  36. González C, Lozano JA, Larrañaga P (2002) Mathematical modeling of UMDAc algorithm with tournament selection. Behaviour on linear and quadratic functions. Int J Approx Reason 31(4):313–340CrossRefzbMATHGoogle Scholar
  37. Grahl J, Minner S, Bosman PA (2008) Learning structure illuminates black boxes—an introduction to estimation of distribution algorithms. In: Advances in metaheuristics for hard optimization. Springer, Berlin, pp 365–395Google Scholar
  38. Harik G (1999) Linkage learning via probabilistic modeling in the ECGA. IlliGAL Report 99010. University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, UrbanaGoogle Scholar
  39. Hartmann AK, Weigt M (2005) Phase transitions in combinatorial optimization problems: basics, algorithms and statistical mechanics. Wiley, WeinheimCrossRefGoogle Scholar
  40. Helmi BH, Rahmani AT, Pelikan M (2014) A factor graph based genetic algorithm. Int J Appl Math Comput Sci 24(3):621–633CrossRefMathSciNetGoogle Scholar
  41. Henrion M (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In: Lemmer JF, Kanal LN (eds) Proceedings of the second annual conference on uncertainty in artificial intelligence. Elsevier, Amsterdam, pp 149–164Google Scholar
  42. Heskes T (2004) On the uniqueness of belief propagation fixed points. Neural Comput 16:2379–2413CrossRefzbMATHGoogle Scholar
  43. Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Ann ArborGoogle Scholar
  44. Höns R (2006) Estimation of distribution algorithms and minimum relative entropy. PhD Thesis, University of Bonn, BonnGoogle Scholar
  45. Höns R (2012) Using maximum entropy and generalized belief propagation in estimation of distribution algorithms. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 175–190CrossRefGoogle Scholar
  46. Höns R, Santana R, Larrañaga P, Lozano JA (2007) Optimization by max-propagation using Kikuchi approximations. Technical Report EHU-KZAA-IK-2/07. Department of Computer Science and Artificial Intelligence, University of the Basque CountryGoogle Scholar
  47. Ihler AT, Fisher J, Willsky AS (2006) Loopy belief propagation: convergence and effects of message errors. J Mach Learn Res 6(1):905MathSciNetGoogle Scholar
  48. Jiroušek R, Přeučil S (1995) On the effective implementation of the iterative proportional fitting procedure. Comput Stat Data Anal 19:177–189CrossRefzbMATHGoogle Scholar
  49. Johnson A, Shapiro JL (2002) The importance of selection mechanisms in distribution estimation algorithms. In: Collet P (ed) Proceedings of EA 2001, lecture notes in computer science, vol 2310. Springer, pp 91–103Google Scholar
  50. Kaban A, Bootkrajang J, Durrant RJ (2013) Towards large scale continuous EDA: a random matrix theory perspective. In: Proceedings of the genetic and evolutionary computation conference GECCO-2013. ACM, Amsterdam, pp 383–390Google Scholar
  51. Karshenas H, Santana R, Bielza C, Larrañaga P (2011) Multi-objective optimization with joint probabilistic modeling of objectives and variables. In: Evolutionary multi-criterion optimization: sixth international conference, EMO 2011, lecture notes in computer science. Springer, Berlin, pp 298–312. doi: 10.1007/978-3-642-19893-9_21
  52. Karshenas H, Santana R, Bielza C, Larrañaga P (2012) Continuous estimation of distribution algorithms based on factorized Gaussian Markov networks. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 157–173. doi: 10.1007/978-3-642-28900-2-10 CrossRefGoogle Scholar
  53. Kim K, McKay BR, Punithan D (2010) Sampling bias in estimation of distribution algorithms for genetic programming using prototype trees. In: PRICAI 2010: trends in artificial intelligence. Springer, Berlin, pp 100–111Google Scholar
  54. Kroc L, Sabharwal A, Selman B (2009) Message-passing and local heuristics as decimation strategies for satisfiability. In: Proceedings of the 2009 ACM symposium on applied computing. ACM, Honolulu, pp 1408–1414Google Scholar
  55. Kschischang FR, Frey BJ, Loeliger HA (2001) Factor graphs and the sum–product algorithm. IEEE Trans Inf Theory 47(2):498–519CrossRefMathSciNetzbMATHGoogle Scholar
  56. Larrañaga P, Lozano JA (eds) (2002) Estimation of distribution algorithms. A new tool for evolutionary computation. Kluwer Academic Publishers, BostonzbMATHGoogle Scholar
  57. Leone M, Weigt S, Weigt M (2007) Clustering by soft-constraint affinity propagation: applications to gene-expression data. Bioinformatics 23(20):2708–2715CrossRefGoogle Scholar
  58. Lima CF (2009) Substructural local search in discrete estimation of distribution algorithms. PhD Thesis, University of AlgarveGoogle Scholar
  59. Lima CF, Pelikan M, Goldberg DE, Lobo FG, Sastry K, Hauschild M (2007) Influence of selection and replacement strategies on linkage learning in BOA. In: Proceedings of the 2007 congress on evolutionary computation CEC-2007. IEEE Press, Los Alamitos, pp 1083–1090Google Scholar
  60. Lima CF, Pelikan M, Lobo FG, Goldberg DE (2009) Loopy substructural local search for the Bayesian optimization algorithm. In: Engineering stochastic local search algorithms. Designing, implementing and analyzing effective heuristics. Springer, Berlin, pp 61–75Google Scholar
  61. Lozano JA, Larrañaga P, Inza I, Bengoetxea E (eds) (2006) Towards a new evolutionary computation: advances on estimation of distribution algorithms. Springer, HeidelbergGoogle Scholar
  62. Mahfoud SW (1995) Niching methods for genetic algorithms. PhD Thesis, University of Illinois at Urbana-Champaign, Urbana. Also IlliGAL Report No. 95001Google Scholar
  63. Malioutov DM, Johnson JK, Willsky AS (2006) Walk-sums and belief propagation in Gaussian graphical models. J Mach Learn Res 7:2031–2064MathSciNetzbMATHGoogle Scholar
  64. Meltzer T, Yanover C, Weiss Y (2005) Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Tenth IEEE international conference on computer vision, pp 428–435Google Scholar
  65. Mendiburu A, Lozano J, Miguel-Alonso J (2005) Parallel implementation of EDAs based on probabilistic graphical models. IEEE Trans Evol Comput 9(4):406–423CrossRefGoogle Scholar
  66. Mendiburu A, Santana R, Bengoetxea E, Lozano, J (2007a) A parallel framework for loopy belief propagation. In: Thierens D et al (eds) Proceedings of the genetic and evolutionary computation conference GECCO-2007, vol II. Companion material. ACM Press, London, pp 2843–2850. http://dl.acm.org/citation.cfm?id=1274084 Accessed 18 Dec 2014
  67. Mendiburu A, Santana R, Lozano JA (2007b) Introducing belief propagation in estimation of distribution algorithms: a parallel framework. Technical Report EHU-KAT-IK-11/07. Department of Computer Science and Artificial Intelligence, University of the Basque CountryGoogle Scholar
  68. Mendiburu A, Santana R, Lozano JA (2012) Fast fitness improvements in estimation of distribution algorithms using belief propagation. In: Santana R, Shakya S (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 141–155. doi: 10.1007/978-3-642-28900-2-9 CrossRefGoogle Scholar
  69. Mézard M, Parisi G, Zechina R (2002) Analytic and algorithmic solution of random satisfiability problems. Science 297:812–812. doi: 10.1126/science.1073287 CrossRefGoogle Scholar
  70. Minka T (2001) A family of algorithms for approximate bayesian inference. PhD Thesis, Massachusetts Institute of TechnologyGoogle Scholar
  71. Minka T (2005) Divergence measures and message passing. Technical Report TR-2005-173. Mitsubishi Electric Research LaboratoriesGoogle Scholar
  72. Mooij JM (2005) Validity estimates for loopy belief propagation on binary real-world networks. In: Advances in neural information processing systems 17. MIT Press, Cambridge, pp 945–952Google Scholar
  73. Mooij J (2010) libDAI: a free and open source C++ library for discrete approximate inference in graphical models. J Mach Learn Res 11:2169–2173zbMATHGoogle Scholar
  74. Mühlenbein H (2012) Convergence theorems of estimation of distribution algorithms. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 91–108CrossRefGoogle Scholar
  75. Mühlenbein H, Höns R (2006) The factorized distributions and the minimum relative entropy principle. In: Pelikan M, Sastry K, Cantú-Paz E (eds) Scalable optimization via probabilistic modeling: from algorithms to applications, studies in computational intelligence. Springer, Berlin, pp 11–38Google Scholar
  76. Mühlenbein H, Mahnig T (2002) Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. Int J Approx Reason 31(3):157–192CrossRefzbMATHGoogle Scholar
  77. Mühlenbein H, Paaß G (1996) From recombination of genes to the estimation of distributions I. Binary parameters. In: Voigt HM, Ebeling W, Rechenberg I, Schwefel HP (eds) Parallel problem solving from nature—PPSN IV, lectures notes in computer science, vol 1141. Springer, Berlin, pp 178–187CrossRefGoogle Scholar
  78. Mühlenbein H, Mahnig T, Ochoa A (1999) Schemata, distributions and graphical models in evolutionary optimization. J Heuristics 5(2):213–247CrossRefGoogle Scholar
  79. Munetomo M, Murao N, Akama K (2008) Introducing assignment functions to Bayesian optimization algorithms. Inf Sci 178(1):152–163CrossRefzbMATHGoogle Scholar
  80. Murphy KP, Weiss Y, Jordan MI (1999) Loopy belief propagation for approximate inference: an empirical study. In: Proceedings of the fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Inc., San Francisco, pp 467–475Google Scholar
  81. Nilsson D (1998) An efficient algorithm for finding the M most probable configurations in probabilistic expert systems. Stat Comput 2:159–173CrossRefGoogle Scholar
  82. Ocenasek J, Schwarz J (2002) Estimation of distribution algorithm for mixed continuous–discrete optimization problems. In: Proceedings of the 2nd Euro-international symposium on computational intelligence. IOS Press, Kosice, pp 227–232Google Scholar
  83. Ochoa A, Höns R, Soto MR, Mühlenbein H (2003) A maximum entropy approach to sampling in EDA—the single connected case. In: Progress in pattern recognition, speech and image analysis, lectures notes in computer science, vol 2905. Springer, Berlin, pp 683–690Google Scholar
  84. Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San MateoGoogle Scholar
  85. Pelikan M, Goldberg DE, Lobo F (2002) A survey of optimization by building and using probabilistic models. Comput Optim Appl 21(1):5–20CrossRefMathSciNetzbMATHGoogle Scholar
  86. Peña J, Lozano JA, Larrañaga P (2005) Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks. Evol Comput 13(1):43–66CrossRefGoogle Scholar
  87. Ponce de León E, Díaz E (2012) Adaptive evolutionary algorithm based on a cliqued gibbs sampling over graphical Markov model structure. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 109–123CrossRefGoogle Scholar
  88. Regnier-Coudert O (2013) Bayesian network structure learning using characteristic properties of permutation representations with applications to prostate cancer treatment. PhD Thesis, Robert Gordon UniversityGoogle Scholar
  89. Rivera JP, Santana R (2000) Design of an algorithm based on the estimation of distributions to generate new rules in the XCS classifier system. Technical Report ICIMAF 2000-100, CEMAFIT 2000-78. Institute of Cybernetics, Mathematics and Physics, HavanaGoogle Scholar
  90. Santana R (2003) Factorized Distribution Algorithms: selection without selected population. Technical Report ICIMAF 2003-240. Institute of Cybernetics, Mathematics and Physics, HavanaGoogle Scholar
  91. Santana R (2006) Advances in probabilistic graphical models for optimization and learning. Applications in protein modelling. PhD Thesis, University of the Basque CountryGoogle Scholar
  92. Santana R, Larrañaga P, Lozano JA (2005) Interactions and dependencies in estimation of distribution algorithms. In: Proceedings of the 2005 congress on evolutionary computation CEC-2005. IEEE Press, Edinburgh, pp 1418–1425. doi: 10.1109/CEC.2005.1554856
  93. Santana R, Mendiburu A, Lozano JA (2008) An empirical analysis of loopy belief propagation in three topologies: grids, small-world networks and random graphs. In: Jaeger M, Nielsen TD (eds) Proceedings of the fourth European workshop on probabilistic graphical models (PGM-2008), pp 249–256Google Scholar
  94. Santana R, Larrañaga P, Lozano JA (2010) Learning factorizations in estimation of distribution algorithms using affinity propagation. Evol Comput 18(4):515–546. http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00002. Accessed 18 Dec 2014
  95. Santana R, Mendiburu A, Lozano JA (2012a) Evolving NK-complexity for evolutionary solvers. In: Companion proceedings of the 2012 genetic and evolutionary computation conference GECCO-2012. ACM Press, Philadelphia, pp 1473–1474. http://dl.acm.org/citation.cfm?id=2330997. Accessed 18 Dec 2014
  96. Santana R, Mendiburu A, Lozano JA (2012b) New methods for generating populations in Markov network based EDAs: decimation strategies and model-based template recombination. Technical Report EHU-KZAA-TR:2012-05. Department of Computer Science and Artificial Intelligence, University of the Basque Country. http://hdl.handle.net/10810/9180. Accessed 18 Dec 2014
  97. Santana R, Mendiburu A, Lozano JA (2013) Message passing methods for estimation of distribution algorithms based on Markov networks. In: Proceedings of the 4th conference on swarm, evolutionary, and memetic computing (SEMCCO-2013), lectures notes in computer science. Springer, Chennai, pp 419–430 (in press)Google Scholar
  98. Sastry K, Abbass HA, Goldberg DE, Johnson D (2005) Sub-structural niching in estimation of distribution algorithms. In: Proceedings of the 2005 conference on genetic and evolutionary computation. ACM, Washington, DC, pp 671–678Google Scholar
  99. Sastry K, Lima CF, Goldberg DE (2006) Evaluation relaxation using substructural information and linear estimation. In: Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM, New York, pp 419–426Google Scholar
  100. Sastry K, Goldberg DE, Llorá X (2007) Towards billion-bit optimization via a parallel estimation of distribution algorithm. In: Thierens D et al (eds) Proceedings of the genetic and evolutionary computation conference GECCO-2007, vol I. ACM Press, London, pp 577–584Google Scholar
  101. Sato H, Hasegawa Y, Bollegala D, Iba H (2012) Probabilistic model building GP with belief propagation. In: 2012 IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8Google Scholar
  102. Shakya S, Santana R, Lozano JA (2012) A Markovianity based optimisation algorithm. Genet Program Evol Mach 13(2):159–195. doi: 10.1007/s10710-011-9149-y CrossRefGoogle Scholar
  103. Soto MR (2003) A single connected factorized distribution algorithm and its cost of evaluation. PhD Thesis, University of Havana, Havana (in Spanish)Google Scholar
  104. Suwannik W, Chongstitvatana P (2008) Solving one-billion-bit noisy OneMax problem using estimation distribution algorithm with arithmetic coding. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 1203–1206Google Scholar
  105. Tanaka K, Shouno H, Okada M, Titterington D (2004) Accuracy of the Bethe approximation for hyperparameter estimation in probabilistic image processing. J Phys A 37(36):8675CrossRefMathSciNetzbMATHGoogle Scholar
  106. Valdez-Peña IS, Hernández-Aguirre A, Botello-Rionda S (2009) Approximating the search distribution to the selection distribution in EDAs. In: Proceedings of the genetic and evolutionary computation conference GECCO-2009. ACM, New York, pp 461–468Google Scholar
  107. Van Hoyweghen C, Goldberg D, Naudts B (2002a) From twomax to the Ising model: easy and hard symmetrical problems. In: Proceedings of the genetic and evolutionary computation conference GECCO-2002. Morgan Kaufmann Publishers, Inc., San Francisco, pp 626–633Google Scholar
  108. Van Hoyweghen C, Naudts B, Goldberg D (2002b) Spin-flip symmetry and synchronization. Evol Comput 10(4):317–344CrossRefGoogle Scholar
  109. Wainwright MJ, Jordan MI (2003) Graphical models, exponential families, and variational inference. Technical Report 649. Department of Statistics, University of California, BerkeleyGoogle Scholar
  110. Wainwright M, Jaakkola T, Willsky A (2002) Exact MAP estimates by (hyper) tree agreement. Adv Neural Inf Process Syst 15:809–816Google Scholar
  111. Wainwright M, Jaakkola T, Willsky A (2004) Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Stat Comput 14:143–166CrossRefMathSciNetGoogle Scholar
  112. Wang Z, Zoghi M, Hutter F, Matheson D, De Freitas N (2013) Bayesian optimization in high dimensions via random embeddings. In: Proceedings of the Twenty-Third international joint conference on Artificial Intelligence. AAAI Press, Chicago, pp 1778–1784Google Scholar
  113. Weiss Y (2000) Correctness of local probability propagation in graphical models with loops. Neural Comput 12:1–41CrossRefzbMATHGoogle Scholar
  114. Weiss Y, Freeman WT (2001) On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans Inf Theory 47(2):723–735CrossRefMathSciNetGoogle Scholar
  115. Welling M (2004) On the choice of regions for generalized belief propagation. In: Proceedings of the 20th conference on uncertainty in artificial intelligence (UAI-2004). Morgan Kaufmann Publishers, Banff, pp 585–592Google Scholar
  116. Wiegerinck W, Heskes T (2003) Fractional belief propagation. In: Advances in neural information processing systems. MIT, Vancouver, pp 455–462Google Scholar
  117. Xing EP, Jordan MI (2003) Graph partition strategies for generalized mean field inference. Technical Report CSD-03-1274. Division of Computer Science, University of California, BerkeleyGoogle Scholar
  118. Yanover C, Weiss Y (2003) Approximate inference and protein-folding. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems 15. MIT Press, Cambridge, pp 1457–1464Google Scholar
  119. Yedidia JS, Freeman WT, Weiss Y (2005) Constructing free energy approximations and generalized belief propagation algorithms. IEEE Trans Inf Theory 51(7):2282–2312CrossRefMathSciNetzbMATHGoogle Scholar
  120. Yuille A (2001) A double-loop algorithm to minimize the Bethe and Kikuchi free energies. Neural Comput 14(6):1691–1722Google Scholar
  121. Zilberstein S (1996) Using anytime algorithms in intelligent systems. AI Mag 17(3):73Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2014

Authors and Affiliations

  • Roberto Santana
    • 1
  • Alexander Mendiburu
    • 1
  • Jose A. Lozano
    • 1
  1. 1.Intelligent Systems Group, Department of Computer Science and Artificial IntelligenceUniversity of the Basque Country (UPV/EHU)San SebastiánSpain

Personalised recommendations