Hierarchical Bayesian Optimization Algorithm

  • Martin Pelikan
  • David E. Goldberg
Part of the Studies in Computational Intelligence book series (SCI, volume 33)

Keywords

Genetic Algorithm Bayesian Network Evolutionary Computation Candidate Solution Simple Genetic Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Arst, R. E. (2002). Which are better, probabilistic model-building genetic algorithms (PMBGAs) or simple genetic algorithms (sGAs)? A com-parison for an optimal groundwater remediation design problem. PhD thesis, University of Illinois at Urbana-Champaign, Department of Civil Engineering, Urbana, ILGoogle Scholar
  2. [2]
    Asoh, H., and Mühlenbein, H. (1994). On the mean convergence time of evolutionary algorithms without selection and mutation. Parallel Problem Solving from Nature, pp. 88-97Google Scholar
  3. [3]
    Baluja, S. (1994). Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Tech. Rep. No. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA Google Scholar
  4. [4]
    Baluja, S. (2002). Using a priori knowledge to create probabilistic models for optimization. International Journal of Approximate Reasoning, 31(3):193-220MATHCrossRefMathSciNetGoogle Scholar
  5. [5]
    Bosman, P. A. N., and Thierens, D. (1999). Linkage information process-ing in distribution estimation algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-99), I:60-67Google Scholar
  6. [6]
    Butz, M. V., Pelikan, M., Llorà, X., and Goldberg, D. E. (2005). Extracted global structure makes local building block processing effective in XCS. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2005), 1:655-662CrossRefGoogle Scholar
  7. [7]
    Cantú-Paz, E. (2000). Efficient and Accurate Parallel Genetic Algorithms. Kluwer, Boston, MAMATHGoogle Scholar
  8. [15]
    Chickering, D. M., Heckerman, D., and Meek, C. (1997). A Bayesian approach to learning Bayesian networks with local structure. Technical Report MSR-TR-97-07, Microsoft Research, Redmond, WAGoogle Scholar
  9. [9]
    Cooper, G. F., and Herskovits, E. H. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9:309-347MATHGoogle Scholar
  10. [10]
    Deb, K., and Goldberg, D. E. (1994). Sufficient conditions for deceptive and easy binary functions. Annals of Mathematics and Artificial Intelligence, 10:385-408MATHCrossRefMathSciNetGoogle Scholar
  11. [11]
    Etxeberria, R., and Larrañaga, P. (1999). Global optimization using Bayesian networks. In Rodriguez, A. A. O., Ortiz, M. R. S., and Hermida, R. S., (Eds.), Second Symposium on Artificial Intelligence (CIMAF-99), pp. 332-339, Habana, Cuba. Institude of Cybernetics, Mathematics, and Physics and Ministry of Science, Technology and EnvironmentGoogle Scholar
  12. [12]
    Friedman, N., and Goldszmidt, M. (1999). Learning Bayesian networks with local structure. In Jordan, M. I., (Ed.), Graphical models, pp. 421-459. MIT, Cambridge, MAGoogle Scholar
  13. [13]
    Friedman, N., and Yakhini, Z. (1996). On the sample complexity of learning Bayesian networks. Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI-96), pp. 274-282Google Scholar
  14. [14]
    Goldberg, D. E. (1994). First flights at genetic-algorithm Kitty Hawk. IlliGAL Report No. 94008, University of Illinois at Urbana-Champaign, Urbana, ILGoogle Scholar
  15. [15]
    Goldberg, D. E. (2002). The design of innovation: Lessons from and for competent genetic algorithms, volume 7 of Genetic Algorithms and Evolutionary Computation. Kluwer, Boston, MAGoogle Scholar
  16. [16]
    Goldberg, D. E., Deb, K., and Clark, J. H. (1992). Genetic algorithms, noise, and the sizing of populations. Complex Systems, 6:333-362MATHGoogle Scholar
  17. [17]
    Goldberg, D. E., Sastry, K., and Latoza, T. (2001). On the supply of building blocks. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 336-342Google Scholar
  18. [18]
    Harik, G. (1999). Linkage learning via probabilistic modeling in the ECGA. IlliGAL Report No. 99010, University of Illinois at UrbanaChampaign, Illinois Genetic Algorithms Laboratory, Urbana, ILGoogle Scholar
  19. [19]
    Harik, G., and Lobo, F. (1999). A parameter-less genetic algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-99), I:258-265Google Scholar
  20. [20]
    Harik, G. R. (1995). Finding multimodal solutions using restricted tournament selection. Proceedings of the International Conference on Genetic Algorithms (ICGA-95), pp. 24-31Google Scholar
  21. [21]
    Harik, G. R., Cantú-Paz, E., Goldberg, D. E., and Miller, B. L. (1997). The gambler’s ruin problem, genetic algorithms, and the sizing of pop-ulations. Proceedings of the International Conference on Evolutionary Computation (ICEC-97), pp. 7-12. Also IlliGAL Report No. 96004Google Scholar
  22. [22]
    Heckerman, D., Geiger, D., and Chickering, D. M. (1994). Learning Bayesian networks: The combination of knowledge and statistical data. Technical Report MSR-TR-94-09, Microsoft Research, Redmond, WAGoogle Scholar
  23. [23]
    Henrion, M. (1988). Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Lemmer, J. F., and Kanal, L. N., (Eds.), Uncertainty in Artificial Intelligence, pp. 149-163. Elsevier, Amsterdam London New YorkGoogle Scholar
  24. [24]
    Holland, J. H.(1975). Adaptation in natural and artificial systems. University of Michigan, Ann Arbor, MIGoogle Scholar
  25. [24]
    Howard, R. A., and Matheson, J. E. (1981). Influence diagrams. In Howard, R. A., and Matheson, J. E., (Eds.), Readings on the principles and applications of decision analysis, volume II, pp. 721-762. Strategic Decisions Group, Menlo Park, CAGoogle Scholar
  26. [26]
    Khan, N. (2003). Bayesian optimization algorithms for multiobjective and hierarchically difficult problems. Master’s thesis, University of Illinois at Urbana-Champaign, Urbana, ILGoogle Scholar
  27. [27]
    Larrañaga, P., and Lozano, J. A., (Eds.) (2002). Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer, Boston, MAMATHGoogle Scholar
  28. [28]
    Laumanns, M., and Ocenasek, J. (2002). Bayesian optimization algorithms for multi-objective optimization. Parallel Problem Solving from Nature, pp. 298-307Google Scholar
  29. [29]
    Li, J., and Aickelin, U. (2003). A Bayesian optimization algorithm for the nurse scheduling problem. Proceedings of the IEEE Congress on Evo-lutionary Computation 2003 (CEC-2003), pp. 2149-2156Google Scholar
  30. [30]
    Lobo, F. G., Goldberg, D. E., and Pelikan, M. (2000). Time complexity of genetic algorithms on exponentially scaled problems. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 151-158. Also IlliGAL Report No. 2000016Google Scholar
  31. [31]
    Looks, M., Goertzel, B., and Pennachin, C.(2005). Learning computer programs with the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2005), 1:747-748CrossRefGoogle Scholar
  32. [32]
    Looks, M., Groetzel, B., and Pennachin, C. (2003). Learning computer programs with the Bayesian optimization algorithm. Technical report, Object Sciences Corporation and Novamente LLC Google Scholar
  33. [33]
    Mühlenbein, H. (1992a). How genetic algorithms really work: I.Mutation and Hillclimbing. Parallel Problem Solving from Nature, pp. 15-25Google Scholar
  34. [30]
    Mühlenbein, H. (1992b). How genetic algorithms really work: I.Mutation and Hillclimbing. In Männer, R., and Manderick, B., (Eds.), Parallel Problem Solving from Nature, pp. 15-25, Elsevier, AmsterdamGoogle Scholar
  35. [35]
    Mühlenbein, H., and Mahnig, T. (2002). Evolutionary optimization and the estimation of search distributions with aplication to graph bipartitioning. International Journal of Approximate Reasoning, 31(3):157-192MATHCrossRefMathSciNetGoogle Scholar
  36. [36]
    Mühlenbein, H., and Paass, G. (1996). From recombination of genes to the estimation of distributions I. Binary parameters. In Eiben, A., Bäck, T., Shoenauer, M., and Schwefel, H., (Eds.), Parallel Problem Solving from Nature, pp. 178-187, Springer, Berlin Heidelberg New YorkGoogle Scholar
  37. [37]
    Mühlenbein, H., and Schlierkamp-Voosen, D. (1993). Predictive models for the breeder genetic algorithm: I. Continuous parameter optimization. Evolutionary Computation, 1(1):25-49CrossRefGoogle Scholar
  38. [38]
    Ocenasek, J., and Schwarz, J. (2000). The parallel Bayesian optimization algorithm. In Proceedings of the European Symposium on Computational Inteligence, pp. 61-67. Physica, Wurzburg (Wien)Google Scholar
  39. [39]
    Ocenasek, J., and Schwarz, J. (2002). Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In 2nd Euro-International Symposium on Computational Intelligence, pp. 227-232Google Scholar
  40. [40]
    Ocenasek, J., Schwarz, J., and Pelikan, M. (2003). Design of multithreaded estimation of distribution algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003), pp. 1247-1258Google Scholar
  41. [32]
    Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann, San Mateo, CAGoogle Scholar
  42. [42]
    Pelikan, M. (2002). Bayesian optimization algorithm: From single level to hierarchy. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL. Also IlliGAL Report No. 2002023Google Scholar
  43. [43]
    Pelikan, M.(2005). Hierarchical Bayesian optimization algorithm: Toward a new generation of evolutionary algorithms. Springer, Berlin Heidelberg New YorkMATHGoogle Scholar
  44. [44]
    Pelikan, M., and Goldberg, D. E. (2000). Hierarchical problem solving by the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), pp. 275-282Google Scholar
  45. [45]
    Pelikan, M., and Goldberg, D. E. (2001). Escaping hierarchical traps with competent genetic algorithms. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 511-518Google Scholar
  46. [46]
    Pelikan, M., and Goldberg, D. E. (2003a). Hierarchical BOA solves Ising spin glasses and MAXSAT. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003), II:1275-1286Google Scholar
  47. [47]
    Pelikan, M., and Goldberg, D. E. (2003b). A hierarchy machine: Learning to optimize from nature and humans. Complexity, 8(5):36-45CrossRefGoogle Scholar
  48. [48]
    Pelikan, M., Goldberg, D. E., and Cantú-Paz, E. (1999). BOA: The Bayesian optimization algorithm. Proceedings of the Genetic and Evo-lutionary Computation Conference (GECCO-99), I:525-532Google Scholar
  49. [49]
    Pelikan, M., Goldberg, D. E., and Cantú-Paz, E. (2000). Linkage problem, distribution estimation, and Bayesian networks. Evolutionary Computation, 8(3):311-341. Also IlliGAL Report No. 98013CrossRefGoogle Scholar
  50. [50]
    Pelikan, M., Goldberg, D. E., and Lobo, F. (2002a). A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1):5-20MATHCrossRefMathSciNetGoogle Scholar
  51. [51]
    Pelikan, M., Goldberg, D. E., and Sastry, K. (2001). Bayesian optimization algorithm, decision graphs, and Occam’s razor. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 519-526. Also IlliGAL Report No. 2000020Google Scholar
  52. [52]
    Pelikan, M., and Lin, T.-K. (2004). Parameter-less hierarchical BOA. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:24-35Google Scholar
  53. [53]
    Pelikan, M., Ocenasek, J., Trebst, S., Troyer, M., and Alet, F. (2004). Computational complexity and simulation of rare events of Ising spin glasses. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:36-47Google Scholar
  54. [54]
    Pelikan, M., and Sastry, K. (2004). Fitness inheritance in the Bayesian optimization algorithm. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2:48-59Google Scholar
  55. [55]
    Pelikan, M., Sastry, K., and Goldberg, D. E. (2002b). Scalability of the Bayesian optimization algorithm. International Journal of Approximate Reasoning, 31(3):221-258MATHCrossRefMathSciNetGoogle Scholar
  56. [56]
    Pelikan, M., Sastry, K., and Goldberg, D. E. (2005a). Multiobjective hBOA, clustering, and scalability. Proceedings of the Genetic and Evolu-tionary Computation Conference (GECCO-2005), pp. 663-670Google Scholar
  57. [57]
    Pelikan, M., Sastry, K., and Goldberg, D. E. (2005b). Sporadic model building for efficiency enhancement of hBOA. Technical report, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms LaboratoryGoogle Scholar
  58. [58]
    Pelikan, M., Goldberg, D. E., and Tsutsui, S. (2003). Getting the best of both worlds: Discrete and continuous genetic and evolutionary algorithms in concert. Information Sciences, 156 (3-4), 147-171CrossRefMathSciNetGoogle Scholar
  59. [59]
    Rissanen, J. J. (1978). Modelling by shortest data description. Automatica, 14:465-471MATHCrossRefGoogle Scholar
  60. [60]
    Rissanen, J. J. (1989). Stochastic complexity in statistical inquiry. World Scientific, SingaporeMATHGoogle Scholar
  61. [61]
    Rissanen, J. J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42(1):40-47MATHCrossRefMathSciNetGoogle Scholar
  62. [62]
    Rothlauf, F. (2001). Towards a theory of representations for genetic and evolutionary algorithms: Development of basic concepts and their application to binary and tree representations. PhD thesis, University of Bayreuth, Beyreuth, GermanyGoogle Scholar
  63. [63]
    Rothlauf, F., Goldberg, D. E., and Heinzl, A. (2000). Bad codings and the utility of well-designed genetic algorithms. IlliGAL Report No. 200007, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, ILGoogle Scholar
  64. [64]
    Santarelli, S., Goldberg, D. E., and Yu, T.-L. (2004). Optimization of a constrained feed network for an antenna array using simple and compe-tent genetic algorithm techniques. Proceedings of the Workshop Military and Security Application of Evolutionary Computation (MSAEC-2004)Google Scholar
  65. [65]
    Sastry, K. (2001). Efficient atomic cluster optimization using a hybrid extended compact genetic algorithm with seeded population. IlliGAL Re-port No. 2001018, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, ILGoogle Scholar
  66. [66]
    Sastry, K., Pelikan, M., and Goldberg, D. E. (2004). Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. Proceedings of the IEEE Conference on Evolutionary Computation, pp. 720-727Google Scholar
  67. [67]
    Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6:461-464MATHCrossRefMathSciNetGoogle Scholar
  68. [68]
    Schwarz, J., and Ocenasek, J. (1999). Experimental study: Hypergraph partitioning based on the simple and advanced algorithms BMDA and BOA. In Proceedings of the International Conference on Soft Computing, pp. 124-130, Brno, Czech Republic. PC-DIRGoogle Scholar
  69. [69]
    Schwarz, J., and Ocenasek, J. (2000). A problem-knowledge based evolutionary algorithm KBOA for hypergraph partitioning. In Proceedings of the Fourth Joint Conference on Knowledge-Based Software Engineering, pp. 51-58, IO, Brno, Czech RepublicGoogle Scholar
  70. [70]
    Simon, H. A. (1968). The Sciences of the Artificial. MIT, Cambridge, MAGoogle Scholar
  71. [71]
    Thierens, D. (1995). Analysis and design of genetic algorithms. PhD thesis, Katholieke Universiteit Leuven, Leuven, BelgiumGoogle Scholar
  72. [72]
    Thierens, D., and Goldberg, D. (1994). Convergence models of genetic algorithm selection schemes. Parallel Problem Solving from Nature, pp. 116-121Google Scholar
  73. [73]
    Thierens, D., Goldberg, D. E., and Pereira, A. G. (1998). Domino convergence, drift, and the temporal-salience structure of problems. Proceedings of the International Conference on Evolutionary Computation (ICEC-98), pp. 535-540Google Scholar
  74. [74]
    Watson, R. A., Hornby, G. S., and Pollack, J. B. (1998). Modeling building-block interdependency. Parallel Problem Solving from Nature, pp. 97-106Google Scholar
  75. [75]
    Wilson, S. W. (1995). Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149-175CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Martin Pelikan
    • 1
  • David E. Goldberg
    • 2
  1. 1.University of Missouri at St. Louis, One University Blvd. St. LouisUSA
  2. 2.University of Illinois at Urbana-Champaign 104 S.USA

Personalised recommendations