Parameter Control of Genetic Algorithms by Learning and Simulation of Bayesian Networks — A Case Study for the Optimal Ordering of Tables

Abstract

Parameter setting for evolutionary algorithms is still an important issue in evolutionary computation. There are two main approaches to parameter setting: parameter tuning and parameter control. In this paper, we introduce self-adaptive parameter control of a genetic algorithm based on Bayesian network learning and simulation. The nodes of this Bayesian network are genetic algorithm parameters to be controlled. Its structure captures probabilistic conditional (in)dependence relationships between the parameters. They are learned from the best individuals, i.e., the best configurations of the genetic algorithm. Individuals are evaluated by running the genetic algorithm for the respective parameter configuration. Since all these runs are time-consuming tasks, each genetic algorithm uses a small-sized population and is stopped before convergence. In this way promising individuals should not be lost. Experiments with an optimal search problem for simultaneous row and column orderings yield the same optima as state-of-the-art methods but with a sharp reduction in computational time. Moreover, our approach can cope with as yet unsolved high-dimensional problems.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    De Jong K A. An analysis of behavior of a class of genetic adaptive systems [Ph.D. Thesis]. University of Michigan, USA, 1975.

  2. [2]

    Grefenstette JJ (1986) Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics 16(1):122–128

    Article  Google Scholar 

  3. [3]

    Wolpert D, Macready W (1997) No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1):67–82

    Article  Google Scholar 

  4. [4]

    Eiben A E, Michalewicz Z, Schoenauer M, Smith J E. Parameter control in evolutionary algorithms. In Studies in Computational Intelligence 54, Lobo F G, Lina C F, Michalewicz Z et al. (eds.), Springer, 2007, pp.19–46.

  5. [5]

    de Lima E B, Pappa G L, de Almeida J M et al. Tuning genetic programming parameters with factorial designs. In Proc. 2010 IEEE Congress on Evolutionary Computation, July 2010.

  6. [6]

    Rojas I, González J, Pomares H et al. Statistical analysis of the main parameters involved in the design of a genetic algorithm. IEEE Transactions on Systems, Man, and CyberneticsPart C, Applications and Reviews, 2002, 32(1): 31–37.

    Google Scholar 

  7. [7]

    Czarn A, MacNish C, Vijayan K et al (2004) Statistical exploratory analysis of genetic algorithms: The importance of interaction. IEEE Trans Evolutionary Computation 8(4):405–421

    Article  Google Scholar 

  8. [8]

    Smit S K, Eiben A E. Parameter tuning of evolutionary algorithms: Generalist vs. specialist. In Lecture Notes in Computer Science 6024, Di Chio C, Cagnoni S, Cotta C et al. (eds.), Springer, 2010, pp.542–551.

  9. [9]

    Rechenberg I. Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Stuttgart, Germany: Frommann-Holzboog, 1973. (In German)

  10. [10]

    Santana R, Larrañaga P, Lozano J A. Adaptive estimation of distribution algorithms. In Studies in Computational Intelligence 136, Cotta C, Sevaux M, Sörensen K et al. (eds.), Springer, 2008, pp.177–197.

  11. [11]

    Kramer O. Self-Adaptive Heuristics for Evolutionary Computation. Berlin, Germany: Springer-Verlag, 2008.

  12. [12]

    Angeline P J. Adaptive and self-adaptive evolutionary computation. In Computational Intelligence: A Dynamic System Perspective, Palaniswami Y, Attikiouzel R, Marks R et al. (eds.), IEEE, 1995, pp.152–161.

  13. [13]

    Hinterding R, Michalewicz Z, Eiben A E. Adaptation in evolutionary computation: A survey. In Proc. the 4th IEEE Conf. Evolutionary Computation, Apr. 1997, pp.65–69.

  14. [14]

    Smith J. Self adaptation in evolutionary algorithms [Ph.D. Thesis]. University of the West of England, UK, 1997.

  15. [15]

    Friesleben B, Hartfelder M. Optimisation of genetic algorithms by genetic algorithms. In Proc. Artificial Neural Networks and Genetic Algorithms, Apr. 1993, pp.392–399.

  16. [16]

    Bäck T. Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. New York, USA: Oxford University Press, 1996.

  17. [17]

    Rechenberg I (1994) Evolutionsstrategie'94. Frommann-Holzboog, Stuttgart, Germany

    Google Scholar 

  18. [18]

    Larrañaga P (2002) Lozano J A. Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, New York, USA

    Book  Google Scholar 

  19. [19]

    Peña J M, Robles V, Larrañaga P et al. GA-EDA: Hybrid evolutionary algorithm using genetic and estimation of distribution algorithms. In Proc. the 17th International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems, May 2004, pp.361–371.

  20. [20]

    Santana R, Larrañaga P, Lozano JA (2008) Combining variable neighborhood search and estimation of distribution algorithms in the protein side chain placement problem. Journal of Heuristics 14(5):519–547

    MATH  Article  Google Scholar 

  21. [21]

    Sun J, Zhang Q, Tsang E (2005) DE/EDA: A new evolutionary algorithm for global optimisation. Information Sciences 169(3/4):249–262

    MathSciNet  Article  Google Scholar 

  22. [22]

    Dong W, Yao X. NichingEDA: Utilizing the diversity inside a population of EDAs for continuous optimization. In Proc. the 2008 IEEE Congress on Evolutionary Computation, June 2008, pp.1260–1267.

  23. [23]

    Nannen V, Smit S K, Eiben A E. Costs and benefits of tuning parameters of evolutionary algorithms. In Proc. the 10th Int. Conf. Parallel Problem Solving from Nature, Sept. 2008, Vol.5199, pp.528–538.

  24. [24]

    Bielza C, Fernández del Pozo JA, Larrañaga P et al (2010) Multidimensional statistical analysis of the parameterization of a genetic algorithm for the optimal ordering of tables. Expert Systems with Applications 37(1):804–815

    Article  Google Scholar 

  25. [25]

    Etxeberria R, Larrañaga P. Global optimization using Bayesian networks. In Proc. the 2nd Symposium on Artificial Intelligence, March 1999, pp.332–339.

  26. [26]

    Pelikan M, Goldberg D E, Cantú-Paz E. BOA: The Bayesian optimization algorithm. In Proc. the Genetic and Evolutionary Computation Conference, July 1999, pp.525–532.

  27. [27]

    Yuan B, Gallagher M. Combining meta-EAs and racing for difficult EA parameter tuning tasks. In Studies in Computational Intelligence 54, Lobo F J, Lima C F, Michalewicz Z (eds.), Springer, 2007, pp.121–142.

  28. [28]

    Lobo F G, Lima C F. Adaptive population sizing schemes in genetic algorithms. In Studies in Computational Intelligence 54, Lobo F, Lima C F, Michalewicz Z (eds.), Springer, 2007, pp.185–204.

  29. [29]

    Friedman N, Yakhini Z. On the sample complexity of learning Bayesian networks. In Proc. the 12th Conference on Uncertainty in Artificial Intelligence, Aug. 1996, pp.274–282.

  30. [30]

    Lam W, Bacchus F (1994) Learning Bayesian belief networks: An approach based on the MDL principle. Computational Intelligence 10(3):269–293

    Article  Google Scholar 

  31. [31]

    Henrion M. Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In Proc. the 2nd Annual Conf. Uncertainty in Artificial Intelligence, Aug. 1986, pp.149–164.

  32. [32]

    Bertin J (1981) Graphics and Graphic Information Processing. Walter de Gruyter & Co., UK

    Book  Google Scholar 

  33. [33]

    Johnson D S, Papadimitriou C H. Computational complexity. In The Traveling Salesman Problem, Lawler E L, Lenstra J K, Rinnooy Kan A et al. (eds.), John Wiley & Sons, 1985, pp.37–85.

  34. [34]

    Niermann S (2005) Optimizing the ordering of tables with evolutionary computation. The American Statistician 59(1):41–46

    MathSciNet  Article  Google Scholar 

  35. [35]

    Larrañaga P, Kuijpers CMH, Murga RH et al (1999) Genetic algorithms for the travelling salesman problem: A review of representations and operators. Artificial Intelligence Review 13(2):129–170

    Article  Google Scholar 

  36. [36]

    Croes GA (1958) A method for solving traveling-salesman problems. Operations Research 6(6):791–812

    MathSciNet  Article  Google Scholar 

  37. [37]

    Pearl J (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco, USA

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Concha Bielza.

Additional information

This work has been partially supported by the Spanish Ministry of Economy and Competitiveness under Grant No. TIN2010-20900-C04-04 and Cajal Blue Brain.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(DOC 38.0 KB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bielza, C., Fernández del Pozo, J.A. & Larrañaga, P. Parameter Control of Genetic Algorithms by Learning and Simulation of Bayesian Networks — A Case Study for the Optimal Ordering of Tables. J. Comput. Sci. Technol. 28, 720–731 (2013). https://doi.org/10.1007/s11390-013-1370-0

Download citation

Keywords

  • genetic algorithm
  • estimation of distribution algorithm
  • parameter control
  • parameter setting
  • Bayesian network