Using Entropy for Parameter Analysis of Evolutionary Algorithms

  • Selmar K. SmitEmail author
  • Agoston E. EibenEmail author


Evolutionary algorithms (EA) form a rich class of stochastic search methods that share the basic principles of incrementally improving the quality of a set of candidate solutions by means of variation and selection (Eiben and Smith 2003, De Jong 2006). Such variation and selection operators often require parameters to be specified. Finding a good set of parameter values is a nontrivial problem in itself. Furthermore, some EA parameters are more relevant than others in the sense that choosing different values for them affects EA performance more than for the other parameters. In this chapter we explain the notion of entropy and discuss how entropy can disclose important information about EA parameters, in particular, about their relevance. We describe an algorithm that is able to estimate the entropy of EA parameters and we present a case study, based on extensive experimentation, to demonstrate the usefulness of this approach and some interesting insights that are gained.


Evolutionary Algorithm Evolutionary Computation Candidate Solution Mutation Operator Shannon Entropy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



All experimental data and results are due to Volker Nannen.


  1. Bäck T (1996) Evolutionary Algorithms in Theory and Practice. Oxford University Press, Oxford, UKzbMATHGoogle Scholar
  2. Bäck T, Fogel D, Michalewicz Z (eds) (2000a) Evolutionary Computation 1: Basic Algorithms and Operators. Institute of Physics Publishing, BristolzbMATHGoogle Scholar
  3. Bäck T, Fogel D, Michalewicz Z (eds) (2000b) Evolutionary Computation 2: Advanced Algorithms and Operators. Institute of Physics Publishing, BristolzbMATHGoogle Scholar
  4. Banzhaf W, Nordin P, Keller R, Francone F (1998) Genetic Programming: An Introduction. Morgan Kaufmann, San Francisco CAzbMATHGoogle Scholar
  5. Bartz-Beielstein T (2003) Experimental Analysis of Evolution Strategies: Overview and Comprehensive Introduction. Tech. Rep. Reihe CI 157/03, SFB 531, Universität Dortmund, Dortmund, Germany, URL http://sfbci.informatik. Scholar
  6. Bartz-Beielstein T (2006) Experimental Research in Evolutionary Computation— The New Experimentalism. Natural Computing Series, SpringerzbMATHGoogle Scholar
  7. Branke J, Chick SE, Schmidt C (2005) New developments in ranking and selection: an empirical comparison of the three main approaches. In: Proceedings of the 37thWinter Simulation Conference (WSC 2005),Winter Simulation Conference, pp 708–717Google Scholar
  8. De Jong K (2006) Evolutionary Computation: A Unified Approach. The MIT PresszbMATHGoogle Scholar
  9. Eiben A, Smith J (2003) Introduction to Evolutionary Computation. Natural Computing Series, SpringerGoogle Scholar
  10. Eiben A, Hinterding R, Michalewicz Z (1999) Parameter Control in Evolutionary Algorithms. IEEE Transactions on Evolutionary Computation 3(2):4–141CrossRefGoogle Scholar
  11. Fogel D (1995) Evolutionary Computation. IEEE PressGoogle Scholar
  12. Fogel D (ed) (1998) Evolutionary Computation: the Fossil Record. IEEE Press, Piscataway,NJzbMATHGoogle Scholar
  13. Fogel L, Owens A, Walsh M (1966) Artificial Intelligence through Simulated Evolution. Wiley, Chichester, UKzbMATHGoogle Scholar
  14. Gallagher M, Yuan B (2006) A general-purpose tunable landscape editor. IEEE Transactions on Evolutionary Computation 10(5): 590–603CrossRefGoogle Scholar
  15. Goldberg D (1989) Genetic Algorithms in Search, Optimization and Machine Learning. Addison-WesleyzbMATHGoogle Scholar
  16. Holland J (1992) Adaption in Natural and Artificial Systems. MIT Press, Cambridge, MA, 1st ed: 1975, The University of Michigan Press, Ann ArborGoogle Scholar
  17. Koza J (1992) Genetic Programming. MIT Press, Cambridge, MAzbMATHGoogle Scholar
  18. Lipson H (ed) (2007) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2007), ACMGoogle Scholar
  19. Mitchell M (1996) An Introduction to Genetic Algorithms. MIT Press, Cambridge,MAGoogle Scholar
  20. Mühlenbein H, Höns R (2005) The estimation of distributions and the minimum relative entropy principle. Evolutionary Computation 13(1):1–27CrossRefGoogle Scholar
  21. Nannen V (April, 2009) Evolutionary agent-based policy analysis in dynamic environments. PhD thesis, Vrije Universiteit AmsterdamGoogle Scholar
  22. Nannen V, Eiben A (2006) A method for parameter calibration and relevance estimation in evolutionary algorithms. In: Keijzer M (ed) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2006), ACM, pp 183–190Google Scholar
  23. Nannen V, Eiben AE (2007a) Efficient Relevance Estimation and Value Calibration of Evolutionary Algorithm Parameters. In: IEEE Congress on Evolutionary Computation, IEEE, pp 103–110Google Scholar
  24. Nannen V, Eiben AE (2007b) Relevance Estimation and Value Calibration of Evolutionary Algorithm Parameters. In: Veloso MM(ed) IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, pp 1034–1039Google Scholar
  25. Nannen V, Smit S, Eiben A (2008) Costs and benefits of tuning parameters of evolutionary algorithms. In: Rudolph G, Jansen T, Lucas SM, Poloni C, Beume N (eds) PPSN, Springer, Lecture Notes in Computer Science, vol 5199, pp 528–538Google Scholar
  26. Rechenberg I (1973) Evolutionstrategie: Optimierung Technisher Systeme nach Prinzipien des Biologischen Evolution. Fromman-Hozlboog Verlag, StuttgartGoogle Scholar
  27. Ridge E, Kudenko D (2007a) Analyzing heuristic performance with response surface models: prediction, optimization and robustness. In: Lipson (2007), pp 150– 157Google Scholar
  28. Ridge E, Kudenko D (2007b) Screening the parameters affecting heuristic performance. In: Lipson (2007), pp 180–180Google Scholar
  29. Samples M, Byom M, Daida J (2007) Parameter sweeps for exploring parameter spaces of genetic and evolutionary algorithms. In: Lobo F, Lima C, Michalewicz Z (eds) Parameter Setting in Evolutionary Algorithms, Springer, pp 161–184CrossRefGoogle Scholar
  30. Schwefel HP (1995) Evolution and Optimum Seeking. Wiley, New York NYGoogle Scholar
  31. Shannon C (1948) A mathematical theory of communication. Bell System Technical Journal 27:379–423, 623–656zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Vrije Universiteit AmsterdamAmsterdamThe Netherlands

Personalised recommendations