Evolutionary Optimization Guided by Entropy-Based Discretization

  • Guleng Sheri
  • David W. Corne
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5484)


The Learnable Evolution Model (LEM) involves alternating periods of optimization and learning, performa extremely well on a range of problems, a specialises in achieveing good results in relatively few function evaluations. LEM implementations tend to use sophisticated learning strategies. Here we continue an exploration of alternative and simpler learning strategies, and try Entropy-based Discretization (ED), whereby, for each parameter in the search space, we infer from recent evaluated samples what seems to be a ‘good’ interval. We find that LEM(ED) provides significant advantages in both solution speed and quality over the unadorned evolutionary algorithm, and is usually superior to CMA-ES when the number of evaluations is limited. It is interesting to see such improvement gained from an easily-implemented approach. LEM(ED) can be tentatively recommended for trial on problems where good results are needed in relatively few fitness evaluations, while it is open to several routes of extension and further sophistication. Finally, results reported here are not based on a modern function optimization suite, but ongoing work confirms that our findings remain valid for non-separable functions.


Learning Phase Distribution Algorithm Function Optimization Problem Good Interval Learning Population 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hussain, F., Liu, H., Tan, C.L., Dash, M.: Discretization: An Enabling Technique. Data Mining and Knowledge Discovery 6(4), 393–423 (2002)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)Google Scholar
  3. 3.
    Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)Google Scholar
  4. 4.
    Sheri, G., Corne, D.: The Simplest Evolution/Learning Hybrid: LEM with KNN. In: Proc. IEEE CEC 2008, pp. 3244–3251 (2008)Google Scholar
  5. 5.
    Bengoextea, E., Miquelez, M., Larranga, P., Lozano, J.A.: Experimental results in function optimization with EDAs in Continuous Domain. In: [12] (2002)Google Scholar
  6. 6.
    Edgington, E.S.: Randomization tests, 3rd edn. Marcel-Dekker, New York (1995)zbMATHGoogle Scholar
  7. 7.
    Auger, A., Hansen, N.: A Restart CMA Evolution Strategy With Increasing Population Size. In: Proc. IEEE CEC 2005, pp. 1769–1776 (2005)Google Scholar
  8. 8.
    Hansen, N.: The CMA Evolution Strategy: A Comparing Review. In: Lozano, J.A., et al. (eds.) Towards a new evolutionary computation. Advances in estimation of distribution algorithms, pp. 75–102. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Goldberg, D.E.: Genetic Algorithms in Search. Optimization and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  10. 10.
    Jourdan, L., Corne, D., Savic, D., Walters, G.: Hybridising rule induction and multiobjective evolutionary search for optimizing water distribution systems. In: Proc. of the 4th Hybrid Intelligent Systems conference, pp. 434–439. IEEE Computer Society Press, Los Alamitos (2005)Google Scholar
  11. 11.
    Kaufmann, K., Michalski, R.S.: Learning from inconsistent and noisy data. In: The AQ18 approach, 11th Int’l. Symp. on Foundations of Intelligent Systems (1999)Google Scholar
  12. 12.
    Larranaga, P., Lozano, J.A. (eds.): Stimulation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (2002)zbMATHGoogle Scholar
  13. 13.
    Michalski, R.S.: Learnable Evolution Model Evolutionary Processes Guided by Machine Learning. Machine Learning 38, 9–40 (2000)CrossRefzbMATHGoogle Scholar
  14. 14.
    Pena, J.M., Robles, V., Larranaga, P., Herves, V., Rosales, F., Perez, M.S.: GA-EDA: Hybrid evolutionary algorithm using genetic and estimation of distribution algorithms. In: Orchard, B., Yang, C., Ali, M. (eds.) IEA/AIE 2004. LNCS (LNAI), vol. 3029, pp. 361–371. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  15. 15.
    Syswerda, G.: Uniform Crossover in Genetic Algorithms. In: Proc. of 3rd International Conference on Genetic Algorithms. Morgan Kaufmann Publishers Inc., San Francisco (1989)Google Scholar
  16. 16.
    Wnek, J., Kaufmann, K., Bloedorn, E., Michalski, R.S.: Inductive Learning System AQ15c: The method and user’s guide. Reports of the Machine Learning and Inference Laboratory, MLI95-4, George Mason University,Fairfax, VA, USA (1995)Google Scholar
  17. 17.
    Wojtusiak, J., Michalski, R.S.: The LEM3 System for Non-Darwinian Evolutionary Computation and Its Application to Complex Function Optimization, Reports of the Machine Learning and Inference Laboratory, MLI 05-2, George Mason University, Fairfax, VA, USA (2005)Google Scholar
  18. 18.
    Wojtusiak, J., Michalski, R.S.: The LEM3 implementation of learnable evolution model and its testing on complex function optimization problems. In: Proc. GECCO 2006 (2006)Google Scholar
  19. 19.
    Zhang, Q., Sun, J., Tsang, E., Ford, J.: Hybrid estimation of distribution algorithm for global optimisation. Engineering Computations 21(1), 91–107 (2003)CrossRefzbMATHGoogle Scholar
  20. 20.
    Zhang, Q., Sun, J., Tsang, E., Ford, J.: Estimation of distribution algorithm with 2-opt Local Search for the Quadratic Assignment Problem. In: Lozana, Larranaga, Inza, Bengoetxea (eds.) Towards a new evolutionary computation: Advances in estimation of distribution algorithms (2006)Google Scholar
  21. 21.
    Suganthan, P.N., Hansen, N., Liang, J.J., Deb, K., Chen, Y.-P., Auger, A., Tiwari, A.: Problem definitions and evaluation criteria for the CEC 2005 Special Session on Real Parameter Optimization. Technical Report 2005005, Nanyang Technological University, Singapore (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Guleng Sheri
    • 1
  • David W. Corne
    • 1
  1. 1.Department of Computer ScienceHeriot-Watt UniversityEdiburghUK

Personalised recommendations