Methodology and Computing in Applied Probability

, Volume 8, Issue 3, pp 383–407 | Cite as

The Cross-Entropy Method for Continuous Multi-Extremal Optimization

  • Dirk P. Kroese
  • Sergey Porotsky
  • Reuven Y. RubinsteinEmail author


In recent years, the cross-entropy method has been successfully applied to a wide range of discrete optimization tasks. In this paper we consider the cross-entropy method in the context of continuous optimization. We demonstrate the effectiveness of the cross-entropy method for solving difficult continuous multi-extremal optimization problems, including those with non-linear constraints.


Cross-entropy Continuous optimization Multi-extremal objective function Dynamic smoothing Constrained optimization Nonlinear constraints Acceptance–rejection Penalty function 

AMS 2000 Subject Classification

Primary 65C05, 65K99 Secondary 94A17 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. E. H. L. Aarts, and J. H. M. Korst, Simulated Annealing and Boltzmann Machines, Wiley: New York, 1989.zbMATHGoogle Scholar
  2. D. Bates, and D.Watts, Nonlinear Regression Analysis and Its Applications, Wiley: New York, 1988.zbMATHGoogle Scholar
  3. Z. Botev, and D. P. Kroese, Global likelihood optimization via the cross-entropy method, with an application to mixture models. In R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, (eds.), Proceedings of the 2004 Winter Simulation Conference, pp. 529–535, IEEE: Washington, District of Columbia, 2004.Google Scholar
  4. J. Bracken, and G. P. McCormick, Selected Applications of Nonlinear Programming. Wiley: New York, 1968.zbMATHGoogle Scholar
  5. P. T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, “A tutorial on the cross-entropy method,” Annals of Operations Research vol. 134(1) pp. 19–67, 2005.zbMATHMathSciNetCrossRefGoogle Scholar
  6. M. Dorigo, V. Maniezzo, and A. Colorni, “The ant system: optimization by a colony of cooperating agents,” IEEE Transactions on Systems, Man, and Cybernetics—Part B vol. 26(1) pp. 29–41, 1996.CrossRefGoogle Scholar
  7. G. Dueck, and T. Scheur, “Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing,” Journal of Computational Physics vol. 90, pp. 161–175, 1990.zbMATHMathSciNetCrossRefGoogle Scholar
  8. F. Glover, and M. L. Laguna, Modern Heuristic Techniques for Combinatorial Optimization, Chapter 3: Tabu search. Blackwell Scientific: London, UK, 1993.Google Scholar
  9. D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley: Reading, Massachusetts, 1989.zbMATHGoogle Scholar
  10. W. B. Gong, Y. C. Ho, and W. Zhai, Stochastic comparison algorithm for discrete optimization with estimation. In Proceedings of the 31st IEEE Conference on Decision and Control, pp. 795–800, 1992.Google Scholar
  11. D. M. Himmelblau, Applied Nonlinear Programming. McGrawHill: New York, 1972.zbMATHGoogle Scholar
  12. W. Hock, and K. Schittkowski, Test Examples for Nonlinear Programming Codes, volume 197. Springer: Berlin Heidelberg New York, 1981.Google Scholar
  13. D. P. Kroese, R. Y. Rubinstein, and T. Taimre, Application of the cross-entropy method to clustering and vector quantization. (To appear in Journal of Global Optimization) 2006.Google Scholar
  14. L. Margolin, “On the convergence of the cross-entropy method,” Annals of Operations Research vol. 134(1) pp. 201–214, 2005.zbMATHMathSciNetCrossRefGoogle Scholar
  15. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer: Berlin Heidelberg New York, 1996.Google Scholar
  16. D. A. Paviani, A New Method for the Solution of the General Nonlinear Programming Problem. Ph.D. thesis, The University of Texas, Austin, Texas, 1969.Google Scholar
  17. R. Y. Rubinstein, “Optimization of computer simulation models with rare events,” European Journal of Operational Research vol. 99 pp. 89–112, 1997.CrossRefGoogle Scholar
  18. R. Y. Rubinstein, The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability vol. 2 pp. 127–190, 1999.zbMATHCrossRefGoogle Scholar
  19. R. Y. Rubinstein, and D. P. Kroese, The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer: Berlin Heidelberg New York, 2004.Google Scholar
  20. K. Schittkowski, Nonlinear Programming Codes, volume 183. Springer: Berlin Heidelberg New York, 1980.Google Scholar
  21. B. V. Sheela, and P. Ramaoorthy, “Swift - a new constrained optimization technique,” Computer Methods in Applied Mechanics and Engineering vol. 6(3) pp. 309–318, 1975.zbMATHMathSciNetCrossRefGoogle Scholar
  22. D. G. Stork, and E. Yom-Tov, Computer Manual to Accompany Pattern Classification. Wiley: New York, 2004.Google Scholar
  23. A. Webb, Statistical Pattern Recognition. Arnold: London, 1999.zbMATHGoogle Scholar
  24. D. White, Markov Decision Process. Wiley: New York, 1992.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  • Dirk P. Kroese
    • 1
  • Sergey Porotsky
    • 2
  • Reuven Y. Rubinstein
    • 3
    Email author
  1. 1.Department of MathematicsThe University of QueenslandBrisbaneAustralia
  2. 2.Optimata Ltd.Ramat GanIsrael
  3. 3.Faculty of Industrial Engineering and ManagementTechnionHaifaIsrael

Personalised recommendations