Abstract
In recent years, the cross-entropy method has been successfully applied to a wide range of discrete optimization tasks. In this paper we consider the cross-entropy method in the context of continuous optimization. We demonstrate the effectiveness of the cross-entropy method for solving difficult continuous multi-extremal optimization problems, including those with non-linear constraints.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E. H. L. Aarts, and J. H. M. Korst, Simulated Annealing and Boltzmann Machines, Wiley: New York, 1989.
D. Bates, and D.Watts, Nonlinear Regression Analysis and Its Applications, Wiley: New York, 1988.
Z. Botev, and D. P. Kroese, Global likelihood optimization via the cross-entropy method, with an application to mixture models. In R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters, (eds.), Proceedings of the 2004 Winter Simulation Conference, pp. 529–535, IEEE: Washington, District of Columbia, 2004.
J. Bracken, and G. P. McCormick, Selected Applications of Nonlinear Programming. Wiley: New York, 1968.
P. T. de Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, “A tutorial on the cross-entropy method,” Annals of Operations Research vol. 134(1) pp. 19–67, 2005.
M. Dorigo, V. Maniezzo, and A. Colorni, “The ant system: optimization by a colony of cooperating agents,” IEEE Transactions on Systems, Man, and Cybernetics—Part B vol. 26(1) pp. 29–41, 1996.
G. Dueck, and T. Scheur, “Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing,” Journal of Computational Physics vol. 90, pp. 161–175, 1990.
F. Glover, and M. L. Laguna, Modern Heuristic Techniques for Combinatorial Optimization, Chapter 3: Tabu search. Blackwell Scientific: London, UK, 1993.
D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley: Reading, Massachusetts, 1989.
W. B. Gong, Y. C. Ho, and W. Zhai, Stochastic comparison algorithm for discrete optimization with estimation. In Proceedings of the 31st IEEE Conference on Decision and Control, pp. 795–800, 1992.
D. M. Himmelblau, Applied Nonlinear Programming. McGrawHill: New York, 1972.
W. Hock, and K. Schittkowski, Test Examples for Nonlinear Programming Codes, volume 197. Springer: Berlin Heidelberg New York, 1981.
D. P. Kroese, R. Y. Rubinstein, and T. Taimre, Application of the cross-entropy method to clustering and vector quantization. (To appear in Journal of Global Optimization) 2006.
L. Margolin, “On the convergence of the cross-entropy method,” Annals of Operations Research vol. 134(1) pp. 201–214, 2005.
Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer: Berlin Heidelberg New York, 1996.
D. A. Paviani, A New Method for the Solution of the General Nonlinear Programming Problem. Ph.D. thesis, The University of Texas, Austin, Texas, 1969.
R. Y. Rubinstein, “Optimization of computer simulation models with rare events,” European Journal of Operational Research vol. 99 pp. 89–112, 1997.
R. Y. Rubinstein, The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability vol. 2 pp. 127–190, 1999.
R. Y. Rubinstein, and D. P. Kroese, The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer: Berlin Heidelberg New York, 2004.
K. Schittkowski, Nonlinear Programming Codes, volume 183. Springer: Berlin Heidelberg New York, 1980.
B. V. Sheela, and P. Ramaoorthy, “Swift - a new constrained optimization technique,” Computer Methods in Applied Mechanics and Engineering vol. 6(3) pp. 309–318, 1975.
D. G. Stork, and E. Yom-Tov, Computer Manual to Accompany Pattern Classification. Wiley: New York, 2004.
A. Webb, Statistical Pattern Recognition. Arnold: London, 1999.
D. White, Markov Decision Process. Wiley: New York, 1992.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Kroese, D.P., Porotsky, S. & Rubinstein, R.Y. The Cross-Entropy Method for Continuous Multi-Extremal Optimization. Methodol Comput Appl Probab 8, 383–407 (2006). https://doi.org/10.1007/s11009-006-9753-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11009-006-9753-0
Keywords
- Cross-entropy
- Continuous optimization
- Multi-extremal objective function
- Dynamic smoothing
- Constrained optimization
- Nonlinear constraints
- Acceptance–rejection
- Penalty function