Theoretically Grounded Acceleration Techniques for Simulated Annealing
Simulated annealing (SA) is a generic optimization method whose popularity stems from its simplicity and its global convergence properties; it emulates the physical process of annealing whereby a solid is heated and then cooled down to eventually reach a minimum energy configuration. Although successfully applied to many difficult problems, SA is widely reported to converge very slowly, and it is common practice to relax some of its convergence conditions as well as to allow extra freedom in its design. However, variations on the theme of annealing usually come without optimal convergence guarantees.
In this paper, we review the fundamentals of SA and we focus on acceleration techniques that come with a rigorous mathematical justification. We discuss the design of the candidate-solution generation mechanism, the issue of finite-time cooling, and the technique of acceleration by concave distortion of the objective function. We also investigate a recently introduced generalization of SA—stochastic continuation—which significantly increases the design flexibility by allowing the candidate-solution generation mechanism and the objective function to vary with temperature.
KeywordsSimulated Annealing Energy Landscape Communication Mechanism Global Convergence Property Communication Matrix
Unable to display preview. Download preview PDF.
- 1.Discussion on the meeting on the Gibbs sampler and other Markov chain Monte Carlo methods. J. Roy. Statist. Soc. Ser. B 55(1), 53–102 (1993)Google Scholar
- 2.Azencott, R.: A common large deviations mathematical framework for sequential annealing and parallel annealing. In: Azencott, R. (ed.) Simulated Annealing: Parallelization Techniques, pp. 11–23. Wiley, New York (1992)Google Scholar
- 3.Azencott, R.: Sequential simulated annealing: speed of convergence and acceleration techniques. In: Azencott, R. (ed.) Simulated Annealing: Parallelization Techniques, pp. 1–10. Wiley, New York (1992)Google Scholar
- 4.Blake, A., Zisserman, A.: Visual reconstruction. The MIT Press (1987)Google Scholar
- 18.van Laarhoven, P.J.M., Aarts, E.H.L.: Simulated annealing: theory and practice. D. Reidel Publishing Company (1987)Google Scholar
- 25.Robini, M.C., Magnin, I.E.: 3-D reconstruction from a few radiographs using the Metropolis dynamics with annealing. In: Proc. IEEE Int. Conf. Image Processing, Kobe, Japan, vol. 3, pp. 876–880 (1999)Google Scholar
- 30.Robini, M.C., Reissman, P.J.: From simulated annealing to stochastic continuation: a new trend in combinatorial optimization. J. Global Optim. (to appear, 2012)Google Scholar
- 31.Robini, M.C., Smekens, F., Sixou, B.: Optimal inverse treatment planning by stochastic continuation. In: Proc. 8th IEEE Int. Symp. Biomedical Imaging, Chicago, IL, pp. 1792–1796 (2011)Google Scholar