Abstract
We present a novel approach, in which parallel annealing processes interact in a manner that expedites the identification of a globally optimal solution. A first annealing process operates at a faster time scale and has a drift function that converges to a non-zero (but relatively small) noise level. A second annealing process (operating at a slower time scale) is subject to a modified drift term in which the steepest descent direction is perturbed with the first annealing process density gradient. This additional term ensures that the second process is “repelled” from regions already explored. As a result, the first annealing process (which quickly identifies locally optimal solutions) allows the second annealing process to bypass locally optimal solutions recently identified, so that it can be made to converge to global optima at a faster rate. We show that, when compared to independent annealing processes, the proposed interactive diffusions can increase the speed of convergence at the expense of minimal additional computational overhead.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
As chips are clocked at higher speeds, it is increasingly difficult to control their temperature and they become much less energy-efficient.
References
Cerný, V.: Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J. Optim. Theory Appl. 45(1), 41–51 (1985)
Kirkpatrick, S., Gelatt, C.D. Jr., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 621–680 (1983)
Geman, S., Hwang, C.R.: Diffusions for global optimization. SIAM J. Control Optim. 24, 1031–1043 (1986)
Chiang, T.S., Hwang, C.R., Sheu, S.J.: Diffusion for global optimization in R n. SIAM J. Control Optim. 25, 737–752 (1987)
Kushner, H.J.: Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: global minimization via Monte Carlo. SIAM J. Appl. Math. 47, 169–185 (1987)
Hwang, C.R., Hwang-Ma, S.Y., Sheu, S.J.: Accelerating Gaussian diffusions. Ann. Appl. Probab. 3(3), 897–913 (1993)
Yin, G., Yin, K.: Global optimization using diffusion perturbations with large noise intensity. Acta Math. Appl. Sin., Engl. Ser. 22(4), 529–542 (2006)
Poliannikov, O., Zhizhina, E., Krim, H.: Global optimization by adapted diffusion. IEEE Trans. Signal Process. 58(12), 6119–6125 (2010)
Coleman, T., Shalloway, D., Wu, Z.: A parallel build-up algorithm for global energy minimizations of molecular clusters using effective energy simulated annealing. J. Glob. Optim. 4, 171–186 (1994)
Wu, Z.: The effective energy transformation scheme as a special continuation approach to global optimization with application to molecular conformation. SIAM J. Optim. 6, 748–768 (1996)
Lau, M., Kwong, C.P.: A smoothing method of global optimization that preserves global minima. J. Glob. Optim. 34, 369–398 (2006)
Khasminskii, R.Z., Yin, G.: Limit behavior of two-time scale diffusions revisited. J. Differ. Equ. 212, 85–113 (2005)
Risken, H.: The Fokker–Planck Equation (Methods of Solution and Applications), 2nd edn. Springer Series in Synergetics. Springer, Berlin (1989). 1989
Pham, H.: Continuous-Time Stochastic Control and Optimization with Financial Applications. Stochastic Modelling and Applied Probability, vol. 61. Springer, Berlin (2009)
Gardiner, C.W.: Handbook of Stochastic Methods for Physics, Chemistry, and the Natural Sciences, 2nd edn. Springer Series in Synergetics, vol. 13. Springer, Berlin (1985)
Ackley, D.H.: A Connectionist Machine for Genetic Hillclimbing. Kluwer Academic Publishers, Boston (1987)
Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin (1992)
Gomes, C.P., Selman, B., Kautz, H.: Boosting combinatorial search through randomization. In: Proceedings of the National Conference on Artificial Intelligence, pp. 431–437. Wiley, New York (1998)
Luby, M., Ertel, W.: Optimal Parallelization of Las Vegas Algorithms. Springer, Berlin (1994)
Shylo, O.V., Middelkoop, T., Pardalos, P.M.: Restart strategies in optimization: parallel and serial cases. Parallel Comput. 37(1), 60–68 (2011)
Acknowledgements
This work was partially funded by Air Force Office of Scientific Research (AFOSR) through grant FA9550-12-1-0163.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Panos M. Pardalos.
Rights and permissions
About this article
Cite this article
Sun, Y., Garcia, A. Interactive Diffusions for Global Optimization. J Optim Theory Appl 163, 491–509 (2014). https://doi.org/10.1007/s10957-013-0394-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-013-0394-5