Journal of Global Optimization

, Volume 57, Issue 3, pp 863–890 | Cite as

An efficient implementation of parallel simulated annealing algorithm in GPUs

  • A. M. Ferreiro
  • J. A. García
  • J. G. López-Salas
  • C. VázquezEmail author


In this work we propose a highly optimized version of a simulated annealing (SA) algorithm adapted to the more recently developed graphic processor units (GPUs). The programming has been carried out with compute unified device architecture (CUDA) toolkit, specially designed for Nvidia GPUs. For this purpose, efficient versions of SA have been first analyzed and adapted to GPUs. Thus, an appropriate sequential SA algorithm has been developed as starting point. Next, a straightforward asynchronous parallel version has been implemented and then a specific and more efficient synchronous version has been developed. A wide appropriate benchmark to illustrate the performance properties of the implementation has been considered. Among all tests, a classical sample problem provided by the minimization of the normalized Schwefel function has been selected to compare the behavior of the sequential, asynchronous and synchronous versions, the last one being more advantageous in terms of balance between convergence, accuracy and computational cost. Also the implementation of a hybrid method combining SA with a local minimizer method has been developed. Note that the generic feature of the SA algorithm allows its application in a wide set of real problems arising in a large variety of fields, such as biology, physics, engineering, finance and industrial processes.


Global optimization Simulated annealing Parallel computing GPUs CUDA 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ackley D.H.: A Connectionist Machine for Genetic Hillclimbing. Kluwer, Boston (1987)CrossRefGoogle Scholar
  2. 2.
    Bäck T.: Evolutionary Algorithms in Theory and Practice. Oxford University Press, Oxford (1996)Google Scholar
  3. 3.
    Bersini, H., Dorigo, M., Langerman, S., Seront, G., Gambardella, L.M.: Results of the first international contest on evolutionary optimisation (1st ICEO). In: Proceedings of IEEE International Conference on Evolutionary Computation, IEEE-EC 96, pp. 611–615. IEEE Press, New York (1996)Google Scholar
  4. 4.
    Bolstad W.M.: Understanding Computational Bayesian Statistics. Wiley, New York (2001)Google Scholar
  5. 5.
    Breiman L., Cutler A.: A deterministic algorithm for global optimization. Math. Program. 58, 179–199 (1993)CrossRefGoogle Scholar
  6. 6.
    Brooks S.B., Morgan B.J.T.: Optimization using simulated annealing. Statistician 44(2), 241–257 (1995)CrossRefGoogle Scholar
  7. 7.
    Chen H., Flann N.S., Watson D.W.: Parallel genetic simulated annealing: a massively parallel SIMD algorithm. IEEE Trans. Parallel Distrib. Syst. 9, 126–136 (1998)CrossRefGoogle Scholar
  8. 8.
    Chen D.J., Lee C.Y., Park C.H., Mendes P.: Parallelizing simulated annealing algorithms based on high-performance computer. J. Glob. Optim. 39, 261–289 (2007)CrossRefGoogle Scholar
  9. 9.
    Dekkers A., Aarts E.: Global optimization and simulated annealing. Math. Program. 50, 367–393 (1991)CrossRefGoogle Scholar
  10. 10.
    De Jong, K.A.: An Analysis of the Behavior of a Glass of Genetic Adaptive Systems. PhD thesis, University of Michigan, Ann Arbor (1975)Google Scholar
  11. 11.
    Debudaj-Grabysz, A., Rabenseifner, R.: Nesting OpenMP in MPI to Implement a Hybrid Communication Method of Parallel Simulated Annealing on a Cluster of SMP Nodes. EuroPVM/MPI, Sorrento, Sept 19, (2005)Google Scholar
  12. 12.
    Dixon L., Szegö G.: Towards Global Optimization. North Holland, New York (1978)Google Scholar
  13. 13.
    Griewank A.O.: General descent for global optimization. J. Optim. Theory Appl. 34, 11–39 (1981)CrossRefGoogle Scholar
  14. 14.
    HimmelBlau D.M.: Applied Linear Programming. McGraw-Hill, New York (1972)Google Scholar
  15. 15.
    Kirkpatrick S., Gelatt C.D., Vecchi M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)CrossRefGoogle Scholar
  16. 16.
    Kliewer, A., Tschöke, S.: A General Parallel Simulated Annealing Library and Its Applications in Industry. Working paper, Department of Mathematics and Computer Science, University of Paderborn, Germany (1998)Google Scholar
  17. 17.
    Lee S.Y., Lee K.G.: Synchronous and asynchronous parallel simulated annealing with multiple Markov Chains. IEEE Trans. Parallel Distrib. Syst. 7(10), 993–1008 (1996)CrossRefGoogle Scholar
  18. 18.
    Levy A.V., Montalvo A.: The tunneling algorithm for the global minimization of functions. SIAM J. Sci. Stat. Comput. 6, 15–29 (1985)CrossRefGoogle Scholar
  19. 19.
    Metropolis N., Rosenbluth A., Rosenbluth M., Teller A., Teller E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953)CrossRefGoogle Scholar
  20. 20.
    Michalewicz Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin (1998)Google Scholar
  21. 21.
    Nvidia Corp.: CUDA 4.0 Programming Guide. (2011)Google Scholar
  22. 22.
    Nvidia Corp.: Whitepaper. NVIDIAs Next Generation CUDA Compute Architecture: FermiGoogle Scholar
  23. 23.
    Nvidia Corp.: CUDA. CURAND Library (2011)Google Scholar
  24. 24.
    Onbasoglu E., Özdamar L.: Parallel simulated annealing algorithms in global optimization. J. Glob. Optim. 19, 27–50 (2001)CrossRefGoogle Scholar
  25. 25.
    Salomon R.: Reevaluating genetic algorithms performance under coordinate rotation of benchmark functions. BioSystems 39(3), 263–278 (1995)CrossRefGoogle Scholar
  26. 26.
    Sanders J., Kandrot E.: CUDA by Example: An Introduction to General-Purpose GPU Programming. Addison-Wesley, Michigan (2011)Google Scholar
  27. 27.
    Storn R., Price K.: Differential evolution: a simple and efficient heuristic for global optimization over continuous spaces. J. Glob. Optim. 11, 341–359 (1997)CrossRefGoogle Scholar
  28. 28.
    Törn, A., ˆilinskas, A.: Global optimization. In: Lecture Notes in Computer Science 350. Springer, Berlin (1996)Google Scholar
  29. 29.
    Thrust library web page:
  30. 30.
    van Laarhoven P.J.M, Aarts E.H.L.: Simulated Annealing: Theory and Applications. Kluwer, Dordrecht (1987)CrossRefGoogle Scholar
  31. 31.
  32. 32.

Copyright information

© Springer Science+Business Media, LLC. 2012

Authors and Affiliations

  • A. M. Ferreiro
    • 1
  • J. A. García
    • 1
  • J. G. López-Salas
    • 1
  • C. Vázquez
    • 1
    Email author
  1. 1.Department of Mathematics, Faculty of InformaticsUniversidade da CoruñaCoruñaSpain

Personalised recommendations