, 6:302 | Cite as

A theoretical framework for simulated annealing

  • Fabio Romeo
  • Alberto Sangiovanni-Vincentelli


Simulated Annealing has been a very successful general algorithm for the solution of large, complex combinatorial optimization problems. Since its introduction, several applications in different fields of engineering, such as integrated circuit placement, optimal encoding, resource allocation, logic synthesis, have been developed. In parallel, theoretical studies have been focusing on the reasons for the excellent behavior of the algorithm. This paper reviews most of the important results on the theory of Simulated Annealing, placing them in a unified framework. New results are reported as well.

Key words

Simulated annealing Combinatorial optimization Markov chains 


  1. [1]
    E. H. L. Aarts and P. J. M. van Laarhoven. Statistical Cooling: A General Approach to Combinatorial Optimization Problems.Philips J. Res.,40:193–226, 1985.MathSciNetGoogle Scholar
  2. [2]
    E. H. L. Aarts and P. J. M. van Laarhoven.Simulated Annealing: Theory and Applications. Reidel, Dordrecht, 1987.MATHGoogle Scholar
  3. [3]
    S. Anily and A. Federgruen. Probabilistic Analysis of Simulated Annealing Methods. Technical Report, Graduate School of Business, Columbia University, New York, 1985.Google Scholar
  4. [4]
    S. Anily and A. Federgruen. Simulated Annealing Methods with General Acceptance Probabilities.J. Appl. Probab.,24: 657–667, 1987.MATHCrossRefMathSciNetGoogle Scholar
  5. [5]
    C. R. Aragon, D. S. Johnson, L. A. McGeoch, and C. Schevon. Simulated Annealing Performance Studies. Workshop on Statistical Physics in Engineering and Biology, April, 1984.Google Scholar
  6. [6]
    P. Banerjee and M. Jones. A Parallel Simulated Annealing Algorithm for Standard Cell Placement on a Hypercube Computer.Proc. ICCAD, pages 34–37, 1986.Google Scholar
  7. [7]
    K. Binder.Monte Carlo Methods in Statistical Physics. Springer Verlag, Berlin, 1978.Google Scholar
  8. [8]
    I. O. Bohachevsky, M. E. Johnson, and M. L. Stein. Generalized Simulated Annealing for Function Optimization.Technometrics,28 (3): 209–217, August, 1986.MATHCrossRefGoogle Scholar
  9. [9]
    A. Casotto, F. Romeo, and A. Sangiovanni-Vincentelli. A Parallel Simulated Annealing Algorithm for the Placement of Macro-Cells.IEEE Trans. Computer-Aided Design,6: 838–847, September, 1987.CrossRefGoogle Scholar
  10. [10]
    A. Casotto and A. Sangiovanni-Vincentelli. Placement of Standard Cells Using Simulated Annealing on the Connection Machine.Proc. ICCAD, pages 350–353, 1987.Google Scholar
  11. [11]
    D. P. Connors and P. R. Kumar. Balance of Recurrence Orders in Time-Inhomogeneous Markov Chains with Applications to Simulated Annealing.Probab. Engrg. Inform. Sci.,2 (2): 157–184, 1988.MATHGoogle Scholar
  12. [12]
    F. Darema-Rogers, S. Kirkpatrick, and V. A. Norton. Simulated Annealing on Shared Memory Parallel Systems.IBM J. Res. Develop., 1987.Google Scholar
  13. [13]
    W. B. Davenport.Probability and Random Processes: An Introduction for Applied Scientists and Engineers. McGraw-Hill, New York, 1970.Google Scholar
  14. [14]
    U. Faigle and R. Schrader. On the Convergence of Stationary Distributions in Simulated Annealing Algorithms.Inform. Process. Lett.,27 (4): 189–194, April, 1988.MATHCrossRefMathSciNetGoogle Scholar
  15. [15]
    D. Freedman.Markov Chains. Holden-Day, San Francisco, CA, 1971.MATHGoogle Scholar
  16. [16]
    S. Gelfand. Analysis of Simulated Annealing Type of Algorithms. Technical Report LIDS-TH-1688, Massachusetts Institute of Technology, Cambridge, MA., 1987.Google Scholar
  17. [17]
    S. Gelfand and S. Mitter. Analysis of Simulated Annealing for Optimization.Proc. 24th Conf. on Decision and Control, pages 779–786, December 1985.Google Scholar
  18. [18]
    S. Gelfand and S. Mitter. Simulated Annealing with Noisy or Imprecise Energy Measurements. Technical Report, Massachusetts Institute of Technology, Cambridge, MA, 1987.Google Scholar
  19. [19]
    S. Geman and D. Geman. Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images.IEEE Trans. Pattern Anal. Mach. Intell.,6: 721–741, 1984.MATHGoogle Scholar
  20. [20]
    B. Gidas. Non-Stationary Markov Chains and Convergence of the Annealing Algorithm.J. Statist. Phys.,39: 73–131, 1984.CrossRefMathSciNetGoogle Scholar
  21. [21]
    B. V. Gnedenko and A. Y. Khinchin.An Elementary Introduction to the Theory of Probability, Freeman, San Francisco, CA, 1961.MATHGoogle Scholar
  22. [22]
    B. Hajek. Cooling Schedules for Optimal Annealing.Math. Oper. Res.,13 (2): 311–329, 1985.MathSciNetCrossRefGoogle Scholar
  23. [23]
    J. M. Hammersley and D. C. Handscomb.Monte Carlo Methods, Wiley, New York, 1964.MATHGoogle Scholar
  24. [24]
    W. K. Hastings. Monte Carlo Sampling Methods Using Markov Chains and Their Applications.Biometrika,57 (1): 97–109, 1970.MATHCrossRefGoogle Scholar
  25. [25]
    D. Hillis.The Connection Machine. MIT Press, Cambridge, MA, 1985.Google Scholar
  26. [26]
    P. G. Hoel.Introduction to Mathematical Statistics. Wiley, New York, 1962.Google Scholar
  27. [27]
    R. Holley and D. Stroock. Simulated Annealing via Sobolev Inequalities.Comm. Math. Phys.,115 (4): 553–569, April, 1988.MATHCrossRefMathSciNetGoogle Scholar
  28. [28]
    M. D. Huang, F. Romeo, and A. Sangiovanni-Vincentelli. An Efficient General Cooling Schedule for Simulated Annealing,Proc. ICCAD, pages 381–384, 1986.Google Scholar
  29. [29]
    S. Hustin. Tim, a New Standard Cell Placement Program Based on the Simulated Annealing Algorithm. Master's thesis, University of California, Berkeley, CA, March, 1988.Google Scholar
  30. [30]
    D. L. Isaacson and R. W. Madsen.Markov Chains: Theory and Applications. Wiley, New York, 1976.MATHGoogle Scholar
  31. [31]
    J. G. Kemeny and J. L. Snell.Finite Markov Chains. Springer-Verlag, New York, 1976.MATHGoogle Scholar
  32. [32]
    S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by Simulated Annealing.Science,220(4598): 671–680, 13 May, 1983.CrossRefMathSciNetGoogle Scholar
  33. [33]
    S. A. Kravitz and R. A. Rutenbar. Multiprocessor-based Placement by Simulated Annealing.Proc. 23rd Design Automation Conf., pages 567–573, 1986.Google Scholar
  34. [34]
    J. Lam and J-M. Delosme. Logic Minimization Using Simulated Annealing.Proc. ICCAD, pages 348–352, 1986.Google Scholar
  35. [35]
    J. Lam and J.-M. Delosme. An Adaptive Annealing Schedule. Technical Report 8608, University of Yale, New Haven, C., 1987.Google Scholar
  36. [36]
    J. Lam and J-M. Delosme. Simulated Annealing: A Fast Heuristic for Some Generic Layout Problems.Proc. ICCAD, pages 510–513, 1988.Google Scholar
  37. [37]
    M. Lundy and A. Mees. Convergence of the Annealing Algorithm.Math. Programming,34: 111–124, 1986.MATHCrossRefMathSciNetGoogle Scholar
  38. [38]
    R. W. Madsen and D. L. Isaacson. Strongly Ergodic Behavior for Non-Stationary Markov Processes.Ann. Prob.,1 (2): 329–335, 1973.MATHCrossRefMathSciNetGoogle Scholar
  39. [39]
    N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, and A. H. Teller. Equation of State Calculations by Fast Computer Machines.J. Chem. Phys.,21: 1087, 1953.CrossRefGoogle Scholar
  40. [40]
    D. Mitra, F. Romeo, and A. Sangiovanni-Vincentelli. Convergence and Finite-Time Behavior of Simulated Annealing.Proc. 24th Conf. on Decision and Control, December, 1985.Google Scholar
  41. [41]
    D. Mitra, F. Romeo, and A. Sangiovanni-Vincentelli. Convergence and Finite-Time Behavior of Simulated Annealing.Adv. Appl. Probab.,18: 747–771, 1986.MATHCrossRefMathSciNetGoogle Scholar
  42. [42]
    J. D. Nulton and P. Salamon. Statistical Mechanics of Combinatorial Optimization.Phys. Rev. A,37(4): 1351–1356, February, 1988.CrossRefMathSciNetGoogle Scholar
  43. [43]
    R. H. J. M. Otten and L. P. P. P. van Ginneken. Floorplan Design Using Simulated Annealing.Proc. ICCAD, pages 96–98, 1984.Google Scholar
  44. [44]
    R. H. Otten and L. P. van Ginneken.Simulated Annealing: The Algorithm. Kluwer, Boston, MA, 1989.Google Scholar
  45. [45]
    P. H. Peskun. Optimum Monte Carlo Sampling Using Markov Chains.Biometrika,60(3): 607–612, 1973.MATHCrossRefMathSciNetGoogle Scholar
  46. [46]
    B. D. Ripley.Stochastic Simulation. Wiley, New York, 1987.MATHGoogle Scholar
  47. [47]
    F. Romeo. Simulated Annealing: Theory and Applications to Layout Problems. Ph. D. thesis, University of California, Berkeley, CA, May 1989.Google Scholar
  48. [48]
    Y. Rossier, M. Troyon, and T. Liebling. Probabilistic Exchange Algorithms and Euclidean Traveling Salesman Problems. Technical Report RO 851125, EPF, Lousanne, 1985.Google Scholar
  49. [49]
    W. Rudin.Principles of Mathematical Analysis. McGraw-Hill, New York, 1976.MATHGoogle Scholar
  50. [50]
    A. Sangiovanni-Vincentelli. Editor's Foreword.Algorithmica, this issue.Google Scholar
  51. [51]
    E. Seneta.Non-negative Matrices and Markov Chains. Springer-Verlag, New York, 1981.MATHGoogle Scholar
  52. [52]
    G. B. Sorkin. Combinatorial Optimization, Simulated Annealing and Fractals. Master's thesis. University of California, Berkeley, CA, December, 1987.Google Scholar
  53. [53]
    G. B. Sorkin. Rapidly Mixing Analysis of Simulated Annealing on Fractals.Proc. of Sixth Annual MIT VLSI Conf., pages 331–351, April, 1990.Google Scholar
  54. [54]
    G. B. Sorkin. Efficient Simulated Annealing on Fractal Energy Landscapes.Algorithmica, this issue, pp. 367–418, 1991.Google Scholar
  55. [55]
    P. N. Strenski. Optimal Annealing Schedules: A Case Study. Technical Report 12923, I.B.M. T. J. Watson Research Center, Yorktown Heights, NY, 1987.Google Scholar
  56. [56]
    P. N. Strenski and S. Kirkpatrick. Analysis of Finite Length Annealing Schedules.Algorithmica, this issue, pp. 346–366, 1991.Google Scholar
  57. [57]
    H. C. Tijms.Stochastic Modeling and Analysis: A Computational Approach, Wiley, Chichester, 1986.Google Scholar
  58. [58]
    J. N. Tsitsiklis.Markov Chains with Rare Transitions and Simulated Annealing. MIT Laboratory for Information and Decision Systems, Cambridge, MA, August, 1985.Google Scholar
  59. [59]
    V. Černy. Thermodynamical Approach to the Traveling Salesman Problem: An Efficient Simulation Algorithm.J. Optim. Theory Appl.,45: 41–51, 1985.MATHCrossRefMathSciNetGoogle Scholar
  60. [60]
    S. White, Concept of Scale in Simulated Annealing.Proc. ICCD, pages 646–651, 1984.Google Scholar

Copyright information

© Springer-Verlag New York Inc. 1991

Authors and Affiliations

  • Fabio Romeo
    • 1
  • Alberto Sangiovanni-Vincentelli
    • 1
  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of CaliforniaBerkeleyUSA

Personalised recommendations