Simulated annealing algorithms and Markov chains with rare transitions

  • Olivier Catoni
Cours Spécialisés
Part of the Lecture Notes in Mathematics book series (LNM, volume 1709)


In these notes, written for a D.E.A. course at University Paris XI during the first term of 1995, we prove the essentials about stochastic optimisation algorithms based on Markov chains with rare transitions, under the weak assumption that the transition matrix obeys a large deviation principle. We present a new simplified line of proofs based on the Freidlin and Wentzell graphical approach. The case of Markov chains with a periodic behaviour at null temperature is considered. We have also included some pages about the spectral gap approach where we follow Diaconis and Stroock [13] and Ingrassia [23] in a more conventional way, except for the application to non reversible Metropolis algorithms (subsection 6.2.2) where we present an original result.


Markov Chain Exit Time Gibbs Distribution Metropolis Algorithm Homogeneous Markov Chain 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Azencott Robert (1988) Simulated Annealing, Séminaire Bourbaki 40ième année, 1987–1988 697 Google Scholar
  2. [2]
    Azencott Robert (1992) Sequential Simulated Annealing: Speed of Convergence and Acceleration Techniques, in Simulated Annealing: Parallelization Techniques, R. Azencott Ed., Wiley Interscience.Google Scholar
  3. [3]
    Azencott Robert (1992) A Common Large Deviations Mathematical Framework for Sequential Annealing and Parallel Annealing, in Simulated Annealing: Parallelization Techniques, R. Azencott Ed., Wiley Interscience.Google Scholar
  4. [4]
    Azencott Robert and Graffigne Christine (1992) Parallel Annealing by Periodically Interacting Multiple Searches: Acceleration Rates, in Simulated Annealing: Parallelization Techniques, R. Azencott Ed., Wiley Interscience.Google Scholar
  5. [5]
    Catoni Olivier (1991) Exponential Triangular Cooling Schedules for Simulated Annealing Algorithms: a case study, Applied Stochastic Analysis, Proceedings of a US-French Workshop, Rutgers University, April 29–May 2, 1991, Karatzas I. and Ocone D. eds., Lecture Notes in Control and Information Sciences No 177, Springer Verlag, 1992.Google Scholar
  6. [6]
    Catoni Olivier (1992) Rough Large Deviation Estimates for Simulated Annealing: Application to Exponential Schedules, The Annals of Probability, Vol. 20, nb. 3, pp. 1109–1146.MathSciNetzbMATHGoogle Scholar
  7. [7]
    Catoni Olivier, (1998) The Energy Transformation Method for the Metropolis Algorithm Compared with Simulated Annealing. Probab. Theory Related Fields 110 (1998), no. 1, pages 69–89.MathSciNetCrossRefzbMATHGoogle Scholar
  8. [8]
    Catoni Olivier and Cerf Raphael (1997) The Exit Path of a Markov Chain with Rare Transitions, ESAIM: P&S, vol. 1, pp. 95–144, http://www Scholar
  9. [9]
    Catoni Olivier (1998) Solving Scheduling Problems by Simulated Annealing. SIAM J. Control Optim. 36, no. 5, (electronic), pages 1539–1575.MathSciNetCrossRefzbMATHGoogle Scholar
  10. [10]
    Catoni Olivier (1996) Metropolis, Simulated Annealing and I.E.T. Algorithms: Theory and Experiments. Journal of Complexity 12, special issue on the conference Foundation of Computational Mathematics, January 5–12 1997, Rio de Janeiro, pages 595–623, December 1996.MathSciNetCrossRefzbMATHGoogle Scholar
  11. [11]
    Cot Cécile and Catoni Olivier (1998) Piecewise constant triangular cooling schedules for generalized simulated annealing algorithms. Ann. Appl. Probab. 8, no. 2, pages 375–396.MathSciNetCrossRefzbMATHGoogle Scholar
  12. [12]
    Deuschel J.D. and Mazza C. (1994) L 2 convergence of time nonhomogeneous Markov processes: I. Spectral Estimates, The annals of Applied Probability, vol. 4, no. 4, 1012–1056.MathSciNetCrossRefzbMATHGoogle Scholar
  13. [13]
    Diaconis Persi and Stroock Daniel (1991) Geometric Bounds for Eigenvalues of Markov Chains, The Annals of Applied Probability, Vol. 1, No. 1, 36–61.MathSciNetCrossRefzbMATHGoogle Scholar
  14. [14]
    Duflo M. (1996) Algorithmes Stochastiques, Mathématiques & Applications (Paris), Springer Verlag.Google Scholar
  15. [15]
    Fill J. A. (1991) Eigenvalue bounds on the convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process, Ann. Applied Probab., 1.Google Scholar
  16. [16]
    Freidlin, M. I. and Wentzell, A. D. (1984). Random Perturbations of Dynamical Systems. Springer, New York.CrossRefzbMATHGoogle Scholar
  17. [17]
    Geman S., Geman D., Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images, I.E.E.E. Transaction on Pattern Analysis and Machine Intelligence, 6, 721–741, 1984.CrossRefzbMATHGoogle Scholar
  18. [18]
    Götze F. (1991) Rate of Convergence of Simulated Annealing Processes, preprint.Google Scholar
  19. [19]
    Graffigne Christine (1992) Parallel Annealing by Periodically Interacting Multiple Searches: An Experimental Study, in Simulated Annealing: Parallelization Techniques, R. Azencott Ed., Wiley InterscienceGoogle Scholar
  20. [20]
    Holley R. and Stroock D. (1988) Annealing via Sobolev inequalities, Comm. Math. Phys., 115:553–559.MathSciNetCrossRefzbMATHGoogle Scholar
  21. [21]
    Holley, R. A., Kusuoka, S. and Stroock, D. W. (1989), Asymptotics of the spectral gap with applications to the theory of simulated annealing, Journal of functional analysis, 83, 333–347.MathSciNetCrossRefzbMATHGoogle Scholar
  22. [22]
    Hwang, C. R. and Sheu, S. J. (1992) Singular perturbed Markov chains and exact behaviour of simulated annealing processes. J. Theoret. Prob., 5, 2, 223–249.MathSciNetCrossRefzbMATHGoogle Scholar
  23. [23]
    Ingrassia S. (1994) On the rate of convergence of the Metropolis algorithm and Gibbs sampler by geometric bounds, Ann. Appl. Probab. 4, no. 2, 347–389.MathSciNetCrossRefzbMATHGoogle Scholar
  24. [24]
    Kirchhoff G. (1847) Über die Auflösung der Gleichungen, auf welche man beider Untersuchung der linearen Verteilung galvanischer Ströme gefuhrt wird, Ann. Phys. Chem., 72, pp. 497–508. (English transl. IRE Trans. Circuit Theory CT-5 (1958) 4–7).CrossRefGoogle Scholar
  25. [25]
    Kirkpatrick, S., Gelatt C. D. and Vecchi M. P., (1983) Optimization by simulated annealing, Science, 220, 621–680, 1983.MathSciNetCrossRefzbMATHGoogle Scholar
  26. [26]
    Miclo Laurent (1991) Evolution de l'énergie libre. Application à l'étude de la convergence des algorithmes du recuit simulé. Doctoral Dissertation, Université d'Orsay, February 1991.Google Scholar
  27. [27]
    Miclo Laurent (1996) Sur les problémes de sortie discrets inhomogènes Ann. Appl. Probab. 6, no 4, 1112–1156.MathSciNetCrossRefGoogle Scholar
  28. [28]
    Miclo Laurent (1995) Sur les temps d'occupations des processus de Markov finis inhomogènes à basse température, submitted to Stochastics and Stochastics Reports.Google Scholar
  29. [29]
    Miclo Laurent (1997) Remarques sur l'hypercontractivité et l'évolution de l'entropie pour des chaînes de Markov finies, Séminaire de Probabilités XXXI, Lecture Notes in Mathematics 1655, Springer.Google Scholar
  30. [30]
    Saloff-Coste, Laurent (1997) Lectures on finite Markov chains Lectures on probability theory and statistics (Saint-Flour, 1996), 301–413, Lecture Notes in Math., 1665, Springer, Berlin.zbMATHGoogle Scholar
  31. [31]
    Trouvé Alain (1993) Parallélisation massive du recuit simulé, Doctoral Dissertation, Université Paris 11, January 5 1993.Google Scholar
  32. [32]
    Trouvé Alain (1994) Cycle Decomposition and Simulated Annealing, S.I.A.M. J. Control Optim., 34(3), 1996.Google Scholar
  33. [33]
    Trouvé, Alain (1995) Rough Large Deviation Estimates for the Optimal Convergence Speed Exponent of Generalized Simulated Annealing Algorithms, Ann. Inst. H. Poincaré, Probab. Statist., 32(2), 1996.MathSciNetGoogle Scholar

Copyright information

© Springer-Verlag 1999

Authors and Affiliations

  • Olivier Catoni

There are no affiliations available

Personalised recommendations