# A Large Deviations Analysis of Certain Qualitative Properties of Parallel Tempering and Infinite Swapping Algorithms

## Abstract

Parallel tempering, or replica exchange, is a popular method for simulating complex systems. The idea is to run parallel simulations at different temperatures, and at a given swap rate exchange configurations between the parallel simulations. From the perspective of large deviations it is optimal to let the swap rate tend to infinity and it is possible to construct a corresponding simulation scheme, known as infinite swapping. In this paper we propose a novel use of large deviations for empirical measures for a more detailed analysis of the infinite swapping limit in the setting of continuous time jump Markov processes. Using the large deviations rate function and associated stochastic control problems we consider a diagnostic based on temperature assignments, which can be easily computed during a simulation. We show that the convergence of this diagnostic to its a priori known limit is a necessary condition for the convergence of infinite swapping. The rate function is also used to investigate the impact of asymmetries in the underlying potential landscape, and where in the state space poor sampling is most likely to occur.

## Notes

### Acknowledgements

J. Doll: Research supported in part by the National Science Foundation (DMS-1317199), and the Defense Advanced Research Projects Agency (W911NF-15-2-0122). P. Dupuis: Research supported in part by the Department of Energy (DE-SC0010539), the National Science Foundation (DMS-1317199), and the Defense Advanced Research Projects Agency (W911NF-15-2-0122). P. Nyquist: Research supported in part by National Science Foundation (DMS-1317199).

## References

- 1.Boué, M., Dupuis, P.: A variational representation for certain functionals of Brownian motion. Ann. Prob.
**26**, 1641–1659 (1998)MathSciNetCrossRefzbMATHGoogle Scholar - 2.Budhiraja, A., Dupuis, P., Maroulas, V.: Variational representations for continuous time processes. Ann. l’Inst. H. Poincaré
**47**, 725–747 (2011)MathSciNetCrossRefzbMATHGoogle Scholar - 3.Doll, J., Dupuis, P.: On performance measures for infinite swapping Monte Carlo methods. J. Chem. Phys
**142**, 024111 (2015)CrossRefGoogle Scholar - 4.Doll, J., Plattner, N., Freeman, D.L., Liu, Y., Dupuis, P.: Rare-event sampling: occupation-based performance measures for parallel tempering and infinite swapping Monte Carlo methods. J. Chem. Phys
**137**, 204112 (2012)CrossRefGoogle Scholar - 5.Dupuis, P., Ellis, R.S.: The large deviation principle for a general class of queueing systems. I. Trans. Am. Math. Soc.
**347**, 2689–2751 (1996)MathSciNetzbMATHGoogle Scholar - 6.Dupuis, P., Ellis, R.S.: A Weak Convergence Approach to the Theory of Large Deviations. Wiley, New York (1997)CrossRefzbMATHGoogle Scholar
- 7.Dupuis, P., Liu, Y.: On the large deviation rate for the empirical measure of a reversible pure jump markov processes. Ann. Probab.
**43**, 1121–1156 (2015)MathSciNetCrossRefzbMATHGoogle Scholar - 8.Dupuis, P., Liu, Y., Plattner, N., Doll, J.D.: On the infinite swapping limit for parallel tempering. SIAM J. Multiscale Model. Simul.
**10**, 986–1022 (2012)MathSciNetCrossRefzbMATHGoogle Scholar - 9.Earl, D.J., Deem, M.W.: Parallel tempering: theory, applications, and new perspectives. Phys. Chem. Chem. Phys.
**7**, 3910–3916 (2005)CrossRefGoogle Scholar - 10.Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley, New York (1986)CrossRefzbMATHGoogle Scholar
- 11.Fleming, W.H.: Exit probabilities and optimal stochastic control. Appl. Math. Optim.
**4**, 329–346 (1978)MathSciNetCrossRefzbMATHGoogle Scholar - 12.Fleming, W.H., Soner, H.M.: Asymptotic expansions for Markov processes with Levy generators. Appl. Math. Optim.
**19**, 203–223 (1989)MathSciNetCrossRefzbMATHGoogle Scholar - 13.Geyer, C.J.: Markov chain Monte Carlo maximum likelihood. In: Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, pp. 156–163. American Statistical Association, New York (1991)Google Scholar
- 14.Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time. Springer-Verlag, New York (2001). Revised Second EditionCrossRefzbMATHGoogle Scholar
- 15.David, G.: Luenberger, Optimization by Vector Space Methods, 1st edn. Wiles, New York (1969)Google Scholar
- 16.Rogers, L.C.G., Williams, D.: Diffusions, Markov Processes, and Martingales, Vol. 2. Cambridge University Press, Cambridge (2000) (Itô calculus, Reprint of the second (1994) edition)Google Scholar
- 17.Stroock, D.W.: An Introduction to Markov Processes. Graduate Texts in Mathematics, vol. 230. Springer, Berlin (2005)zbMATHGoogle Scholar
- 18.Sugita, Y., Okamoto, Y.: The incomplete beta function law for parallel tempering sampling of classical canonical systems. Chem. Phys. Lett.
**314**, 141–151 (1999)CrossRefGoogle Scholar - 19.Swendsen, R.H., Wang, J.S.: Replica Monte Carlo simulation of spin glasses. Phys. Rev. Lett.
**57**, 2607–2609 (1986)MathSciNetCrossRefGoogle Scholar