Rare Event Simulation for Stochastic Dynamics in Continuous Time
 143 Downloads
Abstract
Large deviations for additive path functionals of stochastic dynamics and related numerical approaches have attracted significant recent research interest. We focus on the question of convergence properties for cloning algorithms in continuous time, and establish connections to the literature of particle filters and sequential Monte Carlo methods. This enables us to derive rigorous convergence bounds for cloning algorithms which we report in this paper, with details of proofs given in a further publication. The tilted generator characterizing the large deviation rate function can be associated to nonlinear processes which give rise to several representations of the dynamics and additional freedom for associated numerical approximations. We discuss these choices in detail, and combine insights from the filtering literature and cloning algorithms to compare different approaches and improve efficiency.
Keywords
Dynamic large deviations Interacting particle systems Cloning algorithm Sequential Monte Carlo1 Introduction
Large deviation simulation techniques based on classical ideas of evolutionary algorithms [1, 2] have been proposed under the name of ‘cloning algorithms’ in [3] for discrete and in [4] for continuous time processes, in order to study rare events of dynamic observables of interacting lattice gases. This approach has subsequently been applied in a wide variety of contexts (see e.g. [5, 6, 7, 8] and references therein), and more recently, the convergence properties of the algorithm have become a subject of interest. Analytical approaches so far are based on a branching process interpretation of the algorithm in discrete time [9], with limited and mostly numerical results in continuous time [10]. Systematic errors arise from the correlation structure of the cloning ensemble which can be large in practice, and several variants of the approach have been proposed to address those including e.g. a multicanonical feedback control [7], adaptive sampling methods [11] or systematic resampling [12]. A recent survey of these issues and different variants of cloning algorithms in discrete and continuous time can be found in [13, Sect. 3].
In this paper we provide a novel perspective on the underlying structure of the cloning algorithm, which is in fact well established in the statistics and applied probability literature on Feynman–Kac models and particle filters [14, 15, 16]. The framework we develop here can be used to generalize rigorous convergence results in [17] to the setting of continuoustime cloning algorithms as introduced in [4]. Full mathematical details of this work are published in [18], and here we focus on describing the underlying approach and report the main convergence results. A second motivation is to use different McKean interpretations of Feynman–Kac semigroups (see Sect. 2.2) to highlight several degrees of freedom in the design of cloningtype algorithms that can be used to improve performance. We illustrate this with the example of current large deviations for the inclusion process (originally introduced in [19]), aspects of which have previously been studied [20]. Current fluctuations in stochastic lattice gases have attracted significant recent research interest (see e.g. [21, 22, 23] and references therein), and are one of the main application areas of cloning algorithms which are particularly challenging. In contrast to previous work in the context of cloning algorithms [9, 10], our mathematical approach does not require a time discretization and works in a very general setting of a jump Markov process on a compact state space. This covers in particular any finite state Markov chain or stochastic lattice gas on a finite lattice.
The paper is organized as follows. In Sect. 2 we introduce notation, the Feynman–Kac semigroup and several representations of the associated nonlinear process. In Sect. 3 we describe different particle approximations including the cloning algorithm, and summarize results published in [18] on convergence properties of estimators based on the latter. In Sect. 4 we describe a modification of the cloning algorithm for a particular class of stochastic lattice gases and apply it to the inclusion process as an example.
2 Mathematical Setting
2.1 Large Deviations and the Tilted Generator
We consider a continuoustime Markov jump process \(\big ( X(t) :t\ge 0\big )\) on a compact state space E. To fix ideas we can think of a finite state Markov chain, such as a stochastic lattice gas on a finite lattice \(\Lambda \) with a fixed number of particles M. Here E is of the form \(S^\Lambda \) with a finite set S of local states (e.g. \(S=\{ 0,1\}\) or \(\{ 0,\ldots ,M\}\)), but continuous settings with compact \(E\subset \mathbb {R}^d\) for any \(d\ge 1\) are also included. One can in principle also generalize to separable and locally compact state spaces, including countable Markov chains and lattice gases on finite lattices with open boundaries. But this would require more effort and complicate not only the proof but also the presentation of the main results for technical reasons which we want to avoid here (see [18] for a more detailed discussion).
As mentioned before, the simplest examples covered by our setting are Markov chains with finite state space E. This includes stochastic particle systems on a finite lattice with periodic or closed boundary conditions such as zerorange or inclusion processes [20, 23, 26], and also processes with open boundaries and bounded local state space such as the exclusion process [3]. Choosing g appropriately and \(h\equiv 0\) the functional \(A_T\) can, for example, measure the empirical particle current across a bond of the lattice or within the whole system up to time T.
2.2 McKean Interpretation of the Feynman–Kac Semigroup

\(c=0\) is the default and simplest choice, but is usually not optimal as discussed in Sect. 4.

\(c=\mu _t^k (\mathcal {V}_k )\) corresponding to the average potential: If the system in state x is less fit than c it jumps to state y chosen from the distribution \(\mu _t^k (dy)\) according to (2.27), and independently, the system jumps to states fitter than c irrespective of its current state according to (2.28).

\(c=\sup _{x\in E} \mathcal {V}_k (x)\) or \(\inf _{x\in E} \mathcal {V}_k (x)\), so that \(\mathcal {L}_{k,\mu ,c}^+(f)(x)\equiv 0\) or \(\mathcal {L}_{k,\mu ,c}^ (f)(x)\equiv 0\), respectively, and only one of the two processes has to be implemented in a simulation.
3 Particle Approximations and the Cloning Algorithm
3.1 Basic Particle Approximations
3.2 Essential Properties of Particle Approximations
Recall that the estimator (3.2) for the principal eigenvalue (2.7) is given by an ergodic integral of the average observable \(F(\underline{x})=\mu ^N (\underline{x})(\mathcal {V}_k )\). With (2.12) \(\mathcal {V}_k \in C_b (E)\) and rates are bounded, so \(\mu ^N (\underline{x})\big ( \Gamma _{k,\mu ^N (\underline{x})} \mathcal {V}_k \big )\) is also bounded and the carré du champ (3.16) vanishes as \(N\rightarrow \infty \). Therefore the martingale \(\mathcal {M}_F^N (t)\) also vanishes^{1} for all \(t\ge 0\), leading to a convergence of the measures \(\mu ^N (\underline{X}_k (t))\rightarrow \mu _t^k (t)\) and also of finite time approximations \(\Lambda _k^N (t)\rightarrow \lambda _k (t)\) as reported in the previous subsection. Due to the timenormalization in (3.2) and the assumed ergodicity, corresponding error bounds hold uniformly in \(t\ge 0\). In summary, bounds on the carré du champ are the main ingredient for the proof of convergence results as explained in detail in [18] and references therein. All above properties up to and including (3.15) are generic requirements for any particle approximation. These particle approximations can differ in their correlation structures and this freedom can be used to construct numerically more efficient particle approximations as discussed in the next subsection. To optimize sampling, particles should ideally evolve in as uncorrelated a fashion as possible; it is not possible to achieve completely independent evolution due to the nonlinearity of the underlying McKean process and resulting selection events and meanfield interactions.
3.3 Cloning Algorithms
Here \(q_k^N(x_i):=\sum _{n=0}^N n^2 p^N_{k,x_i}(n)\) denotes the second moment of the number of clones for the particle i, and we use the decomposition (2.32) where \(\widehat{\Gamma }_k f\) is the carré du champ corresponding to the mutation dynamics \(\widehat{\mathcal {L}}_k\), and \(\Gamma _{k,\mu ^N (\underline{x}),c}^+\) the one corresponding to the cloning part (2.28). This estimate holds, of course, only for N large enough that the cloning event is well defined (see Sect. 5). Note also that \(\Gamma _{k,\mu ^N (\underline{x}),c}^+ f(x)\) is proportional to \((\mathcal {V}_k (x)c)^+\) and with (3.20) \((\mathcal {V}_k (x)c)^+ =0\) implies \(m_k^N (x)=0\) for the expectation of the distribution \(p_{k,x}^N\), leading to the indicator function \(\mathbb {1}(\mathcal {V}_k >c) \in \{0,1\}\).
This is sufficient to carry out the full proof of the convergence results mentioned in Sect. 3 based on results in [17]. This is carried out in [18] in full detail, and here we only report the main result of that work. Recall the bounds (2.12) on \(\mathcal {V}_k\) and the total modified exit rate \(w_k\).
Theorem
Remarks

Choosing the normalization of the potential \(c<\inf _{x\in E} \mathcal {V}_k (x)\) the killing rate in (3.6) vanishes and (3.21) describes the full generator \(\mathcal {L}_k^{N,clone}\) for the cloning algorithm. This is computationally cheaper and simpler to implement, since only the mutation process has to be sampled independently for all particles, and cloning events happen simultaneously. However, as is discussed in Sect. 4, this choice in general reduces the accuracy of the estimator.
 A common choice in the physics literature for the distribution \(p_{k,x_i}^N\) of the clone size event (see e.g. [3, 13]) isso the two adjacent integers to the mean are chosen with appropriate probabilities, which minimizes the variance of the distribution for a given mean. This choice therefore minimizes the contribution of the second moment \(q_k^N\) to the bound for the errors in (3.24) and (3.25), and is also simple to implement in practice.$$\begin{aligned} p_{k,x_i}^N (n)=\left\{ \begin{array}{cl} m_k^N (x_i ) \lfloor m_k^N (x_i )\rfloor , &{} \text{ for } n=\lfloor m_k^N (x_i )\rfloor + 1\\ \lfloor m_k^N (x_i )\rfloor + 1 m_k^N (x_i ), &{} \text{ for } n=\lfloor m_k^N (x_i )\rfloor \\ 0,&{} \text{ otherwise }\end{array}\right. , \end{aligned}$$(3.26)

Due to (3.23), trajectories of individual particles follow the same law as the simple particle approximation (3.6) and therefore the same McKean process as explained in Sect. 2.2 The cloning approach can introduce additional correlations between particles due to large cloning events, which is quantified by the second moment \(q_k^N\) entering the error bounds in (3.24) and (3.25).
3.4 The Cloning Factor
4 Efficiency and Application of Particle Approximations
4.1 Efficiency of Algorithms
The correlations introduced by selection are counteracted by mutation dynamics, which occur independently for each particle and decorrelate the ensemble. The dynamics of correlation structures in cloning algorithms has been discussed in some detail recently in [7, 8, 13, 38], and can be understood in terms of ancestry in the generic population dynamics interpretation. Those results also discuss important nonergodicity effects in the measurement of path properties and the interpretation of particle trajectories, which were already pointed out in [3] and are also a subject of recent research [39]. This poses interesting questions for rigorous mathematical investigations which are left to future work. Here we simply conclude with a numerical test in the next subsections, which supports the intuition that approximation (3.7) with minimal selection rates leads to variance reduction in the relevant estimators compared to the cloning algorithm. Since the selection rate in (3.7) depends on potential differences between pairs, implementation is in general more involved than for algorithms based on (3.6). While the scaling \(t N\log N\) of computational complexity with the size N of the clone ensemble is the same, the prefactor and computational cost in practice may be higher and this has to be traded off against gains in accuracy on a case by case basis. For the examples studied below we find a computationally efficient implementation of (3.7) providing a clear improvement over the standard cloning algorithm, which is the main contribution of this paper in this context. Algorithm (3.8), on the other hand, provides only marginal improvement over (3.7), but cannot be implemented as efficiently in our area of interest.
4.2 Current Large Deviations for Lattice Gases
\(\mathbf {Q}_\mathbf {k} >\mathbf {1}\). We sample the ensemble of N clones at a total rate of \(Q_k \mathcal {W}(\underline{\eta })\), and pick a clone i with probability \(w (\eta ^i )/\mathcal {W}(\underline{\eta })\) and a clone j uniformly at random. If \(w(\eta ^j )<w(\eta ^i )\) we replace \(\eta ^j\) by \(\eta ^i\) with probability \(\frac{Q_k 1}{Q_k}\frac{w(\eta ^i )w(\eta ^j )}{w(\eta ^i )}\). Then we mutate clone i as above, combining the mutation and selection event as in the cloning algorithm.
Remarks
Note that \(Q_k =1\) is equivalent to \(k=0\), which corresponds to the original process with \(\lambda _0 =0\) and does not require any estimation. For \(Q_k >1\) we perform mutation and selection events simultaneously, in analogy to the cloning procedure explained in Sect. 3.3, but can use the efficient algorithm (3.7). For \(Q_k <1\) no mutation or selection event occurs with probability \((1Q_k )\frac{w(\eta ^j )}{w(\eta ^i )} \mathbb {1}(w(\eta ^j )<w(\eta ^i ))\), and a high rate of such rejections is not desirable for computational efficiency. But even for very small values of \(Q_k\) the second factor is usually significantly smaller than 1 (or simply 0), since clone i was picked with probability proportional to \(w(\eta ^i )\) and j uniformly at random.
4.3 Details for the Inclusion Process
5 Discussion
We have presented an analytical approach to cloning algorithms based on McKean interpretations of Feynman–Kac semigroups that have been introduced in the applied probability literature. This allows us to establish rigorous error bounds for the cloning algorithm in continuous time, and to suggest a more efficient variant of the algorithm which can be implemented effectively for current large deviations in stochastic lattice gases. The latter is based on minimizing the selection rate in a standard population dynamics interpretation of particle approximations of nonlinear processes. We include a first application of this idea in the context of inclusion processes, but its full potential will be explored in future more systematic studies of optimization of cloningtype algorithms. The rigorous results fully reported in [18] apply under very general conditions, demanding bounded jump rates and existence of a spectral gap for the underlying jump process. These impose no restriction for lattice gases with a fixed number of particles, which are essentially finite state Markov chains. We anticipate that these techniques can also be applied for more general processes including diffusive, piecewise deterministic, or possibly nonMarkovian dynamics (see [42] for first heuristic results in this direction). Another interesting direction would be a rigorous analysis of the detailed ergodic properties of trajectories in the clone ensemble based on recent results in [7, 8, 39].
Footnotes
Notes
Acknowledgements
This work was supported by The Alan Turing Institute under the EPSRC Grant EP/N510129/1 and The Alan Turing Institute–Lloyds Register Foundation Programme on Datacentric Engineering. AP acknowledges support by the National Group of Mathematical Physics (GNFMINdAM), and by Imperial College together with the Data Science Institute and ThomsonReuters Grant No. 45009023973408.
References
 1.Anderson, J.B.: A randomwalk simulation of the Schrödinger equation: \({H}^+_3\). J. Chem. Phys. 63, 1499 (1975)ADSGoogle Scholar
 2.Grassberger, P.: Go with the winners: a general Monte Carlo strategy. Comput. Phys. Commun. 147(1), 64–70 (2002)ADSMathSciNetzbMATHGoogle Scholar
 3.Giardina, C., Kurchan, J., Peliti, L.: Direct evaluation of largedeviation functions. Phys. Rev. Lett. 96(12), 120603 (2006)ADSGoogle Scholar
 4.Lecomte, V., Tailleur, J.: A numerical approach to large deviations in continuous time. J. Stat. Mech.: Theory Exp. 2007(03), P03004 (2007)Google Scholar
 5.Giardina, C., Kurchan, J., Lecomte, V., Tailleur, J.: Simulating rare events in dynamical processes. J. Stat. Phys. 145(4), 787–811 (2011)ADSMathSciNetzbMATHGoogle Scholar
 6.Jack, R.L., Sollich, P.: Large deviations and ensembles of trajectories in stochastic models. Prog. Theor. Phys. Suppl. 184, 304–317 (2010)ADSzbMATHGoogle Scholar
 7.Nemoto, T., Bouchet, F., Jack, R.L., Lecomte, V.: Populationdynamics method with a multicanonical feedback control. Phys. Rev. E 93, 062123 (2016)ADSMathSciNetGoogle Scholar
 8.Hidalgo, E. G.: Cloning algorithms: from large deviations to population dynamics. Ph.D. thesis, Université Sorbonne Paris Cité — Université Paris Diderot 7, (2018)Google Scholar
 9.Nemoto, T., Guevara Hidalgo, E., Lecomte, V.: Finitetime and finitesize scalings in the evaluation of largedeviation functions: analytical study using a birthdeath process. Phys. Rev. E 95, 012102 (2017)ADSMathSciNetGoogle Scholar
 10.Guevara Hidalgo, E., Nemoto, T., Lecomte, V.: Finitetime and finitesize scalings in the evaluation of largedeviation functions: Numerical approach in continuous time. Phys. Rev. E 95, 062134 (2017)ADSGoogle Scholar
 11.Ferré, G., Touchette, H.: Adaptive sampling of large deviations. J. Stat. Phys. 172(6), 1525–1544 (2018)ADSMathSciNetzbMATHGoogle Scholar
 12.Brewer, T., Clark, S.R., Bradford, R., Jack, R.L.: Efficient characterisation of large deviations using population dynamics. J. Stat. Mech. 2018(5), 053204 (2018)MathSciNetGoogle Scholar
 13.PérezEspigares, C., Hurtado, P. I.: Sampling rare events across dynamical phase transitions. arXiv:1902.01276
 14.Del Moral, P., Miclo, L.: A Moran particle system approximation of FeynmanKac formulae. Stoch. Process. Appl. 86(2), 193–216 (2000)MathSciNetzbMATHGoogle Scholar
 15.Del Moral, P., Pierre, M.: Branching and interacting particle systems approximations of FeynmanKac formulae with applications to nonlinear filtering. In: Azema, J., Emery, M., Ledoux, M., Yor, M. (eds.) Seminaire de probabilites, XXXIV, pp. 1–145. Springer, Berlin (2000)Google Scholar
 16.Del Moral, P.: FeynmanKac Formulae. Springer, New York (2004)zbMATHGoogle Scholar
 17.Rousset, M.: On the control of an interacting particle estimation of Schrödinger ground states. SIAM J. Math. Anal. 38(3), 824–844 (2006)MathSciNetzbMATHGoogle Scholar
 18.Angeli, L., Grosskinsky, S., Johansen, A. M.: Limit theorems for cloning algorithms. under review (arxiv:1902.00509)
 19.Giardinà, C., Kurchan, J., Redig, F., Vafayi, K.: Duality and hidden symmetries in interacting particle systems. J. Stat. Phys. 135(1), 25–55 (2009)ADSMathSciNetzbMATHGoogle Scholar
 20.Chleboun, P., Grosskinsky, S., Pizzoferrato, A.: Current large deviations for partially asymmetric particle systems on a ring. J. Phys. A 51(40), 405001 (2018)MathSciNetzbMATHGoogle Scholar
 21.Lazarescu, A.: The physicist’s companion to current fluctuations: onedimensional bulkdriven lattice gases. J. Phys. A 48(50), 503001 (2015)MathSciNetzbMATHGoogle Scholar
 22.Hurtado, P.I., Espigares, C.P., del Pozo, J.J., Garrido, P.L.: Thermodynamics of currents in nonequilibrium diffusive systems: theory and simulation. J. Stat. Phys. 154(1), 214–264 (2014)ADSMathSciNetzbMATHGoogle Scholar
 23.Chleboun, P., Grosskinsky, S., Pizzoferrato, A.: Lower current large deviations for zerorange processes on a ring. J. Stat. Phys. 167(1), 64–89 (2017)ADSMathSciNetzbMATHGoogle Scholar
 24.Chetrite, Rl, Touchette, H.: Nonequilibrium Markov processes conditioned on large deviations. Annales de L’Institut Henri Poincaré 16, 2005–2057 (2015)ADSMathSciNetzbMATHGoogle Scholar
 25.Nyawo, P.T., Touchette, H.: Large deviations of the current for driven periodic diffusions. Phys. Rev. E 94, 032101 (2016)ADSGoogle Scholar
 26.Harris, R.J., Schütz, G.M.: Fluctuation theorems for stochastic dynamics. J. Stat. Mech. 2007(07), P07020 (2007)MathSciNetGoogle Scholar
 27.Den Hollander, F.: Large deviations, volume 14 of Graduate Texts in Mathematics. American Mathematical Society, Providence (2008)Google Scholar
 28.Dembo, A., Zeitouni, O.: Large deviations techniques and applications, vol. 38. Springer, New York (2009)zbMATHGoogle Scholar
 29.Bertini, L., Faggionato, A., Gabrielli, D.: Large deviations of the empirical flow for continuous time Markov chains. Annales de L’Institut Henri Poincaré 51(3), 867–900 (2015)ADSMathSciNetzbMATHGoogle Scholar
 30.Del Moral, P., Miclo, L.: Particle approximations of Lyapunov exponents connected to Schrödinger operators and FeynmanKac semigroups. ESAIM: Prob Stat 7, 171–208 (2003)zbMATHGoogle Scholar
 31.Del Moral, P.: Mean field simulation for Monte Carlo integration. CRC Press, Boca Raton (2013)zbMATHGoogle Scholar
 32.Baker, J. E.: Adaptive selection methods for genetic algorithms. In: Proceedings of an International Conference on Genetic Algorithms and their applications, pp. 101–111 (1985)Google Scholar
 33.Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89(425), 278–288 (1994)zbMATHGoogle Scholar
 34.Liggett, T.M.: Continuous time Markov processes: an introduction, volume 113 of Graduate Texts in Mathematics. American Mathematical Society, Providence (2010)Google Scholar
 35.Grosskinsky, S., Jatuviriyapornchai, W.: Derivation of meanfield equations for stochastic particle systems. Stoch. Process. Appl. 129(4), 1455–1475 (2019)MathSciNetzbMATHGoogle Scholar
 36.Pra, P. D.: Stochastic meanfield dynamics and applications to life sciences (2017). http://www.cirmmath.fr/ProgWeebly/Renc1555/CoursDaiPra.pdf
 37.Chow, Y.S., Teicher, H.: Probability Theory—Independence, Interchangeability, Martingales, 3rd edn. Springer, New Yok (1998)zbMATHGoogle Scholar
 38.Garrahan, J.P., Jack, R.L., Lecomte, V., Pitard, E., van Duijvendijk, Kristina, van Wijland, F.: Firstorder dynamical phase transition in models of glasses: an approach based on ensembles of histories. J. Phys. A 42(7), 075007 (2009)ADSMathSciNetzbMATHGoogle Scholar
 39.Ray, U., Chan, G.K.L., Limmer, D.T.: Importance sampling large deviations in nonequilibrium steady states. i. J Chem Phys 148(12), 124120 (2018)ADSGoogle Scholar
 40.Grosskinsky, S., Redig, F., Vafayi, K.: Dynamics of condensation in the symmetric inclusion process. Electron. J. Prob. 18(66), 1–23 (2013)MathSciNetzbMATHGoogle Scholar
 41.Bianchi, A., Dommers, S., Giardinà, C.: Metastability in the reversible inclusion process. Electron. J. Prob. 22(70), 1–34 (2017)MathSciNetzbMATHGoogle Scholar
 42.Cavallaro, M., Harris, R.J.: A framework for the direct evaluation of large deviations in nonmarkovian processes. J. Phys. A 49(47), 47LT02 (2016)MathSciNetzbMATHGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.