Abstract
We consider the problem of conditioning a Markov process on a rare event and of representing this conditioned process by a conditioning-free process, called the effective or driven process. The basic assumption is that the rare event used in the conditioning is a large deviation-type event, characterized by a convex rate function. Under this assumption, we construct the driven process via a generalization of Doob’s h-transform, used in the context of bridge processes, and show that this process is equivalent to the conditioned process in the long-time limit. The notion of equivalence that we consider is based on the logarithmic equivalence of path measures and implies that the two processes have the same typical states. In constructing the driven process, we also prove equivalence with the so-called exponential tilting of the Markov process, often used with importance sampling to simulate rare events and giving rise, from the point of view of statistical mechanics, to a nonequilibrium version of the canonical ensemble. Other links between our results and the topics of bridge processes, quasi-stationary distributions, stochastic control, and conditional limit theorems are mentioned.
Article PDF
Similar content being viewed by others
References
Doob J.L.: Conditional Brownian motion and the boundary limits of harmonic functions. Bull. Soc. Math. Fr. 85, 431 (1957)
Doob J.L.: Classical Potential Theory and Its Probabilistic Counterpart. Springer, New York (1984)
Rogers L.C.G., Williams D.: Diffusions, Markov Processes and Martingales. Cambridge University Press, Cambridge (2000)
Baudoin F.: Conditioned stochastic differential equations: Theory examples and application to finance. Stoch. Proc. Appl. 100, 109 (2002)
Gasbarra D., Sottinen T., Valkeila E.: Gaussian bridges. In: Benth, F.E., Nunno, G.D., Lindstrøm, T., Øksendal, B., Zhang, T. (eds.) Stochastic Analysis and Applications, Abel Symposia, vol. 2, pp. 361–382. Springer, Berlin (2007)
Sottinen T., Yazigi A.: Generalized Gaussian bridges. Stoch. Proc. Appl. 124, 3084 (2014)
Schrödinger, E.: Über die Umkehrung der Naturgesetze. Sitzungsber. Preuss. Akad. Wiss. Phys. Math. Kl., 144–153 (1931)
Schrödinger E.: Sur la théorie relativiste de l’électron et l’interprétation de la mécanique quantique. Ann. Inst. Henri Poincaré 2, 269 (1932)
Jamison B.: The Markov processes of Schrödinger. Z. Wahrsch. Verw. Gebiete 32, 323 (1975)
Zambrini J.C.: Stochastic mechanics according to Schrödinger. Phys. Rev. A 33, 1532 (1986)
Aebi R.: Schödinger Diffusion Processes. Birkhäuser, Basel (1996)
Darroch J.N., Seneta E.: On quasi-stationary distributions in absorbing discrete-time finite Markov chains. J. Appl. Probab. 2, 88 (1965)
Darroch J.N., Seneta E.: On quasi-stationary distributions in absorbing continuous-time finite Markov chains. J. Appl. Probab. 4, 192 (1967)
Villemonais D.: Quasi-stationary distributions and population processes. Prob. Surveys 9, 340 (2012)
Collet P., Martínez S., Martín J.S.: Quasi-Stationary Distributions. Springer, New York (2013)
van Doorn E.A., Pollett P.K.: Quasi-stationary distributions for discrete-state models. Eur. J. Oper. Res. 230, 1 (2013)
Vasicek O.A.: A conditional law of large numbers. Ann. Probab. 8, 142 (1980)
van Campenhout J.M., Cover T.M.: Maximum entropy and conditional probability. IEEE Trans. Inform. Theory 27, 483 (1981)
Cover T.M., Thomas J.A.: Elements of Information Theory. Wiley, New York (1991)
Csiszar I.: Sanov property, generalized I-projection and a conditional limit theorem. Ann. Probab. 12, 768 (1984)
Dembo A., Zeitouni O.: Large Deviations Techniques and Applications, 2nd edn. Springer, New York (1998)
Diaconis P., Freedman D.: A dozen de Finetti-style results in search of a theory. Ann. Inst. Henri Poincare B Probab. Stat. 23, 397 (1987)
Diaconis P., Freedman D.A.: Conditional limit theorems for exponential families and finite versions of de Finetti’s theorem. J. Theor. Probab. 1, 381 (1988)
Dembo A., Zeitouni O.: Refinements of the Gibbs conditioning principle. Probab. Theor. Relat. Fields 104, 1 (1996)
Csiszár I., Cover T.M., Choi B.-S.: Conditional limit theorems under Markov conditioning. IEEE Trans. Inform. Theory 33, 788 (1987)
Borkar V.S., Juneja S., Kherani A.A.: Peformance analysis conditioned on rare events: an adaptive simulation scheme. Commun. Inform. Syst. 3, 259 (2004)
Bucklew J.A.: Introduction to Rare Event Simulation. Springer, New York (2004)
Feller W.: An Introduction to Probability Theory and its Applications, vol. II. Wiley, New York (1970)
Giardina C., Kurchan J., Peliti L.: Direct evaluation of large-deviation functions. Phys. Rev. Lett. 96, 120603 (2006)
Lecomte V., Tailleur J.: A numerical approach to large deviations in continuous time. J. Stat. Mech. 2007, P03004 (2007)
Tailleur J., Lecomte V.: Simulation of large deviation functions using population dynamics. In: Marro, J., Garrido, P.L., Hurtado, P.I. (eds.) Modeling and Simulation of New Materials: Proceedings of Modeling and Simulation of New Materials, vol. 1091, pp. 212–219. AIP, Melville (2009)
Moral P.D.: Feynman–Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004)
Fleming W.H.: Exit probabilities and optimal stochastic control. Appl. Math. Optim. 4, 329 (1978)
Fleming W.H.: Logarithmic transformations and stochastic control. In: Fleming, W.H., Gorostiza, L.G. (eds.) Advances in Filtering and Optimal Stochastic Control, Lecture Notes in Control and Information Sciences, vol. 42, pp. 131–141. Springer, New York (1982)
Fleming W.H.: A stochastic control approach to some large deviations problems. In: Dolcetto, I.C., Fleming, W.H., Zolezzi, T. (eds.) Recent Mathematical Methods in Dynamic Programming, vol. 1119, pp. 52–66. Springer, New York (1985)
Fleming W.H., Sheu S.-J.: Stochastic variational formula for fundamental solutions of parabolic PDE. Appl. Math. Optim. 13, 193 (1985)
Kac M.: On distributions of certain Wiener functionals. Trans. Am. Math. Soc. 65, 1 (1949)
Stroock D.W., Varadhan R.S.: Multidimensional Diffusion Processes. Springer, New York (1979)
Revuz D., Yor M.: Continuous Martingales and Brownian Motion, 3rd edn. Springer, Berlin (1999)
Fleming W.H., Soner H.M.: Controlled Markov Processes and Viscosity Solutions, Stochastic Modelling and Applied Probability, vol. 25. Springer, New York (2006)
Sheu S.-J.: Stochastic control and principal eigenvalue. Stochastics 11, 191 (1984)
Fleming W.H., Sheub S.J., Soner H.M.: A remark on the large deviations of an ergodic Markov process. Stochastics 22, 187 (1987)
Fleming W.H., Sheu S.-J.: Asymptotics for the principal eigenvalue and eigenfunction of a nearly first-order operator with large potential. Ann. Probab. 25, 1953 (1997)
Fleming W.H., McEneaney W.M.: Risk sensitive optimal control and differential games. In: Duncan, T.E., Pasik-Duncan, B. (eds.) Stochastic Theory and Adaptive Control, Lecture Notes in Control and Information Sciences, vol. 184, pp. 185–197. Springer, New York (1992)
Fleming W., McEneaney W.: Risk-sensitive control on an infinite time horizon. SIAM J. Control Optim. 33, 1881 (1995)
Dupuis P., McEneaney W.: Risk-sensitive and robust escape criteria. SIAM J. Control Optim. 35, 2021 (1997)
Nemoto T., Sasa S.-I.: Thermodynamic formula for the cumulant generating function of time-averaged current. Phys. Rev. E 84, 061113 (2011)
Nemoto T., Sasa S.-I.: Variational formula for experimental determination of high-order correlations of current fluctuations in driven systems. Phys. Rev. E 83, 030105 (2011)
Nemoto T., Sasa S.-I.: Computation of large deviation statistics via iterative measurement-and-feedback procedure. Phys. Rev. Lett. 112, 090602 (2014)
Touchette H.: The large deviation approach to statistical mechanics. Phys. Rep. 478, 1 (2009)
Freidlin M.I., Wentzell A.D.: Random Perturbations of Dynamical Systems, Grundlehren der Mathematischen Wissenschaften, vol. 260. Springer, New York (1984)
Gardiner C.W.: Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences, Springer Series in Synergetics, vol. 13, 2nd edn. Springer, New York (1985)
van Kampen N.G.: Stochastic Processes in Physics and Chemistry. North-Holland, Amsterdam (1992)
Risken H.: The Fokker–Planck equation: Methods of solution and applications, 3rd edn. Springer, Berlin (1996)
Evans R.M.L.: Rules for transition rates in nonequilibrium steady states. Phys. Rev. Lett. 92, 150601 (2004)
Evans R.M.L.: Detailed balance has a counterpart in non-equilibrium steady states. J. Phys. A Math. Gen. 38, 293 (2005)
Evans R.M.L.: Statistical physics of shear flow: a non-equilibrium problem. Contemp. Phys. 51, 413 (2010)
Dellago C., Bolhuis P.G., Geissler P.L.: Transition path sampling. Adv. Chem. Phys. 123, 1 (2003)
Dellago C., Bolhuis P.G., Geissler P.L.: Transition path sampling methods. In: Ferrario, M., Ciccotti, G., Binder, K. (eds.) Computer Simulations in Condensed Matter Systems: From Materials to Chemical Biology, vol. 1, Lecture Notes in Physics, vol. 703, Springer, New York (2006)
Dellago C., Bolhuis P.: Transition path sampling and other advanced simulation techniques for rare events. In: Holm, C., Kremer, K. (eds.) Advanced Computer Simulation Approaches for Soft Matter Sciences III, Advances in Polymer Science, vol. 221, pp. 167–233. Springer, Berlin (2009)
Vanden-Eijnden E.: Transition path theory. In: Ferrario, M., Ciccotti, G., Binder, K. (eds.) Computer Simulations in Condensed Matter Systems From Materials to Chemical Biology, vol. 1, Lecture Notes in Physics, vol. 703, pp. 453–493. Springer, New York (2006)
Chandler D., Garrahan J.P.: Dynamics on the way to forming glass: bubbles in space-time. Annu. Rev. Chem. Phys. 61, 191 (2010)
Hedges L.O., Jack R.L., Garrahan J.P., Chandler D.: Dynamic order-disorder in atomistic models of structural glass formers. Science 323, 1309 (2009)
Jack R.L., Sollich P.: Large deviations and ensembles of trajectories in stochastic models. Prog. Theor. Phys. Suppl. 184, 304 (2010)
Lecomte V., Appert-Rolland C., van Wijland F.: Chaotic properties of systems with Markov dynamics. Phys. Rev. Lett. 95, 010601 (2005)
Lecomte V., Appert-Rolland C., van Wijland F.: Thermodynamic formalism for systems with Markov dynamics. J. Stat. Phys. 127, 51 (2007)
Garrahan J.P., Lesanovsky I.: Thermodynamics of quantum jump trajectories. Phys. Rev. Lett. 104, 160601 (2010)
Garrahan J.P., Armour A.D., Lesanovsky I.: Quantum trajectory phase transitions in the micromaser. Phys. Rev. E 84, 021115 (2011)
Ates C., Olmos B., Garrahan J.P., Lesanovsky I.: Dynamical phases and intermittency of the dissipative quantum ising model. Phys. Rev. A 85, 043620 (2012)
Genway S., Garrahan J.P., Lesanovsky I., Armour A.D.: Phase transitions in trajectories of a superconducting single-electron transistor coupled to a resonator. Phys. Rev. E 85, 051122 (2012)
Hickey J.M., Genway S., Lesanovsky I., Garrahan J.P.: Thermodynamics of quadrature trajectories in open quantum systems. Phys. Rev. A 86, 063824 (2012)
Chetrite R., Touchette H.: Nonequilibrium microcanonical and canonical ensembles and their equivalence. Phys. Rev. Lett. 111, 120601 (2013)
Karatzas I., Shreve S.: Methods of Mathematical Finance, Stochastic Modelling and Applied Probability, vol. 39. Springer, New York (1998)
Berg H.C.: Random Walks in Biology. Princeton University Press, Princeton (1993)
Spohn H.: Large Scale Dynamics of Interacting Particles. Springer, Berlin (1991)
Kipnis C., Landim C.: Scaling Limits of Interacting Particle Systems, Grundlheren der mathematischen Wissenschaften, vol. 320. Springer, Berlin (1999)
Liggett T.M.: Interacting Particle Systems. Springer, New York (2004)
Derrida B.: Non-equilibrium steady states: fluctuations and large deviations of the density and of the current. J. Stat. Mech. 2007, P07023 (2007)
Bertini L., Sole A.D., Gabrielli D., Jona-Lasinio G., Landim C.: Stochastic interacting particle systems out of equilibrium. J. Stat. Mech. 2007, P07014 (2007)
Jacobs K.: Stochastic Processes for Physicists: Understanding Noisy Systems. Cambridge University Press, Cambridge (2010)
Krapivsky P.L., Redner S., Ben-Naim E.: A Kinetic View of Statistical Physics. Cambridge University Press, Cambridge (2010)
Nelson E.: Dynamical Theories of Brownian Motion. Princeton University Press, Princeton (1967)
Anderson W.J.: Continuous-Time Markov Chains: An Applications-Oriented Approach, Series in Statistics. Springer, New York (1991)
Chung K.L., Walsh J.B.: Markov Processes, Brownian Motion, and Time Symmetry, 2nd edn. Springer, Berlin (2005)
Applebaum D.: Lévy Processes and Stochastic Calculus, 2nd edn. Cambridge University Press, Cambridge (2009)
Sato K.: Lévy Processes and Infinite Divisibility, Studies in Advanced Mathematics. Cambridge University Press, Cambridge (1999)
Hörmander L.: Hypoelliptic second order differential equations. Acta. Math. 119, 147 (1967)
Malliavin, P.: Stochastic calculus of variations and hypoelliptic operators. In: Proc. Inter. Symp. Stoch. Diff. Equations, Kyoto, pp. 195–263. Wiley, New York (1978)
Lebowitz J.L., Spohn H.: A Gallavotti–Cohen-type symmetry in the large deviation functional for stochastic dynamics. J. Stat. Phys. 95, 333 (1999)
Maes, C., Netočný, K.: Canonical structure of dynamical fluctuations in mesoscopic nonequilibrium steady states. Europhys. Lett. 82 (2008)
Baiesi M., Maes C., Wynants B.: Fluctuations and response of nonequilibrium states. Phys. Rev. Lett. 103, 010602 (2009)
Jarzynski C.: Equilibrium free-energy differences from nonequilibrium measurements: a master-equation approach. Phys. Rev. E 56, 5018 (1997)
Crooks G.E.: Nonequilibrium measurements of free energy differences for microscopically reversible markovian systems. J. Stat. Phys. 90, 1481 (1998)
Sekimoto K.: Stochastic Energetics, Lect. Notes Phys., vol. 799. Springer, New York (2010)
Maes C., Netočný K., Wynants B.: Steady state statistics of driven diffusions. Phys. A 387, 2675 (2008)
Chernyak V., Chertkov M., Malinin S., Teodorescu R.: Non-equilibrium thermodynamics and topology of currents. J. Stat. Phys. 137, 109 (2009)
Basu A., Bhattacharyya T., Borkar V.S.: A learning algorithm for risk-sensitive cost. Math. Oper. Res. 33, 880 (2008)
den Hollander F.: Large Deviations. Fields Institute Monograph. Math. Soc., Providence (2000)
Ellis R.S.: Entropy, Large Deviations, and Statistical Mechanics. Springer, New York (1985)
Fortelle A.D.L.: Large deviation principle for Markov chains in continuous time. Probab. Inform. Trans. 37, 120 (2001)
Bertini, L., Faggionato, A., Gabrielli, D.: Large deviations of the empirical flow for continuous time Markov chains (2012). arXiv:1210.2004
Roynette B., Yor M.: Penalising Brownian Paths. Springer, New York (2009)
Lewis J.T., Pfister C.-E., Sullivan G.W.: The equivalence of ensembles for lattice systems: some examples and a counterexample. J. Stat. Phys. 77, 397 (1994)
Lewis J.T., Pfister C.-E., Sullivan W.G.: Entropy, concentration of probability and conditional limit theorem. Markov Proc. Relat. Fields 1, 319 (1995)
Touchette, H.: Equivalence and nonequivalence of ensembles: Thermodynamic, macrostate, and measure levels (2014). arXiv:1403.6608.
MacCluer C.R.: The many proofs and applications of Perron’s theorem. SIAM Rev. 42, 487 (2000)
Krein M.G., Rutman M.A.: Linear operators leaving a cone in a Banach space. Am. Math. Soc. Transl. 26, 128 (1950)
Lecomte, V.: Thermodynamique des histoires et fluctuations hors d’équilibre. Ph.D. thesis, Université Paris VII (2007)
Chetrite R., Gupta S.: Two refreshing views of fluctuation theorems through kinematics elements and exponential martingale. J. Stat. Phys. 143, 543 (2011)
Kunita H.: Absolute continuity of Markov processes and generators. Nagoya Math. J. 36, 1 (1969)
Itô K., Watanabe S.: Transformation of Markov processes by multiplicative functionals. Ann. Inst. Fourier 15, 13 (1965)
Palmowski Z., Rolski T.: A technique for exponential change of measure for Markov processes. Bernoulli 8, 767 (2002)
Diaconis P., Miclo L.: On characterizations of Metropolis type algorithms in continuous time. ALEA Lat. Am. J. Probab. Math. Stat. 6, 199 (2009)
Meyer P.A., Zheng W.A.: Construction de processus de Nelson reversibles. In: Azéma, J., Yor, M. (eds.) Séminaire de Probabilités XIX 1983/84, Lecture Notes in Mathematics, vol. 1123, pp. 12–26. Springer, Berlin (1985)
Bernard W., Callen H.B.: Irreversible thermodynamics of nonlinear processes and noise in driven systems. Rev. Mod. Phys. 31, 1017 (1959)
Callen H.B., Welton T.A.: Irreversibility and generalized noise. Phys. Rev. 83, 34 (1951)
Kubo R.: The fluctuation–dissipation theorem. Rep. Prog. Phys. 29, 255 (1966)
Lippiello E., Corberi F., Sarracino A., Zannetti M.: Nonlinear response and fluctuation–dissipation relations. Phys. Rev. E 78, 041120 (2008)
Speck T., Seifert U.: Restoring a fluctuation–dissipation theorem in a nonequilibrium steady state. Europhys. Lett. 74, 391 (2006)
Baiesi M., Maes C., Wynants B.: Nonequilibrium linear response for Markov dynamics I: jump processes and overdamped diffusions. J. Stat. Phys. 137, 1094 (2009)
Baiesi M., Boksenbojm E., Maes C., Wynants B.: Nonequilibrium linear response for Markov dynamics II: Inertial dynamics. J. Stat. Phys. 139, 492 (2010)
Seifert U., Speck T.: Fluctuation–dissipation theorem in nonequilibrium steady states. Europhys. Lett. 89, 10007 (2010)
Chetrite R., Falkovich G., Gawedzki K.: Fluctuation relations in simple examples of non-equilibrium steady states. J. Stat. Mech. 2008, P08005 (2008)
Chetrite R., Gawedzki K.: Eulerian and Lagrangian pictures of non-equilibrium diffusions. J. Stat. Phys. 137, 890 (2009)
Jarzynski C.: Nonequilibrium equality for free energy differences. Phys. Rev. Lett. 78, 2690 (1997)
Evans D.J., Cohen E.G.D., Morriss G.P.: Probability of second law violations in shearing steady states. Phys. Rev. Lett. 71, 2401 (1993)
Gallavotti G., Cohen E.G.D.: Dynamical ensembles in nonequilibrium statistical mechanics. Phys. Rev. Lett. 74, 2694 (1995)
Gallavotti G., Cohen E.G.D.: Dynamical ensembles in stationary states. J. Stat. Phys. 80, 931 (1995)
Dyson F.J.: A Brownian motion model for the eigenvalues of a random matrix. J. Math. Phys. 3, 1191 (1962)
Grabiner D.J.: Brownian motion in a Weyl chamber, non-colliding particles, and random matrices. Ann. Inst. Henri Poincare B Probab. Stat. 35, 177 (1999)
O’Connell N.: Random matrices, non-colliding processes and queues. In: Azéma, J., Émery, M., Ledoux, M., Yor, M. (eds.) Séminaire de Probabilités XXXVI, Lecture Notes in Mathematics, vol. 1801, pp. 165–182. Springer, Berlin (2003)
Touchette H.: Ensemble equivalence for general many-body systems. Europhys. Lett. 96, 50010 (2011)
Rockafellar R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Gingrich, T.R., Vaikuntanathan, S., Geissler, P.L.: Heterogeneity-induced large deviations in activity and (in some cases) entropy production (2014). arXiv:1406.3311
Merhav N., Kafri Y.: Bose–Einstein condensation in large deviations with applications to information systems. J. Stat. Mech. 2010, P02011 (2010)
Harris R.J., Popkov V., Schütz G.M.: Dynamics of instantaneous condensation in the ZRP conditioned on an atypical current. Entropy 15, 5065 (2013)
Szavits-Nossan J., Evans M.R., Majumdar S.N.: Constraint-driven condensation in large fluctuations of linear statistics. Phys. Rev. Lett. 112, 020602 (2014)
Baule A., Evans R.M.L.: Invariant quantities in shear flow. Phys. Rev. Lett. 101, 240601 (2008)
Simha A., Evans R.M.L., Baule A.: Properties of a nonequilibrium heat bath. Phys. Rev. E 77, 031117 (2008)
Baule A., Evans R.M.L.: Nonequilibrium statistical mechanics of shear flow: invariant quantities and current relations. J. Stat. Mech. 2010, P03030 (2010)
Popkov V., Schütz G.M., Simon D.: ASEP on a ring conditioned on enhanced flux. J. Stat. Mech. 2010, P10007 (2010)
Popkov V., Schütz G.: Transition probabilities and dynamic structure function in the ASEP conditioned on strong flux. J. Stat. Phys. 142, 627 (2011)
Jack R.L., Sollich P.: Large deviations of the dynamical activity in the East model: Analysing structure in biased trajectories. J. Phys. A Math. Theor. 47, 015003 (2014)
Knežević M., Evans R.M.L.: Numerical comparison of a constrained path ensemble and a driven quasisteady state. Phys. Rev. E 89, 012132 (2014)
Chen J., Li H., Jian S.: Some limit theorems for absorbing Markov processes. J. Phys. A Math. Theor. 45, 345003 (2012)
Chen J., Deng X.: Large deviations and related problems for absorbing Markov chains. Stoch. Proc. Appl. 123, 2398 (2013)
Bauer, M., Cornu, F.: Affinity and fluctuations in a mesoscopic noria (2014). arXiv:1402.2422
Miller H.D.: A convexity property in the theory of random variables defined on a finite Markov chain. Ann. Math. Stat. 32, 1260 (1961)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Christian Maes.
Rights and permissions
About this article
Cite this article
Chetrite, R., Touchette, H. Nonequilibrium Markov Processes Conditioned on Large Deviations. Ann. Henri Poincaré 16, 2005–2057 (2015). https://doi.org/10.1007/s00023-014-0375-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00023-014-0375-8