Abstract
We propose an algorithm to reduce the variance of Monte Carlo simulation for the class of countable-state, continuous-time Markov chains, or lattice CTMCs. This broad class of systems includes all processes that can be represented using a random-time-change representation, in particular reaction networks. Numerical studies demonstrate order-of-magnitude reduction in MSE for Monte Carlo mean estimates using our approach for both linear and nonlinear systems. The algorithm works by simulating pairs of negatively correlated, identically distributed sample trajectories of the stochastic process and using them to produce variance-reduced, unbiased Monte Carlo estimates, effectively generalizing the method of antithetic variates into the domain of stochastic processes. We define a method to simulate anticorrelated, unit-rate Poisson process paths. We then show how these antithetic Poisson process pairs can be used as the input for random time-change representations of any lattice CTMC, in order to produce anticorrelated trajectories of the desired process. We present three numerical parameter studies. The first examines the algorithm’s performance for the unit-rate Poisson process, and the next two demonstrate the effectiveness of the algorithm in simulating reaction network systems: a gene expression system with affine rate functions and an aerosol particle coagulation system with nonlinear rates. We also prove exact, analytical expressions for the time-resolved and integrated covariance between our antithetic Poisson processes for one technique.
Similar content being viewed by others
References
Anderson DF (2007) A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J Chem Phys 127(21):214107
Anderson DF (2012) An efficient finite difference method for parameter sensitivities of continuous time Markov chains. SIAM J Numer Anal 50(5):2237–2258
Anderson DF, Higham DH (2012) Multilevel Monte Carlo for continuous time Markov chains, with applications in biochemical kinetics. Multiscale Model Simul 10(1):146–179
Anderson DF, Kurtz TG (2015) Stochastic analysis of biochemical systems. Springer, Berlin
Banks HT, Hu S, Joyner M, Broido A, Canter B, Gayvert K, Link K (2012) A comparison of computational efficiencies of stochastic algorithms in terms of two infection models. Math Biosci Eng 9(3):487–526
Bratsun D, Volfson D, Tsimring LS, Hasty J (2005) Delay-induced stochastic oscillations in gene regulation. PNAS 102(41):14593–14598. https://doi.org/10.1073/pnas.0503858102
Briat C, Khammash M (2012) Computer control of gene expression: robust setpoint tracking of protein mean and variance using integral feedback. In: Proceedings of the IEEE conference on decision and control
Cai X (2007) Exact stochastic simulation of coupled chemical reactions with delays. J Chem Phys 126(12):124108. https://doi.org/10.1063/1.2710253
Cao Y, Gillespie DT, Petzold LR (2005) The slow-scale stochastic simulation algorithm. J Chem Phys 122(1):014116. https://doi.org/10.1063/1.1824902
Ethier SN, Kurtz TG (1986) Markov processes: characterization and convergence. Wiley, Hoboken
Gibson M, Bruck J (2000) Efficient exact stochastic simulation of chemical systems with many species and many channels. J Phys Chem A 105:1876
Giles MB (2008) Multilevel Monte Carlo path simulation. Oper Res 56:607–617
Gillespie DT (1975) An exact method for numerically simulating the stochastic coalescence process in a cloud. J Atmos Sci 32:1977–1989
Gillespie DT (1976) A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22:403–434
Gillespie DT (2001) Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115(4):1716–1733
Gillespie DT, Hellander A, Petzold LR (2013) Perspective: Stochastic algorithms for chemical kinetics. J Chem Phys 138(17):170901
Glynn PW, Iglehart DL (1989) Importance sampling for stochastic simulation. Manag Sci 35:1367–1392. https://doi.org/10.1287/mnsc.35.11.1367
Maginnis PA (2011) Variance reduction for Poisson and Markov jump processes. Master’s thesis, University of Illinois at Urbana-Champaign
Maginnis PA, West M, Dullerud GE (2016) Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks. J Comput Phys 322:400–414. https://doi.org/10.1016/j.jcp.2016.06.019
McAdams HH, Arkin A (1997) Stochastic mechanisms in gene expression. Proc Natl Acad Sci 94(3):814–819
Rathinam M, Sheppard PW, Khammash M (2010) Efficient computation of parameter sensitivities of discrete stochastic chemical reaction networks. J Chem Phys 132(3):034103
Riemer N, West M, Zaveri RA, Easter RC (2009) Simulating the evolution of soot mixing state with a particle-resolved aerosol model. J Geophys Res 114:D09202. https://doi.org/10.1029/2008JD011073
Robert CP, Casella G (2004) Monte Carlo statistical methods, 2nd edn. Springer, Berlin
Seinfeld JH, Pandis SN (2006) Atmospheric chemistry and physics. Wiley, Hoboken
Villen-Altamirano M (2012) Rare event simulation: The RESTART methods, In: Proceedings of the international conference on high performance computing and simulation, pp 32–41
Whitt W (1976) Bivariate distributions with given marginals. Ann Stat 4(6):1280–1289
Wilkinson DJ (2011) Stochastic modelling for systems biology. CRC Press, Boca Raton
Acknowledgements
Funding was provided by Air Force Office of Scientific Research (Grant No. FA9550-15-1-0059).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Maginnis, P.A., West, M. & Dullerud, G.E. Exact Variance-Reduced Simulation of Lattice Continuous-Time Markov Chains with Applications in Reaction Networks. Bull Math Biol 81, 3159–3184 (2019). https://doi.org/10.1007/s11538-019-00576-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11538-019-00576-2