Exact Variance-Reduced Simulation of Lattice Continuous-Time Markov Chains with Applications in Reaction Networks

  • P. A. MaginnisEmail author
  • M. West
  • G. E. Dullerud
Special Issue: Gillespie and His Algorithms


We propose an algorithm to reduce the variance of Monte Carlo simulation for the class of countable-state, continuous-time Markov chains, or lattice CTMCs. This broad class of systems includes all processes that can be represented using a random-time-change representation, in particular reaction networks. Numerical studies demonstrate order-of-magnitude reduction in MSE for Monte Carlo mean estimates using our approach for both linear and nonlinear systems. The algorithm works by simulating pairs of negatively correlated, identically distributed sample trajectories of the stochastic process and using them to produce variance-reduced, unbiased Monte Carlo estimates, effectively generalizing the method of antithetic variates into the domain of stochastic processes. We define a method to simulate anticorrelated, unit-rate Poisson process paths. We then show how these antithetic Poisson process pairs can be used as the input for random time-change representations of any lattice CTMC, in order to produce anticorrelated trajectories of the desired process. We present three numerical parameter studies. The first examines the algorithm’s performance for the unit-rate Poisson process, and the next two demonstrate the effectiveness of the algorithm in simulating reaction network systems: a gene expression system with affine rate functions and an aerosol particle coagulation system with nonlinear rates. We also prove exact, analytical expressions for the time-resolved and integrated covariance between our antithetic Poisson processes for one technique.


Stochastic simulation Variance reduction Random time-change Antithetic sampling Monte Carlo Reaction networks 



Funding was provided by Air Force Office of Scientific Research (Grant No. FA9550-15-1-0059).


  1. Anderson DF (2007) A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J Chem Phys 127(21):214107CrossRefGoogle Scholar
  2. Anderson DF (2012) An efficient finite difference method for parameter sensitivities of continuous time Markov chains. SIAM J Numer Anal 50(5):2237–2258MathSciNetCrossRefzbMATHGoogle Scholar
  3. Anderson DF, Higham DH (2012) Multilevel Monte Carlo for continuous time Markov chains, with applications in biochemical kinetics. Multiscale Model Simul 10(1):146–179MathSciNetCrossRefzbMATHGoogle Scholar
  4. Anderson DF, Kurtz TG (2015) Stochastic analysis of biochemical systems. Springer, BerlinCrossRefzbMATHGoogle Scholar
  5. Banks HT, Hu S, Joyner M, Broido A, Canter B, Gayvert K, Link K (2012) A comparison of computational efficiencies of stochastic algorithms in terms of two infection models. Math Biosci Eng 9(3):487–526MathSciNetCrossRefzbMATHGoogle Scholar
  6. Bratsun D, Volfson D, Tsimring LS, Hasty J (2005) Delay-induced stochastic oscillations in gene regulation. PNAS 102(41):14593–14598. CrossRefGoogle Scholar
  7. Briat C, Khammash M (2012) Computer control of gene expression: robust setpoint tracking of protein mean and variance using integral feedback. In: Proceedings of the IEEE conference on decision and controlGoogle Scholar
  8. Cai X (2007) Exact stochastic simulation of coupled chemical reactions with delays. J Chem Phys 126(12):124108. CrossRefGoogle Scholar
  9. Cao Y, Gillespie DT, Petzold LR (2005) The slow-scale stochastic simulation algorithm. J Chem Phys 122(1):014116. CrossRefGoogle Scholar
  10. Ethier SN, Kurtz TG (1986) Markov processes: characterization and convergence. Wiley, HobokenCrossRefzbMATHGoogle Scholar
  11. Gibson M, Bruck J (2000) Efficient exact stochastic simulation of chemical systems with many species and many channels. J Phys Chem A 105:1876CrossRefGoogle Scholar
  12. Giles MB (2008) Multilevel Monte Carlo path simulation. Oper Res 56:607–617MathSciNetCrossRefzbMATHGoogle Scholar
  13. Gillespie DT (1975) An exact method for numerically simulating the stochastic coalescence process in a cloud. J Atmos Sci 32:1977–1989CrossRefGoogle Scholar
  14. Gillespie DT (1976) A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J Comput Phys 22:403–434MathSciNetCrossRefGoogle Scholar
  15. Gillespie DT (2001) Approximate accelerated stochastic simulation of chemically reacting systems. J Chem Phys 115(4):1716–1733CrossRefGoogle Scholar
  16. Gillespie DT, Hellander A, Petzold LR (2013) Perspective: Stochastic algorithms for chemical kinetics. J Chem Phys 138(17):170901CrossRefGoogle Scholar
  17. Glynn PW, Iglehart DL (1989) Importance sampling for stochastic simulation. Manag Sci 35:1367–1392. MathSciNetCrossRefzbMATHGoogle Scholar
  18. Maginnis PA (2011) Variance reduction for Poisson and Markov jump processes. Master’s thesis, University of Illinois at Urbana-ChampaignGoogle Scholar
  19. Maginnis PA, West M, Dullerud GE (2016) Variance-reduced simulation of lattice discrete-time Markov chains with applications in reaction networks. J Comput Phys 322:400–414. MathSciNetCrossRefzbMATHGoogle Scholar
  20. McAdams HH, Arkin A (1997) Stochastic mechanisms in gene expression. Proc Natl Acad Sci 94(3):814–819CrossRefGoogle Scholar
  21. Rathinam M, Sheppard PW, Khammash M (2010) Efficient computation of parameter sensitivities of discrete stochastic chemical reaction networks. J Chem Phys 132(3):034103CrossRefGoogle Scholar
  22. Riemer N, West M, Zaveri RA, Easter RC (2009) Simulating the evolution of soot mixing state with a particle-resolved aerosol model. J Geophys Res 114:D09202. Google Scholar
  23. Robert CP, Casella G (2004) Monte Carlo statistical methods, 2nd edn. Springer, BerlinCrossRefzbMATHGoogle Scholar
  24. Seinfeld JH, Pandis SN (2006) Atmospheric chemistry and physics. Wiley, HobokenGoogle Scholar
  25. Villen-Altamirano M (2012) Rare event simulation: The RESTART methods, In: Proceedings of the international conference on high performance computing and simulation, pp 32–41Google Scholar
  26. Whitt W (1976) Bivariate distributions with given marginals. Ann Stat 4(6):1280–1289MathSciNetCrossRefzbMATHGoogle Scholar
  27. Wilkinson DJ (2011) Stochastic modelling for systems biology. CRC Press, Boca RatonzbMATHGoogle Scholar

Copyright information

© Society for Mathematical Biology 2019

Authors and Affiliations

  1. 1.Department of Mechanical Science and EngineeringUniversity of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations