Statistics and Computing

, Volume 28, Issue 1, pp 131–144 | Cite as

Inference and rare event simulation for stopped Markov processes via reverse-time sequential Monte Carlo

  • Jere KoskelaEmail author
  • Dario Spanò
  • Paul A. Jenkins


We present a sequential Monte Carlo algorithm for Markov chain trajectories with proposals constructed in reverse time, which is advantageous when paths are conditioned to end in a rare set. The reverse time proposal distribution is constructed by approximating the ratio of Green’s functions in Nagasawa’s formula. Conditioning arguments can be used to interpret these ratios as low-dimensional conditional sampling distributions of some coordinates of the process given the others. Hence, the difficulty in designing SMC proposals in high dimension is greatly reduced. Empirically, our method outperforms an adaptive multilevel splitting algorithm in three examples: estimating an overflow probability in a queueing model, the probability that a diffusion follows a narrowing corridor, and the initial location of an infection in an epidemic model on a network.


Intractable likelihood Rare event simulation Sequential Monte Carlo Stopped Markov process Time reversal 

Mathematics Subject Classification

62M05 60J20 60J22 



The authors are grateful to Adam Johansen and Murray Pollock for fruitful conversations on rare event simulation and to Ayalvadi Ganesh for pointing out the problem of inferring initial locations on networks.


  1. Anantharam, V., Heidelberg, P., Tsoucas, P.: Analysis of rare events in continuous time Markov chains via time reversal and fluid approximation. IBM Research Report #RC 16280 (1990)Google Scholar
  2. Barndorff-Nielsen, O.: Exponentially decreasing distributions for the logarithm of particle size. Proc. R. Soc. Lond. A 353, 401–419 (1977)CrossRefGoogle Scholar
  3. Barndorff-Nielsen, O.: Hyperbolic distributions and distributions on hyperbolae. Scand. J. Stat. 5, 151–157 (1978)MathSciNetzbMATHGoogle Scholar
  4. Bibbona, E., Ditlevsen, S.: Estimation in discretely observed Markov processes killed at a threshold. Scand. J. Stat. 40, 274–293 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  5. Bibby, B.M., Sørensen, M.: Hyperbolic processes in finance. In: Rachev, S. (ed.) Handbook of Heavy Tailed Distributions in Finance, pp. 211–248. Elsevier, Amsterdam (2003)CrossRefGoogle Scholar
  6. Birkner, M., Blath, J.: Computing likelihoods for coalescents with multiple collisions in the infinitely many sites model. J. Math. Biol. 57(3), 435–463 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  7. Birkner, M., Blath, J., Steinrücken, M.: Importance sampling for Lambda-coalescents in the infinitely many sites model. Theor. Popul. Biol. 79(4), 155–173 (2011)CrossRefzbMATHGoogle Scholar
  8. Blom, H.A.P., Bakker, G.J., Krystul, J.: Probabilistic reachability analysis for large scale stochastic hybrid systems. In: Proceedings of 46th IEEE Conference on Decision and Control, New Orleans, USA (2007)Google Scholar
  9. Casella, B., Roberts, G.O.: Exact Monte Carlo simulation of killed diffusions. Adv. Appl. Probab. 40, 273–291 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  10. Cérou, F., Del Moral, P., Furon, T., Guyader, A.: Sequential Monte Carlo for rare event estimation. Stat. Comput. 22, 795–808 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  11. Cérou, F., Del Moral, P., Guyader, A.: A non-asymptotic variance theorem for unnormalized Feynman-Kac particle models. Ann. Inst. Henri Poincaré Probab. Stat. 47, 629–649 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  12. Chen, Y., Xie, J., Liu, J.S.: Stopping-time resampling for sequential Monte Carlo methods. J. R. Stat. Soc. B 67, 199–217 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  13. De Iorio, M., Griffiths, R.C.: Importance sampling on coalescent histories I. Adv. Appl. Probab. 36(2), 417–433 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  14. De Iorio, M., Griffiths, R.C.: Importance sampling on coalescent histories II: subdivided population models. Adv. Appl. Probab. 36(2), 434–454 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  15. Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004)CrossRefzbMATHGoogle Scholar
  16. Del Moral, P., Garnier, J.: Genealogical particle analysis of rare events. Ann. Appl. Probab. 15, 2496–2534 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  17. Doucet, A., De Freitas, J.F.G., Gordon, N.J.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001)CrossRefzbMATHGoogle Scholar
  18. Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: fifteen years later. In: Crisan, D., Rozovskii, B. (eds.) The Oxford handbook of Nonlinear Filtering. Oxford University Press, Oxford (2011)Google Scholar
  19. Fearnhead, P.: Computational methods for complex stochastic systems: a review of some alternatives to MCMC. Stat. Comput. 18, 151–171 (2008)MathSciNetCrossRefGoogle Scholar
  20. Frater, M.R., Bitmean, R.R., Kennedy, R.A., Anderson, B.D.O.: Fast simulation of rare events using reverse-time models. Comput. Netw. ISDN 20, 315–321 (1990)CrossRefGoogle Scholar
  21. Frater, M.R., Kennedy, R.A., Anderson, B.D.O.: Reverse-time modelling, optimal control and large deviations. Syst. Control Lett. 12, 351–356 (1989)CrossRefzbMATHGoogle Scholar
  22. Fuchs, M., Yu, P.D.: Rumor source detection for rumor spreading on random increasing trees. Electron. Commun. Probab. 20, 1–12 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  23. Ganesh, A., Massoulie, L., Towsley, D.: The effect of network topology on the spread of epidemics. In: Proceedings of 24th Annual Joint Conference of the IEEE Computer and Communication Societies (INFOCOM), vol. 2, pp. 1455–1466 (2015)Google Scholar
  24. Glasserman, P., Heidelberg, P., Shahabuddin, P., Zajic, T.: Multi-level splitting for estimating rare event probabilities. Oper. Res. 47, 585–600 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  25. Glasserman, P., Wang, Y.: Counterexamples in importance sampling for large deviations probabilities. Ann. Appl. Probab. 7(3), 731–746 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  26. Gobet, E.: Weak approximation of killed diffusions using Euler schemes. Stoch. Proc. Appl. 87(2), 167–197 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  27. Griffiths, R.C., Jenkins, P.A., Song, Y.S.: Importance sampling and the two-locus model with subdivided population structure. Adv. Appl. Probab. 40, 473–500 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  28. Griffiths, R.C., Tavaré, S.: Simulating probability distributions in the coalescent. Theor. Popul. Biol. 46, 131–159 (1994)CrossRefzbMATHGoogle Scholar
  29. Hobolth, A., Uyenoyama, M.K., Wiuf, C.: Importance sampling for the infinite sites model. Stat. Appl. Genet. Mol. Biol. 7(1), 1–26 (2008)Google Scholar
  30. Jarzynski, C.: Rare events and the convergence of exponentially averaged work values. Phys. Rev. E 73 (2006)Google Scholar
  31. Jasra, A., Kantas, N., Persing, A.: Bayesian parameter inference for partially observed stopped processes. Stat. Comput. 24, 1–20 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  32. Jenkins, P.A.: Stopping-time resampling and population genetic inference under coalescent models. Stat. Appl. Genet. Mol. Biol. 11(1), 1–20 (2012)Google Scholar
  33. Johansen, A.M., Del Moral, P., Doucet, A.: Sequential Monte Carlo samplers for rare events. In: Proceedings of the 6th International Workshop on Rare Event Simulation, pp. 256–267 (2006)Google Scholar
  34. Koskela, J., Jenkins, P., Spano, D.: Computational inference beyond Kingman’s coalescent. J. Appl. Probab. 52(2), 519–537 (2015)Google Scholar
  35. Lee, A., Yau, C., Giles, M.B., Doucet, A., Holmes, C.C.: On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. J. Comput. Graph. Stat. 19(4), 769–789 (2010)CrossRefGoogle Scholar
  36. Lezaud, P., Krystul, J., Le Gland, F.: Sampling per mode simulation for switching diffusions. In: Proceedings of 8th International Works Rare–Event Simulation, RESIM, Cambridge (2010)Google Scholar
  37. Lin, M., Chen, R., Mykland, P.: On generating Monte Carlo samples of continuous diffusion bridges. JASA 105(490), 820–838 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  38. Liu, J.S.: Monte Carlo strategies in scientific computing. Springer, New York (2001)zbMATHGoogle Scholar
  39. Meng, X.L., Wong, W.H.: Simulating rations of normalizing constants via a simple identity: a theoretical exploration. Stat. Sin. 6, 831–860 (1996)zbMATHGoogle Scholar
  40. Moore, C., Newman, M.E.J.: Epidemics and percolation in small-world networks. Phys. Rev. E 61, 5678–5682 (2000)CrossRefGoogle Scholar
  41. Pastor-Satorras, R., Vespignani, A.: Epidemic spreading in scale-free networks. Phys. Rev. Lett. 86, 3200–3203 (2001)CrossRefGoogle Scholar
  42. Rogers, L.C.G., Williams, D.: Diffusions, Markov Processes and Martingales, vol. 1, 2nd edn. Wiley, London (1994)zbMATHGoogle Scholar
  43. Rubino, G., Tuffin, B.: Rare Event Simulation Using Monte Carlo Methods. Wiley, London (2009)CrossRefzbMATHGoogle Scholar
  44. Sadowsky, J.S., Bucklew, J.A.: On large deviation theory and asymptotically efficient Monte Carlo estimation. IEEE Trans. Inf. Theory 36, 579–588 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  45. Shah, D., Zaman, T.: Detecting sources of computer viruses in networks: theory and experiment. Proc. ACM Sigmetrics 15, 5249–5262 (2010)Google Scholar
  46. Shah, D., Zaman, T.: Finding rumor sources on random trees. arXiv:1110.6230 (2016)
  47. Shwartz, A., Weiss, A.: Induced rare events: analysis via large deviations and time reversal. Adv. Appl. Probab. 25(3), 667–689 (1993)Google Scholar
  48. Stephens, M., Donnelly, P.: Inference in molecular population genetics. J. R. Stat. Soc. B 62(4), 605–655 (2000)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Institut für MathematikTechnische Universität BerlinBerlinGermany
  2. 2.Department of StatisticsUniversity of WarwickCoventryUK
  3. 3.Departments of Statistics and Computer ScienceUniversity of WarwickCoventryUK

Personalised recommendations