Statistics and Computing

, Volume 18, Issue 2, pp 151–171 | Cite as

Computational methods for complex stochastic systems: a review of some alternatives to MCMC

  • Paul FearnheadEmail author


We consider analysis of complex stochastic models based upon partial information. MCMC and reversible jump MCMC are often the methods of choice for such problems, but in some situations they can be difficult to implement; and suffer from problems such as poor mixing, and the difficulty of diagnosing convergence. Here we review three alternatives to MCMC methods: importance sampling, the forward-backward algorithm, and sequential Monte Carlo (SMC). We discuss how to design good proposal densities for importance sampling, show some of the range of models for which the forward-backward algorithm can be applied, and show how resampling ideas from SMC can be used to improve the efficiency of the other two methods. We demonstrate these methods on a range of examples, including estimating the transition density of a diffusion and of a discrete-state continuous-time Markov chain; inferring structure in population genetics; and segmenting genetic divergence data.


Diffusions Forward-backward algorithm Importance sampling Missing data Particle filter Population genetics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andrieu, C., Doucet, A.: Particle filtering for partially observed Gaussian state space models. J. R. Stat. Soc. Ser. B 64, 827–836 (2002) zbMATHMathSciNetGoogle Scholar
  2. Bahlo, M., Griffiths, R.C.: Inference from gene trees in a subdivided population. Theor. Popul. Biol. 57, 79–95 (1998) Google Scholar
  3. Barry, D., Hartigan, J.A.: Product partition models for change point problems. Ann. Stat. 20, 260–279 (1992) zbMATHMathSciNetGoogle Scholar
  4. Barry, D., Hartigan, J.A.: A Bayesian analysis for change point problems. J. Am. Stat. Soc. 88, 309–319 (1993) zbMATHMathSciNetGoogle Scholar
  5. Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximisation technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41, 164–171 (1970) MathSciNetGoogle Scholar
  6. Beskos, A., Papaspiliopoulos, O., Roberts, G.O.: Monte Carlo maximum likelihood estimation for discretely observed diffusion processes. Ann. Stat. (2007, to appear) Google Scholar
  7. Beskos, A., Papaspiliopoulos, O., Roberts, G.O., Fearnhead, P.: Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J. R. Stat. Soc. Ser. B 68, 333–382 (2006) zbMATHMathSciNetGoogle Scholar
  8. Bhattacharya, S., Gelfand, A.E., Holsinger, K.E.: Model fitting and inference under latent equilibrium processes. Stat. Comput. 17, 193–208 (2007) Google Scholar
  9. Blackwell, D., MacQueen, J.B.: Ferguson distributions via Polya urn schemes. Ann. Stat. 1, 353–355 (1973) zbMATHMathSciNetGoogle Scholar
  10. Boys, R.J., Wilkinson, D.J., Kirkwood, T.B.L.: Bayesian inference for a discretely observed stochastic kinetic model (2007, submitted) Google Scholar
  11. Cappe, O., Guillin, A., Marin, J.M., Robert, C.P.: Population Monte Carlo. J. Comput. Graph. Stat. 13, 907–929 (2004) MathSciNetGoogle Scholar
  12. Carpenter, J., Clifford, P., Fearnhead, P.: An improved particle filter for non-linear problems. IEE Proc. Radar Sonar Navig. 146, 2–7 (1999) Google Scholar
  13. Celeux, G., Marin, J., Robert, C.P.: Iterated importance sampling in missing data problems. Comput. Stat. Data Anal. 50, 3386–3404 (2006) MathSciNetGoogle Scholar
  14. Chen, R., Liu, J.S.: Mixture Kalman filters. J. R. Stat. Soc. Ser. B 62, 493–508 (2000a) zbMATHGoogle Scholar
  15. Chen, Y., Liu, J.S.: Comment on ‘Inference in molecular population genetics’ by M. Stephens and P. Donnelly. J. R. Stat. Soc. Ser. B 62, 644–645 (2000b) Google Scholar
  16. Chen, Y., Xie, J., Liu, J.S.: Stopping-time resampling for Monte Carlo methods. J. R. Stat. Soc. Ser. B 67, 199–217 (2005) zbMATHMathSciNetGoogle Scholar
  17. Chopin, N.: A sequential particle filter method for static models. Biometrika 89, 539–551 (2002) zbMATHMathSciNetGoogle Scholar
  18. Chopin, N.: Inference and model choice for time-ordered hidden Markov models. J. R. Stat. Soc. Ser. B 69, 269–284 (2007) zbMATHMathSciNetGoogle Scholar
  19. Cowell, R.G., Dawid, A.P., Lauritzen, S.L., Spiegelhalter, D.J.: Probabilistic Networks and Expert Systems. Springer, New York (1999) zbMATHGoogle Scholar
  20. Cox, J.C., Ingersoll, J.E. Jr., Ross, S.A.: A theory of the term structure of interest rates. Econometrica 53, 385–407 (1985) MathSciNetGoogle Scholar
  21. De Iorio, M., Griffiths, R.C.: Importance sampling on coalescent histories. I. Adv. Appl. Probab. 36, 417–433 (2004a) zbMATHGoogle Scholar
  22. De Iorio, M., Griffiths, R.C.: Importance sampling on coalescent histories. II: subdivided population models. Adv. Appl. Probab. 36, 434–454 (2004b) zbMATHGoogle Scholar
  23. Del Moral, P.: Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications. Springer, New York (2004) zbMATHGoogle Scholar
  24. Del Moral, P., Doucet, A.: Particle motions in absorbing medium with hard and soft obstacles. Stoch. Anal. Appl. 22, 1175–1207 (2004) zbMATHGoogle Scholar
  25. Del Moral, P., Guionnet, A.: On the stability of interaction processes with applications to filtering and genetic algorithms. Ann. Inst. H. Poincaré Probab. Stat. 37, 155–194 (2001) zbMATHGoogle Scholar
  26. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B 68, 411–436 (2006) zbMATHGoogle Scholar
  27. Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo for Bayesian computation. In: Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (eds.) Bayesian Statistics, vol. 8, pp. 115–148. Oxford University Press, Oxford (2007) Google Scholar
  28. Delyon, B., Hu, Y.: Simulation of conditioned diffusion and application to parameter estimation. Stoch. Process. Appl. 116, 1660–1675 (2006) zbMATHMathSciNetGoogle Scholar
  29. Didelot, X., Achtman, M., Parkhill, J., Thomson, N.R., Falush, D.: A bimodal pattern of relatedness between the salmonella Paratyphi A and Typhi genomes: convergence or divergence by homologous recombination? Genome Res. 17, 61–68 (2007) Google Scholar
  30. Donnelly, P., Kurtz, T.: A countable representation of the Fleming-Viot measure-valued diffusion. Ann. Probab. 24, 698–742 (1996) zbMATHMathSciNetGoogle Scholar
  31. Donnelly, P., Tavaré, S.: Coalescents and genealogical structure under neutrality. Annu. Rev. Genet. 29, 401–421 (1995) Google Scholar
  32. Doucet, A., de Freitas, J.F.G., Gordon, N.J. (eds.): Sequential Monte Carlo Methods in Practice. Springer, New York (2001) zbMATHGoogle Scholar
  33. Doucet, A., Godsill, S.J., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10, 197–208 (2000) Google Scholar
  34. Drummond, A.J., Nicholls, G.K., Rodrigo, A.G., Solomon, W.: Estimating mutation parameters, population history and genealogy simultaneously from temporally spaced sequence data. Genetics 161, 1307–1320 (2002) Google Scholar
  35. Drummond, A.J., Rambaut, A., Shapiro, B., Pybus, O.G.: Bayesian coalescent inference of past population dynamics from molecular sequences. Mol. Biol. Evol. 22, 1185–1192 (2005) Google Scholar
  36. Durbin, R., Eddy, S., Krogh, A., Mitchison, G.: Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press, Cambridge (1998) zbMATHGoogle Scholar
  37. Durham, G.B., Gallant, A.R.: Numerical techniques for maximum likelihood estimation of continuous-time diffusion processes. J. Bus. Econ. Stat. 20, 297–338 (2002) MathSciNetGoogle Scholar
  38. Falush, D., Stephens, M., Pritchard, J.K.: Inference of population structure using multilocus genotype data: linked loci and correlated allele frequencies. Genetics 164, 1567–1587 (2003) Google Scholar
  39. Falush, D., Torpdahl, M., Didelot, X., Conrad, D.F., Wilson, D.J., Achtman, M.: Mismatch induced speciation in Salmonella: model and data. Philos. Trans. R. Soc. Lond. Ser. B 361, 2045–2053 (2006) Google Scholar
  40. Fearnhead, P.: The common ancestor at a non-neutral locus. J. Appl. Probab. 39, 38–54 (2002a) zbMATHMathSciNetGoogle Scholar
  41. Fearnhead, P.: MCMC, sufficient statistics and particle filters. J. Comput. Graph. Stat. 11, 848–862 (2002b) MathSciNetGoogle Scholar
  42. Fearnhead, P.: Particle filters for mixture models with an unknown number of components. Stat. Comput. 14, 11–21 (2004) MathSciNetGoogle Scholar
  43. Fearnhead, P.: Exact Bayesian curve fitting and signal segmentation. IEEE Trans. Signal Process. 53, 2160–2166 (2005a) MathSciNetGoogle Scholar
  44. Fearnhead, P.: Using random quasi-Monte Carlo within particle filters, with application to financial time series. J. Comput. Graph. Stat. 14, 751–769 (2005b) MathSciNetGoogle Scholar
  45. Fearnhead, P.: Exact and efficient inference for multiple changepoint problems. Stat. Comput. 16, 203–213 (2006) MathSciNetGoogle Scholar
  46. Fearnhead, P., Clifford, P.: Online inference for hidden Markov models. J. R. Stat. Soc. Ser. B 65, 887–899 (2003) zbMATHMathSciNetGoogle Scholar
  47. Fearnhead, P., Donnelly, P.: Estimating recombination rates from population genetic data. Genetics 159, 1299–1318 (2001) Google Scholar
  48. Fearnhead, P., Donnelly, P.: Approximate likelihood methods for estimating local recombination rates (with discussion). J. R. Stat. Soc. Ser. B 64, 657–680 (2002) zbMATHMathSciNetGoogle Scholar
  49. Fearnhead, P., Liu, Z.: Online inference for multiple changepoint problems. J. R. Stat. Soc. Ser. B 69, 589–605 (2007) MathSciNetGoogle Scholar
  50. Fearnhead, P., Meligkotsidou, L.: Exact filtering for partially-observed continuous-time Markov models. J. R. Stat. Soc. Ser. B 66, 771–789 (2004) zbMATHMathSciNetGoogle Scholar
  51. Fearnhead, P., Sherlock, C.: Bayesian analysis of Markov modulated Poisson processes. J. R. Stat. Soc. Ser. B 68, 767–784 (2006) zbMATHMathSciNetGoogle Scholar
  52. Fearnhead, P., Vasileiou, D.: Bayesian analysis of isochores (2007, submitted). Available from
  53. Fearnhead, P., Papaspiliopoulos, O., Roberts, G.O.: Particle filters for partially-observed diffusions. J. R. Stat. Soc. Ser. B (2007, submitted) Google Scholar
  54. Felsenstein, J., Churchill, G.A.: A hidden Markov Model approach to variation among sites in rate of evolution. Mol. Biol. Evol. 13, 93–104 (1996) Google Scholar
  55. Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973) zbMATHMathSciNetGoogle Scholar
  56. Gamerman, D.: Markov Chain Monte Carlo: Stochastic Simulation For Bayesian Inference. Taylor and Francis Ltd, London (2006) zbMATHGoogle Scholar
  57. Gilks, W.R., Berzuini, C.: Following a moving target—Monte Carlo inference for dynamic Bayesian models. J. R. Stat. Soc. Ser. B 63, 127–146 (2001) zbMATHMathSciNetGoogle Scholar
  58. Godsill, S.J., Doucet, A., West, M.: Monte Carlo smoothing for non-linear time series. J. Am. Stat. Assoc. 99, 156–168 (2004) zbMATHMathSciNetGoogle Scholar
  59. Golightly, A., Wilkinson, D.J.: Bayesian inference for stochastic kinetic models using a diffusion approximation. Biometrics 61, 781–788 (2005) zbMATHMathSciNetGoogle Scholar
  60. Golightly, A., Wilkinson, D.J.: Bayesian sequential inference for stochastic kinetic biochemical network models. J. Comput. Biol. 13, 838–851 (2006) MathSciNetGoogle Scholar
  61. Gordon, N., Salmond, D., Smith, A.F.M.: Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc. F 140, 107–113 (1993) Google Scholar
  62. Green, P.: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732 (1995) zbMATHMathSciNetGoogle Scholar
  63. Griffiths, R.C., Tavaré, S.: Unrooted genealogical tree probabilities in the infinitely-many-sites model. Math. Biosci. 127, 77–98 (1994) Google Scholar
  64. Hurzeler, M., Kunsch, H.R.: Monte Carlo approximations for general state-space models. J. Comput. Graph. Stat. 7, 175–193 (1998) MathSciNetGoogle Scholar
  65. Jasra, A., Stephens, D.A., Holmes, C.: On population-based simulation for statics inference. Stat. Comput. 17, 263–279 (2007) Google Scholar
  66. Jorde, L.B., Bamshad, M.J., Watkins, W.S., Zenger, R., Fraley, A.E., Krakowiak, P.A., Carpenter, K.D., Soodyall, H., Jenkins, T., Rogers, A.R.: Origins and affinities of modern humans: a comparison of mitochondrial and nuclear genetic data. Am. J. Hum. Genet. 57, 523–538 (1995) Google Scholar
  67. Juang, B.H., Rabiner, L.R.: Hidden Markov models for speech recognition. Technometrics 33, 251–272 (1991) zbMATHMathSciNetGoogle Scholar
  68. Kalman, R., Bucy, R.: New results in linear filtering and prediction theory. J. Basic Eng. Trans. ASME Ser. D 83, 95–108 (1961) MathSciNetGoogle Scholar
  69. Kimura, M., Crow, J.: The number of alleles that can be maintained in a finite population. Genetics 49, 725–738 (1964) Google Scholar
  70. Kingman, J.F.C.: The coalescent. Stoch. Process. Appl. 13, 235–248 (1982) zbMATHMathSciNetGoogle Scholar
  71. Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat. 5, 1–25 (1996) MathSciNetGoogle Scholar
  72. Kloeden, P.E., Platen, E.: Numerical Solution of Stochastic Differential Equations. Springer, New York (1992) zbMATHGoogle Scholar
  73. Kong, A., Liu, J.S., Wong, W.H.: Sequential imputations and Bayesian missing data problems. J. Am. Stat. Assoc. 89, 278–288 (1994) zbMATHGoogle Scholar
  74. Kou, S.C., Zhou, Q., Wong, W.H.: Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Stat. 34, 1581–1619 (2006) MathSciNetGoogle Scholar
  75. Kschischang, F.R., Frey, B.J., Loeliger, H.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory, 47 (2001) Google Scholar
  76. Künsch, H.R.: Monte Carlo filters: algorithms and theoretical analysis. Ann. Stat. 33, 1983–2021 (2005) zbMATHGoogle Scholar
  77. Lander, E.S., Green, P.: Construction of multilocus genetic linkage maps in humans. Proc. Natl. Acad. Sci. USA 84, 2363–2367 (1987) Google Scholar
  78. L’Ecuyer, P., Lécot, C., Tuffin, B.: A randomized quasi-Monte Carlo simulation method for Markov chains. Oper. Res. (2007, to appear) Google Scholar
  79. Liu, J., West, M.: Combined parameter and state estimation in simulation based filtering. In: Doucet, A., de Freitas, J.F.G., Gordon, N.J. (eds.) Sequential Monte Carlo in Practice, pp. 197–223. Springer, New York (2001) Google Scholar
  80. Liu, J.S.: Metropolised independent sampling with comparisons to rejection sampling and importance sampling. Stat. Comput. 6, 113–119 (1996) Google Scholar
  81. Liu, J.S., Chen, R.: Blind deconvolution via sequential imputations. J. Am. Stat. Assoc. 90, 567–576 (1995) zbMATHGoogle Scholar
  82. Liu, J.S., Chen, R.: Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93, 1032–1044 (1998) zbMATHGoogle Scholar
  83. Liu, J.S., Chen, R., Wong, W.H.: Rejection control and sequential importance sampling. J. Am. Stat. Soc. 93, 1022–1031 (1998) zbMATHMathSciNetGoogle Scholar
  84. Liu, J.S., Lawrence, C.E.: Bayesian inference on biopolymer models. Bioinformatics 15, 38–52 (1999) Google Scholar
  85. Maiden, M.C.J., Bygraves, J.A., Feil, E., Morelli, G., Russell, J.E., Urwin, R., Zhang, Q., Zhou, J., Zurth, K., Caugant, D.A., Feavers, I.M., Achtman, M., Spratt, B.G.: Multilocus sequence typing: a portable approach to the identification of clones within populations of pathogenic microorganisms. Proc. Natl. Acad. Sci. USA 95, 3140–3145 (1998) Google Scholar
  86. Meng, X., Wong, W.H.: Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Stat. Sinica 6, 831–860 (1996) zbMATHMathSciNetGoogle Scholar
  87. Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1091 (1953) Google Scholar
  88. Neal, R.M.: (2003). Markov chain sampling for non-linear state space models using embedded hidden Markov models. Available from
  89. Nicholson, G., Smith, A.V., Jónsson, F., Gústafsson, O., Stefánsson, K., Donnelly, P.: Assessing population differentiation and isolation from single-nucleotide polymorphism data. J. R. Stat. Soc. Ser. B 64, 695–715 (2002) zbMATHGoogle Scholar
  90. Papaspilopoulos, O., Roberts, G.O., Sköld, M.: Non-centred parameterisations for hierarchical models and data augmentation (with discussion). In: Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (eds.) Bayesian Statistics, vol. 7. Clarendon, London (2003) Google Scholar
  91. Pedersen, A.R.: A new approach to maximum likelihood estimation of stochastic differential equations based on discrete observations. Scand. J. Stat. 22, 55–71 (1995) zbMATHGoogle Scholar
  92. Pitt, M.: Smooth particle filters for likelihood evaluation and maximisation (2007, submitted). Available from
  93. Pitt, M.K., Shephard, N.: Filtering via simulation: auxiliary particle filters. J. Am. Stat. Assoc. 94, 590–599 (1999) zbMATHMathSciNetGoogle Scholar
  94. Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000a) Google Scholar
  95. Pritchard, J.K., Stephens, M., Rosenberg, N.A., Donnelly, P.: Association mapping in structured populations. Am. J. Hum. Genet. 67, 170–181 (2000b) Google Scholar
  96. Rabiner, L.R., Juang, B.H.: An introduction to hidden Markov models. IEEE ASSP Mag., 4–15 (1986) Google Scholar
  97. Redelings, B.D., Suchard, M.A.: Joint Bayesian estimation of alignment and phylogeny. Syst. Biol. 54, 401–418 (2005) Google Scholar
  98. Ridgeway, G., Madigan, D.: A sequential Monte Carlo method for Bayesian analysis of massive datasets. Data Min. Knowl. Discov. 7, 301–319 (2003) MathSciNetGoogle Scholar
  99. Ripley, B.D.: Stochastic Simulation. Wiley, New York (1987) zbMATHGoogle Scholar
  100. Roberts, G.O., Stramer, O.: On inference for partially observed nonlinear diffusion models using the Metropolis-Hastings algorithm. Biometrika 88, 603–621 (2001) zbMATHMathSciNetGoogle Scholar
  101. Rogers, L.C.G., Williams, D.: Diffusions, Markov processes and Martingales, vol. 1. Cambridge University Press, Cambridge (2000) Google Scholar
  102. Rosenberg, N.A., Pritchard, J.K., Weber, J.L., Cann, H.M., Kidd, K.K., Zhivotovsky, L.A., Feldman, M.W.: Genetic structure of human populations. Science 298, 2381–2385 (2002) Google Scholar
  103. Scott, S.L.: Bayesian analysis of a two state Markov modulated Poisson process. J. Comput. Graph. Stat. 8, 662–670 (1999) Google Scholar
  104. Scott, S.L.: Bayesian methods for hidden Markov models: recursive computing in the 21st century. J. Am. Stat. Assoc. 97, 337–351 (2002) zbMATHGoogle Scholar
  105. Stephens, M.: Problems with computational methods in population genetics. Contribution to the 52nd session of the International Statistical Institute (1999) Google Scholar
  106. Stephens, M.: Times on trees and the age of an allele. Theor. Popul. Biol. 57, 109–119 (2000) zbMATHGoogle Scholar
  107. Stephens, M., Donnelly, P.: Inference in molecular population genetics (with discussion). J. R. Stat. Soc. Ser. B 62, 605–655 (2000) zbMATHMathSciNetGoogle Scholar
  108. Storvik, G.: Particle filters for state-space models with the presence of unknown static parameters. IEEE Trans. Signal Process. 50, 281–289 (2002) MathSciNetGoogle Scholar
  109. Stramer, O., Yan, J.: On simulated likelihood of discretely observed diffusion processes and comparison to closed form approximation. J. Comput. Graph. Stat. 16, 672–691 (2007a) MathSciNetGoogle Scholar
  110. Stramer, O., Yan, J.: Asymptotics of an efficient Monte Carlo estimation for the transition density of diffusion processes. Methodol. Comput. Appl. Probab. 9, 483–496 (2007b) zbMATHGoogle Scholar
  111. Wakeley, J.: Coalescent Theory: An Introduction. Roberts and Company, Denver (2007) Google Scholar
  112. West, M., Harrison, J.: Bayesian Forecasting and Dynamic Models. Springer, New York (1989) zbMATHGoogle Scholar
  113. Wilkinson, D.J.: Stochastic Modelling for Systems Biology. Chapman and Hall/CRC, Boca Raton (2006) zbMATHGoogle Scholar
  114. Wilson, D.J., Gabriel, E., Leatherbarrow, A.J.H., Cheesebrough, J., Gee, S., Bolton, E., Fox, A., Hart, C.A., Diggle, P.J., Fearnhead, P.: Rapid evolution and the importance of recombination to the gastro-enteric pathogen campylobacter jejuni (2007, in preparation) Google Scholar
  115. Yao, Y.: Estimation of a noisy discrete-time step function: Bayes and empirical Bayes approaches. Ann. Stat. 12, 1434–1447 (1984) zbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Mathematics and StatisticsLancaster UniversityLancasterUK

Personalised recommendations