Skip to main content

A flexible particle Markov chain Monte Carlo method

Statistics and Computing volume 30, pages 783–798 (2020)Cite this article

Abstract

Particle Markov Chain Monte Carlo methods are used to carry out inference in nonlinear and non-Gaussian state space models, where the posterior density of the states is approximated using particles. Current approaches usually perform Bayesian inference using either a particle marginal Metropolis–Hastings (PMMH) algorithm or a particle Gibbs (PG) sampler. This paper shows how the two ways of generating variables mentioned above can be combined in a flexible manner to give sampling schemes that converge to a desired target distribution. The advantage of our approach is that the sampling scheme can be tailored to obtain good results for different applications. For example, when some parameters and the states are highly correlated, such parameters can be generated using PMMH, while all other parameters are generated using PG because it is easier to obtain good proposals for the parameters within the PG framework. We derive some convergence properties of our sampling scheme and also investigate its performance empirically by applying it to univariate and multivariate stochastic volatility models and comparing it to other PMCMC methods proposed in the literature.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

Notes

  1. http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/datalibrary.html.

References

  • Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Stat. 37(2), 697–725 (2009)

    MathSciNet  MATH  Article  Google Scholar 

  • Andrieu, C., Doucet, A., Holenstein, R.: Particle Markov chain Monte Carlo methods. J. R. Stat. Soc. Ser. B 72(3), 269–342 (2010)

    MathSciNet  MATH  Article  Google Scholar 

  • Brix, A.F., Lunde, A., Wei, W.: A general Schwartz model for energy spot price—estimation using a particle MCMC method. Energy Econ. 72, 560–582 (2018)

    Article  Google Scholar 

  • Carter, C., Kohn, R.: Markov chain Monte Carlo in conditionally Gaussian state space models. Biometrika 83(3), 589–601 (1996)

    MathSciNet  MATH  Article  Google Scholar 

  • Chib, S., Pitt, M.K., Shephard, N.: Likelihood based inference for diffusion driven models. Working Paper (2004)

  • Chib, S., Nardari, F., Shephard, N.: Analysis of high dimensional multivariate stochastic volatility models. J. Econom. 134(2), 341–371 (2006)

    MathSciNet  MATH  Article  Google Scholar 

  • Dahlin, J., Lindsten, F., Schön, T.: Particle Metropolis–Hastings using gradient and Hessian information. Stat. Comput. 25(1), 81–92 (2015)

    MathSciNet  MATH  Article  Google Scholar 

  • Deligiannidis, G., Doucet, A., Pitt, M.K.: The correlated pseudo-marginal method. J. R. Stat. Soc. Ser. B 80(5), 839–870 (2018)

    MathSciNet  MATH  Article  Google Scholar 

  • Douc, R., Cappé, O.: Comparison of resampling schemes for particle filtering. In: Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005. ISPA 2005, pp. 64–69. IEEE (2005)

  • Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)

    Article  Google Scholar 

  • Durbin, J., Koopman, S.: Time Series Analysis of State Space Methods, second edn. Oxford University Press, Oxford (2012)

    MATH  Book  Google Scholar 

  • Fearnhead, P., Meligkotsidou, L.: Augmentation schemes for particle MCMC. Stat. Comput. 26(6), 1293–1306 (2016)

    MathSciNet  MATH  Article  Google Scholar 

  • Gerlach, R., Carter, C., Kohn, R.: Efficient Bayesian inference for dynamic mixture models. J. Am. Stat. Assoc. 95(451), 819–828 (2000)

    MathSciNet  MATH  Article  Google Scholar 

  • Godsill, S., Doucet, A., West, M.: Monte Carlo smoothing for nonlinear time series. J. Am. Stat. Assoc. 99(465), 156–168 (2004)

    MathSciNet  MATH  Article  Google Scholar 

  • Guo, D., Wang, X., Chen, R.: New sequential Monte Carlo methods for nonlinear dynamic systems. Stat. Comput. 15(2), 135–147 (2005)

    MathSciNet  Article  Google Scholar 

  • Ignatieva, K., Rodrigues, P., Seeger, N.: Empirical analysis of affine versus nonaffine variance specifications in jump-diffusion models for equity indices. J. Bus. Econ. Stat. 33(1), 68–75 (2015)

    MathSciNet  Article  Google Scholar 

  • Kastner, G., Fruhwirth-Schnatter, S., Lopes, H.F.: Efficient Bayesian inference for multivariate factor stochastic volatility models. J. Comput. Graph. Stat. 26(4), 905–917 (2017)

    MathSciNet  Article  Google Scholar 

  • Kim, S., Shephard, N., Chib, S.: Stochastic volatility: likelihood inference and comparison with ARCH models. Rev. Econ. Stud. 65(3), 361–393 (1998)

    MATH  Article  Google Scholar 

  • Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat. 5(1), 1–25 (1996)

    MathSciNet  Google Scholar 

  • Kleppe, T.S., Yu, J., Skaug, H.: Estimating the GARCH diffusion: simulated maximum likelihood in continuous time. SMU Economics and Statistics Working Paper Series, p. 13 (2010)

  • Lindsten, F., B. Schön, T.: On the use of backward simulation in particle Markov chain Monte Carlo methods. arxiv:1110.2873 (2012a)

  • Lindsten, F., Schön, T.B.: On the use of backward simulation in the particle Gibbs sampler. In: Proceedings of the 37th International Conference on Acoustics, Speech, and Signal Processing, pp. 3845–3848. ICASSP (2012b)

  • Lindsten, F., Schon, T.B.: Backward simulation methods for Monte Carlo statistical inference. Found. Trends Mach. Learn. 6(1), 1–143 (2013)

    MATH  Article  Google Scholar 

  • Lindsten, F., Jordan, M.I., Schön, T.B.: Particle Gibbs with ancestor sampling. J. Mach. Learn. Res. 15, 2145–2184 (2014)

    MathSciNet  MATH  Google Scholar 

  • Lindsten, F., Bunch, P., Singh, S.S., Schön, T.B.: Particle ancestor sampling for near-degenerate or intractable state transition models. arxiv:1505.0635v1 (2015)

  • Nemeth, C., Fearnhead, P., Mihaylova, L.: Particle approximations of the score and observed information matrix for parameter estimation in state-space models with linear computational cost. J. Comput. Graph. Stat. 25(4), 1138–1157 (2016a)

    MathSciNet  Article  Google Scholar 

  • Nemeth, C., Sherlock, C., Fearnhead, P.: Particle Metropolis-adjusted Langevin algorithms. Biometrika 103(3), 701–717 (2016b)

    MathSciNet  MATH  Article  Google Scholar 

  • Olsson, J., Ryden, T.: Rao-Blackwellization of particle Markov chain Monte Carlo methods using forward filtering backward sampling. IEEE Trans. Signal Process. 59(10), 4606–4619 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  • Pitt, M.K., Silva, RdS, Giordani, P., Kohn, R.: On some properties of Markov chain Monte Carlo simulation methods based on the particle filter. J. Econom. 171(2), 134–151 (2012)

    MathSciNet  MATH  Article  Google Scholar 

  • Roberts, G.O., Rosenthal, J.S.: Examples of adaptive MCMC. J. Comput. Graph. Stat. 18(2), 349–367 (2009)

    MathSciNet  Article  Google Scholar 

  • Stein, E., Stein, J.: Stock price distributions with stochastic volatility: an analytic approach. Rev. Financ. Stud. 4, 727–752 (1991)

    MATH  Article  Google Scholar 

  • Stramer, O., Bognar, M.: Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach. Bayesian Anal. 6(2), 231–258 (2011)

    MathSciNet  MATH  Article  Google Scholar 

  • Van Der Merwe, R., Doucet, A., De Freitas, N., Wan, E.: The unscented particle filter. Advances in neural information processing systems, pp. 584–590 (2001)

  • Wu, X., Zhou, G., Wang, S.: Estimation of market prices of risks in the G.A.R.C.H. diffusion model. Economic Research-Ekonomska Istraživanja 31(1), 15–36 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

The work of the authors was partially supported by an ARC Research Council Grant DP120104014. The work of Robert Kohn and David Gunawan was also partially supported by the ARC Center of Excellence Grant CE140100049.

Author information

Authors and Affiliations

  1. School of Applied Mathematics, Fundação Getulio Vargas, Rio de Janeiro, Brazil

    Eduardo F. Mendes

  2. School of Economics, University of New South Wales, Sydney, Australia

    Christopher K. Carter, David Gunawan & Robert Kohn

Authors
  1. Eduardo F. Mendes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher K. Carter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Gunawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Kohn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher K. Carter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 494 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mendes, E.F., Carter, C.K., Gunawan, D. et al. A flexible particle Markov chain Monte Carlo method. Stat Comput 30, 783–798 (2020). https://doi.org/10.1007/s11222-019-09916-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-019-09916-7

Keywords

  • Diffusion equation
  • Factor stochastic volatility model
  • Metropolis–Hastings
  • Particle Gibbs sampler