Skip to main content
SpringerLink
Log in

Variance reduction for additive functionals of Markov chains via martingale representations

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

In this paper, we propose an efficient variance reduction approach for additive functionals of Markov chains relying on a novel discrete-time martingale representation. Our approach is fully non-asymptotic and does not require the knowledge of the stationary distribution (and even any type of ergodicity) or specific structure of the underlying density. By rigorously analyzing the convergence properties of the proposed algorithm, we show that its cost-to-variance product is indeed smaller than one of the naive algorithms. The numerical performance of the new method is illustrated for the Langevin-type Markov chain Monte Carlo (MCMC) methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Assaraf, R., Caffarel, M.: Zero-variance principle for Monte Carlo algorithms. Phys. Rev. Lett. 83(23), 4682 (1999)

    Article  Google Scholar 

  • Belomestny, D., Häfner, S., Urusov, M.: Variance reduction for discretised diffusions via regression. J. Math. Anal. Appl. 458, 393–418 (2018)

    Article  MathSciNet  Google Scholar 

  • Belomestny, D., Iosipoi, L., Moulines, E., Naumov, A., Samsonov, S.: Variance reduction for Markov chains with application to MCMC. arXiv preprint arXiv:1910.03643 (2019)

  • Ben Zineb, T., Gobet, E.: Preliminary control variates to improve empirical regression methods. Monte Carlo Methods Appl. 19(4), 331–354 (2013). https://doi.org/10.1515/mcma-2013-0015

    Article  MathSciNet  MATH  Google Scholar 

  • Bortoli, V.D., Durmus, A.: Convergence of diffusions and their discretizations: from continuous to discrete processes and back (2020)

  • Boucheron, S., Lugosi, G., Massart, P.: Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, Oxford (2013)

    Book  Google Scholar 

  • Brosse, N., Durmus, A., Meyn, S., Moulines, E.: Diffusion approximations and control variates for MCMC. arXiv preprint arXiv:1808.01665 (2018)

  • Constantine, G.M.: Combinatorial Theory and Statistical Design. Wiley, New York (1987)

    MATH  Google Scholar 

  • Constantine, G.M., Savits, T.H.: A multivariate faa di bruno formula with applications. Trans. Am. Math. Soc. 348(2), 503–520 (1996)

    Article  MathSciNet  Google Scholar 

  • Dalalyan, A.S.: Theoretical guarantees for approximate sampling from smooth and log-concave densities. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 79(3), 651–676 (2017)

    Article  MathSciNet  Google Scholar 

  • Dellaportas, P., Kontoyiannis, I.: Control variates for estimation based on reversible Markov chain Monte Carlo samplers. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 74(1), 133–161 (2012)

    Article  MathSciNet  Google Scholar 

  • Dimov, I.T.: Monte Carlo Methods for Applied Scientists. World Scientific, Singapore (2008)

    MATH  Google Scholar 

  • Douc, R., Moulines, E., Priouret, P., Soulier, P.: Markov Chains. Springer, New York (2018)

    Book  Google Scholar 

  • Durmus, A., Moulines, E.: Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. Ann. Appl. Probab. 27(3), 1551–1587 (2017). https://doi.org/10.1214/16-AAP1238

    Article  MathSciNet  MATH  Google Scholar 

  • Glasserman, P.: Monte Carlo Methods in Financial Engineering, vol. 53. Springer, Berlin (2013)

    MATH  Google Scholar 

  • Gobet, E.: Monte-Carlo Methods and Stochastic Processes. CRC Press, Boca Raton (2016). (ISBN 978-1-4987-4622-9. From linear to non-linear)

  • Györfi, L., Kohler, M., Krzyzak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer, Berlin (2006)

    MATH  Google Scholar 

  • Heinrich, S., Sindambiwe, E.: Monte Carlo complexity of parametric integration. J. Complex. 15(3), 317–341 (1999)

    Article  MathSciNet  Google Scholar 

  • Henderson, S.G.: Variance reduction via an approximating Markov process. PhD thesis, Stanford University (1997)

  • Henderson, S.G., Simon, B.: Adaptive simulation using perfect control variates. J. Appl. Probab. 41(3), 859–876 (2004). https://doi.org/10.1239/jap/1091543430

    Article  MathSciNet  MATH  Google Scholar 

  • Lamberton, D., Pagès, G.: Recursive computation of the invariant distribution of a diffusion. Bernoulli 8(3), 367–405 (2002)

    MathSciNet  MATH  Google Scholar 

  • Lemaire, V.: An adaptive scheme for the approximation of dissipative systems. Stoch. Process. Appl. 117(10), 1491–1518 (2007). https://doi.org/10.1016/j.spa.2007.02.004

    Article  MathSciNet  MATH  Google Scholar 

  • Liu, J., Zhao, W.: Dynamic analysis of stochastic Lotka–Volterra predator-prey model with discrete delays and feedback control. Complexity 1–15(11), 2019 (2019). https://doi.org/10.1155/2019/4873290

    Article  Google Scholar 

  • Mao, X., Sabanis, S., Renshaw, E.: Asymptotic behaviour of the stochastic Lotka–Volterra model. J. Math. Anal. Appl. 287(1), 141–156 (2003). https://doi.org/10.1016/S0022-247X(03)00539-0

    Article  MathSciNet  MATH  Google Scholar 

  • Mattingly, J., Stuart, A., Higham, D.: Ergodicity for sdes and approximations: locally Lipschitz vector fields and degenerate noise. Stoch. Process. Appl. 101(2), 185–232 (2002). https://doi.org/10.1016/S0304-4149(02)00150-3

    Article  MathSciNet  MATH  Google Scholar 

  • Mengersen, K., Tweedie, R.L.: Rates of convergence of the Hastings and Metropolis algorithms. Ann. Stat. 24, 101–121 (1996)

    Article  MathSciNet  Google Scholar 

  • Mira, A., Solgi, R., Imparato, D.: Zero variance Markov chain Monte Carlo for Bayesian estimators. Stat. Comput. 23(5), 653–662 (2013)

  • Oates, C.J., Girolami, M., Chopin, N.: Control functionals for Monte Carlo integration. J. R. Stat. Soc. Ser. B (Stat. Methodol.) (2016). https://doi.org/10.1111/rssb.12185

  • Oates, C.J., Girolami, M., Chopin, N.: Control functionals for Monte Carlo integration. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 79(3), 695–718 (2017)

    Article  MathSciNet  Google Scholar 

  • Pagès, G., Panloup, F.: Weighted multilevel Langevin simulation of invariant measures. Ann. Appl. Probab. 28(6), 3358–3417 (2018). https://doi.org/10.1214/17-AAP1364

    Article  MathSciNet  MATH  Google Scholar 

  • Rubinstein, R.Y., Kroese, D.P.: Simulation and the Monte Carlo Method, vol. 10. Wiley, New York (2016)

    Book  Google Scholar 

  • South, L.F., Oates, C.J., Mira, A., Drovandi, C.: Regularised zero-variance control variates. arXiv preprint arXiv:1811.05073 (2018)

  • South, L.F., Riabiz, M., Teymur, O., Oates, C. et al.: Post-processing of MCMC. arXiv preprint arXiv:2103.16048 (2021)

Download references

Acknowledgements

The publication was supported by the grant for research centers in the field of AI provided by the Analytical Center for the Government of the Russian Federation (ACRF) in accordance with the agreement on the provision of subsidies (identifier of the agreement 000000D730321P5Q0002) and the agreement with HSE University No. 70-2021-00139.

Author information

Authors and Affiliations

  1. Duisburg-Essen University, Essen, Germany

    D. Belomestny

  2. HSE University, Moscow, Russia

    D. Belomestny, E. Moulines & S. Samsonov

  3. Ecole Polytechnique, Palaiseau, France

    E. Moulines

Authors
  1. D. Belomestny
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. E. Moulines
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Samsonov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Samsonov.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Belomestny, D., Moulines, E. & Samsonov, S. Variance reduction for additive functionals of Markov chains via martingale representations. Stat Comput 32, 16 (2022). https://doi.org/10.1007/s11222-021-10073-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11222-021-10073-z

Keywords

Search

Navigation