Abstract
Bayesian synthetic likelihood (BSL) is now a well-established method for performing approximate Bayesian parameter estimation for simulation-based models that do not possess a tractable likelihood function. BSL approximates an intractable likelihood function of a carefully chosen summary statistic at a parameter value with a multivariate normal distribution. The mean and covariance matrix of this normal distribution are estimated from independent simulations of the model. Due to the parametric assumption implicit in BSL, it can be preferred to its nonparametric competitor, approximate Bayesian computation, in certain applications where a high-dimensional summary statistic is of interest. However, despite several successful applications of BSL, its widespread use in scientific fields may be hindered by the strong normality assumption. In this paper, we develop a semi-parametric approach to relax this assumption to an extent and maintain the computational advantages of BSL without any additional tuning. We test our new method, semiBSL, on several challenging examples involving simulated and real data and demonstrate that semiBSL can be significantly more robust than BSL and another approach in the literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
An, Z., South, L.F., Nott, D.J., Drovandi, C.C.: Accelerating Bayesian synthetic likelihood with the graphical lasso. J. Comput. Graph. Stat. 28(2), 471–475 (2019). https://doi.org/10.1080/10618600.2018.1537928
Anderson, C., Coles, S.: The largest inclusions in a piece of steel. Extremes 5(3), 237–252 (2002)
Andrieu, C., Roberts, G.O.: The pseudo-marginal approach for efficient Monte Carlo computations. Ann. Stat. 37(2), 697–725 (2009)
Barbu, C.M., Sethuraman, K., Billig, E.M.W., Levy, M.Z.: Two-scale dispersal estimation for biological invasions via synthetic likelihood. Ecography 41(4), 661–672 (2017)
Bedford, T., Cooke, R.M.: Vines-a new graphical model for dependent random variables. Ann. Stat. 30(4), 1031–1068 (2002)
Blum, M.G.B., François, O.: Non-linear regression models for approximate Bayesian computation. Stat. Comput. 20(1), 63–73 (2010)
Bortot, P., Coles, S.G., Sisson, S.A.: Inference for stereological extremes. J. Am. Stat. Assoc. 102(477), 84–92 (2007)
Boudt, K., Cornelissen, J., Croux, C.: The Gaussian rank correlation estimator: robustness properties. Stat. Comput. 22(2), 471–483 (2012)
Chambers, J.M., Mallows, C.L., Stuck, B.W.: A method for simulating stable random variables. J. Am. Stat. Assoc. 71(354), 340–344 (1976)
Chiachio, M., Beck, J., Chiachio, J., Rus, G.: Approximate Bayesian computation by subset simulation. SIAM J. Sci. Comput. 36(3), A1339–A1358 (2014)
Drovandi, C.C., Pettitt, A.N., Faddy, M.J.: Approximate Bayesian computation using indirect inference. J. R. Stat. Soc. Ser. C (Appl. Stat.) 60(3), 317–337 (2011)
Drovandi, C.C., Pettitt, A.N., Lee, A.: Bayesian indirect inference using a parametric auxiliary model. Stat. Sci. 30(1), 72–95 (2015)
Dutta, R., Corander, J., Kaski, S., Gutmann, M.U.: Likelihood-free inference by ratio estimation. ArXiv preprint arXiv:1611.10242v3 (2017)
Epanechnikov, V.A.: Non-parametric estimation of a multivariate probability density. Theory Probab. Appl. 14(1), 153–158 (1969)
Everitt, RG.: Bootstrapped synthetic likelihood. ArXiv preprint arXiv:1711.05825v2 (2017)
Fasiolo, M., Wood, S.N., Hartig, F., Bravington, M.V.: An extended empirical saddlepoint approximation for intractable likelihoods. Electron. J. Stat. 12(1), 1544–1578 (2018). https://doi.org/10.1214/18-EJS1433
Fearnhead, P., Prangle, D.: Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 74(3), 419–474 (2012)
Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)
Gleim, A., Pigorsch, C.: Approximate Bayesian computation with indirect summary statistics. Draft paper: http://www.ect-pigorschmeeuni-bonnde/data/research/papers (2013)
Jones, M.C., Pewsey, A.: Sinh-arcsinh distributions. Biometrika 96(4), 761–780 (2009). https://doi.org/10.1093/biomet/asp053
Li, J., Nott, D., Fan, Y., Sisson, S.: Extending approximate Bayesian computation methods to high dimensions via a Gaussian copula model. Comput. Stat. Data Anal. 106, 77–89 (2017)
Liu, H., Lafferty, J., Wasserman, L.: The nonparanormal: semiparametric estimation of high dimensional undirected graphs. J. Mach. Learn. Res. 10, 2295–2328 (2009)
Marchand, P., Boenke, M., Green, D.M.: A stochastic movement model reproduces patterns of site fidelity and long-distance dispersal in a population of Fowler’s toads (Anaxyrus fowleri). Ecol. Model. 360, 63–69 (2017)
Ong, V.M.H., Nott, D.J., Tran, M.N., Sisson, S.A., Drovandi, C.C.: Likelihood-free inference in high dimensions with synthetic likelihood. Comput. Stat. Data Anal. 128, 271–291 (2018a)
Ong, V.M.H., Nott, D.J., Tran, M.N., Sisson, S.A., Drovandi, C.C.: Variational Bayes with synthetic likelihood. Stat. Comput. 28(4), 971–988 (2018b)
Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33(3), 1065–1076 (1962)
Price, L.F., Drovandi, C.C., Lee, A., Nott, D.J.: Bayesian synthetic likelihood. J. Comput. Graph. Stat. 27, 1–11 (2018)
Rosenblatt, M.: Remarks on some nonparametric estimates of a density function. Ann. Math. Stat. 27(3), 832–837 (1956)
Sahu, S.K., Dey, D.K., Branco, M.D.: A new class of multivariate skew distributions with applications to Bayesian regression models. Can. J. Stat. 31(2), 129–150 (2008)
Shestopaloff, AY., Neal, RM.: On Bayesian inference for the M/G/1 queue with efficient MCMC sampling. ArXiv preprint arXiv:1401.5548 (2014)
Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Routledge, Abingdon (2018)
Sisson, S.A., Fan, Y., Beaumont, M.: Handbook of Approximate Bayesian Computation, 1st edn. Chapman and Hall/CRC, Boca Raton (2018)
Sklar, M.: Fonctions de répartition à n dimensions et leurs marges. Inst. Stat. Univ. Paris 8, 229–231 (1959)
Terrell, G.R., Scott, D.W.: Variable kernel density estimation. Ann. Stat. 20(3), 1236–1265 (1992)
Warton, D.I.: Penalized normal likelihood and ridge regularization of correlation and covariance matrices. J. Am. Stat. Assoc. 103(481), 340–349 (2008)
Wood, S.N.: Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466, 1102–1107 (2010)
Acknowledgements
CD was supported by an Australian Research Council’s Discovery Early Career Researcher Award funding scheme (DE160100741). ZA was supported by a scholarship under CDs Grant DE160100741 and a top-up scholarship from the Australian Research Council Centre of Excellence for Mathematical and Statistics Frontiers (ACEMS). DJN was supported by a Singapore Ministry of Education Academic Research Fund Tier 1 Grant (R-155-000-189-114). Computational resources and services used in this work were provided by the HPC and Research Support Group, Queensland University of Technology, Brisbane, Australia. The authors thank Alex Shestopaloff for sharing his code on exact MCMC for the M/G/1 model.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
An, Z., Nott, D.J. & Drovandi, C. Robust Bayesian synthetic likelihood via a semi-parametric approach. Stat Comput 30, 543–557 (2020). https://doi.org/10.1007/s11222-019-09904-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-019-09904-x