Abstract
Most of the existing algorithms for approximate Bayesian computation (ABC) assume that it is feasible to simulate pseudo-data from the model at each iteration. However, the computational cost of these simulations can be prohibitive for high dimensional data. An important example is the Potts model, which is commonly used in image analysis. Images encountered in real world applications can have millions of pixels, therefore scalability is a major concern. We apply ABC with a synthetic likelihood to the hidden Potts model with additive Gaussian noise. Using a pre-processing step, we fit a binding function to model the relationship between the model parameters and the synthetic likelihood parameters. Our numerical experiments demonstrate that the precomputed binding function dramatically improves the scalability of ABC, reducing the average runtime required for model fitting from 71 h to only 7 min. We also illustrate the method by estimating the smoothing parameter for remotely sensed satellite imagery. Without precomputation, Bayesian inference is impractical for datasets of that scale.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beaumont, M.A., Zhang, W., Balding, D.J.: Approximate Bayesian computation in population genetics. Genetics 162(4), 2025–35 (2002)
Beaumont, M.A., Cornuet, J.M., Marin, J.-M., Robert, C.P.: Adaptive approximate Bayesian computation. Biometrika 96(4), 90–983 (2009)
Blum, M.G.B., François, O.: Non-linear regression models for approximate Bayesian computation. Stat. Comput. 20(1), 63–73 (2010)
Cabras, S., Castellanos, M.E., Ruli, E.: A Quasi likelihood approximation of posterior distributions for likelihood-intractable complex models. Metron 72(2), 153–67 (2014)
Cook, S.R., Gelman, A., Rubin, D.B.: Validation of software for Bayesian models using posterior quantiles. J. Comput. Graph. Stat. 15(3), 675–92 (2006)
Cucala, L., Marin, J.-M., Robert, C.P., Titterington, D.M.: A Bayesian reassessment of nearest-neighbor classification. J. Am. Stat. Assoc. 104(485), 73–263 (2009)
Del Moral, P., Doucet, A., Jasra, A.: An adaptive sequential Monte Carlo method for approximate Bayesian computation. Stat. Comput. 22(5), 1009–1020 (2012)
Douc, R., Cappé, O., Moulines, E.: Comparison of resampling schemes for particle filtering. In: Proceedings of 4th International Symposium image and signal processing and analysis (ISPA), pp. 64–69 (2005)
Drovandi, C.C., Pettitt, A.N.: Estimation of parameters for macroparasite population evolution using approximate Bayesian computation. Biometrics 67(1), 225–33 (2011)
Drovandi, C.C., Pettitt, A.N., Faddy, M.J.: Approximate Bayesian computation using indirect inference. J. R. Stat. Soc. Ser. C 60(3), 317–37 (2011)
Drovandi, C.C., Pettitt, A.N., Lee, A.: Bayesian indirect inference using a parametric auxiliary model. Stat. Sci. (2014). http://www.imstat.org/sts/future_papers.html
Eddelbuettel, D., Sanderson, C.: RcppArmadillo: accelerating R with high-performance C++ linear algebra. Comput. Stat. Data Anal. 71, 1054–1063 (2014)
Everitt, R.G.: Bayesian parameter estimation for latent Markov random fields and social networks. J. Comput. Graph. Stat. 21(4), 940–60 (2012)
Filippi, S., Barnes, C.P., Cornebise, J., Stumpf, M.P.H.: On optimality of kernels for approximate Bayesian computation using sequential Monte Carlo. Stat. Appl. Genet. Mol. Biol. 12(1), 87–107 (2013)
Friel, N., Pettitt, A.N.: Classification using distance nearest neighbours. Stat. Comput. 21(3), 431–37 (2011)
Gouriéroux, C., Monfort, A., Renault, E.: Indirect inference. J. Appl. Econom. 8(S1), S85–S118 (1993)
Grelaud, A., Robert, C.P., Marin, J.M., Rodolphe, F., Taly, J.F.: ABC likelihood-free methods for model choice in Gibbs random fields. Bayesian Anal. 4(2), 317–36 (2009)
Higdon, D.M.: Auxiliary variable methods for Markov chain Monte Carlo with applications. J. Am. Stat. Assoc. 93(442), 585–95 (1998)
Hurn, M.A.: Difficulties in the use of auxiliary variables in Markov chain Monte Carlo methods. Stat. Comput. 7, 35–44 (1997)
Jasra, A., Singh, S.S., Martin, J.S., McCoy, E.: Filtering via approximate Bayesian computation. Stat. Comput. 22, 1223–37 (2012)
Kitagawa, G.: Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. J. Comput. Graph. Stat. 5(1), 1–25 (1996)
Liu, J.S.: Monte Carlo Strategies in Scientific Computing. Springer, New York (2001)
McGrory, C.A., Titterington, D., Reeves, R., Pettitt, A.N.: Variational Bayes for estimating the parameters of a hidden Potts model. Stat. Comput. 19(3), 329–40 (2009)
Murray, I., Ghahramani, Z., MacKay, D.J.C.: MCMC for doubly-intractable distributions. In: Proceedings of 22nd Conference UAI, pp. 359–66, AUAI Press, Arlington, VA (2006)
NASA: Landsat 7 science data users handbook. Tech. rep., National Aeronautics and Space Administration (2011). http://www.landsathandbook.gsfc.nasa.gov/
Potts, R.B.: Some generalized order-disorder transformations. Proc. Camb. Philos. Soc. 48, 106–9 (1952)
Pritchard, J.K., Seielstad, M.T., Perez-Lezaun, A., Feldman, M.W.: Population growth of human Y chromosomes: a study of Y chromosome microsatellites. Mol. Biol. Evol. 16(12), 1791–1798 (1999)
Ratmann, O., Camacho, A., Meijer, A., Donker, G.: Statistical modelling of summary values leads to accurate Approximate Bayesian Computations. Techical report (2014). arXiv:1305.4283
Sedki, M., Pudlo, P., Marin, J.-M., Robert, C.P., Cornuet, J.-M.: Efficient learning in ABC algorithms. Technical report (2013). arXiv:1210.1388
Sisson, S.A., Fan, Y., Tanaka, M.M.: Sequential Monte Carlo without likelihoods. Proc. Natl. Acad. Sci. 104(6), 1760–1765 (2007)
Stoehr, J., Pudlo, P., Cucala, L.: Geometric summary statistics for ABC model choice between hidden Gibbs random fields. Stat. Comput. (2014). doi:10.1007/s11222-014-9514-9
Swendsen, R.H., Wang, J.S.: Nonuniversal critical dynamics in Monte Carlo simulations. Phys. Rev. Lett. 58, 86–88 (1987)
Toni, T., Welch, D., Strelkowa, N., Ipsen, A., Stumpf, M.P.H.: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface 6(31), 187–202 (2009)
Wilkinson, R.: accelerating ABC methods using Gaussian processes. In: Proceedings of 17th International Conference AISTATS, JMLR W&CP 33, 1015–23 (2014)
Winkler, G.: Image Analysis, Random Fields and Markov chain Monte Carlo Methods: A Mathematical Introduction, 2nd edn. Springer-Verlag, Heidelberg (2003)
Wood, S.N.: Statistical inference for noisy nonlinear ecological dynamic systems. Nature 466, 1102–1104 (2010)
Acknowledgments
The authors would like to thank the organisers and attendees of the MCMSki conference for their interest and feedback. In particular, we are grateful to D. P. Simpson, A. Mira, and the anonymous reviewers for their thoughtful comments and suggestions on an earlier version of this manuscript. M. T. Moores acknowledges the financial support of Queensland University of Technology and the Australian federal government Department of Education, Science and Training. C.P. Robert’s research is supported by the Agence Nationale de la Recherche (ANR 2011 BS01 010 01 Project Calibration) and an Institut Universitaire de France senior Grant 2010-2016. K. L. Mengersen’s research is funded by a Discovery Project Grant from the Australian Research Council. Landsat imagery courtesy of NASA Goddard Space Flight Center and U.S. Geological Survey. Computational resources and services used in this work were provided by the HPC and Research Support Group, Queensland University of Technology, Brisbane, Australia.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Moores, M.T., Drovandi, C.C., Mengersen, K. et al. Pre-processing for approximate Bayesian computation in image analysis. Stat Comput 25, 23–33 (2015). https://doi.org/10.1007/s11222-014-9525-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-014-9525-6