Abstract
When MCMC methods for Bayesian spatiotemporal modeling are applied to large geostatistical problems, challenges arise as a consequence of memory requirements, computing costs, and convergence monitoring. This article describes the parallelization of a reparametrized and marginalized posterior sampling (RAMPS) algorithm, which is carefully designed to generate posterior samples efficiently. The algorithm is implemented using the Parallel Linear Algebra Package (PLAPACK). The scalability of the algorithm is investigated via simulation experiments that are implemented using a cluster with 25 processors. The usefulness of the method is illustrated with an application to sulfur dioxide concentration data from the Air Quality System database of the U.S. Environmental Protection Agency.
Similar content being viewed by others
References
Adams, N.M., Kirby, S.P.J., Harris, P., Clegg, D.B.: A review of parallel processing for statistical computation. Stat. Comput. 6, 37–49 (1996)
Agarwal, D.K., Gelfand, A.E.: Slice sampling for simulation based fitting of spatial data models. Stat. Comput. 15, 61–69 (2002)
Banerjee, S., Carlin, B.P., Gelfand, A.E.: Hierarchical Modeling and Analysis for Spatial Data. Chapman & Hall/CRC, London (2004)
Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia (1997)
Chib, S.: Bayes regression with autoregressive errors. A Gibbs sampling approach. J. Econom. 58, 275–294 (1993)
Christiansen, C.L., Morris, C.N.: Hierarchical Poisson regression modeling. J. Am. Stat. Assoc. 92, 618–632 (1997)
Cowles, M.K., Zimmerman, D.L., Christ, A., McGinnis, D.L.: Combining snow water equivalent data from multiple sources to estimate spatio-temporal trends and compare measurement systems. J. Agric. Biol. Environ. Stat. 7, 536–557 (2002)
Daniels, M.J.: A prior for the variance in hierarchical models. Can. J. Stat. 27, 567–578 (1999)
Diggle, P., Ribeiro, P.J.: Bayesian inference in Gaussian model-based geostatistics. Geogr. Environ. Model. 6, 129–146 (2002)
Duff, I.S., van der Vorst, H.A.: Developments and trends in the parallel solution of linear systems. Parallel Comput. 25, 1931–1970 (1999)
Finley, A.O., Banerjee, S., Carlin, B.P.: spBayes: spBayes Fits Gaussian Models with Potentially Complex Hierarchical Error Structures. R package version 0.0-1 (2006). http://blue.fr.umn.edu/spatialBayes
Gilks, W.R., Best, N.G., Tan, K.K.C.: Adaptive rejection Metropolis sampling within Gibbs sampling (Corr: 97V46 p. 541–542 with R.M. Neal). Appl. Stat. 44, 455–472 (1995)
Kass, R.E., Carlin, B.P., Gelman, A., Neal, R.M.: Markov chain Monte Carlo in practice: a roundtable discussion. Am. Stat. 52, 93–100 (1998)
Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003)
Nychka, D.: Fields: tools for spatial data. R package version 3.04 (2005). http://www.image.ucar.edu/GSP/Software/Fields
Reed, D.A.: Grids, the teragrid, and beyond. Computer 36, 62–68 (2003)
Rosenthal, J.S.: Parallel computing and Monte Carlo algorithms. Far East J. Theor. Stat. 4, 207–236 (2000)
Rossini, A., Tierney, L., Li, N.: Simple parallel statistical computing in R. Working paper, UW Biostatistics (2003)
Rue, H.: Fast sampling of Gaussian Markov random fields. J. Roy. Stat. Soc. Ser. B: Stat. Methodol. 63, 325–338 (2001)
Rue, H., Martino, S.: Approximate inference for hierarchical Gaussian Markov random field models, Technical report, Department of Mathematical Sciences, NTNU, Norway (2005)
Rue, H., Tjelmeland, H.: Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Stat. 29, 31–49 (2002)
Sargent, D.J., Hodges, J.S., Carlin, B.P.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217–234 (2000)
Schervish, M.J.: Applications of parallel computation to statistical inference. J. Am. Stat. Assoc. 83, 976–983 (1988)
Sylwestrowicz, J.D.: Parallel processing in statistics. In: H. Caussinus, P. Ettinger, R. Tomassone (eds.) COMPSTAT 1982, Proceedings in Computational Statistics, pp. 131–136. Physica-Verlag Ges.m.b.H. (1982)
van de Geijn, R.A.: Using PLAPACK. MIT Press (1997)
Whiley, M., Wilson, S.P.: Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models. Stat. Comput. 14, 171–179 (2004)
Wilkinson, D.J.: Parallel Bayesian computation. In: Kontoghiorghes, E.J. (ed.) Handbook of Parallel Computing and Statistics, pp. 481–512. Dekker/CRC Press, New York (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yan, J., Cowles, M.K., Wang, S. et al. Parallelizing MCMC for Bayesian spatiotemporal geostatistical models. Stat Comput 17, 323–335 (2007). https://doi.org/10.1007/s11222-007-9022-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-007-9022-2