Parallelizing MCMC for Bayesian spatiotemporal geostatistical models


When MCMC methods for Bayesian spatiotemporal modeling are applied to large geostatistical problems, challenges arise as a consequence of memory requirements, computing costs, and convergence monitoring. This article describes the parallelization of a reparametrized and marginalized posterior sampling (RAMPS) algorithm, which is carefully designed to generate posterior samples efficiently. The algorithm is implemented using the Parallel Linear Algebra Package (PLAPACK). The scalability of the algorithm is investigated via simulation experiments that are implemented using a cluster with 25 processors. The usefulness of the method is illustrated with an application to sulfur dioxide concentration data from the Air Quality System database of the U.S. Environmental Protection Agency.

This is a preview of subscription content, access via your institution.


  1. Adams, N.M., Kirby, S.P.J., Harris, P., Clegg, D.B.: A review of parallel processing for statistical computation. Stat. Comput. 6, 37–49 (1996)

    Article  Google Scholar 

  2. Agarwal, D.K., Gelfand, A.E.: Slice sampling for simulation based fitting of spatial data models. Stat. Comput. 15, 61–69 (2002)

    Article  Google Scholar 

  3. Banerjee, S., Carlin, B.P., Gelfand, A.E.: Hierarchical Modeling and Analysis for Spatial Data. Chapman & Hall/CRC, London (2004)

    Google Scholar 

  4. Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia (1997)

    Google Scholar 

  5. Chib, S.: Bayes regression with autoregressive errors. A Gibbs sampling approach. J. Econom. 58, 275–294 (1993)

    MATH  Article  Google Scholar 

  6. Christiansen, C.L., Morris, C.N.: Hierarchical Poisson regression modeling. J. Am. Stat. Assoc. 92, 618–632 (1997)

    MATH  Article  Google Scholar 

  7. Cowles, M.K., Zimmerman, D.L., Christ, A., McGinnis, D.L.: Combining snow water equivalent data from multiple sources to estimate spatio-temporal trends and compare measurement systems. J. Agric. Biol. Environ. Stat. 7, 536–557 (2002)

    Article  Google Scholar 

  8. Daniels, M.J.: A prior for the variance in hierarchical models. Can. J. Stat. 27, 567–578 (1999)

    MATH  Article  Google Scholar 

  9. Diggle, P., Ribeiro, P.J.: Bayesian inference in Gaussian model-based geostatistics. Geogr. Environ. Model. 6, 129–146 (2002)

    Article  Google Scholar 

  10. Duff, I.S., van der Vorst, H.A.: Developments and trends in the parallel solution of linear systems. Parallel Comput. 25, 1931–1970 (1999)

    Article  Google Scholar 

  11. Finley, A.O., Banerjee, S., Carlin, B.P.: spBayes: spBayes Fits Gaussian Models with Potentially Complex Hierarchical Error Structures. R package version 0.0-1 (2006).

  12. Gilks, W.R., Best, N.G., Tan, K.K.C.: Adaptive rejection Metropolis sampling within Gibbs sampling (Corr: 97V46 p. 541–542 with R.M. Neal). Appl. Stat. 44, 455–472 (1995)

    MATH  Article  Google Scholar 

  13. Kass, R.E., Carlin, B.P., Gelman, A., Neal, R.M.: Markov chain Monte Carlo in practice: a roundtable discussion. Am. Stat. 52, 93–100 (1998)

    Article  Google Scholar 

  14. Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003)

    MATH  Article  Google Scholar 

  15. Nychka, D.: Fields: tools for spatial data. R package version 3.04 (2005).

  16. Reed, D.A.: Grids, the teragrid, and beyond. Computer 36, 62–68 (2003)

    Article  Google Scholar 

  17. Rosenthal, J.S.: Parallel computing and Monte Carlo algorithms. Far East J. Theor. Stat. 4, 207–236 (2000)

    MATH  Google Scholar 

  18. Rossini, A., Tierney, L., Li, N.: Simple parallel statistical computing in R. Working paper, UW Biostatistics (2003)

  19. Rue, H.: Fast sampling of Gaussian Markov random fields. J. Roy. Stat. Soc. Ser. B: Stat. Methodol. 63, 325–338 (2001)

    MATH  Article  Google Scholar 

  20. Rue, H., Martino, S.: Approximate inference for hierarchical Gaussian Markov random field models, Technical report, Department of Mathematical Sciences, NTNU, Norway (2005)

  21. Rue, H., Tjelmeland, H.: Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Stat. 29, 31–49 (2002)

    MATH  Article  Google Scholar 

  22. Sargent, D.J., Hodges, J.S., Carlin, B.P.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217–234 (2000)

    Article  Google Scholar 

  23. Schervish, M.J.: Applications of parallel computation to statistical inference. J. Am. Stat. Assoc. 83, 976–983 (1988)

    MATH  Article  Google Scholar 

  24. Sylwestrowicz, J.D.: Parallel processing in statistics. In: H. Caussinus, P. Ettinger, R. Tomassone (eds.) COMPSTAT 1982, Proceedings in Computational Statistics, pp. 131–136. Physica-Verlag Ges.m.b.H. (1982)

  25. van de Geijn, R.A.: Using PLAPACK. MIT Press (1997)

  26. Whiley, M., Wilson, S.P.: Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models. Stat. Comput. 14, 171–179 (2004)

    Article  Google Scholar 

  27. Wilkinson, D.J.: Parallel Bayesian computation. In: Kontoghiorghes, E.J. (ed.) Handbook of Parallel Computing and Statistics, pp. 481–512. Dekker/CRC Press, New York (2005)

    Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Jun Yan.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Yan, J., Cowles, M.K., Wang, S. et al. Parallelizing MCMC for Bayesian spatiotemporal geostatistical models. Stat Comput 17, 323–335 (2007).

Download citation


  • Bayesian inference
  • Markov chain Monte Carlo
  • Parallel computing
  • Spatial modeling