Statistics and Computing

, Volume 17, Issue 4, pp 323–335 | Cite as

Parallelizing MCMC for Bayesian spatiotemporal geostatistical models

  • Jun YanEmail author
  • Mary Kathryn Cowles
  • Shaowen Wang
  • Marc P. Armstrong


When MCMC methods for Bayesian spatiotemporal modeling are applied to large geostatistical problems, challenges arise as a consequence of memory requirements, computing costs, and convergence monitoring. This article describes the parallelization of a reparametrized and marginalized posterior sampling (RAMPS) algorithm, which is carefully designed to generate posterior samples efficiently. The algorithm is implemented using the Parallel Linear Algebra Package (PLAPACK). The scalability of the algorithm is investigated via simulation experiments that are implemented using a cluster with 25 processors. The usefulness of the method is illustrated with an application to sulfur dioxide concentration data from the Air Quality System database of the U.S. Environmental Protection Agency.


Bayesian inference Markov chain Monte Carlo Parallel computing Spatial modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adams, N.M., Kirby, S.P.J., Harris, P., Clegg, D.B.: A review of parallel processing for statistical computation. Stat. Comput. 6, 37–49 (1996) CrossRefGoogle Scholar
  2. Agarwal, D.K., Gelfand, A.E.: Slice sampling for simulation based fitting of spatial data models. Stat. Comput. 15, 61–69 (2002) CrossRefGoogle Scholar
  3. Banerjee, S., Carlin, B.P., Gelfand, A.E.: Hierarchical Modeling and Analysis for Spatial Data. Chapman & Hall/CRC, London (2004) zbMATHGoogle Scholar
  4. Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK Users’ Guide. Society for Industrial and Applied Mathematics, Philadelphia (1997) zbMATHGoogle Scholar
  5. Chib, S.: Bayes regression with autoregressive errors. A Gibbs sampling approach. J. Econom. 58, 275–294 (1993) zbMATHCrossRefGoogle Scholar
  6. Christiansen, C.L., Morris, C.N.: Hierarchical Poisson regression modeling. J. Am. Stat. Assoc. 92, 618–632 (1997) zbMATHCrossRefGoogle Scholar
  7. Cowles, M.K., Zimmerman, D.L., Christ, A., McGinnis, D.L.: Combining snow water equivalent data from multiple sources to estimate spatio-temporal trends and compare measurement systems. J. Agric. Biol. Environ. Stat. 7, 536–557 (2002) CrossRefGoogle Scholar
  8. Daniels, M.J.: A prior for the variance in hierarchical models. Can. J. Stat. 27, 567–578 (1999) zbMATHCrossRefGoogle Scholar
  9. Diggle, P., Ribeiro, P.J.: Bayesian inference in Gaussian model-based geostatistics. Geogr. Environ. Model. 6, 129–146 (2002) CrossRefGoogle Scholar
  10. Duff, I.S., van der Vorst, H.A.: Developments and trends in the parallel solution of linear systems. Parallel Comput. 25, 1931–1970 (1999) CrossRefGoogle Scholar
  11. Finley, A.O., Banerjee, S., Carlin, B.P.: spBayes: spBayes Fits Gaussian Models with Potentially Complex Hierarchical Error Structures. R package version 0.0-1 (2006).
  12. Gilks, W.R., Best, N.G., Tan, K.K.C.: Adaptive rejection Metropolis sampling within Gibbs sampling (Corr: 97V46 p. 541–542 with R.M. Neal). Appl. Stat. 44, 455–472 (1995) zbMATHCrossRefGoogle Scholar
  13. Kass, R.E., Carlin, B.P., Gelman, A., Neal, R.M.: Markov chain Monte Carlo in practice: a roundtable discussion. Am. Stat. 52, 93–100 (1998) CrossRefGoogle Scholar
  14. Neal, R.M.: Slice sampling. Ann. Stat. 31, 705–767 (2003) zbMATHCrossRefGoogle Scholar
  15. Nychka, D.: Fields: tools for spatial data. R package version 3.04 (2005).
  16. Reed, D.A.: Grids, the teragrid, and beyond. Computer 36, 62–68 (2003) CrossRefGoogle Scholar
  17. Rosenthal, J.S.: Parallel computing and Monte Carlo algorithms. Far East J. Theor. Stat. 4, 207–236 (2000) zbMATHGoogle Scholar
  18. Rossini, A., Tierney, L., Li, N.: Simple parallel statistical computing in R. Working paper, UW Biostatistics (2003) Google Scholar
  19. Rue, H.: Fast sampling of Gaussian Markov random fields. J. Roy. Stat. Soc. Ser. B: Stat. Methodol. 63, 325–338 (2001) zbMATHCrossRefGoogle Scholar
  20. Rue, H., Martino, S.: Approximate inference for hierarchical Gaussian Markov random field models, Technical report, Department of Mathematical Sciences, NTNU, Norway (2005) Google Scholar
  21. Rue, H., Tjelmeland, H.: Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Stat. 29, 31–49 (2002) zbMATHCrossRefGoogle Scholar
  22. Sargent, D.J., Hodges, J.S., Carlin, B.P.: Structured Markov chain Monte Carlo. J. Comput. Graph. Stat. 9, 217–234 (2000) CrossRefGoogle Scholar
  23. Schervish, M.J.: Applications of parallel computation to statistical inference. J. Am. Stat. Assoc. 83, 976–983 (1988) zbMATHCrossRefGoogle Scholar
  24. Sylwestrowicz, J.D.: Parallel processing in statistics. In: H. Caussinus, P. Ettinger, R. Tomassone (eds.) COMPSTAT 1982, Proceedings in Computational Statistics, pp. 131–136. Physica-Verlag Ges.m.b.H. (1982) Google Scholar
  25. van de Geijn, R.A.: Using PLAPACK. MIT Press (1997) Google Scholar
  26. Whiley, M., Wilson, S.P.: Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models. Stat. Comput. 14, 171–179 (2004) CrossRefGoogle Scholar
  27. Wilkinson, D.J.: Parallel Bayesian computation. In: Kontoghiorghes, E.J. (ed.) Handbook of Parallel Computing and Statistics, pp. 481–512. Dekker/CRC Press, New York (2005) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Jun Yan
    • 1
    Email author
  • Mary Kathryn Cowles
    • 1
    • 2
  • Shaowen Wang
    • 3
  • Marc P. Armstrong
    • 4
    • 5
  1. 1.Department of Statistics and Actuarial ScienceThe University of IowaIowaUSA
  2. 2.Department of BiostatisticsThe University of IowaIowaUSA
  3. 3.Department of Geography and National Center for Supercomputing ApplicationsUniversity of Illinois at Urbana-ChampaignIllinoisUSA
  4. 4.Department of GeographyThe University of IowaIowaUSA
  5. 5.Program in Applied Mathematical and Computational ScienceThe University of IowaIowaUSA

Personalised recommendations