Iterative numerical methods for sampling from high dimensional Gaussian distributions
- 845 Downloads
Many applications require efficient sampling from Gaussian distributions. The method of choice depends on the dimension of the problem as well as the structure of the covariance- (Σ) or precision matrix (Q). The most common black-box routine for computing a sample is based on Cholesky factorization. In high dimensions, computing the Cholesky factor of Σ or Q may be prohibitive due to accumulation of more non-zero entries in the factor than is possible to store in memory. We compare different methods for computing the samples iteratively adapting ideas from numerical linear algebra. These methods assume that matrix vector products, Qv, are fast to compute. We show that some of the methods are competitive and faster than Cholesky sampling and that a parallel version of one method on a Graphical Processing Unit (GPU) using CUDA can introduce a speed-up of up to 30x. Moreover, one method is used to sample from the posterior distribution of petroleum reservoir parameters in a North Sea field, given seismic reflection data on a large 3D grid.
KeywordsGaussian distribution Krylov methods Numerical linear algebra Sampling
We thank Statoil for permission to use the Norne dataset, François Alouges for helpful discussion of the deformation method and Daniel P. Simpson for insightful discussions on the use of Krylov methods for computing matrix functions.
- Aune, E., Simpson, D.P.: Parameter estimation in high dimensional Gaussian distributions. arXiv:1105.5256v1 [stat.CO] (2011)
- Gray, R.: Toeplitz and Circulant Matrices: A Review. E-book (2006) Google Scholar
- Jegerlehner, B.: Krylov space solvers for shifted linear systems. arXiv:hep-lat/9612014v1 (1996)
- MATLAB: Version 7.11.0 (R2010b). The MathWorks Inc., Natick (2010) Google Scholar
- Parker, A., Fox, C.: Sampling Gaussian distributions in Krylov spaces with conjugate gradients. SIAM J. Sci. Comput. (2011, submitted) Google Scholar
- Shewchuk, J.: An introduction to the conjugate gradient method without the agonizing pain. http://www.cs.colorado.edu/~jessup/csci5646/READINGS/painless-conjugate-gradient.pdf (1994)
- Simpson, D.: Krylov subspace methods for approximating functions of symmetric positive definite matrices with applications to applied statistics and anomalous diffusion. PhD thesis, School of Mathematical Sciences, Queensland Univ of Tech (2008) Google Scholar
- Simpson, D., Turner, I., Pettitt, A.: Fast sampling from a Gaussian Markov random field using Krylov subspace approaches. Technical report, School of Mathematical Sciences, Queensland Univ. of Tech. (2008) Google Scholar
- Zolotarev, E.I.: Applications of elliptic functions to questions of functions deviating least and most from zero. Zap. Imp. Nauk St. Petersb. 30 (1877) Google Scholar