Rate of Convergence of the Gibbs Sampler in the Gaussian Case

Abstract

We show that the Gibbs Sampler in the Gaussian case is closely linked to linear fixed point iterations. In fact stochastic linear iterations converge toward a stationary distribution under the same conditions as the classical linear fixed point one. Furthermore the covariance matrices are shown to satisify a related fixed point iteration, and consequently the Gibbs Sampler in the gaussian case corresponds to the classical Gauss-Seidel iterations on the inverse of the covariance matrix, and the stochastic over-relaxed Gauss-Seidel has the same limiting distribution as the Gibbs Sampler. Then an efficient method to simulate a gaussian vector is proposed. Finally numerical investigations are performed to understand the effect of the different strategies such as the initial ordering, the blocking and the updating order for iterations. The results show that in a geostatistical context the rate of convergence can be improved significantly compared to the standard case.