Fast algorithms for automatic mapping with space-limited covariance functions

Original Paper


In this paper we discuss a fast Bayesian extension to kriging algorithms which has been used successfully for fast, automatic mapping in emergency conditions in the Spatial Interpolation Comparison 2004 (SIC2004) exercise. The application of kriging to automatic mapping raises several issues such as robustness, scalability, speed and parameter estimation. Various ad-hoc solutions have been proposed and used extensively but they lack a sound theoretical basis. In this paper we show how observations can be projected onto a representative subset of the data, without losing significant information. This allows the complexity of the algorithm to grow as O(n m 2), where n is the total number of observations and m is the size of the subset of the observations retained for prediction. The main contribution of this paper is to further extend this projective method through the application of space-limited covariance functions, which can be used as an alternative to the commonly used covariance models. In many real world applications the correlation between observations essentially vanishes beyond a certain separation distance. Thus it makes sense to use a covariance model that encompasses this belief since this leads to sparse covariance matrices for which optimised sparse matrix techniques can be used. In the presence of extreme values we show that space-limited covariance functions offer an additional benefit, they maintain the smoothness locally but at the same time lead to a more robust, and compact, global model. We show the performance of this technique coupled with the sparse extension to the kriging algorithm on synthetic data and outline a number of computational benefits such an approach brings. To test the relevance to automatic mapping we apply the method to the data used in a recent comparison of interpolation techniques (SIC2004) to map the levels of background ambient gamma radiation.



This work was partially supported by the BBSRC contract 92/EGM17737. The SIC2004 data was obtained from []. This work is partially funded by the European Commission, under the Sixth Framework Programme, by the Contract N. 033811 with DG INFSO, action Line IST-2005-2.5.12 ICT for Environmental Risk Management. The views expressed herein are those of the authors and are not necessarily those of the European Commission.


  1. Barry R, Pace RK (1997) Kriging with large data sets using sparse matrix techniques. Commun Stat Comput Simulation 26(2):619–629CrossRefGoogle Scholar
  2. Cornford D, Csató L, Evans DJ, Opper M (2004) Bayesian analysis of the scatterometer wind retieval inverse problems: some new approaches. J Roy Stat Soc Ser B (Statistical Methodology) 66(3):609–626CrossRefGoogle Scholar
  3. Cressie NA (2006) Spatial prediction for massive datasets. In: Mastering the data explosion in the earth and environmental sciencesGoogle Scholar
  4. Csató L (2002) Gaussian processes—iterative sparse approximation. PhD thesis, Neural Computing Research Group,, March 2002
  5. Csató L, Opper M (2001) Sparse representation for Gaussian process models. In: NIPS13ed, editor, NIPS, vol 13. MIT, Cambridge, pp 444–450Google Scholar
  6. Csató L, Opper M (2002) Sparse on-line Gaussian processes. Neural Comput 14(3):641–669CrossRefGoogle Scholar
  7. Davis MW (1976) The practice of kriging. In: Guarascio M, David M, Huijbregts D (eds) Advanced geostatistics in the mining industry. D. Reidell, Boston, p 31Google Scholar
  8. Davis MW, Grivet C (1984) Kriging in a global neighborhood. Math Geol 16:249–265CrossRefGoogle Scholar
  9. Dietrich MW, Newsam GN (1989) A stability analysis of the geostatistical approach to aquifer transmissivity identification. Stochastic Environ Res Risk Assess 3:293–316Google Scholar
  10. Diggle PJ, Tawn JA, Moyeed RA (1998) Model-based geostatistics. Appl Stat 47:299–350Google Scholar
  11. Dubois G, Galmarini S (2005) Spatial interpolation comparison (SIC) 2004: introduction to the exercise and overview of results. In: Automatic mapping algorithms for routine and emergency monitoring dataGoogle Scholar
  12. Fine S, Scheinberf K (2002) Effcient svm training using low-rank kernel representations. J Mach Learn Res 2:243–264CrossRefGoogle Scholar
  13. Furrer R, Genton MG, Nychka D (2006) Covariance tapering for interpolation of large spatial datasets. J Comput Graph Stat 15:502–523CrossRefGoogle Scholar
  14. Gaspari G, Cohn S (1996) Construction of correlation functions in two and three dimensions. Technical Report DAO Office Note 96-03, NASA Goddard Space Flight Center, Greenbelt, Maryland Google Scholar
  15. Gibbs MN (1997) Bayesian Gaussian Processes for Regression and Classification. PhD thesis, Cambridge University, Cambridge.
  16. Golub GH, Van Loan CF (1989) Matrix computations, 2nd edn. Johns Hopkins University Press, BaltimoreGoogle Scholar
  17. Ingram B, Csato L, Evans D (2005) Fast spatial interpolation using sparse gaussian processes. Applied GIS 1(2), Monash University ePressGoogle Scholar
  18. Lawrence ND, Herbrich R (2001) A sparse Bayesian compression scheme—the Informative Vector machine. In NIPS 2001Google Scholar
  19. Lawrence ND, Seeger M, Herbrich R (2002) Fast sparse Gaussian process methods: the Informative Vector machine. In: Becker S, Thrun S, Obermayer K (eds), Advances in neural information processing systems, vol 15. Cambridge, MA: MIT Press, pp 625–632Google Scholar
  20. Liu W-H, Sherman AH (1976) Comparative analysis of the Cuthill–McKee and the reverse Cuthill–McKee ordering algorithms for sparse matrices. SIAM J Numer Anal 13(2):198–213CrossRefGoogle Scholar
  21. Long AE (1994) Cokriging, kernels, and the SVD: toward better geostatistical analysis. PhD thesis, The University of ArizonaGoogle Scholar
  22. MacKay DJC (1998) Introduction to Gaussian processes. In: Bishop CM (ed) Neural networks and machine learning. NATO ASI Series. Kluwer, Dordrecht, pp 133–166Google Scholar
  23. Menzefricke U (1995) On the performance of the gibbs sampler for the multivariate normal distribution. Commun Stat Theory Methods 24:191–213CrossRefGoogle Scholar
  24. Seeger M (2003) Bayesian Gaussian process models: PAC-Bayesian generalisation error bounds and sparse approximations. PhD thesis, University of EdinburghGoogle Scholar
  25. Seeger M, Williams C (2003) Fast forward selection to speed up sparse Gaussian process regression. In Workshop on AI and Statistics 9Google Scholar
  26. Vargas-Guzmán JA, Yeh T-CJ (1999) Sequential kriging and cokriging: two powerful geostatistical approaches. Stochastic Environ Res Risk Assess 13:416–435CrossRefGoogle Scholar
  27. Williams CKI, Rasmussen CE, Schwaighofer A, Tresp V (2002) Observations on the Nyström method for Gaussian process prediction. Technical report, University of EdinburghGoogle Scholar
  28. Williams CKI, Seeger M (2001) Using the nyström method to speed up kernel machines. Neural Inf Process Syst 13:682–688Google Scholar

Copyright information

© Springer-Verlag 2007

Authors and Affiliations

  1. 1.Neural Computing Research GroupAston UniversityBirminghamUK

Personalised recommendations