Advertisement

Computational Statistics

, Volume 34, Issue 4, pp 1889–1909 | Cite as

Four algorithms to construct a sparse kriging kernel for dimensionality reduction

  • Christophette Blanchet-Scalliet
  • Céline Helbert
  • Mélina RibaudEmail author
  • Céline Vial
Original Paper
  • 65 Downloads

Abstract

In the context of computer experiments, metamodels are largely used to represent the output of computer codes. Among these models, Gaussian process regression (kriging) is very efficient see e.g Snelson (Flexible and efficient Gaussian process models for machine learning. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–University of London, University College London, London, 2008). In high dimension that is with a large number of input variables, but with few observations, the estimation of the parameters with a classical anisotropic kriging can be completely inaccurate. Because there are equal numbers of ranges and input variables the optimization space becomes too large compared to available information. One way to overcome this drawback is to use an isotropic kernel that only depends on one parameter. However this model is too restrictive. The aim of this paper is twofold. Our first objective is to propose a smooth kernel with as few parameters as warranted. We introduce a kernel which is a tensor product of few isotropic kernels built on well-chosen subgroup of variables. The main difficulty is to find the number and the composition of the groups. Our second objective is to propose algorithmic strategies to overcome this difficulty. Four forward strategies are proposed. They all start with the simplest isotropic kernel and stop when the best model according to BIC criterion is found. They all show very good accuracy results on simulation test cases. But one of them is more efficient. Tested on a real data set, our kernel shows very good prediction results.

Keywords

Metamodel Isotropic Anisotropic Clustering 

Notes

Acknowledgements

This work benefited from the financial support of the French ANR project “PEPITO” (ANR-14-CE23-0011).

Supplementary material

Open image in new windowFig. 7 Open image in new window Open image in new window
180_2019_874_MOESM1_ESM.R
Supplementary material 1 (R 2 KB)
180_2019_874_MOESM2_ESM.R
Supplementary material 2 (R 3 KB)
180_2019_874_MOESM3_ESM.R
Supplementary material 3 (R 4 KB)
180_2019_874_MOESM4_ESM.R
Supplementary material 4 (R 4 KB)
180_2019_874_MOESM5_ESM.R
Supplementary material 5 (R 3 KB)
180_2019_874_MOESM6_ESM.R
Supplementary material 6 (R 1 KB)
180_2019_874_MOESM7_ESM.R
Supplementary material 7 (R 2 KB)
180_2019_874_MOESM8_ESM.R
Supplementary material 8 (R 4 KB)
180_2019_874_MOESM9_ESM.R
Supplementary material 9 (R 3 KB)
180_2019_874_MOESM10_ESM.R
Supplementary material 10 (R 1 KB)
180_2019_874_MOESM11_ESM.R
Supplementary material 11 (R 11 KB)
180_2019_874_MOESM12_ESM.R
Supplementary material 12 (R 4 KB)

References

  1. Binois M, Ginsbourger D, Roustant O (2015) Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations. Eur J Oper Res 243(2):386–394MathSciNetCrossRefGoogle Scholar
  2. Cornford D, Nabney IT, Williams CKI (2002) Modelling frontal discontinuities in wind fields. J Nonparametr Stat 14(1–2):43–58 Statistical models and methods for discontinuous phenomena (Oslo, 1998)MathSciNetCrossRefGoogle Scholar
  3. Cressie NAC (1993) Statistics for spatial data. Wiley series in probability and mathematical statistics: applied probability and statistics. Wiley, New York. Revised reprint of the 1991 edition, A Wiley-Interscience PublicationCrossRefGoogle Scholar
  4. Dupuy D, Helbert C, Franco J (2015) DiceDesign and DiceEval: two R packages for design and analysis of computer experiments. J Stat Softw 65(11):1–38CrossRefGoogle Scholar
  5. Durrande N (2001) Étude de classes de noyaux adaptées à la simplification et à l’interprétation des modèles d’approximation. Une approche fonctionnelle et probabiliste. PhD thesis, Ecole Nationale Supérieure des Mines de Saint-EtienneGoogle Scholar
  6. Everitt BS, Landau S, Leese M, Stahl D (2011) Cluster analysis. Wiley series in probability and statistics, 5th edn. Wiley, ChichesterzbMATHGoogle Scholar
  7. Fricker TE, Oakley JE, Urban NM (2013) Multivariate Gaussian process emulators with nonseparable covariance structures. Technometrics 55(1):47–56MathSciNetCrossRefGoogle Scholar
  8. Gao J, Gunn S, Kandola J (2002) Adapting kernels by variational approach in SVM. In: AI 2002: advances in artificial intelligence, volume 2557 of Lecture Notes in Comput. Sci., Springer, Berlin, pp 395–406zbMATHGoogle Scholar
  9. Ginsbourger D, Roustant O, Schuhmacher D, Durrande N, Lenz N (2016) On ANOVA decompositions of kernels and Gaussian random field paths. In: Monte Carlo and quasi-Monte Carlo methods, volume 163 of Springer Proc. Math. Stat., Springer, Cham, pp 315–330Google Scholar
  10. Marrel A, Iooss B, Van Dorpe F, Volkova E (2008) An efficient methodology for modeling complex computer codes with Gaussian processes. Comput Statist Data Anal 52(10):4731–4744MathSciNetCrossRefGoogle Scholar
  11. Muehlenstaedt T, Roustant O, Carraro L, Kuhnt S (2012) Data-driven Kriging models based on FANOVA-decomposition. Stat Comput 22(3):723–738MathSciNetCrossRefGoogle Scholar
  12. Paciorek CJ, Schervish MJ (2006) Spatial modelling using a new class of nonstationary covariance functions. Environmetrics 17(5):483–506MathSciNetCrossRefGoogle Scholar
  13. Padonou E, Roustant O (2016) Polar Gaussian processes and experimental designs in circular domains. SIAM/ASA J Uncertain Quantif 4(1):1014–1033MathSciNetCrossRefGoogle Scholar
  14. Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. Adaptive computation and machine learning. MIT Press, Cambridge, MAzbMATHGoogle Scholar
  15. Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer series in statistics. Springer, New YorkCrossRefGoogle Scholar
  16. Schwarz G (1978) Estimating the dimension of a model. Ann Statist 6(2):461–464MathSciNetCrossRefGoogle Scholar
  17. Snelson EL (2008) Flexible and efficient Gaussian process models for machine learning. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–University of London, University College London, LondonGoogle Scholar
  18. Stein ML (1999) Interpolation of spatial data. Springer series in statistics. Springer, New York Some theory for KrigingCrossRefGoogle Scholar
  19. Stitson MO, Gammerman A, Vapnik V, Vovk V, Watkins C, Weston J (1999) Advances in kernel methods. Chapter support vector regression with ANOVA decomposition Kernels, MIT Press, Cambridge, MA, pp 285–291Google Scholar
  20. Sudret B (2012) Meta-models for structural reliability and uncertainty quantification. In: Asian-Pacific symposium on structural reliability and its applications. Singapore, Singapore, pp 1–24Google Scholar
  21. Villa-Vialaneix N, Follador M, Ratto M, Leip A (2012) A comparison of eight metamodeling techniques for the simulation of \({N}_2{O}\) fluxes and N leaching from corn crops. Environ Modell Softw 34:51–66CrossRefGoogle Scholar
  22. Welch WJ, Buck RJ, Sacks J, Wynn HP, Mitchell TJ, Morris MD (1992) Screening, predicting, and computer experiments. Technometrics 34(1):15–25CrossRefGoogle Scholar
  23. Yi G (2009) Variable selection with penalized Gaussian process regression models. PhD thesis, University of Newcastle Upon TyneGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.École centrale de Lyon, CNRS UMR 5208, Institut Camille JordanUniv LyonEcully CedexFrance
  2. 2.CNRS UMR 5208, Institut Camille JordanUniv Lyon, Université Claude Bernard Lyon 1Villeurbanne cedexFrance
  3. 3.INRIAVilleurbanneFrance

Personalised recommendations