# Four algorithms to construct a sparse kriging kernel for dimensionality reduction

- 65 Downloads

## Abstract

In the context of computer experiments, metamodels are largely used to represent the output of computer codes. Among these models, Gaussian process regression (kriging) is very efficient see e.g Snelson (Flexible and efficient Gaussian process models for machine learning. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–University of London, University College London, London, 2008). In high dimension that is with a large number of input variables, but with few observations, the estimation of the parameters with a classical *anisotropic* kriging can be completely inaccurate. Because there are equal numbers of ranges and input variables the optimization space becomes too large compared to available information. One way to overcome this drawback is to use an *isotropic* kernel that only depends on one parameter. However this model is too restrictive. The aim of this paper is twofold. Our first objective is to propose a smooth kernel with as few parameters as warranted. We introduce a kernel which is a tensor product of few isotropic kernels built on well-chosen subgroup of variables. The main difficulty is to find the number and the composition of the groups. Our second objective is to propose algorithmic strategies to overcome this difficulty. Four forward strategies are proposed. They all start with the simplest isotropic kernel and stop when the best model according to BIC criterion is found. They all show very good accuracy results on simulation test cases. But one of them is more efficient. Tested on a real data set, our kernel shows very good prediction results.

## Keywords

Metamodel Isotropic Anisotropic Clustering## Notes

### Acknowledgements

This work benefited from the financial support of the French ANR project “PEPITO” (ANR-14-CE23-0011).

## Supplementary material

## References

- Binois M, Ginsbourger D, Roustant O (2015) Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations. Eur J Oper Res 243(2):386–394MathSciNetCrossRefGoogle Scholar
- Cornford D, Nabney IT, Williams CKI (2002) Modelling frontal discontinuities in wind fields. J Nonparametr Stat 14(1–2):43–58 Statistical models and methods for discontinuous phenomena (Oslo, 1998)MathSciNetCrossRefGoogle Scholar
- Cressie NAC (1993) Statistics for spatial data. Wiley series in probability and mathematical statistics: applied probability and statistics. Wiley, New York. Revised reprint of the 1991 edition, A Wiley-Interscience PublicationCrossRefGoogle Scholar
- Dupuy D, Helbert C, Franco J (2015) DiceDesign and DiceEval: two R packages for design and analysis of computer experiments. J Stat Softw 65(11):1–38CrossRefGoogle Scholar
- Durrande N (2001) Étude de classes de noyaux adaptées à la simplification et à l’interprétation des modèles d’approximation. Une approche fonctionnelle et probabiliste. PhD thesis, Ecole Nationale Supérieure des Mines de Saint-EtienneGoogle Scholar
- Everitt BS, Landau S, Leese M, Stahl D (2011) Cluster analysis. Wiley series in probability and statistics, 5th edn. Wiley, ChichesterzbMATHGoogle Scholar
- Fricker TE, Oakley JE, Urban NM (2013) Multivariate Gaussian process emulators with nonseparable covariance structures. Technometrics 55(1):47–56MathSciNetCrossRefGoogle Scholar
- Gao J, Gunn S, Kandola J (2002) Adapting kernels by variational approach in SVM. In: AI 2002: advances in artificial intelligence, volume 2557 of Lecture Notes in Comput. Sci., Springer, Berlin, pp 395–406zbMATHGoogle Scholar
- Ginsbourger D, Roustant O, Schuhmacher D, Durrande N, Lenz N (2016) On ANOVA decompositions of kernels and Gaussian random field paths. In: Monte Carlo and quasi-Monte Carlo methods, volume 163 of Springer Proc. Math. Stat., Springer, Cham, pp 315–330Google Scholar
- Marrel A, Iooss B, Van Dorpe F, Volkova E (2008) An efficient methodology for modeling complex computer codes with Gaussian processes. Comput Statist Data Anal 52(10):4731–4744MathSciNetCrossRefGoogle Scholar
- Muehlenstaedt T, Roustant O, Carraro L, Kuhnt S (2012) Data-driven Kriging models based on FANOVA-decomposition. Stat Comput 22(3):723–738MathSciNetCrossRefGoogle Scholar
- Paciorek CJ, Schervish MJ (2006) Spatial modelling using a new class of nonstationary covariance functions. Environmetrics 17(5):483–506MathSciNetCrossRefGoogle Scholar
- Padonou E, Roustant O (2016) Polar Gaussian processes and experimental designs in circular domains. SIAM/ASA J Uncertain Quantif 4(1):1014–1033MathSciNetCrossRefGoogle Scholar
- Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. Adaptive computation and machine learning. MIT Press, Cambridge, MAzbMATHGoogle Scholar
- Santner TJ, Williams BJ, Notz WI (2003) The design and analysis of computer experiments. Springer series in statistics. Springer, New YorkCrossRefGoogle Scholar
- Schwarz G (1978) Estimating the dimension of a model. Ann Statist 6(2):461–464MathSciNetCrossRefGoogle Scholar
- Snelson EL (2008) Flexible and efficient Gaussian process models for machine learning. ProQuest LLC, Ann Arbor, MI. Thesis (Ph.D.)–University of London, University College London, LondonGoogle Scholar
- Stein ML (1999) Interpolation of spatial data. Springer series in statistics. Springer, New York Some theory for KrigingCrossRefGoogle Scholar
- Stitson MO, Gammerman A, Vapnik V, Vovk V, Watkins C, Weston J (1999) Advances in kernel methods. Chapter support vector regression with ANOVA decomposition Kernels, MIT Press, Cambridge, MA, pp 285–291Google Scholar
- Sudret B (2012) Meta-models for structural reliability and uncertainty quantification. In: Asian-Pacific symposium on structural reliability and its applications. Singapore, Singapore, pp 1–24Google Scholar
- Villa-Vialaneix N, Follador M, Ratto M, Leip A (2012) A comparison of eight metamodeling techniques for the simulation of \({N}_2{O}\) fluxes and N leaching from corn crops. Environ Modell Softw 34:51–66CrossRefGoogle Scholar
- Welch WJ, Buck RJ, Sacks J, Wynn HP, Mitchell TJ, Morris MD (1992) Screening, predicting, and computer experiments. Technometrics 34(1):15–25CrossRefGoogle Scholar
- Yi G (2009) Variable selection with penalized Gaussian process regression models. PhD thesis, University of Newcastle Upon TyneGoogle Scholar