# Gaussian Process Emulators for Computer Experiments with Inequality Constraints

- 516 Downloads
- 1 Citations

## Abstract

Physical phenomena are observed in many fields (science and engineering) and are often studied by time-consuming computer codes. These codes are analyzed with statistical models, often called emulators. In many situations, the physical system (computer model output) may be known to satisfy inequality constraints with respect to some or all input variables. The aim is to build a model capable of incorporating both data interpolation and inequality constraints into a Gaussian process emulator. By using a functional decomposition, a finite-dimensional approximation of Gaussian processes such that all conditional simulations satisfy the inequality constraints in the entire domain is proposed. To show the performance of the proposed model, some conditional simulations with inequality constraints such as boundedness, monotonicity or convexity conditions in one and two dimensions are given. A simulation study to investigate the efficiency of the method in terms of prediction is included.

## Keywords

Gaussian process emulator Inequality constraints Finite-dimensional approximation Uncertainty quantification Design and modeling of computer experiments## Notes

### Acknowledgements

The authors would like to thank the Associate Editor and the two anonymous referees for their helpful comments. Part of this work has been conducted within the frame of the ReDice Consortium, gathering industrial (CEA, EDF, IFPEN, IRSN, Renault) and academic (École des Mines de Saint-Étienne, INRIA, and the University of Bern) partners around advanced methods for Computer Experiments. The authors also thank Olivier Roustant (ENSM-SE), Laurence Grammont (ICJ, Lyon1) and Yann Richet (IRSN) for helpful discussions as well as the participants of the UCM2014 conference and GRF-Sim2014 workshop.

## References

- Abrahamsen P, Benth FE (2001) Kriging with inequality constraints. Math Geol 33(6):719–744CrossRefGoogle Scholar
- Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 68(3):337–404CrossRefGoogle Scholar
- Bay X, Grammont L, Maatouk H (2015) A new method for interpolating in a convex subset of a Hilbert space Hal-01136466. https://hal.archives-ouvertes.fr/hal-01136466
- Bay X, Grammont L, Maatouk H (2016) Generalization of the Kimeldorf–Wahba correspondence for constrained interpolation. Electron J Stat 10(1):1580–1595CrossRefGoogle Scholar
- Botts C (2013) An accept–reject algorithm for the positive multivariate normal distribution. Comput Stat 28(4):1749–1773CrossRefGoogle Scholar
- Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, New YorkCrossRefGoogle Scholar
- Chopin N (2011) Fast simulation of truncated Gaussian distributions. Stat Comput 21(2):275–288CrossRefGoogle Scholar
- Cousin A, Maatouk H, Rullière D (2016) Kriging of financial term-structures. Eur J Oper Res 255(2):631–648CrossRefGoogle Scholar
- Cramer H, Leadbetter R (1967) Stationary and related stochastic processes: sample function properties and their applications. Wiley series in probability and mathematical statistics. Tracts on probability and statistics. Wiley, HobokenGoogle Scholar
- Cressie N, Johannesson G (2008) Fixed rank kriging for very large spatial data sets. J R Stat Soc Ser B Stat Methodol 70(1):209–226CrossRefGoogle Scholar
- Da Veiga S, Marrel A (2012) Gaussian process modeling with inequality constraints. Ann Fac Sci Toulouse 21(3):529–555CrossRefGoogle Scholar
- Dole D (1999) CoSmo: a constrained scatterplot smoother for estimating convex, monotonic transformations. J Bus Econ Stat 17(4):444–455Google Scholar
- Emery X, Arroyo D, Peláez M (2014) Simulating large Gaussian random vectors subject to inequality constraints by gibbs sampling. Math Geosci 46(3):265–283Google Scholar
- Freulon X, de Fouquet C (1993) Conditioning a Gaussian model with inequalities. In: Geostatistics Troia92. Springer, Netherlands, pp 201–212Google Scholar
- Fritsch FN, Carlson RE (1980) Monotone piecewise cubic interpolation. SIAM J Numer Anal 17(2):238–246CrossRefGoogle Scholar
- Geweke J (1991) Efficient simulation from the multivariate normal and student-t distributions subject to linear constraints and the evaluation of constraint probabilities. In: Computing science and statistics: proceedings of the 23rd symposium on the interface, pp 571–578Google Scholar
- Golchi S, Bingham D, Chipman H, Campbell DA (2015) Monotone emulation of computer experiments. SIAM/ASA J Uncertain Quantif 3(1):370–392CrossRefGoogle Scholar
- Goldfarb D, Idnani A (1982) Dual and primal-dual methods for solving strictly convex quadratic programs. In: Numerical analysis. Springer, Berlin, pp 226–239Google Scholar
- Goldfarb D, Idnani A (1983) A numerically stable dual method for solving strictly convex quadratic programs. Math Program 27(1):1–33CrossRefGoogle Scholar
- Jones DR, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492CrossRefGoogle Scholar
- Kimeldorf GS, Wahba G (1970) A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann Math Stat 41(2):495–502CrossRefGoogle Scholar
- Kleijnen JP, Van Beers WC (2013) Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations. J Oper Res Soc 64(5):708–717CrossRefGoogle Scholar
- Kotecha JH, Djuric PM (1999) Gibbs sampling approach for generation of truncated multivariate Gaussian random variables. In: Proceedings on 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1999, vol 3, pp 1757–1760Google Scholar
- Maatouk H (2015) Correspondence between Gaussian process regression and interpolation splines under linear inequality constraints. Theory and applications. Ph.D. thesis, École des Mines de St-ÉtienneGoogle Scholar
- Maatouk H, Bay X (2016) A new rejection sampling method for truncated multivariate Gaussian random variables restricted to convex sets. In: Nuyens R, Cools R (ed) Monte carlo and quasi-monte carlo methods, vol 163. Springer International Publishing, Cham, pp 521–530Google Scholar
- Maatouk H, Richet Y (2015) constrKriging R package. https://github.com/maatouk/constrKriging
- Maatouk H, Roustant O, Richet Y (2015) Cross-validation estimations of hyper-parameters of Gaussian processes with inequality constraints. Procedia Environ Sci 27:38–44CrossRefGoogle Scholar
- Micchelli C, Utreras F (1988) Smoothing and interpolation in a convex subset of a Hilbert space. SIAM J Sci Stat Comput 9(4):728–746CrossRefGoogle Scholar
- Parzen E (1962) Stochastic processes. Holden-Day series in probability and statistics. Holden-Day, San FranciscoGoogle Scholar
- Philippe A, Robert CP (2003) Perfect simulation of positive Gaussian distributions. Stat Comput 13(2):179–186CrossRefGoogle Scholar
- Ramsay JO (1988) Monotone regression splines in action. Stat Sci 3(4):425–441CrossRefGoogle Scholar
- Rasmussen CE, Williams CK (2005) Gaussian processes for machine learning (adaptive computation and machine learning). The MIT Press, CambridgeGoogle Scholar
- Riihimaki J, Vehtari A (2010) Gaussian processes with monotonicity information. JMLR Proc AISTATS 9:645–652Google Scholar
- Robert CP (1995) Simulation of truncated normal variables. Stat Comput 5(2):121–125CrossRefGoogle Scholar
- Roustant O, Ginsbourger D, Deville Y (2012) DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J Stat Softw 51(1):1–55CrossRefGoogle Scholar
- Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423CrossRefGoogle Scholar
- Villalobos M, Wahba G (1987) Inequality-constrained multivariate smoothing splines with application to the estimation of posterior probabilities. J Am Stat Assoc 82(397):239–248CrossRefGoogle Scholar
- Wolberg G, Alfy I (2002) An energy-minimization framework for monotonic cubic spline interpolation. J Comput Appl Math 143(2):145–188CrossRefGoogle Scholar
- Wright IW, Wegman EJ (1980) Isotonic, convex and related splines. Ann Stat 8(5):1023–1035CrossRefGoogle Scholar
- Xiaojing W (2012) Bayesian modeling using latent structures. Ph.D. thesis, Duke University, Department of Statistical ScienceGoogle Scholar
- Xuming H, Peide S (1996) Monotone B-spline smoothing. J Am Stat Assoc 93:643–650Google Scholar