## Abstract

Physical phenomena are observed in many fields (science and engineering) and are often studied by time-consuming computer codes. These codes are analyzed with statistical models, often called emulators. In many situations, the physical system (computer model output) may be known to satisfy inequality constraints with respect to some or all input variables. The aim is to build a model capable of incorporating both data interpolation and inequality constraints into a Gaussian process emulator. By using a functional decomposition, a finite-dimensional approximation of Gaussian processes such that all conditional simulations satisfy the inequality constraints in the entire domain is proposed. To show the performance of the proposed model, some conditional simulations with inequality constraints such as boundedness, monotonicity or convexity conditions in one and two dimensions are given. A simulation study to investigate the efficiency of the method in terms of prediction is included.

This is a preview of subscription content, log in to check access.

## References

Abrahamsen P, Benth FE (2001) Kriging with inequality constraints. Math Geol 33(6):719–744

Aronszajn N (1950) Theory of reproducing kernels. Trans Am Math Soc 68(3):337–404

Bay X, Grammont L, Maatouk H (2015) A new method for interpolating in a convex subset of a Hilbert space Hal-01136466. https://hal.archives-ouvertes.fr/hal-01136466

Bay X, Grammont L, Maatouk H (2016) Generalization of the Kimeldorf–Wahba correspondence for constrained interpolation. Electron J Stat 10(1):1580–1595

Botts C (2013) An accept–reject algorithm for the positive multivariate normal distribution. Comput Stat 28(4):1749–1773

Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, New York

Chopin N (2011) Fast simulation of truncated Gaussian distributions. Stat Comput 21(2):275–288

Cousin A, Maatouk H, Rullière D (2016) Kriging of financial term-structures. Eur J Oper Res 255(2):631–648

Cramer H, Leadbetter R (1967) Stationary and related stochastic processes: sample function properties and their applications. Wiley series in probability and mathematical statistics. Tracts on probability and statistics. Wiley, Hoboken

Cressie N, Johannesson G (2008) Fixed rank kriging for very large spatial data sets. J R Stat Soc Ser B Stat Methodol 70(1):209–226

Da Veiga S, Marrel A (2012) Gaussian process modeling with inequality constraints. Ann Fac Sci Toulouse 21(3):529–555

Dole D (1999) CoSmo: a constrained scatterplot smoother for estimating convex, monotonic transformations. J Bus Econ Stat 17(4):444–455

Emery X, Arroyo D, Peláez M (2014) Simulating large Gaussian random vectors subject to inequality constraints by gibbs sampling. Math Geosci 46(3):265–283

Freulon X, de Fouquet C (1993) Conditioning a Gaussian model with inequalities. In: Geostatistics Troia92. Springer, Netherlands, pp 201–212

Fritsch FN, Carlson RE (1980) Monotone piecewise cubic interpolation. SIAM J Numer Anal 17(2):238–246

Geweke J (1991) Efficient simulation from the multivariate normal and student-t distributions subject to linear constraints and the evaluation of constraint probabilities. In: Computing science and statistics: proceedings of the 23rd symposium on the interface, pp 571–578

Golchi S, Bingham D, Chipman H, Campbell DA (2015) Monotone emulation of computer experiments. SIAM/ASA J Uncertain Quantif 3(1):370–392

Goldfarb D, Idnani A (1982) Dual and primal-dual methods for solving strictly convex quadratic programs. In: Numerical analysis. Springer, Berlin, pp 226–239

Goldfarb D, Idnani A (1983) A numerically stable dual method for solving strictly convex quadratic programs. Math Program 27(1):1–33

Jones DR, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455–492

Kimeldorf GS, Wahba G (1970) A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Ann Math Stat 41(2):495–502

Kleijnen JP, Van Beers WC (2013) Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations. J Oper Res Soc 64(5):708–717

Kotecha JH, Djuric PM (1999) Gibbs sampling approach for generation of truncated multivariate Gaussian random variables. In: Proceedings on 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1999, vol 3, pp 1757–1760

Maatouk H (2015) Correspondence between Gaussian process regression and interpolation splines under linear inequality constraints. Theory and applications. Ph.D. thesis, École des Mines de St-Étienne

Maatouk H, Bay X (2016) A new rejection sampling method for truncated multivariate Gaussian random variables restricted to convex sets. In: Nuyens R, Cools R (ed) Monte carlo and quasi-monte carlo methods, vol 163. Springer International Publishing, Cham, pp 521–530

Maatouk H, Richet Y (2015) constrKriging R package. https://github.com/maatouk/constrKriging

Maatouk H, Roustant O, Richet Y (2015) Cross-validation estimations of hyper-parameters of Gaussian processes with inequality constraints. Procedia Environ Sci 27:38–44

Micchelli C, Utreras F (1988) Smoothing and interpolation in a convex subset of a Hilbert space. SIAM J Sci Stat Comput 9(4):728–746

Parzen E (1962) Stochastic processes. Holden-Day series in probability and statistics. Holden-Day, San Francisco

Philippe A, Robert CP (2003) Perfect simulation of positive Gaussian distributions. Stat Comput 13(2):179–186

Ramsay JO (1988) Monotone regression splines in action. Stat Sci 3(4):425–441

Rasmussen CE, Williams CK (2005) Gaussian processes for machine learning (adaptive computation and machine learning). The MIT Press, Cambridge

Riihimaki J, Vehtari A (2010) Gaussian processes with monotonicity information. JMLR Proc AISTATS 9:645–652

Robert CP (1995) Simulation of truncated normal variables. Stat Comput 5(2):121–125

Roustant O, Ginsbourger D, Deville Y (2012) DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization. J Stat Softw 51(1):1–55

Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–423

Villalobos M, Wahba G (1987) Inequality-constrained multivariate smoothing splines with application to the estimation of posterior probabilities. J Am Stat Assoc 82(397):239–248

Wolberg G, Alfy I (2002) An energy-minimization framework for monotonic cubic spline interpolation. J Comput Appl Math 143(2):145–188

Wright IW, Wegman EJ (1980) Isotonic, convex and related splines. Ann Stat 8(5):1023–1035

Xiaojing W (2012) Bayesian modeling using latent structures. Ph.D. thesis, Duke University, Department of Statistical Science

Xuming H, Peide S (1996) Monotone B-spline smoothing. J Am Stat Assoc 93:643–650

## Acknowledgements

The authors would like to thank the Associate Editor and the two anonymous referees for their helpful comments. Part of this work has been conducted within the frame of the ReDice Consortium, gathering industrial (CEA, EDF, IFPEN, IRSN, Renault) and academic (École des Mines de Saint-Étienne, INRIA, and the University of Bern) partners around advanced methods for Computer Experiments. The authors also thank Olivier Roustant (ENSM-SE), Laurence Grammont (ICJ, Lyon1) and Yann Richet (IRSN) for helpful discussions as well as the participants of the UCM2014 conference and GRF-Sim2014 workshop.

## Author information

### Affiliations

### Corresponding author

## Appendix: Proofs of Propositions

### Appendix: Proofs of Propositions

### Proof

(Proofs of Propositions 1 and 2) Let \(Y^N\) be defined as in (6) (respectively, in 8). Since \(Y(u_j), \ j=0,\ldots ,N\) are Gaussian variables, then \(Y^N\) is a GP with dimension equal to \(N+1\) and covariance function

The derivative of a GP is also a GP, respectively. For all \(x,x'\in [0,1]\),

To prove the pathwise convergence of \(Y^N\) to *Y*, write more explicitly, for any \(\omega \in \varOmega \)

Hence, the sample paths of the approximating process \(Y^N\) are piecewise linear approximations of the sample paths of the original process *Y*. From \(h_j\ge 0\) and \(\sum _{j=0}^Nh_j(x)=1\), for all \(x\in [0,1]\),

By uniformly continuity of sample paths of the process *Y* on the compact interval [0, 1], this last inequality (13) shows that

with probability one. Let us write, respectively, that for any \(\omega \in \varOmega \)

From Proposition 1, \(\sum _{j=0}^NY'(u_j;\omega )h_j(x)\) converges uniformly pathwise to \(Y'(x;\omega )\) since the realizations of the process are almost surely continuously differentiable. One can conclude that \(Y^N\) converges uniformly to *Y* for almost all \(\omega \in \varOmega \).

The last property done by the fact that \(Y^N\) is a piecewise linear approximation and

\(i=0,\ldots ,N\), (respectively, the fact that the basis functions are non-decreasing and

\(i=0,\ldots ,N\)). \(\square \)

## Rights and permissions

## About this article

### Cite this article

Maatouk, H., Bay, X. Gaussian Process Emulators for Computer Experiments with Inequality Constraints.
*Math Geosci* **49, **557–582 (2017). https://doi.org/10.1007/s11004-017-9673-2

Received:

Accepted:

Published:

Issue Date:

### Keywords

- Gaussian process emulator
- Inequality constraints
- Finite-dimensional approximation
- Uncertainty quantification
- Design and modeling of computer experiments