Approximating Gaussian Process Emulators with Linear Inequality Constraints and Noisy Observations via MC and MCMC

Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 324)


Adding inequality constraints (e.g. positivity, monotonicity, convexity) in Gaussian processes (GPs) leads to more realistic stochastic emulators. Due to the truncated Gaussianity of the posterior, its distribution has to be approximated. In this work, we consider Monte Carlo (MC) and Markov Chain MC (MCMC) methods. However, strictly interpolating the observations may entail expensive computations due to highly restrictive sample spaces. Furthermore, having emulators when data are actually noisy is also of interest for real-world applications. Hence, we introduce a noise term for the relaxation of the interpolation conditions, and we develop the corresponding approximation of GP emulators under linear inequality constraints. We demonstrate on various synthetic examples that the performance of MC and MCMC samplers improves when considering noisy observations. Finally, on 2D and 5D coastal flooding applications, we show that more flexible and realistic emulators are obtained by considering noise effects and by enforcing the inequality constraints.


Linear Inequality Constraints Gaussian Processes Emulators Monte Carlo and Markov Chain Monte Carlo Methods Uncertainty Quantification with Noisy Observations 



This research was conducted within the frame of the Chair in Applied Mathematics OQUAIDO, gathering partners in technological research (BRGM, CEA, IFPEN, IRSN, Safran, Storengy) and academia (CNRS, Ecole Centrale de Lyon, Mines Saint-Etienne, Univ. Grenoble, Univ. Nice, Univ. Toulouse) around advanced methods for computer experiments.


  1. 1.
    Azzimonti, D.: profExtrema: Compute and visualize profile extrema functions. R package version 0.2.0 (2018)Google Scholar
  2. 2.
    Azzimonti, D., Ginsbourger, D., Rohmer, J., Idier, D.: Profile extrema for visualizing and quantifying uncertainties on excursion regions. Application to coastal flooding. Technometrics 0(ja), 1–26 (2019)Google Scholar
  3. 3.
    Bay, X., Grammont, L., Maatouk, H.: Generalization of the Kimeldorf-Wahba correspondence for constrained interpolation. Electron. J. Stat. 10(1), 1580–1595 (2016)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Botev, Z.I.: The normal law under linear restrictions: simulation and estimation via minimax tilting. J. R. Stat. Soc.: Ser. B 79(1), 125–148 (2017)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Cousin, A., Maatouk, H., Rullière, D.: Kriging of financial term-structures. Eur. J. Oper. Res. 255(2), 631–648 (2016)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Deville, Y., Ginsbourger, D., Durrande, N., Roustant, O.: kergp: Gaussian process laboratory. R package version 0.2.0 (2015)Google Scholar
  7. 7.
    Dupuy, D., Helbert, C., Franco, J.: DiceDesign and DiceEval: two R packages for design and analysis of computer experiments. J. Stat. Softw. 65(11), 1–38 (2015)CrossRefGoogle Scholar
  8. 8.
    Geyer, C.J.: Practical Markov Chain Monte Carlo. Stat. Sci. 7(4), 473–483 (1992)CrossRefGoogle Scholar
  9. 9.
    Golchi, S., Bingham, D.R., Chipman, H., Campbell, D.A.: Monotone emulation of computer experiments. SIAM/ASA J. Uncertain. Quantif. 3(1), 370–392 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Goldfarb, D., Idnani, A.: Dual and primal-dual methods for solving strictly convex quadratic programs. Numerical Analysis, pp. 226–239. Springer, New York (1982)Google Scholar
  11. 11.
    Gong, L., Flegal, J.M.: A practical sequential stopping rule for high-dimensional Markov Chain Monte Carlo. J. Comput. Graph. Stat. 25(3), 684–700 (2016)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Lan, S., Shahbaba, B.: Sampling constrained probability distributions using spherical augmentation. Algorithmic Advances in Riemannian Geometry and Applications, pp. 25–71. Springer, New York (2016)Google Scholar
  13. 13.
    Larson, M.G., Bengzon, F.: The Finite Element Method: Theory, Implementation, and Applications. Springer, New York (2013)CrossRefGoogle Scholar
  14. 14.
    López-Lopera, A.F.: lineqGPR: Gaussian process regression models with linear inequality constraints. R package version 0.0.3 (2018)Google Scholar
  15. 15.
    López-Lopera, A.F., Bachoc, F., Durrande, N., Roustant, O.: Finite-dimensional Gaussian approximation with linear inequality constraints. SIAM/ASA J. Uncertain. Quantif. 6(3), 1224–1255 (2018)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Maatouk, H., Bay, X.: A new rejection sampling method for truncated multivariate Gaussian random variables restricted to convex sets. Monte Carlo and Quasi-Monte Carlo Methods, pp. 521–530. Springer, New York (2016)Google Scholar
  17. 17.
    Maatouk, H., Bay, X.: Gaussian process emulators for computer experiments with inequality constraints. Math. Geosci. 49(5), 557–582 (2017)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Murphy, K.P.: Machine Learning: A Probabilistic Perspective. Adaptive Computation and Machine Learning Series. The MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  19. 19.
    Pakman, A., Paninski, L.: Exact Hamiltonian Monte Carlo for truncated multivariate Gaussians. J. Comput. Graph. Stat. 23(2), 518–542 (2014)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. The MIT Press, Cambridge (2005)CrossRefGoogle Scholar
  21. 21.
    Rohmer, J., Idier, D.: A meta-modelling strategy to identify the critical offshore conditions for coastal flooding. Nat. Hazards Earth Syst. Sci. 12(9), 2943–2955 (2012)CrossRefGoogle Scholar
  22. 22.
    Roustant, O., Ginsbourger, D., Deville, Y.: DiceKriging, DiceOptim: two R packages for the analysis of computer experiments by Kriging-based metamodeling and optimization. J. Stat. Softw. 51(1), 1–55 (2012)CrossRefGoogle Scholar
  23. 23.
    Taylor, J., Benjamini, Y.: restrictedMVN: multivariate normal restricted by affine constraints. R package version 1.0 (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Mines Saint-Étienne, CNRS, UMR 6158 LIMOS, Institut Henri FayolUniversité Clermont AuvergneSaint-ÉtienneFrance
  2. 2.Institut de Mathématiques de Toulouse, Université Paul SabatierToulouseFrance
  3. 3.PROWLER.ioCambridgeUK
  4. 4.BRGMDRP/R3COrléans cédex 2France

Personalised recommendations