Skip to main content

Accounting for Gaussian Process Imprecision in Bayesian Optimization

  • 567 Accesses

Part of the Lecture Notes in Computer Science book series (LNAI,volume 13199)


Bayesian optimization (BO) with Gaussian processes (GP) as surrogate models is widely used to optimize analytically unknown and expensive-to-evaluate functions. In this paper, we propose Prior-mean-RObust Bayesian Optimization (PROBO) that outperforms classical BO on specific problems. First, we study the effect of the Gaussian processes’ prior specifications on classical BO’s convergence. We find the prior’s mean parameters to have the highest influence on convergence among all prior components. In response to this result, we introduce PROBO as a generalization of BO that aims at rendering the method more robust towards prior mean parameter misspecification. This is achieved by explicitly accounting for GP imprecision via a prior near-ignorance model. At the heart of this is a novel acquisition function, the generalized lower confidence bound (GLCB). We test our approach against classical BO on a real-world problem from material science and observe PROBO to converge faster. Further experiments on multimodal and wiggly target functions confirm the superiority of our method.


  • Bayesian optimization
  • Imprecise Gaussian process
  • Imprecise probabilities
  • Prior near-ignorance
  • Model imprecision
  • Robust optimization

Julian Rodemann would like to thank the scholarship program of Evangelisches Studienwerk Villigst for the support of his studies and Lars Kotthoff for providing data as well as Christoph Jansen and Georg Schollmeyer for valuable remarks.

Open Science: Code to reproduce all findings and figures presented in this paper is available on a public repository:

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

    Also referred to as infill criterion.

  2. 2.

    Focus search shrinks the search space and applies random search, see [4, p. 7].

  3. 3.

    BO’s computational complexity depends on the SM. In case of GPs, it is \(\mathcal {O}(n^{3})\) due to the required inversion of the covariance matrix, where n is total number of target function evaluations.

  4. 4.

    Also called covariance function or kernel function.

  5. 5.

    To the best of our knowledge, this is the very first systematic assessment of GP prior’s influence on BO.

  6. 6.

    Note that neither accumulated differences (Definition 3) nor mean optimization paths (Definition 2) are scale-invariant.

  7. 7.

    Note that from a decision-theoretic point of view, LCB violates the dominance principle. GLCB inherits this property.

  8. 8.

    Further note that with expensive target functions to optimize, the computational costs of surrogate models and acquisition functions in BO can be regarded as negligible. The computational complexity of PROBO is the same as for BO with GP.


  1. Augustin, T., Coolen, F.P., de Cooman, G., Troffaes, M.C.M.: Introduction to Imprecise Probabilities. Wiley, Chichester (2014)

    Google Scholar 

  2. Awal, M.A., Masud, M., Hossain, M.S., Bulbul, A.A., Mahmud, S.M.H., Bairagi, A.K.: A novel Bayesian optimization-based machine learning framework for COVID-19 detection from inpatient facility data. IEEE Access 9, 10263–10281 (2021)

    CrossRef  Google Scholar 

  3. Benavoli, A., Zaffalon, M.: Prior near ignorance for inferences in the k-parameter exponential family. Statistics 49(5), 1104–1140 (2015)

    CrossRef  MathSciNet  Google Scholar 

  4. Bischl, B., Richter, J., Bossek, J., Horn, D., Thomas, J., Lang, M.: mlrMBO: a modular framework for model-based optimization of expensive black-box functions. arXiv preprint arXiv:1703.03373 (2017)

  5. Bossek, J.: smoof: Single- and multi-objective optimization test functions. R J. 9(1), 103–113 (2017)

    Google Scholar 

  6. Cox, D.D., John, S.: A statistical method for global optimization. In: Proceedings of 1992 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1241–1246. IEEE (1992)

    Google Scholar 

  7. Duvenaud, D.: Automatic model construction with Gaussian processes. Ph.D. thesis. University of Cambridge (2014)

    Google Scholar 

  8. Frazier, P.I., Wang, J.: Bayesian optimization for materials design. In: Lookman, T., Alexander, F.J., Rajan, K. (eds.) Information Science for Materials Discovery and Design. SSMS, vol. 225, pp. 45–75. Springer, Cham (2016).

    CrossRef  Google Scholar 

  9. Horn, D., Wagner, T., Biermann, D., Weihs, C., Bischl, B.: Model-based multi-objective optimization: taxonomy, multi-point proposal, toolbox and benchmark. In: Gaspar-Cunha, A., Henggeler Antunes, C., Coello, C.C. (eds.) EMO 2015. LNCS, vol. 9018, pp. 64–78. Springer, Cham (2015).

    CrossRef  Google Scholar 

  10. Malkomes, G., Garnett, R.: Automating Bayesian optimization with Bayesian optimization. Adv. Neural. Inf. Process. Syst. 31, 5984–5994 (2018)

    Google Scholar 

  11. Mangili, F.: A prior near-ignorance Gaussian process model for nonparametric regression. In: ISIPTA 2015: Proceedings of the 9th International Symposium on Imprecise Probability: Theories and Applications, pp. 187–196 (2015)

    Google Scholar 

  12. Mangili, F.: A prior near-ignorance Gaussian process model for nonparametric regression. Int. J. Approx. Reason. 78, 153–171 (2016)

    CrossRef  MathSciNet  Google Scholar 

  13. McKay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1), 55–61 (2000)

    CrossRef  Google Scholar 

  14. Močkus, J.: On Bayesian methods for seeking the extremum. In: Marchuk, G.I. (ed.) Optimization Techniques 1974. LNCS, vol. 27, pp. 400–404. Springer, Heidelberg (1975).

    CrossRef  Google Scholar 

  15. Nguyen, V.: Bayesian optimization for accelerating hyper-parameter tuning. In: 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), pp. 302–305. IEEE (2019)

    Google Scholar 

  16. Pyzer-Knapp, E.O.: Bayesian optimization for accelerated drug discovery. IBM J. Res. Dev. 62(6), 2:1-2:7 (2018)

    CrossRef  Google Scholar 

  17. R Core Team: R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2020)

    Google Scholar 

  18. Rasmussen, C.E.: Gaussian processes in machine learning. In: Bousquet, O., von Luxburg, U., Rätsch, G. (eds.) ML -2003. LNCS (LNAI), vol. 3176, pp. 63–71. Springer, Heidelberg (2004).

    CrossRef  Google Scholar 

  19. Insua, D.R., Ruggeri, F.: Robust Bayesian Analysis. Springer, New York (2000)

    Google Scholar 

  20. Rodemann, J.: Robust generalizations of stochastic derivative-free optimization. Master’s thesis. LMU Munich (2021)

    Google Scholar 

  21. Schmidt, A.M., Conceição, M.F.G., Moreira, G.A.: Investigating the sensitivity of Gaussian processes to the choice of their correlation function and prior specifications. J. Stat. Comput. Simul. 78(8), 681–699 (2008)

    Google Scholar 

  22. Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian optimization of machine learning algorithms. Adv. Neural. Inf. Process. Syst. 25, 2951–2959 (2012)

    Google Scholar 

  23. Utkin, L., Kovalev, M., Meldo, A., Coolen, F.: Imprecise extensions of random forests and random survival forests. In: International Symposium on Imprecise Probabilities: Theories and Applications, pp. 404–413. PMLR (2019)

    Google Scholar 

  24. Wahab, H., Jain, V., Tyrrell, A.S., Seas, M.A., Kotthoff, L., Johnson, P.A.: Machine-learning-assisted fabrication: Bayesian optimization of laser-induced graphene patterning using in-situ Raman analysis. Carbon 167, 609–619 (2020)

    CrossRef  Google Scholar 

  25. Walter, G., Augustin, T.: Imprecision and prior-data conflict in generalized Bayesian inference. J. Stat. Theory Pract. 3(1), 255–271 (2009)

    CrossRef  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Julian Rodemann .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rodemann, J., Augustin, T. (2022). Accounting for Gaussian Process Imprecision in Bayesian Optimization. In: Honda, K., Entani, T., Ubukata, S., Huynh, VN., Inuiguchi, M. (eds) Integrated Uncertainty in Knowledge Modelling and Decision Making. IUKM 2022. Lecture Notes in Computer Science(), vol 13199. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98017-7

  • Online ISBN: 978-3-030-98018-4

  • eBook Packages: Computer ScienceComputer Science (R0)