Abstract
This chapter discusses the case of constrained optimization, and the additional steps needed to perform Bayesian Optimization in the presence of constraints. Different acquisition functions are needed, and four examples are explored in more detail: constrained expected improvement, asymmetric entropy, augmented Lagrangian, and barrier methods. These four are demonstrated on both simple and more complex examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ariafar S, Coll-Font J, Brooks D, Dy J (2019) ADMMBO: Bayesian optimization with unknown constraints using ADMM. J Mach Learn Res 20(123):1–26
Bertsekas DP (1982) Constrained optimization and Lagrange multiplier methods. Academic, New York
Breiman L (2001) Random forests. Mach Learn 45:5–32
Chipman HA, George EI, McCulloch RE (2010) BART: Bayesian additive regression trees. Ann Appl Stat 4(1):266–298
Cunningham JP, Hennig P, Lacoste-Julien S (2013) Gaussian probabilities and expectation propagation. arXiv:1111.6832
Eriksson D, Poloczek M (2021) Scalable constrained bayesian optimization. In: Banerjee A, Fukumizu K (eds) Proceedings of the 24th international conference on artificial intelligence and statistics, PMLR. Proceedings of machine learning research, vol 130, pp 730–738
Gardner JR, Kusner MJ, Xu Z, Weinberger KQ, Cunningham JP (2014) Bayesian optimization with inequality constraints. In: Proceedings of the 24th international conference on machine learning, ICML ’14, pp 937–945
Gramacy RB, Lee HKH (2011) Optimization under unknown constraints. In: Bernardo J, Bayarri S, Berger J, Dawid A, Heckerman D, Smith A, West M (eds) Bayesian statistics, vol 9. Oxford University Press, Oxford, pp 229–256
Gramacy RB, Gray GA, Le Digabel S, Lee HKH, Ranjan P, Wells G, Wild SM (2016) Modeling an augmented lagrangian for blackbox constrained optimization. Technometrics 58(1):1–11
Letham B, Karrer B, Ottoni G, Bakshy E (2019) Constrained Bayesian optimization with noisy experiments. Bayesian Anal 14(2):495–519
Lindberg D, Lee HKH (2015) Optimization under constraints by applying an asymmetric entropy measure. J Comput Graph Stat 24:379–393
Marcellin S, Zighed DA, Ritschard G (2006) An asymmetric entropy measure for decision trees. In: 11th information processing and management of uncertainty in knowledge-based systems (IPMU 06), pp 1292–1299
Moustaki I, Knott M (2000) Generalized latent trait models. Psychometrika 65:391–411
Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New York, NY
Picheny V (2014) A stepwise uncertainty reduction approach to constrained global optimization. Proceedings of the seventeenth international conference on artificial intelligence and statistics, pp 787–795
Picheny V, Gramacy RB, Wild S, Le Digabel S (2016) Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian. In: Lee D, Sugiyama M, Luxburg U, Guyon I, Garnett R (eds) Advances in neural information processing systems, vol 29. Curran Associates, Inc., Red Hook, pp 1435–1443
Pourmohamad T, Lee HKH (2016) Multivariate stochastic process models for correlated responses of mixed type. Bayesian Anal 11(3):797–820
Pourmohamad T, Lee HKH (2020) The statistical filter approach to constrained optimization. Technometrics 62(3):303–312
Pourmohamad T, Lee HKH (2021) Bayesian optimization via barrier functions. J Comput Graph Stat (to appear). https://www.tandfonline.com/doi/full/10.1080/10618600.2021.1935270
Rasmussen CE, Williams CKI (2006) Gaussian processes for machine learning. The MIT Press, Cambridge, MA
Sammel MD, Ryan LM, Legler JM (1997) Latent variable models for mixed discrete and continuous outcomes. J R Stat Soc Ser B (Stat Methodol) 59:667–678
Schonlau M, Welch W, Jones D (1998) Global versus local search in constrained optimization of computer models. In: New Developments and applications in experimental design, vol 34. IMS lecture notes. Institute of mathematical statistics, pp 11–25
Wackernagel H (2003) Multivariate geostatistics: an introduction with applications. Springer, New York, NY
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Pourmohamad, T., Lee, H.K.H. (2021). Constrained Optimization. In: Bayesian Optimization with Application to Computer Experiments. SpringerBriefs in Statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-82458-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-82458-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82457-0
Online ISBN: 978-3-030-82458-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)