Abstract
The types of constraints encountered in black-box simulation-based optimization problems differ significantly from those addressed in nonlinear programming. We introduce a characterization of constraints to address this situation. We provide formal definitions for several constraint classes and present illustrative examples in the context of the resulting taxonomy. This taxonomy, denoted KARQ, is useful for modeling and problem formulation, as well as optimization software development and deployment. It can also be used as the basis for a dialog with practitioners in moving problems to increasingly solvable branches of optimization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achtziger W, Kanzow C (2008) Mathematical programs with vanishing constraints: optimality conditions and constraint qualifications. Math Program 114(1):69–99. https://doi.org/10.1007/s10107-006-0083-3
Adcock B, Cardenas J, Dexter N (2023) An adaptive sampling and domain learning strategy for multivariate function approximation on unknown domains. SIAM J Sci Comput 45(1):A200–A225. https://doi.org/10.1137/22M1472693
Aggarwal V, Aneja Y, Nair K (1982) Minimal spanning tree subject to a side constraint. Comput Oper Res 9(4):287–296. https://doi.org/10.1016/0305-0548(82)90026-0
Alarie S, Audet C, Garnier V, Le Digabel S, Leclaire LA (2013) Snow water equivalent estimation using Blackbox optimization. Pac J Optim 9(1):1–21
Alexandrov N, Lewis R (1999) Comparative properties of collaborative optimization and other approaches to MDO. ICASE Report 99–24, Institute for Computer Applications in Science and Engineering. http://techreports.larc.nasa.gov/ltrs/PDF/1999/mtg/NASA-99-asmo-nma.pdf
Audet C, Béchard V, Le Digabel S (2008) Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J Global Optim 41(2):299–318. https://doi.org/10.1007/s10898-007-9234-1
Audet C, Dang CK, Orban D (2010) Algorithmic parameter optimization of the DFO method with the OPAL framework. In: Naono K, Teranishi K, Cavazos J, Suda R (eds), Software automatic tuning: from concepts to state-of-the-art results, chap. 15, Springer, pp 255–274. https://doi.org/10.1007/978-1-4419-6935-4_15
Audet C, Dang CK, Orban D (2013) Efficient use of parallelism in algorithmic parameter optimization applications. Optim Lett 7(3):421–433. https://doi.org/10.1007/s11590-011-0428-6
Audet C, Dennis JE Jr (2009) A progressive barrier for derivative-free nonlinear programming. SIAM J Optim 20(1):445–472. https://doi.org/10.1137/070692662
Audet C, Hare W (2017) Derivative-free and Blackbox optimization. Springer series in operations research and financial engineering. Springer, Cham, Switzerland. https://doi.org/10.1007/978-3-319-68913-5
Audet C, Le Digabel S, Rochon Montplaisir V, Tribes C (2022) Algorithm 1027: NOMAD version 4: nonlinear optimization with the MADS algorithm. ACM Trans Math Softw 48(3):35:1-35:22. https://doi.org/10.1145/3544489
Avriel M, Wilde D (1967) Optimal condenser design by geometric programming. Ind Eng Chem Process Design Dev 6(2):256–263. https://doi.org/10.1021/i260022a018
Bachoc F, Helbert C, Picheny V (2020) Gaussian process optimization with failures: classification and convergence proof. J Global Optim 78(3):483–506. https://doi.org/10.1007/s10898-020-00920-0
Balaprakash P, Wild S, Hovland P (2011) Can search algorithms save large-scale automatic performance tuning? Procedia Comput Sci (ICCS 2011) 4:2136–2145. https://doi.org/10.1016/j.procs.2011.04.234
Brooke A, Kendrick D, Meeraus A (1988) GAMS: a users’ guide. The Scientific Press, Danvers
Carter R, Gablonsky J, Patrick A, Kelley C, Eslinger O (2001) Algorithms for noisy problems in gas transmission pipeline optimization. Optim Eng 2:139–157. https://doi.org/10.1023/A:1013123110266
Charnes A, Cooper W, Symonds G (1958) Cost horizons and certainty equivalents: an approach to stochastic programming of heating oil. Manag Sci 4(3):235–263. https://doi.org/10.1287/mnsc.4.3.235
Choi T, Eslinger O, Kelley C, David J, Etheridge M (2000) Optimization of automotive valve train components with implicit filtering. Optim Eng 1:9–27. https://doi.org/10.1023/A:1010071821464
Choi T, Kelley C (2000) Superlinear convergence and implicit filtering. SIAM J Optim 10(4):1149–1162. https://doi.org/10.1137/S1052623499354096
Conn A, Scheinberg K, Toint PL (1998) A Derivative Free Optimization Algorithm in Practice. In: Proceedings of 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization. http://perso.fundp.ac.be/~phtoint/pubs/TR98-11.ps
Conn A, Scheinberg K, Toint PL (2001) DFO (Derivative Free Optimization Software). https://projects.coin-or.org/Dfo
Conn A, Scheinberg K, Vicente L (2009) Introduction to derivative-free optimization. MOS-SIAM series on optimization. SIAM, Philadelphia. https://doi.org/10.1137/1.9780898718768
Custódio A, Vicente L (2005) SID-PSM: a pattern search method guided by simplex derivatives for use in derivative-free optimization. http://www.mat.uc.pt/sid-psm
Diouane Y, Gratton S, Vicente L (2015) Globally convergent evolution strategies for constrained optimization. Comput Optim Appl 62(2):323–346. https://doi.org/10.1007/s10589-015-9747-3
Fiacco A, McCormick G (1990) Nonlinear programming: sequential unconstrained minimization techniques. Classics in applied mathematics, vol 4. SIAM, Philadelphia. https://doi.org/10.1137/1.9781611971316
Fourer R, Gay D, Kernighan B (2003) AMPL: a modeling language for mathematical programming, 2nd edn. Thomson/Brooks/Cole, Pacific Grove
Fowler K, Reese J, Kees C, Dennis J Jr, Kelley C, Miller C, Audet C, Booker A, Couture G, Darwin R, Farthing M, Finkel D, Gablonsky J, Gray G, Kolda T (2008) Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv Water Resour 31(5):743–757. https://doi.org/10.1016/j.advwatres.2008.01.010
Ghattas O, Biegler L, Heinkenschloss M, van Bloemen Wanders B (2003) Large-scale PDE-constrained optimization: an introduction. In: Biegler L, Ghattas O, Heinkenschloss M, van Bloemen Wanders B (eds) Large-scale PDE-constrained optimization. Springer, New York, pp 3–13
Gould N, Orban D, Toint P (2015) CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60(3):545–557. https://doi.org/10.1007/s10589-014-9687-3
Gramacy R, Lee H (2011) Optimization under unknown constraints. In: Bernardo J, Bayarri S, Berger J, Dawid A, Heckerman D, Smith A, West M (eds) Proceedings of the Ninth Valencia international meetings on Bayesian statistics. Oxford University Press, Oxford, pp 229–256. https://doi.org/10.1093/acprof:oso/9780199694587.003.0008
Gramacy R, Taddy M, Wild S (2013) Variable selection and sensitivity analysis via dynamic trees with an application to computer code performance tuning. Ann Appl Stat 7(1):51–80. https://doi.org/10.1214/12-AOAS590
Gratton S, Vicente L (2014) A merit function approach for direct search. SIAM J Optim 24(4):1980–1998. https://doi.org/10.1137/130917661
Griffin J, Kolda T (2010) Nonlinearly constrained optimization using heuristic penalty methods and asynchronous parallel generating set search. Appl Math Res Express 2010(1):36–62. https://doi.org/10.1093/amrx/abq003
Griva I, Nash S, Sofer A (2009) Linear and nonlinear optimization. SIAM, Philadelphia
Huyer W, Neumaier A (2008) SNOBFIT—stable noisy optimization by branch and fit. ACM Trans Math Softw 35(2):9:1-9:25. https://doi.org/10.1145/1377612.1377613
Kannan A, Wild S (2012) Benefits of deeper analysis in simulation-based groundwater optimization problems. In: Proceedings of the XIX international conference on computational methods in water resources (CMWR 2012). http://www.mcs.anl.gov/~wild/papers/2012/AKSW12.pdf
Kelley C (2011) Implicit Filtering. SIAM, Philadephia, PA. https://doi.org/10.1137/1.9781611971903
Koch T (2004) Rapid Mathematical Prototyping. Ph.D. thesis, Technische Universität Berlin
Kortelainen M, Lesinski T, Moré J, Nazarewicz W, Sarich J, Schunck N, Stoitsov MV, Wild SM (2010) Nuclear energy density optimization. Phys Rev C 82(2):024,313. https://doi.org/10.1103/PhysRevC.82.024313
Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear Optimization with the MADS algorithm. ACM Trans Math Softw 37(4):44:1-44:15. https://doi.org/10.1145/1916461.1916468
Lee H, Gramacy R, Linkletter C, Gray G (2011) Optimization subject to hidden constraints via statistical emulation. Pac J Optim 7(3):467–478
Lemyre Garneau M (2015) Modelling of a solar thermal power plant for benchmarking blackbox optimization solvers. Master’s thesis, Polytechnique Montréal. https://publications.polymtl.ca/1996/
Luo ZQ, Pang JS, Ralph D (1996) Mathematical programs with equilibrium constraints. Cambridge University Press, Cambridge. https://doi.org/10.1017/CBO9780511983658
March A, Willcox K (2012) Constrained multifidelity optimization using model calibration. Struct Multidiscip Optim 46(1):93–109. https://doi.org/10.1007/s00158-011-0749-1
Martelli E, Amaldi E (2014) PGS-COM: a hybrid method for constrained non-smooth black-box optimization problems: brief review, novel algorithm and comparative evaluation. Comput Chem Eng 63:108–139. https://doi.org/10.1016/j.compchemeng.2013.12.014
Martins J, Marriage C, Tedford N (2009) pyMDO: an object-oriented framework for multidisciplinary design optimization. ACM Trans Math Softw 36(4):20:1-20:25. https://doi.org/10.1145/1555386.1555389
Matott L, Leung K, Sim J (2011) Application of MATLAB and Python optimizers to two case studies involving groundwater flow and contaminant transport modeling. Comput Geosci 37(11):1894–1899. https://doi.org/10.1016/j.cageo.2011.03.017
Müller J, Day M (2019) Surrogate optimization of computationally expensive black-box problems with hidden constraints. INFORMS J Comput 31(4):689–702. https://doi.org/10.1287/ijoc.2018.0864
Nemhauser G, Savelsbergh M, Sigismondi G (1992) Constraint classification for mixed integer programming formulations. COAL Bull 20:8–12
Plantenga T (2009) HOPSPACK 2.0 User Manual. Tech. Rep. SAND2009-6265, Sandia National Laboratories, Livermore, California. http://www.sandia.gov/hopspack/HopspackUserManual_2_0_2.pdf
Rossi F, van Beek P, Walsh T (eds.) (2006) Handbook of constraint programming. Elsevier. https://www.elsevier.com/books/handbook-of-constraint-programming/rossi/978-0-444-52726-4
Stripinis L, Paulavičius R (2021) A new DIRECT-GLh algorithm for global optimization with hidden constraints. Optim Lett 15(6):1865–1884. https://doi.org/10.1007/s11590-021-01726-z
Tosserams S, Etman L, Rooda J (2009) A classification of methods for distributed system optimization based on formulation structure. Struct Multidiscip Optim 39:503–517. https://doi.org/10.1007/s00158-008-0347-z
Acknowledgements
This material was based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research applied mathematics program under contract nos. DE-AC02-05CH11231 and DE-AC02-06CH11357; by the Air Force Office of Scientific Research Grant FA9550-12-1-0198; and by the Natural Sciences and Engineering Research Council of Canada Discovery Grant 418250. The authors thank the American Institute of Mathematics for the SQuaRE workshops that allowed them to initiate their discussions on the taxonomy; the authors thank the participants: Bobby Gramacy, Genetha Gray, Herbie Lee, and Garth Wells. Thanks also to Charles Audet, John Dennis, Warren Hare, Sven Leyffer, and Paul Patience, for their useful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Le Digabel, S., Wild, S.M. A taxonomy of constraints in black-box simulation-based optimization. Optim Eng (2023). https://doi.org/10.1007/s11081-023-09839-3
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11081-023-09839-3