Journal of Global Optimization

, Volume 69, Issue 1, pp 117–136 | Cite as

GOSAC: global optimization with surrogate approximation of constraints

  • Juliane MüllerEmail author
  • Joshua D. Woodbury


We introduce GOSAC, a global optimization algorithm for problems with computationally expensive black-box constraints and computationally cheap objective functions. The variables may be continuous, integer, or mixed-integer. GOSAC uses a two-phase optimization approach. The first phase aims at finding a feasible point by solving a multi-objective optimization problem in which the constraints are minimized simultaneously. The second phase aims at improving the feasible solution. In both phases, we use cubic radial basis function surrogate models to approximate the computationally expensive constraints. We iteratively select sample points by minimizing the computationally cheap objective function subject to the constraint function approximations. We assess GOSAC’s efficiency on computationally cheap test problems with integer, mixed-integer, and continuous variables and two environmental applications. We compare GOSAC to NOMAD and a genetic algorithm (GA). The results of the numerical experiments show that for a given budget of allowed expensive constraint evaluations, GOSAC finds better feasible solutions more efficiently than NOMAD and GA for most benchmark problems and both applications. GOSAC finds feasible solutions with a higher probability than NOMAD and GOSAC.


Black-box optimization Computationally expensive constraints Surrogate models Radial basis functions 



Global optimization with surrogate approximation of constraints


Nonlinear optimization by mesh adaptive direct search, version 3.6.2


Radial basis function

List of symbols

\({\mathbf {x}}\)

Decision variable vector (may be continuous, integer, binary, mixed-integer)

\({\mathbf {x}}^T\)

Transpose of \({\mathbf {x}}\)


Problem dimension

\(x_i^l\), \(x_i^u\), \(i = 1\ldots d\)

Variables’ lower and upper bounds

\(\varOmega \)

Feasible domain defined by variables’ lower and upper bounds

\(\varOmega _c\)

Feasible domain defined by all constraints

\({{\mathbb {I}}}\)

Index set of integer variables

\(f(\cdot )\)

Objective function, computationally cheap

\(c_j(\cdot ), j = 1\ldots m\)

Computationally expensive black-box constraint functions

\(s_j(\cdot )\), \(j = 1\ldots m\)

Response surface for the jth constraint function

\({\mathcal {S}}\)

Set of already evaluated points, \({\mathcal {S}}=\{{\mathbf {x}}_1,\ldots ,{\mathbf {x}}_n\}\)


Number of points in initial experimental design

Mathematics Subject Classification

90C26 90C30 90C56 90C59 

Supplementary material

10898_2017_496_MOESM1_ESM.pdf (227 kb)
Supplementary material 1 (pdf 226 KB)


  1. 1.
    Arnold, J., Srinivasan, R., Muttiah, R., Williams, J.: Large area hydrologic modeling and assessment part 1: model development. J. Am. Water Resour. Assoc. 34, 73–89 (1998)CrossRefGoogle Scholar
  2. 2.
    Audet, C., Dennis Jr., J.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Audet, C., Dennis Jr., J.: A progressive barrier approach for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Basudhar, A., Dribusch, C., Lacaze, S.S.M.: Constrained efficient global optimization with support vector machines. Struct. Multidiscip. Optim. 46(2), 201–221 (2012)CrossRefzbMATHGoogle Scholar
  5. 5.
    Bertsekas, D.: Constrained Optimization and Lagrange Multiplier Methods. Academic Press, New York (1982)zbMATHGoogle Scholar
  6. 6.
    Booker, A., Dennis Jr., J., Frank, P., Serafini, D., Torczon, V., Trosset, M.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidisc. Optim. 17, 1–13 (1999)CrossRefGoogle Scholar
  7. 7.
    Conn, A., Scheinberg, K., Vicente, L.: Introduction to Derivative-Free Optimization. SIAM. 1, 1–12 (2009). doi: 10.1137/1.9780898718768.ch1 Google Scholar
  8. 8.
    Currie, J., Wilson, D.: Foundations of Computer-Aided Process Operations, Chap. OPTI: Lowering the Barrier Between Open Source Optimizers and the Industrial MATLAB User. Savannah, Georgia, USA (2012)Google Scholar
  9. 9.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Progr. Ser. A 91(2), 239–269 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Forrester, A., Sóbester, A., Keane, A.: Engineering Design via Surrogate Modelling: A Practical Guide. Wiley, London (2008)CrossRefGoogle Scholar
  11. 11.
    Friedman, J.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–141 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Goel, T., Haftka, R.T., Shyy, W., Queipo, N.V.: Ensemble of surrogates. Struct. Multidiscip. Optim. 33, 199–216 (2007)CrossRefGoogle Scholar
  13. 13.
    Gramacy, R., Gray, G., Le Digabel, S., Lee, H., Ranjan, P., Wells, G., Wild, S.: Modeling an augmented Lagrangian for blackbox constrained optimization. Technometrics 58(1), 1–11 (2016)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Gutmann, H.: A radial basis function method for global optimization. J. Global Optim. 19, 201–227 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Harbaugh, A., McDonald, M.: User’s documentation for MODFLOW-96: an update to the U.S. Geological Survey modular finite-difference ground-water flow model. U.S. Geological Survey Open-File Report 96-485, p. 56 (1996)Google Scholar
  16. 16.
    Jones, D., Schonlau, M., Welch, W.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13, 455–492 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Le Digabel, S.: Algorithm 909: NOMAD—nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37, 44 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Le Digabel, S., Wild, S.: A Taxonomy of Constraints in Simulation-Based Optimization (2015). ArXiv:1505.07881
  19. 19.
    Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for mixed-integer constrained optimization problems. J. Optim. Theory Appl. 164, 933–965 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Martin, J.D., Simpson, T.W.: Use of kriging models to approximate deterministic computer models. AIAA J. 43, 853–863 (2005)CrossRefGoogle Scholar
  21. 21.
    Matheron, G.: Principles of geostatistics. Econ. Geol. 58, 1246–1266 (1963)CrossRefGoogle Scholar
  22. 22.
    Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Müller, J.: MISO: mixed-integer surrogate optimization framework. Optim. Eng. 17(1), 177–203 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Müller, J., Krityakierne, T., Shoemaker, C.: SO-MODS: Optimization for high dimensional computationally expensive multi-modal functions with surrogate search. In: 2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China (2014)Google Scholar
  25. 25.
    Müller, J., Piché, R.: Mixture surrogate models based on Dempster–Shafer theory for global optimization problems. J. Global Optim. 51, 79–104 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Müller, J., Shoemaker, C.: Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J. Global Optim. 60(2), 123–144 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Müller, J., Shoemaker, C., Piché, R.: SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J. Global Optim. 59(4), 865–889 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Müller, J., Shoemaker, C., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40, 1383–1400 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Myers, R., Montgomery, D.: Response Surface Methodology, Process and Product Optimization Using Designed Experiments. Wiley, Hoboken (1995)zbMATHGoogle Scholar
  30. 30.
    Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2006)zbMATHGoogle Scholar
  31. 31.
    Parr, J.: Improvement criteria for constraint handling and multiobjective optimization. Ph.D. thesis, University of Southampton, University Road, Southampton SO17 1BJ (2013)Google Scholar
  32. 32.
    Powell, M.: The theory of radial basis function approximation in 1990. Advances in Numerical Analysis, vol. 2: Wavelets, Subdivision Algorithms and Radial Basis Functions. Oxford University Press, Oxford, pp. 105-210 (1992)Google Scholar
  33. 33.
    Regis, R.: Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 38, 837–853 (2011)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Regis, R., Shoemaker, C.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19, 497–509 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Regis, R., Shoemaker, C.: Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng. Optim. 45, 529–555 (2013)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Regis, R., Shoemaker, C.: A quasi-multistart framework for global optimization of expensive functions using response surface models. J. Global Optim. 56(4), 1719–1753 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Talgorn, B., Le Digabel, S., Kokkolaras, M.: Statistical surrogate formulations for simulation-based design optimization. J. Mech. Des. 137(2), MD-14-1128 (2015)Google Scholar
  38. 38.
    Tolson, B., Asadzadeh, M., Maier, H., Zecchin, A.: Hybrid discrete dynamically dimensioned search (HD-DDS) algorithm for water distribution system design optimization. Water Resour. Res. 45(12), W12416 (2009)CrossRefGoogle Scholar
  39. 39.
    Tolson, B., Shoemaker, C.: Cannonsville reservoir watershed SWAT2000 model development, calibration and validation. J. Hydrol. 337, 68–86 (2007)CrossRefGoogle Scholar
  40. 40.
    Tolson, B., Shoemaker, C.: Dynamically dimensioned search algorithm for computationally efficient watershed model calibration. Water Resour. Res. W01413(413), 16 (2007)Google Scholar
  41. 41.
    Viana, F.A.: Multiple Surrogates for Prediction and Optimization. Ph.D. thesis, University of Florida (2011)Google Scholar
  42. 42.
    Wild, S., Regis, R., Shoemaker, C.: ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Wild, S., Shoemaker, C.: Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev. 55, 349–371 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Zheng, C., Wang, P.: MT3DMS: A Modular Three-Dimensional Multispecies Transport Model. US Army Corps of Engineers (1999)Google Scholar

Copyright information

© Springer Science+Business Media New York (outside the USA) 2017

Authors and Affiliations

  1. 1.Lawrence Berkeley National Laboratory, Computational Research Division, Center for Computational Sciences and EngineeringBerkeleyUSA
  2. 2.Swiss ReArmonkUSA

Personalised recommendations