Optimization and Engineering

, Volume 17, Issue 1, pp 177–203 | Cite as

MISO: mixed-integer surrogate optimization framework

  • Juliane MüllerEmail author


We introduce MISO, the mixed-integer surrogate optimization framework. MISO aims at solving computationally expensive black-box optimization problems with mixed-integer variables. This type of optimization problem is encountered in many applications for which time consuming simulation codes must be run in order to obtain an objective function value. Examples include optimal reliability design and structural optimization. A single objective function evaluation may take from several minutes to hours or even days. Thus, only very few objective function evaluations are allowable during the optimization. The development of algorithms for this type of optimization problems has, however, rarely been addressed in the literature. Because the objective function is black-box, derivatives are not available and numerically approximating the derivatives requires a prohibitively large number of function evaluations. Therefore, we use computationally cheap surrogate models to approximate the expensive objective function and to decide at which points in the variable domain the expensive objective function should be evaluated. We develop a general surrogate model framework and show how sampling strategies of well-known surrogate model algorithms for continuous optimization can be modified for mixed-integer variables. We introduce two new algorithms that combine different sampling strategies and local search to obtain high-accuracy solutions. We compare MISO in numerical experiments to a genetic algorithm, NOMAD version 3.6.2, and SO-MI. The results show that MISO is in general more efficient than NOMAD and the genetic algorithm with respect to finding improved solutions within a limited budget of allowable evaluations. The performance of MISO depends on the chosen sampling strategy. The MISO algorithm that combines a coordinate perturbation search with a target value strategy and a local search performs best among all algorithms.


Mixed-integer optimization Surrogate models Black-box optimization Radial basis functions Derivative-free Global optimization 



DYnamic COordinate search using response surface models (Regis and Shoemaker 2013 )


Efficient global optimization (Jones et al. 1998)


Mixed-integer surrogate optimization


MISO-coordinate perturbation


MISO-expected improvement


MISO-random sampling


MISO-surface minimum


MISO-target value


MISO with combination of CP and TV


MISO with combination of CP, TV, and a local search


MISO-CPTV-local that uses fmincon as local optimizer


MISO-CPTV-local that uses ORBIT (Wild et al. 2007) as local optimizer


Nonlinear optimization by mesh-adaptive direct search (Le Digabel 2011), version 3.6.2


Radial basis function


Surrogate optimization-mixed integer (Müller et al. 2013b)


Stochastic radial basis function algorithm (Regis and Shoemaker 2007)


Computationally expensive objective function


Problem dimension


Number of integer variables


Number of continuous variables

\({\mathbf {z}}\)

Variable vector

\(z_i^l\), \(z_i^u\)

Lower and upper variable bounds of the ith variable

\({\mathcal {Z}}\)

Set of evaluated points, \({\mathcal {Z}}=\{{\mathbf {z}}_1,\ldots ,{\mathbf {z}}_n\}\)

\({\mathcal {Z}}_l\)

Set of points evaluated during the local search (l-step)


Number of points in the initial experimental design


Number of function evaluations done in the local search (l-step)


Number of already evaluated points

\(s_n(\cdot )\)

Surrogate model fit to n data points

\({\mathbb {I}}\)

Indices of the integer variables



This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Applied Mathematics program under contract number DE-AC02005CH11231. Partial support was also provided by NSF CISE 1116298 to Prof. Shoemaker at Cornell University, School of Civil and Environmental Engineering.

Supplementary material

11081_2015_9281_MOESM1_ESM.pdf (96 kb)
Supplementary material 1 (PDF 96 kb)


  1. Abramson M, Audet C, Chrissis J, Walston J (2009) Mesh adaptive direct search algorithms for mixed variable optimization. Optim Lett 3:35–47CrossRefMathSciNetzbMATHGoogle Scholar
  2. Booker A, Dennis J Jr, Frank P, Serafini D, Torczon V, Trosset M (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Multidiscip Optim 17:1–13CrossRefGoogle Scholar
  3. Conn A, Scheinberg K, Vicente L (2009) Introduction to Derivative-Free Optimization. SIAMGoogle Scholar
  4. Currie J, Wilson D (2012) Foundations of computer-aided process operations. OPTI: lowering the barrier between open source optimizers and the industrial MATLAB user. Savannah, Georgia, USAGoogle Scholar
  5. Davis E, Ierapetritou M (2009) Kriging based method for the solution of mixed-integer nonlinear programs containing black-box functions. J Glob Optim 43:191–205CrossRefMathSciNetzbMATHGoogle Scholar
  6. Forrester A, Sóbester A, Keane A (2008) Engineering design via surrogate modelling—a practical guide. Wiley, ChichesterCrossRefGoogle Scholar
  7. Friedman J (1991) Multivariate adaptive regression splines. Ann Stat 19:1–141CrossRefzbMATHGoogle Scholar
  8. Giunta A, Balabanov V, Haim D, Grossman B, Mason W, Watson L, Haftka R (1997) Aircraft multidisciplinary design optimisation using design of experiments theory and response surface modelling. Aeronaut J 101:347–356Google Scholar
  9. Glaz B, Friedmann P, Liu L (2008) Surrogate based optimization of helicopter rotor blades for vibration reduction in forward flight. Struct Multidiscip Optim 35:341–363CrossRefGoogle Scholar
  10. Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33:199–216CrossRefGoogle Scholar
  11. Gutmann H (2001) A radial basis function method for global optimization. J Glob Optim 19:201–227CrossRefMathSciNetzbMATHGoogle Scholar
  12. Hemker T, Fowler K, Farthing M, von Stryk O (2008) A mixed-integer simulation-based optimization approach with surrogate functions in water resources management. Optim Eng 9:341–360CrossRefMathSciNetzbMATHGoogle Scholar
  13. Holmström K (2008a) An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J Glob Optim 41:447–464CrossRefzbMATHGoogle Scholar
  14. Holmström K (2008b) An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer global optimization. J Glob Optim 9:311–339zbMATHGoogle Scholar
  15. Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455–492CrossRefMathSciNetzbMATHGoogle Scholar
  16. Koziel S, Leifsson L (2013) Surroagte-based modeling and optimization: applications in engineering. Springer, New YorkCrossRefGoogle Scholar
  17. Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the mads algorithm. ACM Trans Math Softw 37:44CrossRefMathSciNetGoogle Scholar
  18. Li R, Emmerich M, Eggermont J, Bovenkamp E, Back T, Dijkstra J, Reiber H (2008) Metamodel-assisted mixed-integer evolution strategies and their applications to intravascular ultrasound image analysis. In: IEEE World Congress on Computational Intelligence, IEEE, pp 2764–2771Google Scholar
  19. Marsden A, Wang M, Dennis J Jr, Moin P (2004) Optimal aeroacoustic shape design using the surrogate management framework. Optim Eng 5:235–262CrossRefMathSciNetzbMATHGoogle Scholar
  20. Moré J, Wild S (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20:172–191CrossRefMathSciNetzbMATHGoogle Scholar
  21. Müller J (2014) MATSuMoTo: The MATLAB surrogate model toolbox for computationally expensive black-box global optimization problems. arXiv:14044261Google Scholar
  22. Müller J, Shoemaker C, Piché R (2013a) SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J Glob Optim 59:865–889. doi: 10.1007/s10,898-013-0101-y CrossRefGoogle Scholar
  23. Müller J, Shoemaker C, Piché R (2013b) SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput Oper Res 40:1383–1400CrossRefMathSciNetGoogle Scholar
  24. Müller J, Piché R (2011) Mixture surrogate models based on Dempster–Shafer theory for global optimization problems. J Glob Optim 51:79–104CrossRefzbMATHGoogle Scholar
  25. Müller J, Shoemaker C (2014) Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J Glob Optim 60:123–144. doi: 10.1007/s10898-014-0184-0 CrossRefzbMATHGoogle Scholar
  26. Myers R, Montgomery D (1995) Response surface methodology: process and product optimization using designed experiments. Wiley-Interscience Publication, HobokenzbMATHGoogle Scholar
  27. Powell M (1992) Advances in numerical analysis, vol. 2: wavelets, subdivision algorithms and radial basis functions. In: Light WA (ed) The theory of radial basis function approximation in 1990. Oxford University Press, Oxford, pp 105–210Google Scholar
  28. Rashid S, Ambani S, Cetinkaya E (2012) An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization. Eng Optim 45:185–206. doi: 10.1080/0305215X.2012.665450 CrossRefMathSciNetGoogle Scholar
  29. Regis R, Shoemaker C (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19:497–509CrossRefMathSciNetzbMATHGoogle Scholar
  30. Regis R, Shoemaker C (2009) Parallel stochastic global optimization using radial basis functions. INFORMS J Comput 21:411–426CrossRefMathSciNetzbMATHGoogle Scholar
  31. Regis R, Shoemaker C (2013) Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng Optim 45:529–555CrossRefMathSciNetGoogle Scholar
  32. Simpson T, Mauery T, Korte J, Mistree F (2001) Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39:2233–2241CrossRefGoogle Scholar
  33. Viana F, Haftka R, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39:439–457CrossRefGoogle Scholar
  34. Wild S, Regis R, Shoemaker C (2007) ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J Sci Comput 30:3197–3219CrossRefMathSciNetGoogle Scholar
  35. Wild S, Shoemaker C (2013) Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev 55:349–371CrossRefMathSciNetzbMATHGoogle Scholar
  36. Zhuang L, Tang K, Jin Y (2013) Metamodel assisted mixed-integer evolution strategies based on Kendall rank correlation coefficient. In: Hea Yin (ed) IDEAL 2013. Springer-Verlag, Berlin, pp 366–375Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Center for Computational Sciences and EngineeringLawrence Berkeley National LaboratoryBerkeleyUSA

Personalised recommendations