Skip to main content

Advertisement

SpringerLink
Log in

SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

This paper presents the surrogate model based algorithm SO-I for solving purely integer optimization problems that have computationally expensive black-box objective functions and that may have computationally expensive constraints. The algorithm was developed for solving global optimization problems, meaning that the relaxed optimization problems have many local optima. However, the method is also shown to perform well on many local optimization problems, and problems with linear objective functions. The performance of SO-I, a genetic algorithm, Nonsmooth Optimization by Mesh Adaptive Direct Search (NOMAD), SO-MI (Müller et al. in Comput Oper Res 40(5):1383–1400, 2013), variable neighborhood search, and a version of SO-I that only uses a local search has been compared on 17 test problems from the literature, and on eight realizations of two application problems. One application problem relates to hydropower generation, and the other one to throughput maximization. The numerical results show that SO-I finds good solutions most efficiently. Moreover, as opposed to SO-MI, SO-I is able to find feasible points by employing a first optimization phase that aims at minimizing a constraint violation function. A feasible user-supplied point is not necessary.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. http://www.i2c2.aut.ac.nz/Wiki/OPTI/.

  2. http://www.iim.csic.es/~gingproc/meigo.html.

  3. Note that SO-I is used in order to find a feasible solution from which both algorithms could start. Thus, the number of failed trials for SO-I, SO-MI, and VNS-ii are identical.

    Table 2 Constrained problems
    Full size table
    Table 3 Unconstrained problems
    Full size table
    Table 4 Application problems
    Full size table

  4. The Matlab codes provided on http://www.mcs.anl.gov/~more/dfo/ have been used for creating the plots.

Abbreviations

GA:

Genetic algorithm

NOMAD:

Nonsmooth Optimization by Mesh Adaptive Direct Search

RBF:

Radial basis function

SEM:

Standard error of means

SO-I:

Surrogate Optimization-Integer

SO-MI:

Surrogate Optimization-Mixed Integer

local-SO-I:

local-Surrogate Optimization-Integer

VNS:

Variable neighborhood search

\(\mathbb R \) :

Real numbers

\(\mathbb Z \) :

Integer numbers

\(\mathbf u \) :

Discrete decision variable vector, see Eq. (1d)

\(f(\cdot )\) :

Objective function, see Eq. (1a)

\(c_j(\cdot )\) :

\(j\)th constraint function, \(j=1,\ldots , m\), see Eq. (1b)

\(m\) :

Number of constraints

\(k\) :

Problem dimension

\(j\) :

Index for the constraints

\(i\) :

Index for the variables

\(u_i^l,\,u_i^u\) :

Lower and upper bounds for the \(i\)th variable, see Eq. (1c)

\(\varOmega _b\) :

Box-constrained variable domain

\(\varOmega \) :

Feasible variable domain

\(\mathcal S \) :

Set of already evaluated points

\(n_0\) :

Number of points in initial experimental design

\(q(\cdot )\) :

Auxiliary function for minimizing constraint violation in phase 1, see Eq. (5)

\(f_p(\cdot )\) :

Objective function value augmented with penalty term, see Eq. (6)

\(f_\text {max}\) :

Objective function value of the worst feasible point found so far, see Eq. (6)

\(p\) :

Penalty factor, see Eq. (6)

\(v(\cdot )\) :

Squared constraint violation function, see Eq. (7)

\({\varvec{\chi }}_\jmath \) :

\(\jmath \)th candidate point for next sample site, \(\jmath =1,\ldots , t\)

\(n\) :

Number of already sampled points

\(s(\cdot )\) :

Radial basis function interpolant

\(V(\cdot )\) :

Weighted score, see Eq. (8)

\(\omega _R, \omega _D\) :

Weights for response surface and distance criteria, respectively

References

  1. Abramson, M.A., Audet, C.: Convergence of mesh adaptive direct search to second-order stationary points. SIAM J. Optim. 17, 606–619 (2006)

    Article  Google Scholar 

  2. Abramson, M.A., Audet, C., Chrissis, J., Walston, J.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3, 35–47 (2009)

    Article  Google Scholar 

  3. Abramson, M.A., Audet, C., Couture, G., Dennis, J.E. Jr., Le Digabel, S.: The NOMAD project. Software available at http://www.gerad.ca/nomad (2011)

  4. Abramson, M.A., Audet, C., Dennis Jr, J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. SIAM J. Optim. 11, 573–594 (2004)

    Google Scholar 

  5. Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Global Optim. 41, 299–318 (2008)

    Article  Google Scholar 

  6. Bäck, T.: Evolutionary Algorithms in Theory and Practice. Oxford University Press, Oxford (1996)

    Google Scholar 

  7. Bäck, T., Fogel, D., Michalewicz, Z.: Handbook of Evolutionary Computation. Oxford University Press, Oxford (1997)

    Book  Google Scholar 

  8. Booker, A.J., Dennis Jr, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17, 1–13 (1999)

    Article  Google Scholar 

  9. Bussieck, M.R., Drud, A.S., Meeraus, A.: MINLPLib—a collection of test models for mixed-integer nonlinear programming. INFORMS J. Comput. 15, 114–119 (2003)

    Article  Google Scholar 

  10. Coit, D.W., Smith, A.E.: Reliability optimization of series-parallel systems using a genetic algorithm. IEEE Trans. Reliab. 45, 254–266 (1996)

    Article  Google Scholar 

  11. Currin, C., Mitchell, T., Morris, M., Ylvisaker, D.: A Bayesian approach to the design and analysis of computer experiments. Technical report, Oak Ridge National Laboratory, Oak Ridge, TN (1988)

  12. Davis, E., Ierapetritou, M.: Kriging based method for the solution of mixed-integer nonlinear programs containing black-box functions. J. Global Optim. 43, 191–205 (2009)

    Article  Google Scholar 

  13. Dorigo, M.: Optimization, Learning and Natural Algorithms. Ph.D. thesis, Politecnico di Milano, Italie (1992)

  14. Duchon, J.: Constructive Theory of Functions of Several Variables. Springer, Berlin (1977)

    Google Scholar 

  15. Fiechter, C.-N.: A parallel tabu search algorithm for large traveling salesman problems. Discret. Appl. Math. 51, 243–267 (1994)

    Article  Google Scholar 

  16. Friedman, J.H.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–141 (1991)

    Article  Google Scholar 

  17. Gershwin, S.B., Schor, J.E.: Efficient algorithms for buffer space allocation. Ann. Oper. Res. 93, 117–144 (2000)

    Article  Google Scholar 

  18. Glaz, B., Friedmann, P.P., Liu, L.: Surrogate based optimization of helicopter rotor blades for vibration reduction in forward flight. Struct. Multidiscip. Optim. 35, 341–363 (2008)

    Article  Google Scholar 

  19. Glover, F.W.: Heuristics for integer programming using surrogate constraints. Decis. Sci. 8, 156–166 (1977)

    Article  Google Scholar 

  20. Glover, F.W.: Tabu search—part 1. ORSA J. Comput. 1, 190–206 (1989)

    Article  Google Scholar 

  21. Glover, F.W.: Tabu search—part 2. ORSA J. Comput. 2, 4–32 (1990)

    Article  Google Scholar 

  22. Glover, F.W.: A Template for Scatter Search and Path Relinking. Springer, Heidelberg (1998)

    Google Scholar 

  23. Gutmann, H.-M.: A radial basis function method for global optimization. J. Global Optim. 19, 201–227 (2001)

    Article  Google Scholar 

  24. Hansen, P., Mladenović, N.: Variable neighborhood search: principles and applications. Eur. J. Oper. Res. 130, 449–467 (2001)

    Article  Google Scholar 

  25. Haupt, R.L.: Antenna design with a mixed integer genetic algorithm. IEEE Trans. Antennas Propag. 55, 577–582 (2007)

    Article  Google Scholar 

  26. Hesser, J., Männer, R.: Towards an optimal mutation probability for genetic algorithms. Lect. Notes Comput. Sci. 496, 23–32 (1991)

    Article  Google Scholar 

  27. Holland, J.H.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  28. Holmström, K.: An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J. Global Optim. 41, 447–464 (2008)

    Article  Google Scholar 

  29. Hu, J., Ogras, U.Y., Marculescu, R.: System-level buffer allocation for application-specific networks-on-chip router design. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 25, 2919–2933 (2006)

    Article  Google Scholar 

  30. Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21, 345–383 (2001)

    Article  Google Scholar 

  31. Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13, 455–492 (1998)

    Article  Google Scholar 

  32. Jouhaud, J.-C., Sagaut, P., Montagnac, M., Laurenceau, J.: A surrogate-model based multidisciplinary shape optimization method with application to a 2D subsonic airfoil. Comput. Fluids 36, 520–529 (2007)

    Article  Google Scholar 

  33. Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on, Neural Networks, vol. 4, pp. 1942–1948 (1995)

  34. Kennedy, J., Eberhart, R.: Swarm Intelligence. Morgan Kaufmann, San Francisco (2001)

    Google Scholar 

  35. Koziel, S., Michalewicz, Z.: Evolutionary algorithms, homomorphous mappings, and constrained parameter optimization. Evolut. Comput. 7, 19–44 (1999)

    Article  Google Scholar 

  36. Laguna, M., Martí, R.: Experimental testing of advanced scatter search designs for global optimization of multimodal functions. Technical report, University of Colorado at Boulder (2000)

  37. Lam, X.B., Kim, Y.S., Hoang, A.D., Park, C.W.: Coupled aerostructural design optimization using the kriging model and integrated multiobjective optimization algorithm. J. Optim. Theory Appl. 142, 533–556 (2009)

    Article  Google Scholar 

  38. Land, A.H., Doig, A.G.: An automatic method of solving discrete programming problems. Econometrica 28, 497–520 (1960)

    Article  Google Scholar 

  39. Laskari, E.C., Parsopoulos, K.E., Vrahatis, M.N.: Particle swarm optimization for integer programming. In: Proceedings of the 2002 Congress on Evolutionary Computation vol. 2, pp. 1582–1587 (2002)

  40. Li, C., Wang, F.-L., Chang, Y.-Q., Liu, Y.: A modified global optimization method based on surrogate model and its application in packing profile optimization of injection molding process. Int. J. Adv. Manuf. Technol. 48, 505–511 (2010)

    Article  Google Scholar 

  41. Li, F., Shoemaker, C.A., Wei, J., Fu, X.: Estimating maximal annual energy given heterogeneous hydropower generating units with application to the Three Gorges System. J. Water Resour. Plan. Manag. doi:10.1061/(ASCE)WR.1943-5452.0000250 (2012)

  42. Liang, Y.-C.: An ant colony optimization algorithm for the redundancy allocation problem (RAP). IEEE Trans. Reliab. 53, 417–423 (2004)

    Article  Google Scholar 

  43. Malek, M., Guruswamy, M., Pandya, M., Owens, H.: Serial and parallel simulated annealing and tabu search algorithms for the traveling salesman problem. Ann. Oper. Res. 21, 59–84 (1989)

    Article  Google Scholar 

  44. Michalewicz, Z., Fogel, D.: How to Solve it: Modern Heuristics. Springer, New York (2004)

    Book  Google Scholar 

  45. Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24, 1097–1100 (1997)

    Article  Google Scholar 

  46. Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20, 172–191 (2009)

    Article  Google Scholar 

  47. Müller, J., Piché, R.: Mixture surrogate models based on Dempster-Shafer theory for global optimization problems. J. Global Optim. 51, 79–104 (2011)

    Article  Google Scholar 

  48. Müller, J., Shoemaker, C.A., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40, 1383–1400 (2013)

    Article  Google Scholar 

  49. Myers, R.H., Montgomery, D.C.: Response Surface Methodology, Process and Product Optimization Using Designed Experiments. Wiley-Interscience Publication, New York (1995)

    Google Scholar 

  50. Pichitlamken, J., Nelson, B.L., Hong, L.J.: A sequential procedure for neighborhood selection-of-the-best in optimization via simulation. Eur. J. Oper. Res. 173, 283–298 (2006)

    Article  Google Scholar 

  51. Poli, R.: Analysis of the publications on the applications of particle swarm optimisation. J. Artif. Evolut. Appl. 1–10, 2008 (2008)

    Google Scholar 

  52. Powell, M.J.D.: The theory of radial basis function approximation in 1990. In: Advances in Numerical Analysis, Wavelets, Subdivision Algorithms and Radial basis Functions, vol. 2, pp. 105–210. Oxford University Press, Oxford (1992)

  53. Rashid, K., Ambani, S., Cetinkaya, E.: An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization. Eng. Optim. doi:10.1080/0305215X.2012.665450 (2012)

  54. Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black-box functions using radial basis functions. J. Global Optim. 31, 153–171 (2005)

    Article  Google Scholar 

  55. Regis, R.G., Shoemaker, C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19, 497–509 (2007)

    Article  Google Scholar 

  56. Regis, R.G., Shoemaker, C.A.: Improved strategies for radial basis function methods for global optimization. J. Global Optim. 37, 113–135 (2007)

    Article  Google Scholar 

  57. Schoen, F.: A wide class of test functions for global optimization. J. Global Optim. 3, 133–137 (1993)

    Article  Google Scholar 

  58. Shan, S., Wang, G.G.: Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct. Multidiscip. Optim. 41, 219–241 (2010)

    Article  Google Scholar 

  59. Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: Proceedings of IEEE International Conference on Evolutionary Computation, pp. 69–73 (1998)

  60. Spinellis, D.D., Papadopoulos, C.T.: A simulated annealing approach for buffer allocation in reliable production lines. Ann. Oper. Res. 93, 373–384 (2000)

    Article  Google Scholar 

  61. Srinivas, M., Patnaik, L.M.: Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Trans. Syst. Man Cybern. 24, 565–667 (1994)

    Article  Google Scholar 

  62. Törn, A., Zilinskas, A.: Global Optimization. In: Lecture Notes in Computer Science, 350. Springer, Berlin (1989)

  63. Wild, S.M., Regis, R.G., Shoemaker, C.A.: ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30, 3197–3219 (2007)

    Article  Google Scholar 

  64. Wu, Q.H., Cao, Y.J., Wen, J.Y.: Optimal reactive power dispatch using an adaptive genetic algorithm. Int. J. Electr. Power Energy Syst. 20, 563–569 (1998)

    Article  Google Scholar 

  65. Ye, K.Q., Li, W., Sudjianto, A.: Algorithmic construction of optimal symmetric Latin hypercube designs. J. Stat. Plan. Inference 90, 145–159 (2000)

    Article  Google Scholar 

  66. Zhang, J., Chung, H.S.-H., Lo, W.-L.: Clustering-based adaptive crossover and mutation probabilities for genetic algorithms. IEEE Trans. Evol. Comput. 11, 326–335 (2007)

    Article  Google Scholar 

Download references

Acknowledgments

The project has been supported by NSF CISE: AF 1116298 and CPA-CSA-T 0811729. The first author thanks the Finnish Academy of Science and Letters, Vilho, Yrjö and Kalle Väisälä Foundation for the financial support. The authors thank the anonymous reviewers for their helpful comments and suggestions.

Author information

Authors and Affiliations

  1. School of Civil and Environmental Engineering, Cornell University, 209 Hollister Hall, Ithaca, NY , 14853-3501, USA

    Juliane Müller

  2. School of Civil and Environmental Engineering, Center of Applied Mathematics, Cornell University, 210 Hollister Hall, Ithaca, NY , 14853-3501, USA

    Christine A. Shoemaker

  3. School of Operations Research and Information Engineering, Center of Applied Mathematics, Cornell University, 210 Hollister Hall, Ithaca, NY , 14853-3501, USA

    Christine A. Shoemaker

  4. Department of Mathematics, Tampere University of Technology, P.O. Box 553, 33101 , Tampere, Finland

    Robert Piché

Authors
  1. Juliane Müller
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christine A. Shoemaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Robert Piché
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juliane Müller.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 99 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Müller, J., Shoemaker, C.A. & Piché, R. SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J Glob Optim 59, 865–889 (2014). https://doi.org/10.1007/s10898-013-0101-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-013-0101-y

Keywords

Search

Navigation