Abstract
A trust-region-based derivative free algorithm for solving bound constrained mixed integer nonlinear programs is developed in this paper. The algorithm is proven to converge to a local minimum after a finite number of function evaluations. In addition, an improved definition of local minima of mixed integer programs is proposed. Computational results showing the effectiveness of the derivative free algorithm are presented.
Similar content being viewed by others
Notes
\(s\) is excluded since HEMBOQA can never replace the best interpolation point.
References
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM (2009)
Gumma, E.A.E.: Derivative-Free Algorithm for Linearly Constrained Optimization Problems. Ph.D. dissertation, University of Khartoum, Khartoum (2010)
Hemker, T.: Derivative Free Surrogate Optimization for Mixed-Integer Nonlinear Black Box Problems in Engineering. Ph.D. dissertation, Technischen Universität Darmstadt (2008)
Holmström, K., Quttineh, N.-H., Edvall, M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9, 311–339 (2008)
Audet, C., Dennis Jr, J.E.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11, 573–594 (2001)
Costa, L., Oliveira, P.: Evolutionary algorithms approach to the solution of mixed integer non-linear programming problems. Comput. Chem. Eng. 25, 257–266 (2001)
Deep, K., Singh, K.P., Kansal, M.L., Mohan, C.: A real coded genetic algorithm for solving integer and mixed integer optimization problems. Appl. Math. Comput. 212, 505–518 (2009)
Kitayama, S., Arakawa, M., Yamazaki, K.: Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization. Struct. Multidiscip. Optim. 32, 191–202 (2006)
Romeijn, H.E., Zabinsky, Z.B., Graesser, D.L., Neogi, S.: New reflection generator for simulated annealing in mixed-integer/continuous global optimization. J. Optim. Theory Appl. 101, 403–427 (1999)
Wah, B., Chen, Y., Wang, T.: Simulated annealing with asymptotic convergence for nonlinear constrained optimization. J. Glob. Optim. 39, 1–37 (2007)
Yiqing, L., Xigang, Y., Yongjian, L.: An improved PSO algorithm for solving non-convex NLP/MINLP problems with equality constraints. Comput. Chem. Eng. 31, 153–162 (2007)
Müller, J., Shoemaker, C.A., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40, 1383–1400 (2013)
Abramson, M., Audet, C., Chrissis, J., Walston, J.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3, 35–47 (2009)
Abramson, M., Audet, C., Dennis Jr, J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3, 477–500 (2007)
Abramson, M.A.: Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems. Ph.D. dissertation, Rice University (2002)
Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for bound constrained mixed-integer optimization. Comput. Optim. Appl. 53, 502–526 (2012)
Lucidi, S., Piccialli, V., Sciandrone, M.: An algorithm model for mixed variable programming. SIAM J. Optim. 15, 1057–1084 (2005)
Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)
Vicente, L.N., Custódio, A.L.: Analysis of direct searches for discontinuous functions. Math. Program. 133, 299–325 (2012)
Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18, 537–555 (2007)
Custódio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46, 265–278 (2010)
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)
Sriver, T.A., Chrissis, J.W., Abramson, M.A.: Pattern search ranking and selection algorithms for mixed variable simulation-based optimization. Eur. J. Oper. Res. 198, 878–890 (2009)
Abramson, M.A., Audet, C., Couture, G., Dennis, Jr., J.E., Le Digabel, S.: The NOMAD project. Software available at http://www.gerad.ca/nomad (2013)
Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37, 1–15 (2011)
Audet, C., Dennis Jr, J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217 (2006)
Lucidi, S., Sciandrone, M.: A derivative-free algorithm for bound constrained optimization. Comput. Optim. Appl. 21, 119–142 (2002)
Vicente, L.N.: Implicitly and densely discrete black-box optimization problems. Optim. Lett. 3, 475–482 (2009)
Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivatives. Technical Report DAMTP 2009/NA06, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (2009)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first-and second-order critical points. SIAM J. Optim. 20, 387–415 (2009)
Conn, A.R., Scheinberg, K., Toint, Ph L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization: Tributes to M. J. D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)
Newby, E.: General solution methods for mixed integer quadratic programming and derivative free mixed integer non-linear programming problems. Ph.D. dissertation, University of the Witwatersrand (2013)
Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)
Wolsey, L.A.: Integer Programming. Wiley, New York (1998)
Li, D., Sun, X.: Nonlinear Integer Programming. Springer, New York (2006)
Piccialli, V.: Methods for Solving Mixed Variable Programming Problems. Ph.D. dissertation, University of Rome, Rome (2003)
GAMS: CPLEX 12 user manual (2011)
Billionnet, A., Elloumi, S., Lambert, A.: Extending the QCR method to general mixed-integer programs. Math. Program. 131, 381–401 (2012)
Abramson, M.A.: NOMADm version 4.6 user’s guide. Technical report, Department of Mathematics and Statistics, Air Force Institute of Technology (2007)
Ali, M.M., Khompatraporn, C., Zabinsky, Z.B.: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Glob. Optim. 31, 635–672 (2005)
Yang, X.-S.: Engineering Optimization: An Introduction with Metaheuristic Applications. Wiley, New York (2010)
DeJong, K.A.: An Analysis of the Behavior of a Class of Genetic Adaptive Systems. Ph.D. dissertation, University of Michigan, Michigan (1975)
Levy, A.V., Montalvo, A.: The tunneling algorithm for the global optimization of functions. SIAM J. Sci. Stat. Comput. 6, 15–29 (1985)
Molga, M., Smutnicki, C.: Test functions for optimization needs. Technical report, Zakład Systemów Dyskretnych (2005)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Diniz-Ehrhardt, M.A., Martínez, J.M., Pedroso, L.G.: Derivative-free methods for nonlinear programming with general lower-level constraints. J. Comput. Appl. Math. 30, 19–52 (2010)
Lewis, R.M., Torczon, V.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075–1089 (2002)
Bueno, L.F., Friedlander, A., Martínez, J.M.: Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J. Optim. 23, 1189–1213 (2013)
Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20, 445–472 (2009)
Audet, C., Dennis Jr, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14, 980–1010 (2004)
Zhu, W., Ali, M.M.: Solving nonlinearly constrained global optimization problem via an auxiliary function method. J. Comput. Appl. Math. 230, 491–503 (2009)
Acknowledgments
The authors would like to thank the anonymous reviewers and the associate editor for their helpful comments. The authors were supported by the National Research Foundation of South Africa under Grant CPR2010030300009918.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Newby, E., Ali, M.M. A trust-region-based derivative free algorithm for mixed integer programming. Comput Optim Appl 60, 199–229 (2015). https://doi.org/10.1007/s10589-014-9660-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-014-9660-1