Skip to main content
Log in

A trust-region-based derivative free algorithm for mixed integer programming

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

A trust-region-based derivative free algorithm for solving bound constrained mixed integer nonlinear programs is developed in this paper. The algorithm is proven to converge to a local minimum after a finite number of function evaluations. In addition, an improved definition of local minima of mixed integer programs is proposed. Computational results showing the effectiveness of the derivative free algorithm are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. \(s\) is excluded since HEMBOQA can never replace the best interpolation point.

References

  1. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM (2009)

  2. Gumma, E.A.E.: Derivative-Free Algorithm for Linearly Constrained Optimization Problems. Ph.D. dissertation, University of Khartoum, Khartoum (2010)

  3. Hemker, T.: Derivative Free Surrogate Optimization for Mixed-Integer Nonlinear Black Box Problems in Engineering. Ph.D. dissertation, Technischen Universität Darmstadt (2008)

  4. Holmström, K., Quttineh, N.-H., Edvall, M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9, 311–339 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  5. Audet, C., Dennis Jr, J.E.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11, 573–594 (2001)

    Article  MathSciNet  Google Scholar 

  6. Costa, L., Oliveira, P.: Evolutionary algorithms approach to the solution of mixed integer non-linear programming problems. Comput. Chem. Eng. 25, 257–266 (2001)

    Article  Google Scholar 

  7. Deep, K., Singh, K.P., Kansal, M.L., Mohan, C.: A real coded genetic algorithm for solving integer and mixed integer optimization problems. Appl. Math. Comput. 212, 505–518 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  8. Kitayama, S., Arakawa, M., Yamazaki, K.: Penalty function approach for the mixed discrete nonlinear problems by particle swarm optimization. Struct. Multidiscip. Optim. 32, 191–202 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  9. Romeijn, H.E., Zabinsky, Z.B., Graesser, D.L., Neogi, S.: New reflection generator for simulated annealing in mixed-integer/continuous global optimization. J. Optim. Theory Appl. 101, 403–427 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  10. Wah, B., Chen, Y., Wang, T.: Simulated annealing with asymptotic convergence for nonlinear constrained optimization. J. Glob. Optim. 39, 1–37 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  11. Yiqing, L., Xigang, Y., Yongjian, L.: An improved PSO algorithm for solving non-convex NLP/MINLP problems with equality constraints. Comput. Chem. Eng. 31, 153–162 (2007)

    Article  Google Scholar 

  12. Müller, J., Shoemaker, C.A., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40, 1383–1400 (2013)

    Article  MathSciNet  Google Scholar 

  13. Abramson, M., Audet, C., Chrissis, J., Walston, J.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3, 35–47 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  14. Abramson, M., Audet, C., Dennis Jr, J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3, 477–500 (2007)

    MATH  MathSciNet  Google Scholar 

  15. Abramson, M.A.: Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems. Ph.D. dissertation, Rice University (2002)

  16. Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for bound constrained mixed-integer optimization. Comput. Optim. Appl. 53, 502–526 (2012)

    Article  MathSciNet  Google Scholar 

  17. Lucidi, S., Piccialli, V., Sciandrone, M.: An algorithm model for mixed variable programming. SIAM J. Optim. 15, 1057–1084 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  18. Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7, 1–25 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  19. Vicente, L.N., Custódio, A.L.: Analysis of direct searches for discontinuous functions. Math. Program. 133, 299–325 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  20. Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. 18, 537–555 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  21. Custódio, A.L., Rocha, H., Vicente, L.N.: Incorporating minimum Frobenius norm models in direct search. Comput. Optim. Appl. 46, 265–278 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  22. Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385–482 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  23. Sriver, T.A., Chrissis, J.W., Abramson, M.A.: Pattern search ranking and selection algorithms for mixed variable simulation-based optimization. Eur. J. Oper. Res. 198, 878–890 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  24. Abramson, M.A., Audet, C., Couture, G., Dennis, Jr., J.E., Le Digabel, S.: The NOMAD project. Software available at http://www.gerad.ca/nomad (2013)

  25. Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37, 1–15 (2011)

    Article  Google Scholar 

  26. Audet, C., Dennis Jr, J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  27. Lucidi, S., Sciandrone, M.: A derivative-free algorithm for bound constrained optimization. Comput. Optim. Appl. 21, 119–142 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  28. Vicente, L.N.: Implicitly and densely discrete black-box optimization problems. Optim. Lett. 3, 475–482 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  29. Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivatives. Technical Report DAMTP 2009/NA06, Department of Applied Mathematics and Theoretical Physics, University of Cambridge (2009)

  30. Conn, A.R., Scheinberg, K., Vicente, L.N.: Geometry of interpolation sets in derivative free optimization. Math. Program. 111, 141–172 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  31. Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first-and second-order critical points. SIAM J. Optim. 20, 387–415 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  32. Conn, A.R., Scheinberg, K., Toint, Ph L.: On the convergence of derivative-free methods for unconstrained optimization. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization: Tributes to M. J. D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)

    Google Scholar 

  33. Newby, E.: General solution methods for mixed integer quadratic programming and derivative free mixed integer non-linear programming problems. Ph.D. dissertation, University of the Witwatersrand (2013)

  34. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (2006)

    MATH  Google Scholar 

  35. Wolsey, L.A.: Integer Programming. Wiley, New York (1998)

    MATH  Google Scholar 

  36. Li, D., Sun, X.: Nonlinear Integer Programming. Springer, New York (2006)

    MATH  Google Scholar 

  37. Piccialli, V.: Methods for Solving Mixed Variable Programming Problems. Ph.D. dissertation, University of Rome, Rome (2003)

  38. GAMS: CPLEX 12 user manual (2011)

  39. Billionnet, A., Elloumi, S., Lambert, A.: Extending the QCR method to general mixed-integer programs. Math. Program. 131, 381–401 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  40. Abramson, M.A.: NOMADm version 4.6 user’s guide. Technical report, Department of Mathematics and Statistics, Air Force Institute of Technology (2007)

  41. Ali, M.M., Khompatraporn, C., Zabinsky, Z.B.: A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Glob. Optim. 31, 635–672 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  42. Yang, X.-S.: Engineering Optimization: An Introduction with Metaheuristic Applications. Wiley, New York (2010)

    Book  Google Scholar 

  43. DeJong, K.A.: An Analysis of the Behavior of a Class of Genetic Adaptive Systems. Ph.D. dissertation, University of Michigan, Michigan (1975)

  44. Levy, A.V., Montalvo, A.: The tunneling algorithm for the global optimization of functions. SIAM J. Sci. Stat. Comput. 6, 15–29 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  45. Molga, M., Smutnicki, C.: Test functions for optimization needs. Technical report, Zakład Systemów Dyskretnych (2005)

  46. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  47. Diniz-Ehrhardt, M.A., Martínez, J.M., Pedroso, L.G.: Derivative-free methods for nonlinear programming with general lower-level constraints. J. Comput. Appl. Math. 30, 19–52 (2010)

    Google Scholar 

  48. Lewis, R.M., Torczon, V.: A globally convergent augmented lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12, 1075–1089 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  49. Bueno, L.F., Friedlander, A., Martínez, J.M.: Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J. Optim. 23, 1189–1213 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  50. Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20, 445–472 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  51. Audet, C., Dennis Jr, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14, 980–1010 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  52. Zhu, W., Ali, M.M.: Solving nonlinearly constrained global optimization problem via an auxiliary function method. J. Comput. Appl. Math. 230, 491–503 (2009)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous reviewers and the associate editor for their helpful comments. The authors were supported by the National Research Foundation of South Africa under Grant CPR2010030300009918.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Eric Newby.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Newby, E., Ali, M.M. A trust-region-based derivative free algorithm for mixed integer programming. Comput Optim Appl 60, 199–229 (2015). https://doi.org/10.1007/s10589-014-9660-1

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-014-9660-1

Keywords

Mathematics Subject Classification

Navigation