Skip to main content

An algorithmic framework based on primitive directions and nonmonotone line searches for black-box optimization problems with integer variables


In this paper, we develop a new algorithmic framework to solve black-box problems with integer variables. The strategy included in the framework makes use of specific search directions (so called primitive directions) and a suitably developed nonmonotone line search, thus guaranteeing a high level of freedom when exploring the integer lattice. First, we describe and analyze a version of the algorithm that tackles problems with only bound constraints on the variables. Then, we combine it with a penalty approach in order to solve problems with simulation constraints. In both cases we prove finite convergence to a suitably defined local minimum of the problem. We report extensive numerical experiments based on a test bed of both bound-constrained and generally-constrained problems. We show the effectiveness of the method when compared to other state-of-the-art solvers for black-box integer optimization.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. A new version of NOMAD with better functionalities, e.g. the Nelder Mead search and the possibility to specify the direction type for poll intensification, was released while this paper was under review (for further details visit the website


  1. Abramson, M.A., Audet, C., Chrissis, J.W., Walston, J.C.: Mesh adaptive direct search algorithms for mixed variable optimization. Optim. Lett. 3(1), 35–47 (2009)

    Article  MathSciNet  Google Scholar 

  2. Abramson, M.A., Audet, C., Couture, G., Dennis Jr., J.E., Le Digabel, S.: The NOMAD project.

  3. Abramson, M.A., Audet, C., Dennis Jr., J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)

    MathSciNet  MATH  Google Scholar 

  4. Abramson, M.A., Audet, C., Dennis Jr., J.E., Le Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948–966 (2009)

    Article  MathSciNet  Google Scholar 

  5. Audet, C., Dennis Jr., J.E.: Pattern search algorithms for mixed variable programming. SIAM J. Optim. 11(3), 573–594 (2001)

    Article  MathSciNet  Google Scholar 

  6. Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer, Berlin (2017)

    Book  Google Scholar 

  7. Audet, C., Le Digabel, S., Tribes, C.: The mesh adaptive direct search algorithm for granular and discrete variables. SIAM J. Optim. 29(2), 1164–1189 (2019)

    Article  MathSciNet  Google Scholar 

  8. Blum, C., Roli, A.: Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput. Surv. CSUR 35(3), 268–308 (2003)

    Article  Google Scholar 

  9. Conn, A., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization, vol. 8. SIAM, Philadelphia (2009)

    Book  Google Scholar 

  10. Costa, A., Nannicini, G.: RBFOpt: an open-source library for black-box optimization with costly function evaluations. Math. Program. Comput. 10(4), 597–629 (2018).

    Article  MathSciNet  MATH  Google Scholar 

  11. Davis, L.: Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York (1991)

    Google Scholar 

  12. Diniz-Ehrhardt, M., Martínez, J., Raydán, M.: A derivative-free nonmonotone line-search technique for unconstrained optimization. J. Comput. Appl. Math. 219(2), 383–397 (2008)

    Article  MathSciNet  Google Scholar 

  13. Fasano, G., Liuzzi, G., Lucidi, S., Rinaldi, F.: A linesearch-based derivative-free approach for nonsmooth constrained optimization. SIAM J. Optim. 24(3), 959–992 (2014)

    Article  MathSciNet  Google Scholar 

  14. García-Palomares, U.M., Rodríguez, J.F.: New sequential and parallel derivative-free algorithms for unconstrained minimization. SIAM J. Optim. 13(1), 79–96 (2002)

    Article  MathSciNet  Google Scholar 

  15. Gendreau, M., Potvin, J.Y.: Handbook of Metaheuristics, vol. 2. Springer, Berlin (2010)

    Book  Google Scholar 

  16. Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning, 1st edn. Addison-Wesley Longman Publishing Co. Inc, Boston, MA (1989)

    MATH  Google Scholar 

  17. Grippo, L., Rinaldi, F.: A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations. Comput. Optim. Appl. 60(1), 1–33 (2015)

    Article  MathSciNet  Google Scholar 

  18. Halton, J.: On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer. Math. 2, 84–90 (1960)

    Article  MathSciNet  Google Scholar 

  19. Larson, J., Leyffer, S., Palkar, P., Wild, S.M.: A method for convex black-box integer global optimization. arXiv preprint arXiv:1903.11366 (2019)

  20. Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)

    Article  MathSciNet  Google Scholar 

  21. Le Digabel, S., Wild, S.M.: A taxonomy of constraints in simulation-based optimization. arXiv preprint arXiv:1505.07881 (2015)

  22. Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for bound constrained mixed-integer optimization. Comput. Optim. Appl. 53(2), 505–526 (2012)

    Article  MathSciNet  Google Scholar 

  23. Liuzzi, G., Lucidi, S., Rinaldi, F.: Derivative-free methods for mixed-integer constrained optimization problems. J. Optim. Theory Appl. 164(3), 933–965 (2015)

    Article  MathSciNet  Google Scholar 

  24. Liuzzi, G., Lucidi, S., Rinaldi, F.: DFLINT—an algorithm for black-box inequality and box constrained integer nonlinear programming problems (2020).

  25. Lourenço, H.R., Martin, O.C., Stützle, T.: Iterated local search. In: Glover, F., Kochenberger, G.A. (eds.) Handbook of Metaheuristics, pp. 320–353. Springer, Boston, MA (2003).

    Chapter  Google Scholar 

  26. Lucidi, S., Piccialli, V., Sciandrone, M.: An algorithm model for mixed variable programming. SIAM J. Optim. 15(4), 1057–1084 (2005)

    Article  MathSciNet  Google Scholar 

  27. Lukšan, V., Vlček, J.: Test problems for nonsmooth unconstrained and linearly constrained optimization. Technical report VT798-00, Institute of Computer Science, Academy of Sciences of the Czech Republic (2000)

  28. Mladenović, N., Hansen, P.: Variable neighborhood search. Comput. Oper. Res. 24(11), 1097–1100 (1997)

    Article  MathSciNet  Google Scholar 

  29. Moré, J., Wild, S.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)

    Article  MathSciNet  Google Scholar 

  30. Müller, J.: MISO: mixed-integer surrogate optimization framework. Optim. Eng. 17(1), 177–203 (2016)

    Article  MathSciNet  Google Scholar 

  31. Müller, J., Shoemaker, C.A., Piché, R.: SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput. Oper. Res. 40(5), 1383–1400 (2013)

    Article  MathSciNet  Google Scholar 

  32. Müller, J., Shoemaker, C.A., Piché, R.: SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J. Glob. Optim. 59(4), 865–889 (2014)

    Article  MathSciNet  Google Scholar 

  33. Newby, E., Ali, M.M.: A trust-region-based derivative free algorithm for mixed integer programming. Comput. Optim. Appl. 60(1), 199–229 (2015)

    Article  MathSciNet  Google Scholar 

  34. Porcelli, M., Toint, P.L.: BFO, a trainable derivative-free brute force optimizer for nonlinear bound-constrained optimization and equilibrium computations with continuous and discrete variables. ACM Trans. Math. Softw. 44(1), 1–25 (2017)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Giampaolo Liuzzi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liuzzi, G., Lucidi, S. & Rinaldi, F. An algorithmic framework based on primitive directions and nonmonotone line searches for black-box optimization problems with integer variables. Math. Prog. Comp. 12, 673–702 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Derivative free optimization
  • Black box problems
  • Integer variables
  • Nonmonotone line search

Mathematics Subject Classification

  • 90C56
  • 90C10
  • 90C30