Computational Optimization and Applications

, Volume 71, Issue 2, pp 307–329 | Cite as

A progressive barrier derivative-free trust-region algorithm for constrained optimization

  • Charles Audet
  • Andrew R. Conn
  • Sébastien Le Digabel
  • Mathilde PeyregaEmail author


We study derivative-free constrained optimization problems and propose a trust-region method that builds linear or quadratic models around the best feasible and around the best infeasible solutions found so far. These models are optimized within a trust region, and the progressive barrier methodology handles the constraints by progressively pushing the infeasible solutions toward the feasible domain. Computational experiments on 40 smooth constrained problems indicate that the proposed method is competitive with COBYLA, and experiments on two nonsmooth multidisciplinary optimization problems from mechanical engineering show that it can be competitive with the NOMAD software.


Derivative-free optimization Trust-region algorithms Progressive barrier 

Mathematics Subject Classification

90C30 90C56 



The authors would like to thank two anonymous referees for their careful reading and helpful comments and suggestions.


  1. 1.
    Abramson, M.A., Audet, C., Dennis Jr., J.E.: Filter pattern search algorithms for mixed variable constrained optimization problems. Pac. J. Optim. 3(3), 477–500 (2007)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Arouxét, M.B., Echebest, N.E., Pilotta, E.A.: Inexact restoration method for nonlinear optimization without derivatives. J. Comput. Appl. Math. 290, 26–43 (2015)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Audet, C.: A survey on direct search methods for blackbox optimization and their applications, chapter 2. In: Pardalos, P.M., Rassias, T.M. (eds.) Mathematics Without Boundaries: Surveys in Interdisciplinary Research, pp. 31–56. Springer, Berlin (2014)Google Scholar
  4. 4.
    Audet, C., Béchard, V., Le Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299–318 (2008)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Audet, C., Dennis Jr., J.E.: Analysis of generalized pattern searches. SIAM J. Optim. 13(3), 889–903 (2003)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Audet, C., Dennis Jr., J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Audet, C., Dennis Jr., J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445–472 (2009)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Audet, C., Dennis Jr., J.E., Le Digabel, S.: Globalization strategies for mesh adaptive direct search. Comput. Optim. Appl. 46(2), 193–215 (2010)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer International Publishing, Berlin (2017)CrossRefGoogle Scholar
  11. 11.
    Audet, C., Ianni, A., Le Digabel, S., Tribes, C.: Reducing the number of function evaluations in mesh adaptive direct search algorithms. SIAM J. Optim. 24(2), 621–642 (2014)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Audet, C., Le Digabel, S., Peyrega, M.: Linear equalities in blackbox optimization. Comput. Optim. Appl. 61(1), 1–23 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Augustin, F., Marzouk, Y.M.: NOWPAC: a provably convergent derivative-free nonlinear optimizer with path-augmented constraints. Technical report, arXiv (2014)Google Scholar
  14. 14.
    Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Convergence of trust-region methods based on probabilistic models. SIAM J. Optim. 24(3), 1238–1264 (2014)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Bertsekas, D.P.: Constrained Optimization and Lagrangian Multiplier Methods. Academic, New York (1982)zbMATHGoogle Scholar
  16. 16.
    Conejo, P.D., Karas, E.W., Pedroso, L.G.: A trust-region derivative-free algorithm for constrained optimization. Optim. Methods Softw. 30(6), 1126–1145 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Conn, A.R., Gould, N.I.M., Toint, PhL: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28(2), 545–572 (1991)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Conn, A.R., Gould, N.I.M., Toint, Ph.L.: Trust-Region Methods. SIAM, MPS-SIAM Series on Optimization (2000)Google Scholar
  19. 19.
    Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139–158 (2013)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Global convergence of general derivative-free trust-region algorithms to first and second order critical points. SIAM J. Optim. 20(1), 387–415 (2009)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MOS-SIAM Series on Optimization. SIAM, Philadelphia (2009)CrossRefGoogle Scholar
  22. 22.
    Custódio, A.L., Scheinberg, K., Vicente, L.N.: Methodologies and software for derivative-free optimization, chapter 37. In: Terlaky, T., Anjos, M.F., Ahmed, S. (eds.) Advances and Trends in Optimization with Engineering Applications. MOS-SIAM Book Series on Optimization. SIAM, Philadelphia (2017)Google Scholar
  23. 23.
    Dennis Jr., J.E., Price, C.J., Coope, I.D.: Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5(2), 123–144 (2004)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Echebest, N., Schuverdt, M.L., Vignau, R.P.: An inexact restoration derivative-free filter method for nonlinear programming. Comput. Appl. Math. 36(1), 693–718 (2017)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. Ser. A 91, 239–269 (2002)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Gould, N.I.M., Orban, D., Toint, Ph.L.: CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization. Comput. Optim. Appl. 60(3):545–557 (2015). Accessed 2015MathSciNetCrossRefGoogle Scholar
  27. 27.
    Gould, N.I.M., Toint, PhL: Nonlinear programming without a penalty function or a filter. Math. Program. 122(1), 155–196 (2010)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Gumma, E.A.E., Hashim, M.H.A., Ali, M.M.: A derivative-free algorithm for linearly constrained optimization problems. Comput. Optim. Appl. 57(3), 599–621 (2014)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Kolda, T.G., Lewis, R.M., Torczon, V.: A generating set direct search augmented Lagrangian algorithm for optimization with a combination of general and linear constraints. Technical Report SAND2006-5315, Sandia National Laboratories, USA (2006)Google Scholar
  30. 30.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Stationarity results for generating set search for linearly constrained optimization. SIAM J. Optim. 17(4), 943–968 (2006)MathSciNetCrossRefGoogle Scholar
  31. 31.
    Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. 37(4), 44:1–44:15 (2011)MathSciNetCrossRefGoogle Scholar
  32. 32.
    Le Digabel, S., Wild, S.M.: A Taxonomy of Constraints in Simulation-Based Optimization. Technical Report G-2015-57, Les cahiers du GERAD (2015)Google Scholar
  33. 33.
    Lewis, R.M., Shepherd, A., Torczon, V.: Implementing generating set search methods for linearly constrained minimization. SIAM J. Sci. Comput. 29(6), 2507–2530 (2007)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Lewis, R.M., Torczon, V.: Active set identification for linearly constrained minimization without explicit derivatives. SIAM J. Optim. 20(3), 1378–1405 (2009)MathSciNetCrossRefGoogle Scholar
  35. 35.
    Liuzzi, G., Lucidi, S.: A derivative-free algorithm for inequality constrained nonlinear programming via smoothing of an \(\ell_\infty \) penalty function. SIAM J. Optim. 20(1), 1–29 (2009)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Liuzzi, G., Lucidi, S., Sciandrone, M.: Sequential penalty derivative-free methods for nonlinear constrained optimization. SIAM J. Optim. 20(5), 2614–2635 (2010)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Moré, J.J., Wild, S.M.: Benchmarking derivative-free optimization algorithms. SIAM J. Optim. 20(1), 172–191 (2009)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Perez, R., Liu, H.H.T., Behdinan, K.: Evaluation of multidisciplinary optimization approaches for aircraft conceptual design. In: AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Albany, NY, September (2004)Google Scholar
  39. 39.
    Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Gomez, S., Hennart, J.-P. (eds.) Advances in Optimization and Numerical Analysis. Mathematics and Its Applications, vol. 275, pp. 51–67. Springer, Dordrecht (1994)CrossRefGoogle Scholar
  40. 40.
    Powell, M.J.D.: On fast trust region methods for quadratic models with linear constraints. Math. Program. Comput. 7(3), 237–267 (2015)MathSciNetCrossRefGoogle Scholar
  41. 41.
    Sampaio, PhR, Toint, PhL: A derivative-free trust-funnel method for equality-constrained nonlinear optimization. Comput. Optim. Appl. 61(1), 25–49 (2015)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Sampaio, PhR, Toint, PhL: Numerical experience with a derivative-free trust-funnel method for nonlinear optimization problems with general nonlinear constraints. Optim. Methods Softw. 31(3), 511–534 (2016)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Sobieszczanski-Sobieski, J., Agte, J.S., Sandusky Jr., R.R.: Bilevel integrated system synthesis. AIAA J. 38(1), 164–172 (2000)CrossRefGoogle Scholar
  44. 44.
    Tribes, C., Dubé, J.-F., Trépanier, J.-Y.: Decomposition of multidisciplinary optimization problems: formulations and application to a simplified wing design. Eng. Optim. 37(8), 775–796 (2005)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Tröltzsch, A.: A sequential quadratic programming algorithm for equality-constrained optimization without derivatives. Optim. Lett. 10(2), 383–399 (2016)MathSciNetCrossRefGoogle Scholar
  46. 46.
    Xue, D., Sun, W.: On convergence analysis of a derivative-free trust region algorithm for constrained optimization with separable structure. Sci. China Math. 57(6), 1287–1302 (2014)MathSciNetCrossRefGoogle Scholar
  47. 47.
    Yuan, Y.: Recent advances in trust region algorithms. Math. Program. 151(1), 249–281 (2015)MathSciNetCrossRefGoogle Scholar
  48. 48.
    Yuan, Y.-X.: An example of non-convergence of trust region algorithms. In: Yuan, Y.-X. (ed.) Advances in Nonlinear Programming, pp. 205–215. Kluwer Academic, Dordercht (1998)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.GERAD and Département de mathématiques et génie industrielÉcole Polytechnique de MontréalMontréalCanada
  2. 2.IBM Business Analytics and Mathematical SciencesIBM T J Watson Research CenterYorktown HeightsUSA

Personalised recommendations