Skip to main content
SpringerLink
Log in

A derivative-free trust-funnel method for equality-constrained nonlinear optimization

Computational Optimization and Applications volume 61, pages 25–49 (2015)Cite this article

Abstract

A new derivative-free method is proposed for solving equality-constrained nonlinear optimization problems. This method is of the trust-funnel variety and is also based on the use of polynomial interpolation models. In addition, it uses a self-correcting geometry procedure in order to ensure that the interpolation problem is well defined in the sense that the geometry of the set of interpolation points does not differ too much from the ideal one. The algorithm is described in detail and some encouraging numerical results are presented.

Graphical Abstract

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3

References

  1. Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization. Math. Program. 134(1), 223–257 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  2. Colson, B.: Trust-Region Algorithms for Derivative-Free Optimization and Nonlinear Bilevel Programming. PhD Thesis, Department of Mathematics, FUNDP-University of Namur, Namur, Belgium, (2004)

  3. Conn, A.R., Gould, N.I.M., Toint, PhL: Trust-Region Methods. MOS-SIAM Series on Optimization. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  4. Conn, A.R., Scheinberg, K., Toint, PhL: On the convergence of derivative-free methods for unconstrained optimization. In: Iserles, A., Buhmann, M. (eds.) Approximation Theory and Optimization: Tributes to M.J.D. Powell, pp. 83–108. Cambridge University Press, Cambridge (1997)

  5. Conn, A.R., Scheinberg, K., Toint, Ph.L.: A derivative-free optimization algorithm in practice. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, (1998)

  6. Conn, A.R., Scheinberg, K., Vincente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Book Series on Optimization. SIAM, Philadelphia (2009)

    Book  MATH  Google Scholar 

  7. Conn, A.R., Scheinberg, K., Zang, H.: A derivative-free algorithm for least-squares minimization. SIAM J. Opt. 20(6), 3555–3576 (2010)

    Article  MATH  Google Scholar 

  8. Dennis, J.E., Schnabel, R.B., Methods, Numerical, for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, NJ (1983). Reprinted as Classics in Applied Mathematics, vol. 16. SIAM, Philadelphia (1996)

  9. Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. 91(2), 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fasano, G., Nocedal, J., Morales, J.-L.: On the geometry phase in model-based algorithms for derivative-free optimization. Opt. Methods Softw. 24(1), 145–154 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  11. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91(2), 239–269 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  12. Gill, P.E., Murray, W., Wright, M.H.: Pract. Opt. Academic Press, London (1981)

    Google Scholar 

  13. Gould, N.I.M., Orban, D., Toint, PhL: CUTEr, a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  14. Gould, N.I.M., Toint, PhL: Nonlinear programming without a penalty function or a filter. Math. Program. 122(1), 155–196 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  15. Gould, N.I.M., Hribar, M.E., Nocedal, J.: On the solution of equality constrained quadratic programming problems arising in optimization. SIAM J. Sci. Comput. 23(4), 1376–1395 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  16. Gratton, S., Toint, PhL, Tröltzsch, A.: An active-set trust-region method for bound-constrained nonlinear optimization without derivatives. Opt. Methods Softw. 26(4–5), 875–896 (2011)

    Google Scholar 

  17. Lewis, R.M., Torczon, V.: Pattern search algorithms for bound constrained minimization. SIAM J. Opt. 9, 1082–1099 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  18. Lewis, R.M., Torczon, V.: Pattern search algorithms for linearly constrained minimization. SIAM J. Opt. 10, 917–941 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  19. Lewis, R.M., Torczon, V.: A globally convergent augmented langragian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Opt. 12(4), 1075–1089 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  20. Lewis, R.M., Torczon, V.: Active set identification for linearly constrained minimization without explicit derivatives. SIAM J. Opt. 20(3), 1378–1405 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  21. Lewis, R.M., Torczon, V., Trosset, M.W.: Direct search methos: then and now. J. Comput. Appl. Math. 124(1–2), 191–207 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  22. Moré, J.J.: The Levenberg–Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.) Numerical Analysis. Lecture Notes in Mathematics, chapter, vol. 10. Springer, Berlin (1978)

  23. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965)

    Article  MATH  Google Scholar 

  24. Nocedal, J., Wright, S.J.: Numerical Optimization. Series in Operations Research. Springer, Heidelberg (1999)

    Book  MATH  Google Scholar 

  25. Omojokun, E.O.: Trust region algorithms for optimization with nonlinear equality and inequality constraints. PhD Thesis, University of Colorado, Boulder, Colorado (1989)

  26. Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Advances in Optimization and Numerical Analysis, Proceedings of the Sixth Workshop on Optimization and Numerical Analysis, Oaxaca, Mexico, vol. 275, pp. 51–67. Kluwer Academic Publishers, Dordrecht, The Netherlands (1994)

  27. Powell, M.J.D.: Direct search algorithms for optimization calculations. Acta Numerica 7, 287–336 (1998)

    Article  Google Scholar 

  28. Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic interpolation. Math. Program. Ser. A 92, 555–582 (2002)

    Article  MATH  Google Scholar 

  29. Powell, M.J.D.: Least Frobenius norm updating of quadratic models that satisfy interpolation conditions. Math. Program. Ser. B 100(1), 183–215 (2004)

    Article  MATH  Google Scholar 

  30. Powell, M.J.D.: The newuoa software for unconstrained optimization without derivatives. In: Pillo, G., Roma, M., Pardalos, P. (eds.) Large Scale Nonlinear Optimization, Nonconvex Optimization and Its Applications, vol. 83, pp. 255–297. Springer, New York (2006)

  31. Powell, M.J.D.: Developments of NEWUOA for minimization without derivatives. IMA J. Numer. Anal. 28(4), 649–664 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  32. Powell, M.J.D.: The BOBYQA algorithm for bound constrained optimization without derivatives. Department of Applied Mathematics and Theoretical Physics, Cambridge University, Cambridge, England, Technical report (2009)

  33. Scheinberg, K., Toint, PhL: Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization. SIAM J. Opt. 20(6), 3512–3532 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  34. Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20(3), 626–637 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  35. Stewart, G.W.: A modification of davidon’s minimization method to accept difference approximations of derivatives. J. ACM 14(1), 72–83 (1967)

    Article  MATH  Google Scholar 

  36. Toint, PhL: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses. pp. 57–88. Academic Press, London (1981)

  37. Tröltzsch, A.: An active-set trust-region method for bound-constrained nonlinear optimization without derivatives applied to noisy aerodynamic design problems. PhD Thesis, University of Toulouse (2011)

  38. Winfield, D.: Function and functional optimization by interpolation in data tables. PhD Thesis, Harvard University, Cambridge (1969)

  39. Winfield, D.: Function minimization by interpolation in a data table. J. Inst. Math. Appl. 12, 339–347 (1973)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The first author gratefully acknowledges a CERUNA-UNamur scholarship.

Author information

Authors and Affiliations

  1. Namur Center for Complex Systems (naXys) and Department of Mathematics, University of Namur, 61, rue de Bruxelles, 5000, Namur, Belgium

    Ph. R. Sampaio & Ph. L. Toint

Authors
  1. Ph. R. Sampaio
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ph. L. Toint
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ph. R. Sampaio.

Appendix

Appendix

See the Table 1, 2

Table 1 Number of function evaluations required by DEFT-FUNNEL, CDFO and COBYLA methods to converge in the first type of comparison
Full size table
Table 2 Number of function evaluations required by DEFT-FUNNEL, CDFO and COBYLA methods to converge in the second type of comparison
Full size table

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sampaio, P.R., Toint, P.L. A derivative-free trust-funnel method for equality-constrained nonlinear optimization. Comput Optim Appl 61, 25–49 (2015). https://doi.org/10.1007/s10589-014-9715-3

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-014-9715-3

Keywords

search

Navigation