Optimization Letters

, Volume 7, Issue 3, pp 447–465 | Cite as

On the convergence of an inexact Gauss–Newton trust-region method for nonlinear least-squares problems with simple bounds

  • Margherita Porcelli
Original Paper


We introduce an inexact Gauss–Newton trust-region method for solving bound-constrained nonlinear least-squares problems where, at each iteration, a trust-region subproblem is approximately solved by the Conjugate Gradient method. Provided a suitable control on the accuracy to which we attempt to solve the subproblems, we prove that the method has global and asymptotic fast convergence properties. Some numerical illustration is also presented.


Bound-constrained nonlinear least-squares Simple bounds Trust-region methods Convergence theory Affine scaling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alsaifi N.M., Englezos P.: Prediction of multiphase equilibrium using the PC-SAFT equation of state and simultaneous testing of phase stability. Fluid Phase Equilib. 302, 169–178 (2011)CrossRefGoogle Scholar
  2. 2.
    Andreani R., Friedlander A., Mello M.P., Santos S.A.: Box-constrained minimization reformulations of complementarity problems in second-order cones. J. Glob. Optim. 40(4), 505–527 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Bellavia S., Macconi M., Morini B.: An affine scaling trust-region approach to bound-constrained nonlinear systems. Appl. Numer. Math. 44, 257–280 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Bellavia S., Morini B.: An interior global method for nonlinear systems with simple bounds. Optim. Methods Softw. 20, 1–22 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bellavia S., Morini B.: Subspace trust-region methods for large bound-constrained nonlinear equations. SIAM J. Numer. Anal. 44, 1535–1555 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Bencini, L., Fantacci, R., Maccari, L.: Analytical model for performance analysis of IEEE 802.11 DCF mechanism in multi-radio wireless networks. In: Proceedings of ICC 2010, pp. 1–5, Cape Town, South Africa (2010)Google Scholar
  7. 7.
    Branch M.A., Coleman T.F., Li Y.: A subspace, interior and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21, 1–23 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Cartis C., Gould N.I.M., Toint Ph.L.: Trust-region and other regularisations of linear least-squares problems. BIT 49, 21–53 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Coleman T.F., Li Y.: An interior trust-region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Dembo R.S., Eisenstat S.C., Steihaug T.: Inexact Newton methods. SIAM J. Numer. Anal. 19, 400–408 (1982)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Dolan E.D., Moré J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–221 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Francisco J.B., Krejić N., Martínez J.M.: An interior-point method for solving box-constrained underdetermined nonlinear systems. J. Comput. Appl. Math. 177, 67–88 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  13. 13.
    Gould N.I.M., Orban D., Toint Ph.L.: CUTEr, a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Gould N.I.M., Orban D., Toint Ph.L.: GALAHAD—a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29, 353–372 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  15. 15.
    Hestenes M.R., Stiefel E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Hestenes M.R.: Pseudoinverses and conjugate gradients. Commun. ACM 18, 40–43 (1975)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Horn R.A., Johnson C.R.: Matrix Analysis. The Cambridge University Press, Cambridge (1985)zbMATHGoogle Scholar
  18. 18.
    Kaiser, M., Klamroth, K., Thekale, A., Toint, Ph.L.: Solving Structured Nonlinear Least-Squares and Nonlinear Feasibility Problems with Expensive Functions, Report NAXYS-07-2010. Department of Mathematics, FUNDP, Namur (B) (2010)Google Scholar
  19. 19.
    Kanzow C., Klug A.: An interior-point affine-scaling trust-region method for semismooth equations with box constraints. Comput. Optim. Appl. 37, 329–353 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Kanzow C., Petra S.: Projected filter trust region methods for a semismooth least-squares formulation of mixed complementarity problems. Optim. Methods Softw. 22, 713–735 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Macconi M., Morini B., Porcelli M.: Trust-region quadratic methods for nonlinear systems of mixed equalities and inequalities. Appl. Numer. Math. 59, 859–876 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Macconi M., Morini B., Porcelli M.: A Gauss–Newton method for solving bound-constrained underdetermined nonlinear systems. Optim. Methods Softw. 24, 219–235 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Moré J.J., Sorensen D.C.: Computing a trust-region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)zbMATHCrossRefGoogle Scholar
  24. 24.
    Morini, B., Porcelli, M.: TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities. Comput. Optim. Appl. (2010). doi: 10.1007/s10589-010-9327-5
  25. 25.
    Pardalos P.M., Resende M.G.C.: Handbook of Applied Optimization. Oxford University Press, Oxford (2002)zbMATHGoogle Scholar
  26. 26.
    Steihaug T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Toint Ph.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57–88. Academic Press, London (1981)Google Scholar
  28. 28.
    Zhu D.: Affine scaling interior LevenbergMarquardt method for bound-constrained semismooth equations under local error bound conditions. J. Comput. Appl. Math. 219, 198–215 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Yuan Y.: On the truncated conjugate-gradient method. Math. Program. Ser. A 87(3), 561–573 (2000)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Namur Center for Complex Systems (NAXYS)FUNDP-University of NamurNamurBelgium

Personalised recommendations