Optimization Letters

, Volume 7, Issue 3, pp 447–465 | Cite as

On the convergence of an inexact Gauss–Newton trust-region method for nonlinear least-squares problems with simple bounds

  • Margherita Porcelli
Original Paper


We introduce an inexact Gauss–Newton trust-region method for solving bound-constrained nonlinear least-squares problems where, at each iteration, a trust-region subproblem is approximately solved by the Conjugate Gradient method. Provided a suitable control on the accuracy to which we attempt to solve the subproblems, we prove that the method has global and asymptotic fast convergence properties. Some numerical illustration is also presented.


Bound-constrained nonlinear least-squares Simple bounds Trust-region methods Convergence theory Affine scaling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alsaifi N.M., Englezos P.: Prediction of multiphase equilibrium using the PC-SAFT equation of state and simultaneous testing of phase stability. Fluid Phase Equilib. 302, 169–178 (2011)CrossRefGoogle Scholar
  2. 2.
    Andreani R., Friedlander A., Mello M.P., Santos S.A.: Box-constrained minimization reformulations of complementarity problems in second-order cones. J. Glob. Optim. 40(4), 505–527 (2008)MathSciNetMATHCrossRefGoogle Scholar
  3. 3.
    Bellavia S., Macconi M., Morini B.: An affine scaling trust-region approach to bound-constrained nonlinear systems. Appl. Numer. Math. 44, 257–280 (2003)MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Bellavia S., Morini B.: An interior global method for nonlinear systems with simple bounds. Optim. Methods Softw. 20, 1–22 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Bellavia S., Morini B.: Subspace trust-region methods for large bound-constrained nonlinear equations. SIAM J. Numer. Anal. 44, 1535–1555 (2006)MathSciNetMATHCrossRefGoogle Scholar
  6. 6.
    Bencini, L., Fantacci, R., Maccari, L.: Analytical model for performance analysis of IEEE 802.11 DCF mechanism in multi-radio wireless networks. In: Proceedings of ICC 2010, pp. 1–5, Cape Town, South Africa (2010)Google Scholar
  7. 7.
    Branch M.A., Coleman T.F., Li Y.: A subspace, interior and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J. Sci. Comput. 21, 1–23 (1999)MathSciNetMATHCrossRefGoogle Scholar
  8. 8.
    Cartis C., Gould N.I.M., Toint Ph.L.: Trust-region and other regularisations of linear least-squares problems. BIT 49, 21–53 (2009)MathSciNetMATHCrossRefGoogle Scholar
  9. 9.
    Coleman T.F., Li Y.: An interior trust-region approach for nonlinear minimization subject to bounds. SIAM J. Optim. 6, 418–445 (1996)MathSciNetMATHCrossRefGoogle Scholar
  10. 10.
    Dembo R.S., Eisenstat S.C., Steihaug T.: Inexact Newton methods. SIAM J. Numer. Anal. 19, 400–408 (1982)MathSciNetMATHCrossRefGoogle Scholar
  11. 11.
    Dolan E.D., Moré J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–221 (2002)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Francisco J.B., Krejić N., Martínez J.M.: An interior-point method for solving box-constrained underdetermined nonlinear systems. J. Comput. Appl. Math. 177, 67–88 (2005)MathSciNetMATHCrossRefGoogle Scholar
  13. 13.
    Gould N.I.M., Orban D., Toint Ph.L.: CUTEr, a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)MathSciNetMATHCrossRefGoogle Scholar
  14. 14.
    Gould N.I.M., Orban D., Toint Ph.L.: GALAHAD—a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization. ACM Trans. Math. Softw. 29, 353–372 (2003)MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Hestenes M.R., Stiefel E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Hestenes M.R.: Pseudoinverses and conjugate gradients. Commun. ACM 18, 40–43 (1975)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Horn R.A., Johnson C.R.: Matrix Analysis. The Cambridge University Press, Cambridge (1985)MATHGoogle Scholar
  18. 18.
    Kaiser, M., Klamroth, K., Thekale, A., Toint, Ph.L.: Solving Structured Nonlinear Least-Squares and Nonlinear Feasibility Problems with Expensive Functions, Report NAXYS-07-2010. Department of Mathematics, FUNDP, Namur (B) (2010)Google Scholar
  19. 19.
    Kanzow C., Klug A.: An interior-point affine-scaling trust-region method for semismooth equations with box constraints. Comput. Optim. Appl. 37, 329–353 (2007)MathSciNetMATHCrossRefGoogle Scholar
  20. 20.
    Kanzow C., Petra S.: Projected filter trust region methods for a semismooth least-squares formulation of mixed complementarity problems. Optim. Methods Softw. 22, 713–735 (2007)MathSciNetMATHCrossRefGoogle Scholar
  21. 21.
    Macconi M., Morini B., Porcelli M.: Trust-region quadratic methods for nonlinear systems of mixed equalities and inequalities. Appl. Numer. Math. 59, 859–876 (2009)MathSciNetMATHCrossRefGoogle Scholar
  22. 22.
    Macconi M., Morini B., Porcelli M.: A Gauss–Newton method for solving bound-constrained underdetermined nonlinear systems. Optim. Methods Softw. 24, 219–235 (2009)MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Moré J.J., Sorensen D.C.: Computing a trust-region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)MATHCrossRefGoogle Scholar
  24. 24.
    Morini, B., Porcelli, M.: TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities. Comput. Optim. Appl. (2010). doi: 10.1007/s10589-010-9327-5
  25. 25.
    Pardalos P.M., Resende M.G.C.: Handbook of Applied Optimization. Oxford University Press, Oxford (2002)MATHGoogle Scholar
  26. 26.
    Steihaug T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)MathSciNetMATHCrossRefGoogle Scholar
  27. 27.
    Toint Ph.L.: Towards an efficient sparsity exploiting Newton method for minimization. In: Duff, I.S. (ed.) Sparse Matrices and Their Uses, pp. 57–88. Academic Press, London (1981)Google Scholar
  28. 28.
    Zhu D.: Affine scaling interior LevenbergMarquardt method for bound-constrained semismooth equations under local error bound conditions. J. Comput. Appl. Math. 219, 198–215 (2008)MathSciNetMATHCrossRefGoogle Scholar
  29. 29.
    Yuan Y.: On the truncated conjugate-gradient method. Math. Program. Ser. A 87(3), 561–573 (2000)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  1. 1.Namur Center for Complex Systems (NAXYS)FUNDP-University of NamurNamurBelgium

Personalised recommendations