Skip to main content
SpringerLink
Log in

Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

This paper presents a class of Levenberg–Marquardt methods for solving the nonlinear least-squares problem. Explicit algebraic rules for computing the regularization parameter are devised. In addition, convergence properties of this class of methods are analyzed. We prove that all accumulation points of the generated sequence are stationary. Moreover, q-quadratic convergence for the zero-residual problem is obtained under an error bound condition. Illustrative numerical experiments with encouraging results are presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Birgin, E.G., Gentil, J.M.: Evaluating bound-constrained minimization software. Comput. Optim. Appl. 53(2), 347–373 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  2. Dan, H., Yamashita, N., Fukushima, M.: Convergence properties of the inexact Levenberg-Marquardt method under local error bound conditions. Optim. Methods Softw. 17(4), 605–626 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  3. Dennis Jr., J.E., Schnabel, R.B.: Numerical methods for unconstrained optimization and nonlinear equations. In: Classics in Applied Mathematics, vol 16. Society for Industrial and Applied Mathematics (SIAM). Philadelphia, PA (1996). Corrected reprint of the 1983 original

  4. Fan, J.: Accelerating the modified Levenberg-Marquardt method for nonlinear equations. Math. Compd. 83(287), 1173–1187 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  5. Fan, J., Pan, J.: Convergence properties of a self-adaptive Levenberg-Marquardt algorithm under local error bound condition. Comput. Optim. Appl. 34(1), 47–62 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Fan, J., Yuan, Y.-X.: On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption. Computing 74(1), 23–39 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  7. Fischer, A.: Local behavior of an iterative framework for generalized equations with nonisolated solutions. Math. Progr. 94((1, Ser. A)), 91–124 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  8. Fischer, A., Shukla, P.K., Wang, M.: On the inexactness level of robust Levenberg-Marquardt methods. Optimization 59(2), 273–287 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  9. Gould, N., Orban, D., Toint, P.: Cutest: a constrained and unconstrained testing environment with safe threads. Technical Report RAL-TR-2013-005. Rutherford Appleton Laboratory, Oxfordshire (2013)

  10. Gratton, S., Lawless, A.S., Nichols, N.K.: Approximate Gauss-Newton methods for nonlinear least squares problems. SIAM J. Optim. 18(1), 106–132 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  11. Karas, E.W., Santos, S.A., Svaiter, B.F.: Algebraic rules for quadratic regularization of Newton’s method. Comput. Optim. Appl. 60, 343–376 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  12. Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math. 2, 164–168 (1944)

    MathSciNet  MATH  Google Scholar 

  13. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  14. Moré, J.J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Watson, G.A. (ed.), Lecture Notes in Mathematics 630: Numerical Analysis, pp. 105–116. Springer, New York (1978)

  15. Morrison, D.D.: Methods for nonlinear least squares problems and convergence proofs. In: Lorell, J., Yagi, F. (eds.), Proceedings of the Seminar on Tracking Programs and Orbit Determination, pp. 1–9. Jet Propulsion Laboratory, Pasadena (1960)

  16. Ueda, K., Yamashita, N.: On a global complexity bound on the Levenberg-Marquardt method. J. Optimiz. Theory App. 147, 443–453 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  17. Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marquardt method. In: Alefeld, G., Chen, X. (eds.) Topics in Numerical Analysis: With Special Emphasis on Nonlinear Problems, Computing Supplement, vol. 15, pp. 239–249. Springer, Vienna (2001)

  18. Yuan, Y.-X.: Recent advances in numerical methods for nonlinear equations and nonlinear least squares. Numer. Algebra Control Optim. 1(1), 15–34 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The authors are thankful to Abel S. Siqueira for installing the CUTEst and preparing our machine to run it within Matlab. We are also grateful to the anonymous referees whose suggestions led to improvements in the paper. Elizabeth W. Karas was partially supported by CNPq Grants 477611/2013-3 and 308957/2014-8. Sandra A. Santos was partially supported by CNPq Grant 304032/2010-7, FAPESP Grants 2013/05475-7 and 2013/07375-0 and PRONEX Optimization. Benar F. Svaiter was partially supported by CNPq Grants 302962/2011-5, 474944/2010-7, FAPERJ Grant E-26/102.940/2011 and PRONEX Optimization.

Author information

Authors and Affiliations

  1. Department of Mathematics, Federal University of Paraná, CP 19081, Curitiba, PR, 81531-980, Brazil

    Elizabeth W. Karas

  2. Department of Applied Mathematics, University of Campinas, Campinas, SP, Brazil

    Sandra A. Santos

  3. IMPA, Estrada Dona Castorina, 110, Rio de Janeiro, 22460-320, Brazil

    Benar F. Svaiter

Authors
  1. Elizabeth W. Karas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandra A. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benar F. Svaiter
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elizabeth W. Karas.

Appendix

Appendix

The complete computational results are presented next. The outcomes of the four solvers CH, QR, FP and LM, respectively, are displayed row by row for each problem.

Table 1 Numerical results
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karas, E.W., Santos, S.A. & Svaiter, B.F. Algebraic rules for computing the regularization parameter of the Levenberg–Marquardt method. Comput Optim Appl 65, 723–751 (2016). https://doi.org/10.1007/s10589-016-9845-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-016-9845-x

Keywords

Mathematics Subject Classification

Search

Navigation