Skip to main content
SpringerLink
Log in

Convergence Properties of a Self-adaptive Levenberg-Marquardt Algorithm Under Local Error Bound Condition

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

We propose a new self-adaptive Levenberg-Marquardt algorithm for the system of nonlinear equations F(x) = 0. The Levenberg-Marquardt parameter is chosen as the product of ‖F k ‖δ with δ being a positive constant, and some function of the ratio between the actual reduction and predicted reduction of the merit function. Under the local error bound condition which is weaker than the nonsingularity, we show that the Levenberg-Marquardt method converges superlinearly to the solution for δ∈ (0, 1), while quadratically for δ∈ [1, 2]. Numerical results show that the new algorithm performs very well for the nonlinear equations with high rank deficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. H. Dan, N. Yamashita, and M. Fukushima, “Convergence properties of the inexact Levenberg-Marquardt method under local error bound,” Optimization Methods and Software, vol. 17, pp. 605–626, 2002.,

  2. J.Y. Fan and Y.X. Yuan, “On the quadratic convergence of the Levenberg-Marquardt method without nonsingularity assumption,” Computing, vol. 74, no. 1, pp. 23–39, 2005.,

  3. J.Y. Fan, “A modified Levenberg-Marquardt algorithm for singular system of nonlinear equations,” Journal of Computational Mathematics, vol. 21, no. 5, pp. 625–636, 2003.,

  4. L. Hei, “A self-adaptive trust region algorithm,” Report, AMSS, Chinese Academy of Sciences, 2000.,

  5. K. Levenberg, “A method for the solution of certain nonlinear problems in least squares,” Quart. Appl. Math., vol. 2, pp. 164–166, 1944.,

  6. D.W. Marquardt, “An algorithm for least-squares estimation of nonlinear inequalities,” SIAM J. Appl. Math., vol. 11, pp. 431–441, 1963.,

  7. J.J. Moré, “The Levenberg-Marquardt algorithm: implementation and theory,” in Lecture Notes in Mathematics 630: Numerical Analysis, G.A. Watson (Ed.), Springer-Verlag: Berlin, 1978, pp. 105–116.,

  8. J.J. Moré, “Recent developments in algorithms and software for trust region methods,” in Mathematical Programming: The State of Art, A. Bachem, M. Grötschel and B. Korte (Eds.), Springer: Berlin, 1983, pp. 258–287.,

  9. J.J. Moré, B.S. Garbow, and K.H. Hillstrom, “Testing unconstrained optimization software,” ACM Trans. Math. Software, vol. 7, pp. 17–41, 1981.,

  10. J. Nocedal and Y.X. Yuan, “Combining trust region and line search techniques,” in Advances in Nonlinear Programming, Y. Yuan (Ed.), Kluwer, pp. 153–175, 1998.,

  11. M.J.D. Powell, “Convergence properties of a class of minimization algorithms,” in Nonlinear Programming 2, O.L. Mangasarian, R.R. Meyer, and S.M. Robinson (Eds.), Academic Press: New York, 1975, pp. 1–27.,

  12. R.B. Schnabel and P.D. Frank, “Tensor methods for nonlinear equations,” SIAM J. Numer. Anal., vol. 21, pp. 815–843, 1984.,

  13. G.W. Stewart and J.G. Sun, Matrix Perturbation Theory. Academic Press: San Diego, CA, 1990.,

  14. N. Yamashita and M. Fukushima, “On the rate of convergence of the Levenberg-Marquardt method,” Computing (Suppl), vol. 15, pp. 239–249, 2001.,

  15. Y.X. Yuan, “Trust region algorithms for nonlinear programming,” in Contemporary Mathematics, Z.C. Shi (Ed.) vol. 163, American Mathematics Society, 1994, pp. 205–225.,

  16. Y.X. Yuan, “Trust region algorithms for nonlinear equations,” Information, vol. 1, pp. 7–20, 1998.,

  17. Y.X. Yuan, “A review of trust region algorithms for optimization,” in ICM99: Proceedings of the Fourth International Congress on Industrial and Applied Mathematics, J.M. Ball and J.C.R. Hunt (Eds.), Oxford University Press, 2000 pp. 271–282.,

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Shanghai Jiao Tong University, Shanghai, 200240, P.R. China

    Jinyan Fan

  2. Division of Computational Science, E-Institute of Shanghai Universities, SJTU, Shanghai, P.R. China

    Jinyan Fan

  3. Department of Mathematics, East China Normal University, Shanghai, 200062, P.R. China

    Jianyu Pan

Authors
  1. Jinyan Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianyu Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This work is supported by Chinese NSFC grants 10401023 and 10501013, Research Grants for Young Teachers of Shanghai Jiao Tong University, and E-Institute of Shanghai Municipal Education Commission, N. E03004.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fan, J., Pan, J. Convergence Properties of a Self-adaptive Levenberg-Marquardt Algorithm Under Local Error Bound Condition. Comput Optim Applic 34, 47–62 (2006). https://doi.org/10.1007/s10589-005-3074-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-005-3074-z

Keywords

Search

Navigation