Abstract
Recently, Yamashita and Fukushima [11] established an interesting quadratic convergence result for the Levenberg-Marquardt method without the nonsingularity assumption. This paper extends the result of Yamashita and Fukushima by using μ k =||F(x k )||δ, where δ∈[1,2], instead of μ k =||F(x k )||2 as the Levenberg-Marquardt parameter. If ||F(x)|| provides a local error bound for the system of nonlinear equations F(x)=0, it is shown that the sequence {x k } generated by the new method converges to a solution quadratically, which is stronger than dist(x k ,X*)→0 given by Yamashita and Fukushima. Numerical results show that the method performs well for singular problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Kelley, C. T: Iterative methods for optimization (Frontiers in Applied Mathematics 18). Philadelphia: SIAM 1999.
Levenberg, K.: A method for the solution of certain nonlinear problems in least squares. Quart. Appl. Math. 2, 164–166 (1944).
Marquardt, D. W.: An algorithm for least-squares estimation of nonlinear inequalities. SIAM J. Appl. Math. 11, 431–441 (1963).
Moré, J. J.: The Levenberg-Marquardt algorithm: implementation and theory. In: Lecture Notes in Mathematics 630: Numerical analysis (Watson, G. A., ed.), pp. 105–116. Berlin: Springer 1978.
Moré, J. J.: Recent developments in algorithms and software for trust region methods. In: Mathematical programming: The state of art. (Bachem, A., Grötschel, M., Korte, B., eds.), pp. 258–287. Berlin: Springer 1983.
Moré, J. J., Garbow, B. S., Hillstrom, K. H.: Testing unconstrained optimization software. ACM Trans. Math. Software 7, 17–41 (1981).
Nocedal, J., Yuan, Y.X.: Combining trust region and line search techniques. In: Advances in nonlinear programming (Yuan, Y., ed.), pp. 153–175. Kluwer 1998.
Powell, M. J. D.: An iterative method for finding stationary values of a function of several variables. Comput. J. 5, 147–151 (1962).
Schnabel, R. B., Frank, P.D.: Tensor methods for nonlinear equations. SIAM J. Numer. Anal. 21, 815–843 (1984).
Stewart, G. W., Sun, J. G.: Matrix perturbation theory. San Diego: Academic Press 1990.
Yamashita, N., Fukushima, M.: On the rate of convergence of the Levenberg-Marquardt method. Computing (Suppl. 15): 237–249 (2001).
Yuan, Y. X.: Trust region algorithms for nonlinear equations. Information 1, 7–20 (1998).
Yuan, Y. X.: Problems on convergence of unconstrained optimization algorithms. In: Numerical linear algebra and optimization (Yuan, Y. X., ed.), pp. 95–107. Beijing: Science Press 1999.
Yuan, Y. X.: A review of trust region algorithms for optimization. In: ICM99: Proc. 4th Int. Congress on Industrial and Applied Mathematics (Ball, J. M., Hunt, J. C. R., eds.), pp. 271–282. Oxford: Oxford University Press 2000.
Acknowledgments.
Supported by Chinese NSF grants 19731010, 10231060 and the Knowledge Innovation Program of CAS. It was pointed out by Prof. C.T. Keller (private communication) that the choice μ k =||F(x k )|| was suggested in his book [1]. We would like to thank two anonymous referees for their valuable comments.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fan, Jy., Yuan, Yx. On the Quadratic Convergence of the Levenberg-Marquardt Method without Nonsingularity Assumption. Computing 74, 23–39 (2005). https://doi.org/10.1007/s00607-004-0083-1
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00607-004-0083-1