Abstract
In this work, a new stabilization scheme for the Gauss-Newton method is defined, where the minimum norm solution of the linear least-squares problem is normally taken as search direction and the standard Gauss-Newton equation is suitably modified only at a subsequence of the iterates. Moreover, the stepsize is computed by means of a nonmonotone line search technique. The global convergence of the proposed algorithm model is proved under standard assumptions and the superlinear rate of convergence is ensured for the zero-residual case. A specific implementation algorithm is described, where the use of the pure Gauss-Newton iteration is conditioned to the progress made in the minimization process by controlling the stepsize. The results of a computational experimentation performed on a set of standard test problems are reported.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ben-Israel, A., A Newton-Raphson Method for the Solution of Systems of Equations, Journal of Mathematical Analysis and Applications, Vol. 15, pp. 243-252, 1966.
Boggs, P. T., The Convergence of the Ben-Israel Iteration for Nonlinear Least Squares Problems, Mathematics of Computation, Vol. 30, pp. 512-522, 1976.
Menzel, R., On Solving Nonlinear Least-Squares Problems in Case of Rank-Deficient Jacobians, Computing, Vol. 34, pp. 63-72, 1985.
Grippo, L., Lampariello, F., and Lucidi, S., A Nonmonotone Line Search Technique for Newton's Method, SIAM Journal on Numerical Analysis, Vol. 23, pp. 707-716, 1986.
Grippo, L., Lampariello, F., and Lucidi, S., A Class of Nonmonotone Stabilization Methods in Unconstrained Optimization, Numerische Mathematik, Vol. 59, pp. 779-805, 1991.
Ferris, M. C., Lucidi, S., and Roma, M., Nonmonotone Curvilinear Line Search Methods for Unconstrained Optimization, Computational Optimization and Applications, Vol. 6, pp. 117-136, 1996.
Ferris, M. C., and Lucidi, S., Nonmonotone Stabilization Methods for Nonlinear Equations, Journal of Optimization Theory and Applications, Vol. 81, pp. 815-832, 1994.
Zhang, J. Z., and Chen, L. H., Nonmonotone Levenberg-Marquardt Algorithms and Their Convergence Analysis, Journal of Optimization Theory and Applications, Vol. 92, pp. 393-418, 1997.
Golub, G. H., and Van loan, C. F., Matrix Computations, 2nd Edition, The John Hopkins University Press, Baltimore, Maryland, 1989.
Hestenes, M. R., Conjugate Direction Methods in Optimization, Springer Verlag, Berlin, Germany, 1980.
Barzilai, J., and Borwein, J. M., Two-Point Stepsize Gradient Methods, IMA Journal of Numerical Analysis, Vol. 8, pp. 141-148, 1988.
Fletcher, R., On the Barzilai-Borwein Method, Dundee Numerical Analysis Report NA/207, University of Dundee, 2001.
BjÖ rck, A., Numerical Methods for Least Squares Problems, SIAM, Philadelphia, Pennsylvania, 1996.
Lampariello, F., and Sciandrone, M., Global Convergence Technique for the Newton Method with Periodic Hessian Evaluation, Journal of Optimization Theory and Applications, Vol. 111, pp. 341-358, 2001.
De Leone, R., Gaudioso, M., and Grippo, L., Stopping Criteria for Line-Search Methods without Derivatives, Mathematical Programming, Vol. 30, pp. 285-300, 1984.
Bertsekas, D. P., Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, NY, 1980.
MorÉ, J. J., Garbow, B. S., and Hillstrom, K.E., Testing Unconstrained Optimization Software, ACM Transactions on Mathematical Software, Vol. 7, pp. 17-41, 1981.
Schittkowski, K., More Test Examples for Nonlinear Programming Codes, Springer Verlag, Berlin, Germany, 1987.
Wolfe, M. A., Numerical Methods for Unconstrained Optimization, Van Nostrand Reinhold, New York, NY, 1978.
Gill, P. E., and Murray, W., Algorithms for the Solution of the Nonlinear Least-Squares Problem, SIAM Journal on Numerical Analysis, Vol. 15, pp. 977-992, 1978.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Lampariello, F., Sciandrone, M. Use of the Minimum-Norm Search Direction in a Nonmonotone Version of the Gauss-Newton Method. Journal of Optimization Theory and Applications 119, 65–82 (2003). https://doi.org/10.1023/B:JOTA.0000005041.99777.af
Issue Date:
DOI: https://doi.org/10.1023/B:JOTA.0000005041.99777.af