Abstract
A new globalization strategy for solving an unconstrained minimization problem is proposed based on the idea of combining Newton's direction and the steepest descent direction WITHIN each iteration. Global convergence is guaranteed with an arbitrary initial point. The search direction in each iteration is chosen to be as close to the Newton's direction as possible and could be the Newton's direction itself. Asymptotically the Newton step will be taken in each iteration and thus the local convergence is quadratic. Numerical experiments are also reported. Possible combination of a Quasi-Newton direction with the steepest descent direction is also considered in our numerical experiments. The differences between the proposed strategy and a few other strategies are also discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. Akaike, “On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method,” Ann. Inst. Stat. Math. Tokyo, vol. 11, pp. 1–16, 1959.
L. Armijo, “Minimization of functions having Lipschitz continuous first partial derivatives,” Pacific J. Math., vol. 16, no. 1, pp. 1–3, 1996.
D.P. Bertsekas, Nonlinear Programming, Athena Scientific: Belmont, Massachusetts, 1995.
J.E. Dennis, Jr. and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Inc.: Engleword Cliffs, New Jersey, 1983.
S.C. Eisenstat and H.F. Walker, “Globally convergent inexact newton methods,” SIAM J. on Optimization, vol. 4, pp. 393–422, 1994.
R. Fletcher, Practical Methods of Optimization, John Wiley & Sons, 1987.
A.A. Goldstein, “On steepest descent,” SIAM J. On Control, vol. 3, pp. 147–151, 1965.
A.A. Goldstein, Constructive Real Analysis, Harpers & Row: New York, 1967.
D.G. Luenberger, Linear and Nonlinear Programming, Addison-Wesley Publishing, 1984.
Lukšan, “Inexact trust region method for large sparse systems of nonlinear equations,” J. of Optimization Theory and Applications, vol. 81, pp. 569–590, 1994.
J.J. Moré, B.S. Garbow, and K.E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions On Mathematical Software, vol. 7, pp. 17–41, 1981.
F.A. Potra and Y. Shi, “Efficient line search algorithm for unconstrained optimization,” J. Optimization Theory and Application, vol. 85, pp. 677–704, 1995.
M.J.D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line searches,” in SIAM-AMS Proceedings, vol. IX, R.W. Cottle and C.E. Lemke (Eds.). SIAM Publications: Philadelphia, 1976.
Y. Shi, “Aglobalization procedure for solving nonlinear systems of equations,” Numerical Algorithms, vol. 12, pp. 272–286, 1996.
P. Wolfe, “Convergence conditions for ascent methods,” SIAM Rev., vol. 11, pp. 226–235, 1968.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Shi, Y. Globally Convergent Algorithms for Unconstrained Optimization. Computational Optimization and Applications 16, 295–308 (2000). https://doi.org/10.1023/A:1008772414083
Issue Date:
DOI: https://doi.org/10.1023/A:1008772414083