Abstract
A new line search algorithm for smooth unconstrained optimization is presented that requires only one gradient evaluation with an inaccurate line search and at most two gradient evaluations with an accurate line search. It terminates in finitely many operations and shares the same theoretical properties as the standard line search rules like the Armijo-Goldstein-Wolfe-Powell rules. This algorithm is especially appropriate for the situation when gradient evaluations are very expensive relative to function evaluations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Dennis, J. E. andSchnabel, R. B.,Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, New Jersey, 1983.
Fletcher, R.,Practical Methods of Optimization, John Wiley and Sons, New York, New York, 1987.
Goldstein, A. A.,On Steepest Descent, SIAM Journal on Control, Vol. 3, pp. 147–151, 1965.
Goldstein, A. A.,Constructive Real Analysis, Harpers and Row, New York, New York, 1967.
Armijo, L.,Minimization of Functions Having Lipschitz Continuous First Partial Derivatives, Pacific Journal of Mathematics, Vol. 16, pp. 1–3, 1966.
Wolfe, P.,Convergence Conditions for Ascent Methods, SIAM Review, Vol. 11, pp. 226–235, 1968.
Powell, M. J. D.,Some Global Convergence Properties of a Variable-Metric Algorithm for Minimization without Exact Line Searches, SIAM-AMS Proceedings, SIAM Publications, Philadelphia, Pennsylvania, Vol. 9, pp. 53–72, 1976.
Luenberger, D. G.,Linear and Nonlinear Programming, Addison-Wesley Publishing Company, Reading, Massachusetts, 1984.
Boggs, P. T., andSchnabel, R. B.,Lecture Notes for a Short Course on Numerical Optimization, Manuscript, Houston, Texas, 1987.
Gill, E. P., Murray, W., Saunders, A. M., andWright, H. M. A Note on a Sufficient-Decrease Criterion for a Nonderivative Steplength Procedure, Mathematical Programming, Vol. 23, pp. 349–352, 1982.
Ortega, J. M., andRheinboldt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, London, England, 1970.
Brent, R. P.,Algorithms for Minimization without Derivatives, Prentice-Hall, Englewood Cliffs, New Jersey, 1972.
Byrd, R. H., Nocedal, andYuan, Y. X.,Global Convergence of a Class of Quasi-Newton Methods on Convex Problems, SIAM Journal on Numerical Analysis, Vol. 24, pp. 1171–1190, 1987.
Moré, J. J., Garbow, B. S., andHillstrom, K. E.,Testing Unconstrained Optimization Software, ACM Transactions on Mathematical Software, Vol. 7, pp. 17–41, 1981.
Prapasri, A.,Application of Conjugate Directions and Quasi-Newton Methods in Parallel Unconstrained Optimization, PhD Thesis, University of Iowa, 1989.
Prapasri, A., andPotra, F. A.,Parallel Line Search Algorithms for Solving Convex Unconstrained Minimization Problems, Libertas Mathematica, Vol. 8, pp. 31–46, 1988.
Author information
Authors and Affiliations
Additional information
The authors would like to thank Margaret Wright and Jorge Moré for valuable comments on earlier versions of this paper.
Rights and permissions
About this article
Cite this article
Potra, F.A., Shi, Y. Efficient line search algorithm for unconstrained optimization. J Optim Theory Appl 85, 677–704 (1995). https://doi.org/10.1007/BF02193062
Issue Date:
DOI: https://doi.org/10.1007/BF02193062