Abstract
In this paper we deal with the iterative computation of negative curvature directions of an objective function, within large scale optimization frameworks. In particular, suitable directions of negative curvature of the objective function represent an essential tool, to guarantee convergence to second order critical points. However, an “adequate” negative curvature direction is often required to have a good resemblance to an eigenvector corresponding to the smallest eigenvalue of the Hessian matrix. Thus, its computation may be a very difficult task on large scale problems. Several strategies proposed in literature compute such a direction relying on matrix factorizations, so that they may be inefficient or even impracticable in a large scale setting. On the other hand, the iterative methods proposed either need to store a large matrix, or they need to rerun the recurrence.
On this guideline, in this paper we propose the use of an iterative method, based on a planar Conjugate Gradient scheme. Under mild assumptions, we provide theory for using the latter method to compute adequate negative curvature directions, within optimization frameworks. In our proposal any matrix storage is avoided, along with any additional rerun.
Similar content being viewed by others
References
Bank, R., Chan, T.: A composite step bi-conjugate gradient algorithm for nonsymmetric linear systems. Numer. Algorithms 7, 1–16 (1994)
Boman, E., Murray, W.: An iterative approach to computing a direction of negative curvature. Presented at Copper Mountain conference, March 1998. Available at the url: www-sccm.stanford.edu/students/boman/papers.shtml
Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS–SIAM Series on Optimization. SIAM, Philadelphia (2000)
Cullum, J., Willoughby, R.: Lanczos Algorithms for Large Symmetric Eigenvalue Computations. Birkhäuser, Boston (1985)
Dixon, L., Ducksbury, P., Singh, P.: A new three-term conjugate gradient method. Technical report 130, Numerical Optimization Centre, Hatfield Polytechnic, Hatfield, Hertfordshire, UK (1985)
Facchinei, F., Lucidi, S.: Convergence to second order stationary points in inequality constrained optimization. Math. Oper. Res. 93, 746–766 (1998)
Fasano, G.: Use of conjugate directions inside Newton-type algorithms for large scale unconstrained optimization. PhD thesis, Università di Roma “La Sapienza”, Roma, Italy (2001)
Fasano, G.: Lanczos-conjugate gradient method and pseudoinverse computation, on indefinite and singular systems. J. Optim. Theory Appl. DOI 10.1007/s10957-006-91193
Fasano, G.: Planar-conjugate gradient algorithm for large-scale unconstrained optimization, part 1: theory. J. Optim. Theory Appl. 125, 523–541 (2005)
Fasano, G.: Planar-conjugate gradient algorithm for large-scale unconstrained optimization, part 2: application. J. Optim. Theory Appl. 125, 543–558 (2005)
Fasano, G., Roma, M.: Iterative computation of negative curvature directions in large scale optimization: theory and preliminary numerical results, Technical report 12-05, Dipartimento di Informatica e Sistemistica “A. Ruberti”, Roma, Italy (2005)
Ferris, M., Lucidi, S., Roma, M.: Nonmonotone curvilinear linesearch methods for unconstrained optimization. Comput. Optim. Appl. 6, 117–136 (1996)
Golub, G., Van Loan, C.: Matrix Computations, 3rd edn. John Hopkins University Press, Baltimore (1996).
Gould, N.I.M., Lucidi, S., Roma, M., Toint, P.L.: Solving the trust-region subproblem using the Lanczos method. SIAM J. Optim. 9, 504–525 (1999)
Gould, N.I.M., Lucidi, S., Roma, M., Toint, P.L.: Exploiting negative curvature directions in linesearch methods for unconstrained optimization. Optim. Methods Softw. 14, 75–98 (2000)
Gould, N.I.M., Orban, D., Toint, P.: \(\mathsf{CUTEr}\) (and \(\mathsf{SifDec}\) ), a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29, 373–394 (2003)
Hestenes, M.: Conjugate Direction Methods in Optimization. Springer, New York (1980)
Liu, Y., Storey, C.: Efficient generalized conjugate gradient algorithm, part 1. J. Optim. Theory Appl. 69, 129–137 (1991)
Lucidi, S., Rochetich, F., Roma, M.: Curvilinear stabilization techniques for truncated Newton methods in large scale unconstrained optimization. SIAM J. Optim. 8, 916–939 (1998)
Lucidi, S., Roma, M.: Numerical experiences with new truncated Newton methods in large scale unconstrained optimization. Comput. Optim. Appl. 7, 71–87 (1997)
McCormick, G.: A modification of Armijo’s step-size rule for negative curvature. Math. Program. 13, 111–115 (1977)
Miele, A., Cantrell, J.: Study on a memory gradient method for the minimization of functions. J. Optim. Theory Appl. 3, 459–470 (1969)
Moré, J., Sorensen, D.: On the use of directions of negative curvature in a modified Newton method. Math. Program. 16, 1–20 (1979)
Moré, J., Sorensen, D.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)
Nash, S.: A survey of truncated-Newton methods. J. Comput. Appl. Math. 124, 45–59 (2000)
Paige, C., Saunders, M.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12, 617–629 (1975)
Parlett, B.: The Symmetric Eigenvalue Problem. Prentice-Hall Series in Computational Mathematics. Prentice-Hall, Englewood Cliffs (1980)
Shultz, G., Schnabel, R., Byrd, R.: A family of trust-region-based algorithms for unconstrained minimization. SIAM J. Numer. Anal. 22, 47–67 (1985)
Stoer, J.: Solution of large linear systems of equations by conjugate gradient type methods. In: Bachem A., Grötschel M., Korte B. (eds.) Mathematical Programming. The State of the Art, pp. 540–565. Springer, Berlin/Heidelberg (1983)
Trefethen, L., Bau, D.: Numerical Linear Algebra. SIAM, Philadelphia (1997)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fasano, G., Roma, M. Iterative computation of negative curvature directions in large scale optimization. Comput Optim Appl 38, 81–104 (2007). https://doi.org/10.1007/s10589-007-9034-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-007-9034-z