Abstract
The paper describes a new conjugate gradient algorithm for large scale nonconvex problems with box constraints. In order to speed up the convergence the algorithm employs a scaling matrix which transforms the space of original variables into the space in which Hessian matrices of functionals describing the problems have more clustered eigenvalues. This is done efficiently by applying limited memory BFGS updating matrices. Once the scaling matrix is calculated, the next few iterations of the conjugate gradient algorithms are performed in the transformed space. The box constraints are treated by the projection as previously used in [R. Pytlak, The efficient algorithm for large-scale problems with simple bounds on the variables, SIAM J. on Optimization, Vol. 8, 532–560, 1998]. We believe that the preconditioned conjugate gradient algorithm gives more flexibility in achieving balance between the computing time and the number of function evaluations in comparison to a limited memory BFGS algorithm. The numerical results show that the proposed method is competitive to L-BFGS-B procedure.
Chapter PDF
Similar content being viewed by others
keywords
References
D.P. Bertsekas, Projected Newton methods for optimization problems with simple constraints. SIAM J. Control and Optimiz. 20:221–245. 1982.
I. Bongartz, A.R. Conn, N.I.M. Gould, Ph.L. Toint, CUTE: Constrained and Unconstrained Testing Environment. Research Report RC 18860, IBM T.J. Watson Research Center, Yorktown Heights, NY, USA, 1994.
R. Byrd, J. Nocedal, R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods. Technical Report NAM-03, 1996.
Y. Dai, Y. Yuan, Global convergence of the method of shortest residuals. Numerische Mathematik 83:581–598, 1999.
P.E. Gill, M.W. Leonard, Reduced-Hessian methods for unconstrained optimization. SIAM J. Optimiz. 12:209–237, 2001.
P.E. Gill, M.W. Leonard, Limited-memory reduced hessian methods for large-scale unconstrained optimization. SIAM J. Optimiz. 14:380–401, 2003.
C. Lemaréchal, An Extension of Davidon methods to nondifferentiable Problem. In Mathematical Programming Study 3. North-Holland, Amsterdam, 1975.
J. Nocedal, S.J. Wright, Numerical optimization. Springer-Verlag, New York, 1999.
R. Pytlak, On the convergence of conjugate gradient algorithms. IMA Journal of Numerical Analysis. 14:443–460, 1994.
R. Pytlak, An efficient algorithm for large-scale nonlinear programming problems with simple bounds on the variables. SIAM J. on Optimiz. 8:532–560, 1998.
R. Pytlak, T. Tarnawski, The preconditione conjugate gradient algorithm for nonconvex problems. Research Report, Military University of Technology, Faculty of Cybernetics, N.-1, 2005.
D. Siegel, Modifying the BFGS update by a new column scaling technique. Mathematical Programming. 66:45–78, 1994.
P. Wolfe, A Method of Conjugate Subgradients for Minimizing Nondifferentiable Functions. In Mathematical Programming Study 3. North-Holland, Amsterdam, 1975.
C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, Algorithm 778: L-BFGS-B, FORTRAN subroutines for large scale bound constrained optimization. ACM Transactions on Mathematical Software. 23:550–560, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 International Federation for Information Processing
About this paper
Cite this paper
Pytlak, R., Tarnawski, T. (2006). Preconditioned Conjugate Gradient Algorithms for Nonconvex Problems with Box Constraints. In: Ceragioli, F., Dontchev, A., Futura, H., Marti, K., Pandolfi, L. (eds) System Modeling and Optimization. CSMO 2005. IFIP International Federation for Information Processing, vol 199. Springer, Boston, MA. https://doi.org/10.1007/0-387-33006-2_11
Download citation
DOI: https://doi.org/10.1007/0-387-33006-2_11
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-32774-7
Online ISBN: 978-0-387-33006-8
eBook Packages: Computer ScienceComputer Science (R0)