Preconditioned Conjugate Gradient Algorithms for Nonconvex Problems with Box Constraints

  • R. Pytlak
  • T. Tarnawski
Conference paper
Part of the IFIP International Federation for Information Processing book series (IFIPAICT, volume 199)


The paper describes a new conjugate gradient algorithm for large scale nonconvex problems with box constraints. In order to speed up the convergence the algorithm employs a scaling matrix which transforms the space of original variables into the space in which Hessian matrices of functionals describing the problems have more clustered eigenvalues. This is done efficiently by applying limited memory BFGS updating matrices. Once the scaling matrix is calculated, the next few iterations of the conjugate gradient algorithms are performed in the transformed space. The box constraints are treated by the projection as previously used in [R. Pytlak, The efficient algorithm for large-scale problems with simple bounds on the variables, SIAM J. on Optimization, Vol. 8, 532–560, 1998]. We believe that the preconditioned conjugate gradient algorithm gives more flexibility in achieving balance between the computing time and the number of function evaluations in comparison to a limited memory BFGS algorithm. The numerical results show that the proposed method is competitive to L-BFGS-B procedure.


bound constrained nonlinear optimization problems conjugate gradient algorithms quasi-Newton methods 


  1. [1]
    D.P. Bertsekas, Projected Newton methods for optimization problems with simple constraints. SIAM J. Control and Optimiz. 20:221–245. 1982.MATHMathSciNetCrossRefGoogle Scholar
  2. [2]
    I. Bongartz, A.R. Conn, N.I.M. Gould, Ph.L. Toint, CUTE: Constrained and Unconstrained Testing Environment. Research Report RC 18860, IBM T.J. Watson Research Center, Yorktown Heights, NY, USA, 1994.Google Scholar
  3. [3]
    R. Byrd, J. Nocedal, R. B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods. Technical Report NAM-03, 1996.Google Scholar
  4. [4]
    Y. Dai, Y. Yuan, Global convergence of the method of shortest residuals. Numerische Mathematik 83:581–598, 1999.MathSciNetGoogle Scholar
  5. [5]
    P.E. Gill, M.W. Leonard, Reduced-Hessian methods for unconstrained optimization. SIAM J. Optimiz. 12:209–237, 2001.MathSciNetCrossRefGoogle Scholar
  6. [6]
    P.E. Gill, M.W. Leonard, Limited-memory reduced hessian methods for large-scale unconstrained optimization. SIAM J. Optimiz. 14:380–401, 2003.MathSciNetCrossRefGoogle Scholar
  7. [7]
    C. Lemaréchal, An Extension of Davidon methods to nondifferentiable Problem. In Mathematical Programming Study 3. North-Holland, Amsterdam, 1975.Google Scholar
  8. [8]
    J. Nocedal, S.J. Wright, Numerical optimization. Springer-Verlag, New York, 1999.Google Scholar
  9. [9]
    R. Pytlak, On the convergence of conjugate gradient algorithms. IMA Journal of Numerical Analysis. 14:443–460, 1994.MATHMathSciNetGoogle Scholar
  10. [10]
    R. Pytlak, An efficient algorithm for large-scale nonlinear programming problems with simple bounds on the variables. SIAM J. on Optimiz. 8:532–560, 1998.MATHMathSciNetCrossRefGoogle Scholar
  11. [11]
    R. Pytlak, T. Tarnawski, The preconditione conjugate gradient algorithm for nonconvex problems. Research Report, Military University of Technology, Faculty of Cybernetics, N.-1, 2005.Google Scholar
  12. [12]
    D. Siegel, Modifying the BFGS update by a new column scaling technique. Mathematical Programming. 66:45–78, 1994.MATHMathSciNetCrossRefGoogle Scholar
  13. [13]
    P. Wolfe, A Method of Conjugate Subgradients for Minimizing Nondifferentiable Functions. In Mathematical Programming Study 3. North-Holland, Amsterdam, 1975.Google Scholar
  14. [14]
    C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, Algorithm 778: L-BFGS-B, FORTRAN subroutines for large scale bound constrained optimization. ACM Transactions on Mathematical Software. 23:550–560, 1997.MathSciNetCrossRefGoogle Scholar

Copyright information

© International Federation for Information Processing 2006

Authors and Affiliations

  • R. Pytlak
    • 1
  • T. Tarnawski
    • 2
  1. 1.Faculty of CyberneticsMilitary University of TechnologyWarsawPoland
  2. 2.Faculty of CyberneticsMilitary University of TechnologyWarsawPoland

Personalised recommendations