Abstract
Every Newton step in an interior-point method for optimization requires a solution of a symmetric indefinite system of linear equations. Most of today's codes apply direct solution methods to perform this task. The use of logarithmic barriers in interior point methods causes unavoidable ill-conditioning of linear systems and, hence, iterative methods fail to provide sufficient accuracy unless appropriately preconditioned. Two types of preconditioners which use some form of incomplete Cholesky factorization for indefinite systems are proposed in this paper. Although they involve significantly sparser factorizations than those used in direct approaches they still capture most of the numerical properties of the preconditioned system. The spectral analysis of the preconditioned matrix is performed: for convex optimization problems all the eigenvalues of this matrix are strictly positive. Numerical results are given for a set of public domain large linearly constrained convex quadratic programming problems with sizes reaching tens of thousands of variables. The analysis of these results reveals that the solution times for such problems on a modern PC are measured in minutes when direct methods are used and drop to seconds when iterative methods with appropriate preconditioners are used.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
A. Altman and J. Gondzio, “Regularized symmetric indefinite systems in interior point methods for linear and quadratic optimization,” Optimization Methods and Software, vol. 11/12, pp. 275-302, 1999.
E.D. Andersen, J. Gondzio, C. Mészáros, and X. Xu, “Implementation of interior point methods for large scale linear programming,” in Interior Point Methods in Mathematical Programming, T. Terlaky (Ed.), Kluwer Academic Publishers, pp. 189-252, 1996.
M. Arioli, I.S. Duff, and P.P.M. de Rijk, “On the augmented system approach to sparse least-squares problems,” Numerische Mathematik, vol. 55, pp. 667-684, 1989.
V. Baryamureeba, “Solution of large-scale weighted least-squares problems,” Numerical Linear Algebra with Applications, vol. 9, pp. 93-106, 2002.
V. Baryamureeba and T. Steihaug, “On the convergence of an inexact primal-dual interior point method for linear programming,” Tech. Rep. 188, Department of Informatics, University of Bergen, Norway, 2000.
S. Bellavia, “An inexact interior point method,” Journal of Optimization Theory and Applications, vol. 96, pp. 109-121, 1998.
L. Bergamaschi and G. Zilli, “Parallel inexact Newton and interior point method,” Ann. Univ. Ferrara-Sez VII-Sc. Mat., XLV, pp. 467-478, 2000.
J. Castro, “A specialized interior-point algorithm for multicommodity network flows,” SIAM Journal on Optimization, vol. 10, pp. 852-877, 2000.
R.S. Dembo, S.C. Eisenstat, and T. Steihaug, “Inexact Newton methods,” SIAM Journal on Numerical Analysis, vol. 19, pp. 400-408, 1982.
I.S. Duff, A.M. Erisman, and J.K. Reid, “Direct Methods for Sparse Matrices,” Oxford University Press: New York, 1987.
C. Durazzi and V. Ruggiero, “Indefinitely preconditioned conjugate gradient method for large sparse equality and inequality constrained quadratic problems,” Numerical Linear Algebra with Applications, vol. 10, pp. 673-688, 2003.
R. Fletcher, “Conjugate Gradient Methods for Indefinite Systems,” in Numerical Analysis Dundee 1975, G. Watson (Ed.), Springer-Verlag: Berlin, New York, 1976, pp. 73-89.
R.W. Freund and N.M. Nachtigal, “Software for simplified Lanczos and QMR algorithms,” Applied Numerical Mathematics, vol. 19, pp. 319-341, 1995.
P.E. Gill, W. Murray, D.B. Ponceleon, and M.A. Saunders, “Preconditioners for indefinite systems arising in optimization,” SIAM Journal on Matrix Analysis and Applications, vol. 13, pp. 292-311, 1992.
G.H. Golub and A.J. Wathen, “An iteration for indefinite systems and its application to the Navier-Stokes equations,” SIAM Journal on Scientific Computing, vol. 19, pp. 530-539, 1998.
J. Gondzio,“HOPDM(version 2.12)-a fast LP solver based on a primal-dual interior point method,” European Journal of Operational Research, vol. 85, pp. 221-225, 1995.
N.I.M. Gould, M.E. Hribar, and J. Nocedal, “On the solution of equality constrained quadratic problems arising in optimization,” SIAM Journal on Scientific Computing, vol. 23, pp. 1375-1394, 2001.
J.J. JÚdice, J. Patricio, L.F. Portugal, M.G.C. Resende, and G. Veiga, “A study of preconditioners for network interior point methods,” Computational Optimization and Applications, vol. 24, pp. 5-35, 2003.
C. Keller, N.I.M. Gould, and A.J. Wathen, “Constraint preconditioning for indefinite linear systems,” SIAM Journal on Matrix Analysis and Applications, vol. 21, pp. 1300-1317, 2000.
C.T. Kelley, Iterative Methods for Linear and Nonlinear Equations, vol. 16 of Frontiers in Applied Mathematics, SIAM, Philadelphia, 1995.
L. Lukšan and J. Vlček, “Indefinitely preconditioned inexact Newton method for large sparse equality constrained nonlinear programming problems,” Numerical Linear Algebra with Applications, vol. 5, pp. 219-247, 1998.
L. Lukšan and J. Vlček, “Interior point method for nonlinear nonconvex optimization,” Tech. Rep. 836, Institute of Computer Science, Academy of Sciences of the Czech Republic, 18207 Prague, Czech Republic, 2001.
A.R.L. Oliveira, A new class of preconditioners for large-scale linear systems from interior point methods for linear programming, PhD thesis, Department of Computational and Applied Mathematics, Rice University, Houston, TX 77005, 1997.
I. Perugia and V. Simoncini, “Block-diagonal and indefinite symmetric preconditioners for mixed finite elements formulations,” Numerical Linear Algebra with Applications, vol. 7, pp. 585-616, 2000.
M. Rozlozník and V. Simoncini, “Krylov subspace methods for saddle point problems with indefinite preconditioning,” SIAM Journal of Matrix Analysis and Applications, vol. 24, pp. 368-391, 2002.
Y. Saad and M. Schultz, “GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems,” SIAM Journal on Scientific and Statistical Computing, vol. 7, pp. 856-869, 1986.
H.A. Van der Vorst, “Bi-CGSTAB: A fast and smoothly converging variant to Bi-CG for the solution of nonlinear systems,” SIAM Journal on Scientific and Statistical Computing, vol. 13, pp. 631-644, 1992.
R.J. Vanderbei, “Symmetric quasidefinite matrices,” SIAM Journal on Optimization, vol. 5, pp. 100-113, 1995.
W. Wang and D.P. O'Leary, “Adaptive use of iterative methods in predictor-corrector interior point methods for linear programming,” Numerical Algorithms, vol. 25, pp. 387-406, 2000.
S.J. Wright, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, 1997.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bergamaschi, L., Gondzio, J. & Zilli, G. Preconditioning Indefinite Systems in Interior Point Methods for Optimization. Computational Optimization and Applications 28, 149–171 (2004). https://doi.org/10.1023/B:COAP.0000026882.34332.1b
Issue Date:
DOI: https://doi.org/10.1023/B:COAP.0000026882.34332.1b