Numerical Algorithms

, Volume 70, Issue 1, pp 9–41 | Cite as

Limited-memory LDL factorization of symmetric quasi-definite matrices with application to constrained optimization

Original Paper

Abstract

We propose a generalization of the limited-memory Cholesky factorization of Lin and Moré (SIAM J. Sci. Comput. 21(1), 24–45, 1999) to the symmetric indefinite case with special interest in symmetric quasi-definite matrices. We use this incomplete factorization to precondition two formulations of linear systems arising from regularized interior-point methods for quadratic optimization. An advantage of the limited-memory approach is predictable memory requirements. We establish existence of incomplete factors when the input matrix is an H-matrix but our numerical results illustrate that the factorization succeeds more generally. An appropriate diagonal shift is applied whenever the input matrix is not quasi definite. As the memory parameter increases an efficiency measure of the preconditioner suggested by Scott and Tůma (2013) improves. The combination of the 3×3 block formulation analyzed by Greif, Moulding, and Orban (SIAM J. Optim. 24(1), 49–83, 2014), the SYMAMD ordering, and a moderate memory parameter results in encouraging performance.

Keywords

Preconditioning Symmetric quasi definite Incomplete factorization Limited-memory factorization Interior-point methods 

Mathematics Subject Classifications (2000)

15A06 15A23 15B57 90C06 90C20 65F08 65F10 65F22 65F50 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Amestoy, P., Davis, T., Duff, I.: An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17(4), 886–905 (1996). doi:10.1137/S0895479894278952 MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Anderson, B.D.O., Moore, J.B.: Optimal Filtering. Prentice-Hall, Englewood Cliffs (1979)Google Scholar
  3. 3.
    Aravkin, A.Y., Burke, J.V., Pillonetto, G.: Optimization viewpoint on kalman smoothing with applications to robust and sparse estimation. In: Compressed Sensing & Sparse Filtering, pp. 237–280. Springer (2014)Google Scholar
  4. 4.
    Arioli, M., Orban, D.: Iterative methods for symmetric quasi-definite linear systems—Part I: Theory. Cahier du GERAD G-2013-32, GERAD, Montréal, QC, Canada (2013)Google Scholar
  5. 5.
    Axelsson, O.: Iterative Solution Methods. Cambridge University Press, Cambridge (1994)CrossRefGoogle Scholar
  6. 6.
    Bunch, J.R., Kaufman, L.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31(137), 163–179 (1977)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Buoni, J.J.: A stable method for the incomplete factorization of H-matrices. Linear Algebra Appl. 129, 143–154 (1990). doi:10.1016/0024-3795(90)90302-S MathSciNetCrossRefGoogle Scholar
  8. 8.
    Byrd, R.H., Nocedal, J., Waltz, R.A.: KNITRO: An integrated package for nonlinear optimization. In G. di Pillo and M. Roma, editors, Large-Scale Nonlinear Optimization, volume 83 of Nonconvex Optimization and Its Applications, pp. 35–59. Springer Verlag (2006)Google Scholar
  9. 9.
    Chan, W.M., George, A.: A linear time implementation of the reverse Cuthill-McKee algorithm. BIT Numer. Math. 20(1), 8–14 (1980). doi:10.1007/BF01933580 MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Courtier, P.: Dual formulation of four-dimensional variational assimilation. Q. J. R. Meteorol. Soc. 123, 2449–2461 (1997). doi:10.1002/qj.49712354414 CrossRefMATHGoogle Scholar
  11. 11.
    Cuthill, E., McKee, J.: Reducing the bandwidth of sparse symmetric matrices. In: Proceedings of the 1969 24th national conference, ACM ’69, pp. 157–172 (1969). doi:10.1145/800195.805928
  12. 12.
    Dolan, E., Moré, J.: Benchmarking optimization software with performance profiles. Math. Program. Ser. B 91, 201–213 (2002). doi:10.1007/s101070100263 CrossRefMATHGoogle Scholar
  13. 13.
    Dollar, H.S., Gould, N.I.M., Schilders, W.H.A., Wathen, A.J.: Implicit-factorization preconditioning and iterative solvers for regularized saddle-point systems. SIAM J. Matrix Anal. Appl. 28(1), 170–189 (2006). doi:10.1137/05063427X MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Elman, H., Silvester, D., Wathen, A.: Finite Elements and Fast Iterative Solvers with Applications in Incompressible Fluid Dynamics. Numerical Mathematics and Scientific Computation. Oxford University Press, Oxford (2005)Google Scholar
  15. 15.
    Forsgren, A., Gill, Ph. E., Wright, M.H.: Interior methods for nonlinear optimization. SIAM Rev. 44(4), 525–597 (2002). doi:10.1137/S0036144502414942 MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Friedlander, M.P., Orban, D.: A primal-dual regularized interior-point method for convex quadratic programs. Math. Program. Comput. 4(1), 71–107 (2012). doi:10.1007/s12532-012-0035-2 MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Friedlander, M.P., van Den Berg, E.: Spot—a linear-operator toolbox (2013). www.cs.ubc.ca/labs/scl/spot
  18. 18.
    Gill, P.E., Saunders, M.A., Shinnerl, J.R.: On the stability of Cholesky factorization for symmetric quasidefinite systems. SIAM J. Optim. 17(1), 35–46 (1996). doi:10.1137/S0895479893252623 MathSciNetMATHGoogle Scholar
  19. 19.
    Golub, G.H., Van Loan, C.F., 3rd: Matrix Computations. Johns Hopkins Series in the Mathematical Sciences. Johns Hopkins, Baltimore (1996)Google Scholar
  20. 20.
    Gould, N.I.M., Orban, D., Toint, Ph. L.: CUTEr and SifDec, a Constrained and Unconstrained Testing Environment, revisited. ACM Trans. Math. Softw. 29(4), 373–394 (2003)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Gould, N.I.M., Orban, D., Toint, Ph. L.: Numerical methods for large-scale nonlinear optimization. Acta Numerica 14, 299–361 (2005). doi:10.1017/S0962492904000248 MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Greif, C., He, S., Liu, P.: sym-ildl: Incomplete L D L T factorizations of indefinite symmetric matrices (2013). URL www.cs.ubc.ca/inutard/html
  23. 23.
    Greif, C., Moulding, E., Orban, D.: Bounds on the eigenvalues of block matrices arising from interior-point methods. SIAM J. Optim. 24(1), 49–83 (2014). doi:10.1137/120890600 MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Gustafsson, I.: A class of first order factorization methods. BIT 18(2), 142–156 (1978). doi:10.1007/BF01931691 MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Higham, N.J., 2nd: Accuracy and Stability of Numerical Algorithms. Number OT80 in Other Titles in Applied Mathematics. SIAM, Philadelphia (2002)CrossRefGoogle Scholar
  26. 26.
    HSL: A collection of Fortran codes for large-scale scientific computation. www.hsl.rl.ac.uk (2013)
  27. 27.
    Jones, M.T., Plassmann, P.E.: An improved incomplete Cholesky factorization. ACM Trans. Math. Softw. 21(1), 5–17 (1995). doi:10.1145/200979.200981 MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Kaporin, I.E.: High quality preconditioning of a general symmetric positive definite matrix based on its \(U^{T U + U^{T} R + R^{T} U}\) decomposition. Numer. Linear Algebra Appl. 5(6), 483–509 (1998). doi:10.1002/(SICI)1099-1506(199811/12)5:6<483::AID-NLA156>3.0.CO;2-7 MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Li, N., Saad, Y., Chow, E.: Crout versions of ILU for general sparse matrices. SIAM J. Sci. Comput. 25(2), 716–728 (2003). doi:10.1137/S1064827502405094 MathSciNetCrossRefGoogle Scholar
  30. 30.
    Li, N., Saad, Y.: Crout versions of ILU factorization with pivoting for sparse symmetric matrices. Electron. Trans. Numer. Anal. 20, 75–85 (2005)MathSciNetGoogle Scholar
  31. 31.
    Lin, C.-J., Moré, J.J.: Incomplete Cholesky factorizations with limited memory. SIAM J. Sci. Comput. 21(1), 24–45 (1999). doi:10.1137/S1064827597327334 MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Manteuffel, T.A.: Shifted incomplete Cholesky factorization. In: Duff, I. S., Stewart, G. W. (eds.) Sparse Matrix Proceedings, pp. 41–61. SIAM, Philadelphia (1979)MATHGoogle Scholar
  33. 33.
    Manteuffel, T.A.: An incomplete factorization technique for positive definite linear systems. Math. Comput. 34(150), 473–497 (1980)MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Maros, I., Mészáros, C.: A repository of convex quadratic programming problems. Optim. Methods Softw. 11 & 12, 671–681 (1999). Special Issue on Interior Point Methods.CrossRefMATHGoogle Scholar
  35. 35.
    Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM J. Optim. 2(4), 575–601 (1992). doi:10.1137/0802028 MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Meijerink, J.A., van der Vorst, H.A.: An iterative solution method for linear equations systems of which the coefficient matrix is a symmetric M-matrix. Math. Comput. 137, 148–162 (1977)MathSciNetGoogle Scholar
  37. 37.
    Messaoudi, A.: On the stability of the incomplete LU-factorizations and characterizations of H-matrices. Numer. Math. 69(3), 321–331 (1995). doi:10.1007/s002110050095 MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Munksgaard, N.: Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients. ACM Trans. Math. Softw. 6(2), 206–219 (1980). doi:10.1145/355887.355893 CrossRefMATHGoogle Scholar
  39. 39.
    Ortega, J.M.: Introduction to Parallel and Vector Solution of Linear Systems. Fontiers of Computer Science. Springer Science+Business Media, New York (1988). Originally published by Plenum PressCrossRefGoogle Scholar
  40. 40.
    Paige, C.C., Saunders, M.A.: Solution of sparse indefinite systems of linear equations. SIAM J. Numer. Anal. 12(4), 617–629 (1975). doi:10.1137/0712047 MathSciNetCrossRefGoogle Scholar
  41. 41.
    Saad, Y.: ILUT: A dual threshdol incomplete LU factorization. Numer Linear Algebra Appl 1(4), 387–402 (1994). doi:10.1002/nla.1680010405 MathSciNetCrossRefMATHGoogle Scholar
  42. 42.
    Saunders, M.A.: PDCO—primal-dual interior methods. Class notes #7 for Large-Scale Numerical Optimization, MS&amp;E 318/CME 338, Stanford University, Spring 2013. http://web.stanford.edu/group/SOL/software/pdco
  43. 43.
    Scott, J., Tůma, M.: HSL_MI28: an efficient and robust limited-memory incomplete Cholesky factorization code. Preprint RAL-P-2013-004. Rutherford Appleton Laboratory, Didcot (2013)Google Scholar
  44. 44.
    Tismenetsky, M.: A new preconditioning technique for solving large sparse linear systems. Linear Algebra Appl. 154–156, 331–353 (1991). doi:10.1016/0024-3795(91)90383-8 MathSciNetCrossRefGoogle Scholar
  45. 45.
    Tůma, M.: A note on the L D L T decomposition of matrices from saddle-point problems. SIAM J. Matrix Anal. Appl. 23(4), 903–915 (2002). doi:10.1137/S0895479897321088 MathSciNetCrossRefMATHGoogle Scholar
  46. 46.
    Vanderbei, R.J.: Symmetric quasi-definite matrices. SIAM J. Optim. 5(1), 100–113 (1995). doi:10.1137/0805005 MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Vanderbei, R.J., Shanno, D.F.: An interior point algorithm for nonconvex nonlinear programming. Comput. Optim. Appl. 13(1–3), 231–252 (1999). doi:10.1023/A:1008677427361 MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. (2006). doi:10.1007/s10107-004-0559-y

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.GERAD and Department of Mathematics and Industrial EngineeringÉcole PolytechniqueMontréalCanada

Personalised recommendations