Skip to main content
SpringerLink
Log in

Matrix-free preconditioning using partial matrix estimation

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

We consider matrix-free solver environments where information about the underlying matrix is available only through matrix vector computations which do not have access to a fully assembled matrix. We introduce the notion of partial matrix estimation for constructing good algebraic preconditioners used in Krylov iterative methods in such matrix-free environments, and formulate three new graph coloring problems for partial matrix estimation. Numerical experiments utilizing one of these formulations demonstrate the viability of this approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. A. Ajiz and A. Jennings, A robust incomplete Choleski-conjugate gradient algorithm, Int. J. Numer. Methods Eng., 20 (1984), pp. 949–966.

    Article  MATH  MathSciNet  Google Scholar 

  2. O. Axelsson, Iterative Solution Methods, Cambridge University Press, Cambridge, 1994.

    MATH  Google Scholar 

  3. O. Axelsson and L. Kolotilina, Diagonally compensated reduction and related preconditioning methods, Numer. Lin. Algebra Appl., 1 (1994), pp. 155–177.

    Article  MATH  MathSciNet  Google Scholar 

  4. M. W. Benson, Iterative Solution of Large-Scale Linear Systems, Master’s Thesis, Lakehead University, Thunder Bay, 1973.

  5. M. Benzi and M. Tůma, A sparse approximate inverse preconditioner for nonsymmetric linear systems, SIAM J. Sci. Comput., 19 (1998), pp. 968–994.

    Article  MATH  MathSciNet  Google Scholar 

  6. E. Chow, A priori sparsity patterns for parallel sparse approximate inverse preconditioners, SIAM J. Sci. Comput., 21 (2000), pp. 1804–1822.

    Article  MATH  MathSciNet  Google Scholar 

  7. E. Chow, ParaSails user’s guide, tech. rep. UCRL-MA-137863, Lawrence Livermore National Laboratory, Livermore, CA, 2000.

  8. E. Chow, Parallel implementation and practical use of sparse approximate inverses with a priori sparsity patterns, Int. J. High Perf. Comput. Appl., 15 (2001), pp. 56–74.

    Article  Google Scholar 

  9. T. F. Coleman and J. Cai, The cyclic coloring problem and estimation of sparse Hessian matrices, SIAM J. Discrete Alg. Methods, 7 (1986), pp. 221–235.

    MATH  MathSciNet  Google Scholar 

  10. T. F. Coleman, B. S. Garbow, and J. J. Moré, Software for estimation sparse Jacobian matrices, ACM Trans. Math. Softw., 10 (1984), pp. 329–345.

    Article  MATH  Google Scholar 

  11. T. F. Coleman and J. J. Moré, Estimation of sparse Jacobian matrices and graph coloring problems, SIAM J. Numer. Anal., 20 (1983), pp. 187–209.

    Article  MATH  MathSciNet  Google Scholar 

  12. T. F. Coleman and J. J. Moré, Estimation of sparse Hessian matrices and graph coloring problems, Math. Program., 28 (1984), pp. 243–270.

    MATH  Google Scholar 

  13. T. F. Coleman and A. Verma, Structure and efficient Jacobian calculation, in Computational Differentiation: Techniques, Applications and Tools, Berz et al., eds., pp. 149–159, SIAM, Philadelphia, 1996.

  14. T. F. Coleman and A. Verma, The efficient computation of sparse Jacobian matrices using automatic differentiation, SIAM J. Sci. Comput., 19 (1998), pp. 1210–1233.

    Article  MATH  MathSciNet  Google Scholar 

  15. A. R. Curtis, M. J. D. Powell, and J. K. Reid, On the estimation of sparse Jacobian matrices, J. Inst. Math. Appl., 13 (1974), pp. 117–119.

    MATH  Google Scholar 

  16. T. Davis, Sparse matrix collection, NA Digest, 42 (1994), http://www.cise.ufl.edu/∼davis/sparse/.

  17. T. F. Dupont, R. P. Kendall, and H. H. Rachford, An approximate factorization procedure for solving self-adjoint elliptic difference equations, SIAM J. Numer. Anal., 5 (1968), pp. 559–573.

    Article  MATH  MathSciNet  Google Scholar 

  18. A. H. Gebremedhin, F. Manne, and A. Pothen, What color is your Jacobian? Graph coloring for computing derivatives, SIAM Rev., 47 (2005), pp. 629–705.

    Article  MATH  MathSciNet  Google Scholar 

  19. K. Georg, Matrix-free numerical continuation and bifurcation, Numer. Funct. Anal. Optimization, 22 (2001), pp. 303–320.

    Article  MATH  MathSciNet  Google Scholar 

  20. A. Griewank, Direct calculation of Newton steps without accumulating Jacobians, in Large-Scale Numerical Optimization, T. F. Coleman and Y. Li, eds., pp. 115–137, SIAM, Philadelphia, 1990.

    Google Scholar 

  21. A. Griewank, Some bounds on complexity of gradients, Jacobians and Hessians, in Complexity in Numerical Optimization, P. M. Pardalos, ed., World Scientific Publishing Company, River Edge, NJ, 1993.

  22. I. Gustafsson, A class of first order factorization methods, BIT, 18 (1978), pp. 142–156.

    Article  MATH  MathSciNet  Google Scholar 

  23. S. Hossain, On the computation of sparse Jacobian matrices and Newton steps, tech. rep. No. 146, Department of Informatics, University of Bergen, 1998.

  24. S. Hossain and T. Steihaug, Graph coloring and the estimation of sparse Jacobian matrices with segmented columns, tech. rep. No. 72, Department of Informatics, University of Bergen, 1997.

  25. S. Hossain and T. Steihaug, Sparsity issues in the computation of Jacobian matrices, in ACM Proceedings of the 2002 International Symposium on Symbolic and Algebraic Computation, T. Mora, ed., pp. 123–130, ACM Press, New York, 2002.

    Chapter  Google Scholar 

  26. S. Hossain and T. Steihaug, Reducing the number of AD passes for computing of a sparse Jacobian matrix, in Automatic Differentiation: From Simulation to Optimization, G. Corliss, C. Faure, A. Griewank, L. Hascoët, and U. Naumann, eds., Computer and Information Science, Springer, New York, 2002.

  27. T. Huckle, Approximate sparsity patterns for the inverse of a matrix and preconditioning, in Proceedings of the 15th IMACS World Congress 1997 on Scientific Computation, Modelling and Applied Mathematics, Vol. II, pp. 569–574, IMACS, New Brunswick, 1997.

  28. D. Hysom and A. Pothen, A scalable parallel algorithm for incomplete factor preconditioning, SIAM J. Sci. Comput., 22 (2001), pp. 2194–2215.

    Article  MATH  MathSciNet  Google Scholar 

  29. D. Hysom and A. Pothen, Efficient parallel computation of ILU(k) preconditioners, in Proceedings Supercomputing ’99, pp. 1–19, IEEE Press, Piscataway, NJ, 1999.

  30. G. Karypis and V. Kumar, Parallel threshold-based ILU factorization, tech. rep. TR-96-061, University of Minnesota, 1996.

  31. C. T. Kelley, Solving Nonlinear Equations with Newton’s Method, SIAM, Philadelphia, 2003.

    Google Scholar 

  32. D. E. Keyes, Terascale implicit methods for partial differential equations, Contemp. Math., 306 (2001), pp. 29–84.

    MathSciNet  Google Scholar 

  33. D. A. Knoll and D. E. Keyes, Jacobian-free Newton–Krylov methods: a survey of approaches and applications, J. Comput. Phys., 193 (2004), pp. 357–397.

    Article  MATH  MathSciNet  Google Scholar 

  34. L. Y. Kolotilina, A. A. Nikishin, and A. Yu. Yeremin, Factorized sparse approximate inverse preconditionings. IV: simple approaches to rising efficiency, Numer. Linear Algebra Appl., 6 (1999), pp. 515–531.

    Article  MATH  MathSciNet  Google Scholar 

  35. G. Meurant, Computer Solution of Large Linear Systems, Studies in Mathematics and Its Applications, no. 28, Elsevier, Amsterdam–Lausanne–New-York–Oxford–Shannon–Singapore–Tokyo, 1999.

    Google Scholar 

  36. J. L. Morales and J. Nocedal, Automatic preconditioning by limited memory quasi-Newton updates, SIAM J. Optim., 10 (2000), pp. 1079–1096.

    Article  MATH  MathSciNet  Google Scholar 

  37. J. L. Morales and J. Nocedal, Algorithm 809: PREQN: Fortran 77 preconditioning of the conjugate gradient method, ACM Trans. Math. Softw., 27 (2001), pp. 83–91.

    Article  MATH  Google Scholar 

  38. P. E. Plassman, Sparse Jacobian estimation and factorization on a multiprocessor, in Large-Scale Optimization, T. F. Coleman and Y. Li, eds., pp. 152–179, SIAM, Philadelphia, 1990.

    Google Scholar 

  39. Y. Saad, A flexible inner-outer preconditioned GMRES algorithm, SIAM J. Sci. Stat. Comput., 14 (1993), pp. 461–469.

    Article  MATH  MathSciNet  Google Scholar 

  40. Y. Saad, ILUT: a dual threshold incomplete LU factorization, Numer. Linear Algebra Appl., 1 (1994), pp. 387–402.

    Article  MATH  MathSciNet  Google Scholar 

  41. Y. Saad, Iterative Methods for Sparse Linear Systems, 2nd ed., SIAM, Philadelphia, 2003.

    MATH  Google Scholar 

  42. P. L. Toint, On sparse and symmetric matrix updating subject to a linear equation, Math. Comput., 31 (1977), pp. 954–961.

    Article  MATH  MathSciNet  Google Scholar 

  43. D. A. Korzekwa and A. David, Current status of solid mechanics capability in Truchas, tech. rep. LA-UR-04-4092, Los Alamos National Laboratory, 2004.

Download references

Author information

Authors and Affiliations

  1. Los Alamos National Laboratory, 87545, Los Alamos, NM, USA

    J. K. Cullum

  2. Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod Vodárenskou věží 2, 18207, Praha 8, Libeň, Czech Republic

    M. Tůma

Authors
  1. J. K. Cullum
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Tůma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Tůma.

Additional information

AMS subject classification (2000)

65F10, 65F50, 49M37, 90C06

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cullum, J., Tůma, M. Matrix-free preconditioning using partial matrix estimation . Bit Numer Math 46, 711–729 (2006). https://doi.org/10.1007/s10543-006-0094-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-006-0094-8

Key words

Search

Navigation