Automatic Sparsity Detection Implemented as a Source-to-Source Transformation

  • Ralf Giering
  • Thomas Kaminski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3994)


An implementation of Automatic Sparsity Detection (ASD) as a new source-to-source transformation is presented. Given a code for evaluation of a function, ASD generates code to evaluate the sparsity pattern of the function’s Jacobian by operations on bit-vectors. Similar to Automatic Differentiation (AD), there are forward and reverse modes of ASD. As ASD code has significantly fewer required variables than AD, ASD should be operated in pure mode, i.e. without an evaluation of the underlying function included in the ASD code. In a performance comparison of ASD to AD on five small test problems, ASD is about two orders of magnitude faster than AD. Hence, for a particular class of sparse Jacobians, it is efficient to determine first the sparsity patten via ASD. In a subsequent AD step, this allows to reduce the effective dimension for the evaluation of the Jacobian by avoiding the evaluation of zero elements via a selection of seed matrices according to the sparsity pattern.


Reverse Mode Pure Mode Sparsity Pattern Sparsity Structure Function Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Averick, B.M., Carter, R.G., Moré, J.J., Xue, G.-L.: The MINPACK-2 test problem collection. Preprint MCS–P153–0692, ANL/MCS–TM–150, Rev. 1, Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Ill (1992)Google Scholar
  2. 2.
    Bischof, C., Carle, A., Khademi, P., Mauer, A.: ADIFOR 2.0: Automatic differentiation of Fortran 77 programs. IEEE Computational Science & Engineering 3(3), 18–32 (1996)CrossRefGoogle Scholar
  3. 3.
    Bischof, C.H., Khademi, P.M., Bouaricha, A., Carle, A.: Efficient computation of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation. Optimization Methods and Software 7, 1–39 (1997)MATHCrossRefGoogle Scholar
  4. 4.
    Coleman, T.F., Verma, A.: Structure and efficient Jacobian calculation. In: Berz, M., Bischof, C., Corliss, G., Griewank, A. (eds.) Computational Differentiation: Techniques, Applications, and Tools, pp. 149–159. SIAM, Philadelphia (1996)Google Scholar
  5. 5.
    Creusillet, B., Irigoin, F.: Interprocedural Array Region Analysis. Rapport CRI, A-282, Ecole des Mines de Paris, FRANCE (January 1996)Google Scholar
  6. 6.
    Curtis, A.R., Powell, M.J.D., Reid, J.K.: On the estimation of sparse Jacobian matrices. J. Inst. Math. Appl. 13, 117–119 (1974)MATHGoogle Scholar
  7. 7.
    Geitner, U., Utke, J., Griewank, A.: Automatic Computation of Sparse Jacobians by Applying the Method of Newsam and Ramsdell. In: Berz, M., Bischof, C., Corliss, G., Griewank, A. (eds.) Computational Differentiation: Techniques Applications, and Tools, pp. 161–172. SIAM, Philadelphia (1996)Google Scholar
  8. 8.
    Giering, R., Kaminski, T.: Recipes for Adjoint Code Construction. ACM Trans. Math. Software 24(4), 437–474 (1998)MATHCrossRefGoogle Scholar
  9. 9.
    Giering, R., Kaminski, T.: Recomputations in reverse mode AD. In: Corliss, G., Griewank, A., Fauré, C., Hascoet, L., Naumann, U. (eds.) Automatic Differentiation of Algorithms: From Simulation to Optimization, ch. 33, pp. 283–291. Springer, Heidelberg (2002)Google Scholar
  10. 10.
    Giering, R., Kaminski, T.: Applying TAF to generate efficient derivative code of Fortran 77-95 programs. PAMM 2(1), 54–57 (2003)CrossRefGoogle Scholar
  11. 11.
    Giering, R., Kaminski, T., Slawig, T.: Applying TAF to a Navier-Stokes solver that simulates an Euler flow around an airfoil. Future Generation Computer Systems 21(8), 1345–1355 (2005)CrossRefGoogle Scholar
  12. 12.
    Kaminski, T., Giering, R., Scholze, M., Rayner, P., Knorr, W.: An example of an automatic differentiation-based modelling system. In: Kumar, V., Gavrilova, M.L., Tan, C.J.K., L’Ecuyer, P. (eds.) ICCSA 2003. LNCS, vol. 2668, pp. 95–104. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Newsam, G.N., Ramsdell, J.D.: Estimation of sparse Jacobian matrices. SIAM J. Alg. Disc. Meth. 4(3), 404–417 (1983)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Rayner, P., Scholze, M., Knorr, W., Kaminski, T., Giering, R., Widmann, H.: Two decades of terrestrial Carbon fluxes from a Carbon Cycle Data Assimilation System (CCDAS). Global Biogeochemical Cycles 19 (2005), doi:10.1029/2004GB002254Google Scholar
  15. 15.
    Scholze, M.: Model studies on the response of the terrestrial carbon cycle on climate change and variability. Examensarbeit, Max-Planck-Institut für Meteorologie, Hamburg, Germany (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ralf Giering
    • 1
  • Thomas Kaminski
    • 1
  1. 1.FastOptHamburgGermany

Personalised recommendations