Linearity Analysis for Automatic Differentiation

  • Michelle Mills Strout
  • Paul Hovland
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3994)


Linearity analysis determines which variables depend on which other variables and whether the dependence is linear or nonlinear. One of the many applications of this analysis is determining whether a loop involves only linear loop-carried dependences and therefore the adjoint of the loop may be reversed and fused with the computation of the original function. This paper specifies the data-flow equations that compute linearity analysis. In addition, the paper describes using linearity analysis with array dependence analysis to determine whether a loop-carried dependence is linear or nonlinear.


Linearity Analysis Basic Block Nonlinear Dependence Variable Pair Dependence Class 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Averick, B.M., Moré, J.J., Bischof, C.H., Carle, A., Griewank, A.: Computing large sparse Jacobian matrices using automatic differentiation. SIAM J. Sci. Comput. 15(2), 285–294 (1994)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Banerjee, U.: Dependence analysis for supercomputing. The Kluwer international series in engineering and computer science. Parallel processing and fifth generation computing. Kluwer Academic, Boston (1988)Google Scholar
  3. 3.
    Cousot, P., Halbwachs, N.: Automatic discovery of linear restraints among variables of a program. In: POPL 1978: Proceedings of the 5th ACM SIGACT-SIGPLAN symposium on Principles of programming languages, pp. 84–96. ACM Press, New York (1978)CrossRefGoogle Scholar
  4. 4.
    Feautrier, P.: Dataflow analysis of array and scalar references. International Journal of Parallel Programming 20(1) (February 1991)Google Scholar
  5. 5.
    Gould, N., Toint, P.L.: Preprocessing for quadratic programming. Math. Programming 100(1), 95–132 (2004)MATHMathSciNetGoogle Scholar
  6. 6.
    Hascoët, L., Fidanova, S., Held, C.: Adjoining independent computations. In: Corliss, G., Faure, C., Griewank, A., Hascoët, L., Naumann, U. (eds.) Automatic Differentiation of Algorithms: From Simulation to Optimization, Computer and Information Science, ch. 35, pp. 299–304. Springer, New York (2001)Google Scholar
  7. 7.
    Hascoët, L., Naumann, U., Pascual, V.: To be recorded analysis in reverse-mode automatic differentiation. Future Generation Computer Systems 21(8) (2005)Google Scholar
  8. 8.
    Kaminski, T., Giering, R., Voßbeck, M.: Efficient sensitivities for the spin-up phase. In: Bücker, H.M., Corliss, G., Hovland, P., Naumann, U., Norris, B. (eds.) Automatic Differentiation: Applications, Theory, and Tools. Lecture Notes in Computational Science and Engineering. Springer, Heidelberg (2005)Google Scholar
  9. 9.
    Karr, M.: Affine relationships among variables of a program. Acta Informatica 6(2), 133–151 (1976)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Kildall, G.A.: A unified approach to global program optimization. In: ACM Symposium on Principles of Programming Languages, October 1973, pp. 194–206 (1973)Google Scholar
  11. 11.
    Kirby, R., Scott, R.: Personal communication (2004)Google Scholar
  12. 12.
    Müller-Olm, M., Seidl, H.: Precise interprocedural analysis through linear algebra. In: POPL 2004: Proceedings of the 31st ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 330–341. ACM Press, New York (2004)CrossRefGoogle Scholar
  13. 13.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Heidelberg (1999)MATHCrossRefGoogle Scholar
  14. 14.
    Pugh, W.: Omega test: A practical algorithm for exact array dependency analysis. Comm. of the ACM 35(8), 102 (1992)CrossRefGoogle Scholar
  15. 15.
    Rodriguez-Carbonell, E., Kapur, D.: Automatic generation of polynomial loop invariants: Algebraic foundations. In: ISSAC 2004: Proceedings of the 2004 international symposium on Symbolic and algebraic computation, pp. 266–273. ACM Press, New York (2004)CrossRefGoogle Scholar
  16. 16.
    Strout, M.M., Mellor-Crummey, J., Hovland, P.: Representation-independent program analysis. In: Proceedings of the Sixth ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (2005)Google Scholar
  17. 17.
    Weiser, M.: Program slicing. IEEE Trans. Software Eng. 10(4), 352–357 (1984)CrossRefGoogle Scholar
  18. 18.
    Wolfe, M., Tseng, C.W.: The power test for data dependence. IEEE Trans. Parallel Distrib. Syst. 3(5), 591–601 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Michelle Mills Strout
    • 1
  • Paul Hovland
    • 2
  1. 1.Colorado State UniversityFort Collins
  2. 2.Argonne National LaboratoryArgonne

Personalised recommendations