Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation

  • Mike B. Giles
Part of the Lecture Notes in Computational Science and Engineering book series (LNCSE, volume 64)


This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation. It highlights in particular the remarkable contribution of a 1948 paper by Dwyer and Macphail which derives the linear and adjoint sensitivities of a matrix product, inverse and determinant, and a number of related results motivated by applications in multivariate analysis in statistics.


Forward mode reverse mode numerical linear algebra 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bischof, C., Bücker, H., Lang, B., Rasch, A., Vehreschild, A.: Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs. In: Proceedings of the Second IEEE International Workshop on Source Code Analysis and Manipulation (SCAM 2002), pp. 65–72. IEEE Computer Society (2002)Google Scholar
  2. 2.
    Coleman, T., Verma, A.: ADMIT-1: Automatic differentiation and MATLAB interface toolbox. ACM Transactions on Mathematical Software 26(1), 150–175 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Dwyer, P.: Some applications of matrix derivatives in multivariate analysis. Journal of the American Statistical Association 62(318), 607–625 (1967)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Dwyer, P., Macphail, M.: Symbolic matrix derivatives. The Annals of Mathematical Statistics 19(4), 517–534 (1948)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Forth, S.: An efficient overloaded implementation of forward mode automatic differentiation in MATLAB. ACM Transactions on Mathematical Software 32(2), 195–222 (2006)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Giles, M.: An extended collection of matrix derivative results for forward and reverse mode algorithmic differentiation. Tech. Rep. NA08/01, Oxford University Computing Laboratory (2008)Google Scholar
  7. 7.
    Glasserman, P.: Monte Carlo Methods in Financial Engineering. Springer-Verlag, New York (2004)zbMATHGoogle Scholar
  8. 8.
    Griewank, A.: Evaluating derivatives: Principles and techniques of algorithmic differentiation. SIAM (2000)Google Scholar
  9. 9.
    Higham, N.: The scaling and squaring method for the matrix exponential revisited. SIAM Journal on Matrix Analysis and Applications 26(4), 1179–1193 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Kubota, K.: Matrix inversion algorithms by means of automatic differentiation. Applied Mathematics Letters 7(4), 19–22 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Magnus, J., Neudecker, H.: Matrix differential calculus with applications in statistics and econometrics. John Wiley & Sons (1988)Google Scholar
  12. 12.
    Mathai, A.: Jacobians of matrix transformations and functions of matrix argument. World Scientific, New York (1997)zbMATHGoogle Scholar
  13. 13.
    Minka, T.: Old and new matrix algebra useful for statistics. (2000)
  14. 14.
    Rao, C.: Linear statistical inference and its applications. Wiley, New York (1973)zbMATHCrossRefGoogle Scholar
  15. 15.
    Rogers, G.: Matrix derivatives. Marcel Dekker, New York (1980)zbMATHGoogle Scholar
  16. 16.
    Squire, W., Trapp, G.: Using complex variables to estimate derivatives of real functions. SIAM Review 10(1), 110–112 (1998)CrossRefMathSciNetGoogle Scholar
  17. 17.
    Srivastava, M., Khatri, C.: An introduction to multivariate statistics. North Holland, New York (1979)zbMATHGoogle Scholar
  18. 18.
    Verma, A.: ADMAT: automatic differentiation in MATLAB using object oriented methods. In: SIAM Interdiscplinary Workshop on Object Oriented Methods for Interoperability, pp. 174–183. SIAM (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Mike B. Giles
    • 1
  1. 1.Oxford University Computing LaboratoryOxfordUK

Personalised recommendations