Summary
This paper collects together a number of matrix derivative results which are very useful in forward and reverse mode algorithmic differentiation. It highlights in particular the remarkable contribution of a 1948 paper by Dwyer and Macphail which derives the linear and adjoint sensitivities of a matrix product, inverse and determinant, and a number of related results motivated by applications in multivariate analysis in statistics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bischof, C., Bücker, H., Lang, B., Rasch, A., Vehreschild, A.: Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs. In: Proceedings of the Second IEEE International Workshop on Source Code Analysis and Manipulation (SCAM 2002), pp. 65–72. IEEE Computer Society (2002)
Coleman, T., Verma, A.: ADMIT-1: Automatic differentiation and MATLAB interface toolbox. ACM Transactions on Mathematical Software 26(1), 150–175 (2000)
Dwyer, P.: Some applications of matrix derivatives in multivariate analysis. Journal of the American Statistical Association 62(318), 607–625 (1967)
Dwyer, P., Macphail, M.: Symbolic matrix derivatives. The Annals of Mathematical Statistics 19(4), 517–534 (1948)
Forth, S.: An efficient overloaded implementation of forward mode automatic differentiation in MATLAB. ACM Transactions on Mathematical Software 32(2), 195–222 (2006)
Giles, M.: An extended collection of matrix derivative results for forward and reverse mode algorithmic differentiation. Tech. Rep. NA08/01, Oxford University Computing Laboratory (2008)
Glasserman, P.: Monte Carlo Methods in Financial Engineering. Springer-Verlag, New York (2004)
Griewank, A.: Evaluating derivatives: Principles and techniques of algorithmic differentiation. SIAM (2000)
Higham, N.: The scaling and squaring method for the matrix exponential revisited. SIAM Journal on Matrix Analysis and Applications 26(4), 1179–1193 (2005)
Kubota, K.: Matrix inversion algorithms by means of automatic differentiation. Applied Mathematics Letters 7(4), 19–22 (1994)
Magnus, J., Neudecker, H.: Matrix differential calculus with applications in statistics and econometrics. John Wiley & Sons (1988)
Mathai, A.: Jacobians of matrix transformations and functions of matrix argument. World Scientific, New York (1997)
Minka, T.: Old and new matrix algebra useful for statistics. http://research.microsoft.com/~minka/papers/matrix/ (2000)
Rao, C.: Linear statistical inference and its applications. Wiley, New York (1973)
Rogers, G.: Matrix derivatives. Marcel Dekker, New York (1980)
Squire, W., Trapp, G.: Using complex variables to estimate derivatives of real functions. SIAM Review 10(1), 110–112 (1998)
Srivastava, M., Khatri, C.: An introduction to multivariate statistics. North Holland, New York (1979)
Verma, A.: ADMAT: automatic differentiation in MATLAB using object oriented methods. In: SIAM Interdiscplinary Workshop on Object Oriented Methods for Interoperability, pp. 174–183. SIAM (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Giles, M.B. (2008). Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation. In: Bischof, C.H., Bücker, H.M., Hovland, P., Naumann, U., Utke, J. (eds) Advances in Automatic Differentiation. Lecture Notes in Computational Science and Engineering, vol 64. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68942-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-68942-3_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68935-5
Online ISBN: 978-3-540-68942-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)