Skip to main content
SpringerLink
Log in

Fast matrix multiplication is stable

  • Published:
Numerische Mathematik Aims and scope Submit manuscript

Abstract

We perform forward error analysis for a large class of recursive matrix multiplication algorithms in the spirit of Bini and Lotti [Numer. Math. 36:63–72, 1980]. As a consequence of our analysis, we show that the exponent of matrix multiplication (the optimal running time) can be achieved by numerically stable algorithms. We also show that new group-theoretic algorithms proposed in Cohn and Umans [Foundations of Computer Science, 44th Annual IEEE Symposium, pp. 438–449, 2003] and Cohn et al. [Foundations of Computer Science, 46th Annual IEEE Symposium, pp. 379–388, 2005] are all included in the class of algorithms to which our analysis applies, and are therefore numerically stable. We perform detailed error analysis for three specific fast group-theoretic algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Anderson E., Bai Z., Bischof C., Demmel J., Dongarra J., Du Croz J., Greenbaum A., Hammarling S., McKenney A., Blackford S. and Sorensen D. (1999). LAPACK Users’ Guide. SIAM, Philadelphia

    Google Scholar 

  2. Bini D. and Lotti D. (1980). Stability of fast algorithms for matrix multiplication. Numer. Math. 36: 63–72

    Article  MATH  MathSciNet  Google Scholar 

  3. Blackford L.S., Choi J., Cleary A., D’Azevedo E., Demmel J., Dhillon I., Dongarra J., Hammarling S., Henry G., Petitet A., Stanley K., Walker D. and Whaley R.C. (1997). ScaLAPACK Users’ Guide. SIAM, Philadelphia

    MATH  Google Scholar 

  4. Brent, R.P.: Algorithms for matrix multiplication. Report CS 157, Computer Science Department, Stanford University (1970)

  5. Brockett R.W. and Dobkin D. (1978). On the optimal evaluation of a set of bilinear forms. Linear Algebra Appl. 19(3): 207–235

    Article  MATH  MathSciNet  Google Scholar 

  6. Bürgisser, P., Clausen, M., Shokrollahi, M.A.: Algebraic complexity theory. In: der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 315. Springer, Berlin (1997)

  7. Cohn, H., Kleinberg, R., Szegedy, B., Umans, C.: Group-theoretic algorithms for matrix multiplication. In: Foundations of Computer Science. 46th Annual IEEE Symposium, 23–25 October 2005, pp. 379–388 (2005)

  8. Cohn, H., Kleinberg, R., Szegedy, B., Umans, C.: Implementing group-theoretic algorithms for matrix multiplication using the abelian discrete Fourier transform (in preparation)

  9. Cohn, H., Umans, C.: A group-theoretic approach to matrix multiplication. In: Foundations of Computer Science. 44th Annual IEEE Symposium, pp. 438–449 (2003)

  10. Coppersmith D. and Winograd S. (1990). Matrix multiplication via arithmetic progressions. J. Symbolic Comput. 9(3): 251–280

    Article  MATH  MathSciNet  Google Scholar 

  11. Demmel, J., Dumitriu, I., Holtz, O.: Fast linear algebra is stable (2006, in preparation)

  12. Demmel J. and Higham N.J. (1992). Stability of block algorithms with fast level 3 BLAS. ACM Trans. Math. Soft. 18: 274–291

    Article  MATH  Google Scholar 

  13. Higham N.J. (1990). Exploiting fast matrix multiplication within the level 3 BLAS. ACM Trans. Math. Soft. 16: 352–368

    Article  MATH  MathSciNet  Google Scholar 

  14. Higham N.J. (2002). Accuracy and Stability of Numerical Algorithms. SIAM, Philadelphia

    MATH  Google Scholar 

  15. Maslen D.K. and Rockmore D.N. (2001). The Cooley–Tukey FFT and group theory. Not. Am. Math. Soc. 48(10): 1151–1160

    MathSciNet  MATH  Google Scholar 

  16. Miller W. (1975). Computational complexity and numerical stability. SIAM J. Comput. 4(2): 97–107

    Article  MATH  MathSciNet  Google Scholar 

  17. Raz R. (2003). On the complexity of matrix product. SIAM J. Comput. 32(5): 1356–1369. electronic

    Article  MATH  MathSciNet  Google Scholar 

  18. Schönhage A. and Strassen V. (1971). Schnelle Multiplikation grosser Zahlen. Computing (Arch. Elektron. Rechnen) 7: 281–292

    MATH  MathSciNet  Google Scholar 

  19. Strassen V. (1969). Gaussian elimination is not optimal. Numer. Math. 13: 354–356

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mathematics Department, University of California, Berkeley, CA, 94720, USA

    James Demmel, Ioana Dumitriu & Olga Holtz

  2. Computer Science Division, University of California, Berkeley, CA, 94720, USA

    James Demmel & Robert Kleinberg

Authors
  1. James Demmel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioana Dumitriu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Olga Holtz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Kleinberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Olga Holtz.

Additional information

J. Demmel acknowledges support of NSF under grants CCF-0444486, ACI-00090127, CNS-0325873 and of DOE under grant DE-FC02-01ER25478.

I. Dumitriu acknowledges support of the Miller Institute for Basic Research in Science.

R. Kleinberg is supported by an NSF Mathematical Sciences Postdoctoral Research Fellowship

Rights and permissions

Reprints and permissions

About this article

Cite this article

Demmel, J., Dumitriu, I., Holtz, O. et al. Fast matrix multiplication is stable. Numer. Math. 106, 199–224 (2007). https://doi.org/10.1007/s00211-007-0061-6

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00211-007-0061-6

Keywords

Search

Navigation