Skip to main content
SpringerLink
Log in
Book cover

Mediterranean Conference on Algorithms

MedAlg 2012: Design and Analysis of Algorithms pp 13–36Cite as

Graph Expansion Analysis for Communication Costs of Fast Rectangular Matrix Multiplication

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7659))

Abstract

Graph expansion analysis of computational DAGs is useful for obtaining communication cost lower bounds where previous methods, such as geometric embedding, are not applicable. This has recently been demonstrated for Strassen’s and Strassen-like fast square matrix multiplication algorithms. Here we extend the expansion analysis approach to fast algorithms for rectangular matrix multiplication, obtaining a new class of communication cost lower bounds. These apply, for example to the algorithms of Bini et al. (1979) and the algorithms of Hopcroft and Kerr (1971). Some of our bounds are proved to be optimal.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alon, N., Schwartz, O., Shapira, A.: An elementary construction of constant-degree expanders. Combinatorics, Probability & Computing 17(3), 319–327 (2008)

    MathSciNet  MATH  Google Scholar 

  2. Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O.: Brief announcement: strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds. In: Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2012, pp. 77–79. ACM, New York (2012)

    Chapter  Google Scholar 

  3. Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O.: Communication-optimal parallel algorithm for Strassen’s matrix multiplication. In: Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2012, pp. 193–204. ACM, New York (2012)

    Chapter  Google Scholar 

  4. Ballard, G., Demmel, J., Holtz, O., Schwartz, O.: Minimizing communication in numerical linear algebra. SIAM J. Matrix Analysis Applications 32(3), 866–901 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Ballard, G., Demmel, J., Holtz, O., Schwartz, O.: Graph expansion and communication costs of fast matrix multiplication. J. ACM (accepted, 2012)

    Google Scholar 

  6. Ballard, G., Demmel, J., Lipshitz, B., Schwartz, O.: Communication-avoiding parallel Strassen: Implementation and performance. In: Proceedings of 2012 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2012, ACM, New York (2012)

    Google Scholar 

  7. Beling, P., Megiddo, N.: Using fast matrix multiplication to find basic solutions. Theoretical Computer Science 205(1-2), 307–316 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bilardi, G., Pietracaprina, A., D’Alberto, P.: On the Space and Access Complexity of Computation DAGs. In: Brandes, U., Wagner, D. (eds.) WG 2000. LNCS, vol. 1928, pp. 47–58. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  9. Bilardi, G., Preparata, F.: Processor-time tradeoffs under bounded-speed message propagation: Part II, lower boundes. Theory of Computing Systems 32(5), 1432–4350 (1999)

    Article  MathSciNet  Google Scholar 

  10. Bini, D.: Relations between exact and approximate bilinear algorithms. applications. Calcolo 17, 87–97 (1980), doi:10.1007/BF02575865

    Article  MathSciNet  MATH  Google Scholar 

  11. Bini, D., Capovani, M., Romani, F., Lotti, G.: O(n 2.7799) complexity for n ×n approximate matrix multiplication. Information Processing Letters 8(5), 234–235 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  12. Bűrgisser, P., Clausen, M., Shokrollahi, M.A.: Algebraic Complexity Theory. Grundlehren der mathematischen Wissenschaften, vol. 315. Springer (1997)

    Google Scholar 

  13. Coppersmith, D.: Rapid multiplication of rectangular matrices. SIAM Journal on Computing 11(3), 467–471 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  14. Coppersmith, D.: Rectangular matrix multiplication revisited. J. Complex. 13, 42–49 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  15. Fischer, P., Probert, R.: Efficient Procedures for Using Matrix Algorithms. In: Loeckx, J. (ed.) ICALP 1974. LNCS, vol. 14, pp. 413–427. Springer, Heidelberg (1974)

    Chapter  Google Scholar 

  16. Galil, Z., Pan, V.: Parallel evaluation of the determinant and of the inverse of a matrix. Information Processing Letters 30(1), 41–45 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  17. Hong, J.W., Kung, H.T.: I/O complexity: The red-blue pebble game. In: STOC 1981: Proceedings of the Thirteenth Annual ACM Symposium on Theory of Computing, pp. 326–333. ACM, New York (1981)

    Google Scholar 

  18. Hopcroft, J., Musinski, J.: Duality applied to the complexity of matrix multiplications and other bilinear forms. In: Proceedings of the Fifth Annual ACM Symposium on Theory of Computing, STOC 1973, pp. 73–87. ACM, New York (1973)

    Chapter  Google Scholar 

  19. Hopcroft, J.E., Kerr, L.R.: On minimizing the number of multiplications necessary for matrix multiplication. SIAM Journal on Applied Mathematics 20(1), 30–36 (1971)

    Article  MathSciNet  MATH  Google Scholar 

  20. Huang, X., Pan, V.Y.: Fast rectangular matrix multiplications and improving parallel matrix computations. In: Proceedings of the Second International Symposium on Parallel Symbolic Computation, PASCO 1997, pp. 11–23. ACM, New York (1997)

    Chapter  Google Scholar 

  21. Huang, X., Pan, V.Y.: Fast rectangular matrix multiplication and applications. J. Complex. 14, 257–299 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  22. Irony, D., Toledo, S., Tiskin, A.: Communication lower bounds for distributed-memory matrix multiplication. J. Parallel Distrib. Comput. 64(9), 1017–1026 (2004)

    Article  MATH  Google Scholar 

  23. Kaplan, H., Sharir, M., Verbin, E.: Colored intersection searching via sparse rectangular matrix multiplication. In: Proceedings of the Twenty-Second Annual Symposium on Computational Geometry, SCG 2006, pp. 52–60. ACM, New York (2006)

    Chapter  Google Scholar 

  24. Ke, S., Zeng, B., Han, W., Pan, V.: Fast rectangular matrix multiplication and some applications. Science in China Series A: Mathematics 51, 389–406 (2008), doi:10.1007/s11425-007-0169-2

    Article  MathSciNet  MATH  Google Scholar 

  25. Knight, P.: Fast rectangular matrix multiplication and QR decomposition. Linear Algebra and its Applications 221, 69–81 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  26. Koucky, M., Kabanets, V., Kolokolova, A.: Expanders made elementary (2007) (in preparation), http://www.cs.sfu.ca/~kabanets/papers/expanders.pdf

  27. Kratsch, D., Spinrad, J.: Between O(nm) and O(n)?. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2003, pp. 709–716. Society for Industrial and Applied Mathematics, Philadelphia (2003)

    Google Scholar 

  28. Lev, G., Valiant, L.G.: Size bounds for superconcentrators. Theoretical Computer Science 22(3), 233–251 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lotti, G., Romani, F.: On the asymptotic complexity of rectangular matrix multiplication. Theoretical Computer Science 23(2), 171–185 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  30. Mihail, M.: Conductance and convergence of Markov chains: A combinatorial treatment of expanders. In: Proceedings of the Thirtieth Annual IEEE Symposium on Foundations of Computer Science, pp. 526–531 (1989)

    Google Scholar 

  31. Reingold, O., Vadhan, S., Wigderson, A.: Entropy waves, the zig-zag graph product, and new constant-degree expanders. Annals of Mathematics 155(1), 157–187 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  32. Savage, J.: Space-time tradeoffs in memory hierarchies. Technical report, Brown University, Providence, RI, USA (1994)

    Google Scholar 

  33. Strassen, V.: Gaussian elimination is not optimal. Numer. Math. 13, 354–356 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  34. Yuster, R., Zwick, U.: Detecting short directed cycles using rectangular matrix multiplication and dynamic programming. In: Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2004, pp. 254–260. Society for Industrial and Applied Mathematics, Philadelphia (2004)

    Google Scholar 

  35. Yuster, R., Zwick, U.: Fast sparse matrix multiplication. ACM Trans. Algorithms 1(1), 2–13 (2005)

    Article  MathSciNet  Google Scholar 

  36. Zwick, U.: All pairs shortest paths using bridging sets and rectangular matrix multiplication. J. ACM 49, 289–317 (2002)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. EECS Department, University of California, Berkeley, CA, 94720, US

    Grey Ballard, Benjamin Lipshitz & Oded Schwartz

  2. Mathematics Department and CS Division, University of California, Berkeley, CA, 94720, USA

    James Demmel

  3. Departments of Mathematics, University of California, Berkeley, USA

    Olga Holtz

  4. Technische Universität, Berlin, Germany

    Olga Holtz

Authors
  1. Grey Ballard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Demmel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Olga Holtz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Lipshitz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oded Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Electrical Engineering, Tel-Aviv University, 67789, Tel Aviv, Israel

    Guy Even

  2. School of Electrical Engineering, Tel-Aviv University, 67789, Tel-Aviv, Israel

    Dror Rawitz

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O. (2012). Graph Expansion Analysis for Communication Costs of Fast Rectangular Matrix Multiplication. In: Even, G., Rawitz, D. (eds) Design and Analysis of Algorithms. MedAlg 2012. Lecture Notes in Computer Science, vol 7659. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34862-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-34862-4_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-34861-7

  • Online ISBN: 978-3-642-34862-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation