Abstract
Graph expansion analysis of computational DAGs is useful for obtaining communication cost lower bounds where previous methods, such as geometric embedding, are not applicable. This has recently been demonstrated for Strassen’s and Strassen-like fast square matrix multiplication algorithms. Here we extend the expansion analysis approach to fast algorithms for rectangular matrix multiplication, obtaining a new class of communication cost lower bounds. These apply, for example to the algorithms of Bini et al. (1979) and the algorithms of Hopcroft and Kerr (1971). Some of our bounds are proved to be optimal.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alon, N., Schwartz, O., Shapira, A.: An elementary construction of constant-degree expanders. Combinatorics, Probability & Computing 17(3), 319–327 (2008)
Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O.: Brief announcement: strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds. In: Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2012, pp. 77–79. ACM, New York (2012)
Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O.: Communication-optimal parallel algorithm for Strassen’s matrix multiplication. In: Proceedings of the 24th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA 2012, pp. 193–204. ACM, New York (2012)
Ballard, G., Demmel, J., Holtz, O., Schwartz, O.: Minimizing communication in numerical linear algebra. SIAM J. Matrix Analysis Applications 32(3), 866–901 (2011)
Ballard, G., Demmel, J., Holtz, O., Schwartz, O.: Graph expansion and communication costs of fast matrix multiplication. J. ACM (accepted, 2012)
Ballard, G., Demmel, J., Lipshitz, B., Schwartz, O.: Communication-avoiding parallel Strassen: Implementation and performance. In: Proceedings of 2012 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2012, ACM, New York (2012)
Beling, P., Megiddo, N.: Using fast matrix multiplication to find basic solutions. Theoretical Computer Science 205(1-2), 307–316 (1998)
Bilardi, G., Pietracaprina, A., D’Alberto, P.: On the Space and Access Complexity of Computation DAGs. In: Brandes, U., Wagner, D. (eds.) WG 2000. LNCS, vol. 1928, pp. 47–58. Springer, Heidelberg (2000)
Bilardi, G., Preparata, F.: Processor-time tradeoffs under bounded-speed message propagation: Part II, lower boundes. Theory of Computing Systems 32(5), 1432–4350 (1999)
Bini, D.: Relations between exact and approximate bilinear algorithms. applications. Calcolo 17, 87–97 (1980), doi:10.1007/BF02575865
Bini, D., Capovani, M., Romani, F., Lotti, G.: O(n 2.7799) complexity for n ×n approximate matrix multiplication. Information Processing Letters 8(5), 234–235 (1979)
Bűrgisser, P., Clausen, M., Shokrollahi, M.A.: Algebraic Complexity Theory. Grundlehren der mathematischen Wissenschaften, vol. 315. Springer (1997)
Coppersmith, D.: Rapid multiplication of rectangular matrices. SIAM Journal on Computing 11(3), 467–471 (1982)
Coppersmith, D.: Rectangular matrix multiplication revisited. J. Complex. 13, 42–49 (1997)
Fischer, P., Probert, R.: Efficient Procedures for Using Matrix Algorithms. In: Loeckx, J. (ed.) ICALP 1974. LNCS, vol. 14, pp. 413–427. Springer, Heidelberg (1974)
Galil, Z., Pan, V.: Parallel evaluation of the determinant and of the inverse of a matrix. Information Processing Letters 30(1), 41–45 (1989)
Hong, J.W., Kung, H.T.: I/O complexity: The red-blue pebble game. In: STOC 1981: Proceedings of the Thirteenth Annual ACM Symposium on Theory of Computing, pp. 326–333. ACM, New York (1981)
Hopcroft, J., Musinski, J.: Duality applied to the complexity of matrix multiplications and other bilinear forms. In: Proceedings of the Fifth Annual ACM Symposium on Theory of Computing, STOC 1973, pp. 73–87. ACM, New York (1973)
Hopcroft, J.E., Kerr, L.R.: On minimizing the number of multiplications necessary for matrix multiplication. SIAM Journal on Applied Mathematics 20(1), 30–36 (1971)
Huang, X., Pan, V.Y.: Fast rectangular matrix multiplications and improving parallel matrix computations. In: Proceedings of the Second International Symposium on Parallel Symbolic Computation, PASCO 1997, pp. 11–23. ACM, New York (1997)
Huang, X., Pan, V.Y.: Fast rectangular matrix multiplication and applications. J. Complex. 14, 257–299 (1998)
Irony, D., Toledo, S., Tiskin, A.: Communication lower bounds for distributed-memory matrix multiplication. J. Parallel Distrib. Comput. 64(9), 1017–1026 (2004)
Kaplan, H., Sharir, M., Verbin, E.: Colored intersection searching via sparse rectangular matrix multiplication. In: Proceedings of the Twenty-Second Annual Symposium on Computational Geometry, SCG 2006, pp. 52–60. ACM, New York (2006)
Ke, S., Zeng, B., Han, W., Pan, V.: Fast rectangular matrix multiplication and some applications. Science in China Series A: Mathematics 51, 389–406 (2008), doi:10.1007/s11425-007-0169-2
Knight, P.: Fast rectangular matrix multiplication and QR decomposition. Linear Algebra and its Applications 221, 69–81 (1995)
Koucky, M., Kabanets, V., Kolokolova, A.: Expanders made elementary (2007) (in preparation), http://www.cs.sfu.ca/~kabanets/papers/expanders.pdf
Kratsch, D., Spinrad, J.: Between O(nm) and O(n)?. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2003, pp. 709–716. Society for Industrial and Applied Mathematics, Philadelphia (2003)
Lev, G., Valiant, L.G.: Size bounds for superconcentrators. Theoretical Computer Science 22(3), 233–251 (1983)
Lotti, G., Romani, F.: On the asymptotic complexity of rectangular matrix multiplication. Theoretical Computer Science 23(2), 171–185 (1983)
Mihail, M.: Conductance and convergence of Markov chains: A combinatorial treatment of expanders. In: Proceedings of the Thirtieth Annual IEEE Symposium on Foundations of Computer Science, pp. 526–531 (1989)
Reingold, O., Vadhan, S., Wigderson, A.: Entropy waves, the zig-zag graph product, and new constant-degree expanders. Annals of Mathematics 155(1), 157–187 (2002)
Savage, J.: Space-time tradeoffs in memory hierarchies. Technical report, Brown University, Providence, RI, USA (1994)
Strassen, V.: Gaussian elimination is not optimal. Numer. Math. 13, 354–356 (1969)
Yuster, R., Zwick, U.: Detecting short directed cycles using rectangular matrix multiplication and dynamic programming. In: Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2004, pp. 254–260. Society for Industrial and Applied Mathematics, Philadelphia (2004)
Yuster, R., Zwick, U.: Fast sparse matrix multiplication. ACM Trans. Algorithms 1(1), 2–13 (2005)
Zwick, U.: All pairs shortest paths using bridging sets and rectangular matrix multiplication. J. ACM 49, 289–317 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ballard, G., Demmel, J., Holtz, O., Lipshitz, B., Schwartz, O. (2012). Graph Expansion Analysis for Communication Costs of Fast Rectangular Matrix Multiplication. In: Even, G., Rawitz, D. (eds) Design and Analysis of Algorithms. MedAlg 2012. Lecture Notes in Computer Science, vol 7659. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34862-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-34862-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34861-7
Online ISBN: 978-3-642-34862-4
eBook Packages: Computer ScienceComputer Science (R0)