Skip to main content

The Input/Output Complexity of Sparse Matrix Multiplication

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 8737)


We consider the problem of multiplying sparse matrices (over a semiring) where the number of non-zero entries is larger than main memory. In the classical paper of Hong and Kung (STOC ’81) it was shown that to compute a product of dense U ×U matrices, \(\Theta \left( U^3 / (B \sqrt{M}) \right)\) I/Os are necessary and sufficient in the I/O model with internal memory size M and memory block size B.

In this paper we generalize the upper and lower bounds of Hong and Kung to the sparse case. Our bounds depend of the number N = nnz(A)+nnz(C) of nonzero entries in A and C, as well as the number Z =nnz(AC) of nonzero entries in AC.

We show that using \(\tilde{O} \left( \tfrac{N}{B} \min\left(\sqrt{\tfrac{Z}{M}},\tfrac{N}{M}\right) \right)\) I/Os, AC can be computed with high probability. This is tight (up to polylogarithmic factors) when only semiring operations are allowed, even for dense rectangular matrices: We show a lower bound of \(\Omega \left( \tfrac{N}{B} \min\left(\sqrt{\tfrac{Z}{M}},\tfrac{N}{M}\right) \right)\) I/Os.

While our lower bound uses fairly standard techniques, the upper bound makes use of “compressed matrix multiplication” sketches, which is new in the context of I/O-efficient algorithms, and a new matrix product size estimation technique that avoids the “no cancellation” assumption.


  • Matrix Multiplication
  • Nonzero Entry
  • Sparse Matrix
  • Elementary Product
  • Matrix Multiplication Algorithm

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This work is supported by the Danish National Research Foundation under the Sapere Aude program.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Aggarwal, A., Vitter, J.S.: The Input/Output complexity of sorting and related problems. Commun. ACM 31(9), 1116–1127 (1988)

    CrossRef  MathSciNet  Google Scholar 

  2. Alon, N., Yuster, R., Zwick, U.: Color-coding. J. ACM (4), 844–856

    Google Scholar 

  3. Alon, N., Yuster, R., Zwick, U.: Finding and counting given length cycles. Algorithmica 17(3), 209–223 (1997)

    CrossRef  MATH  MathSciNet  Google Scholar 

  4. Amossen, R.R., Campagna, A., Pagh, R.: Better size estimation for sparse matrix products. APPROX/RANDOM 2010, 406–419 (2010)

    Google Scholar 

  5. Amossen, R.R., Pagh, R.: Faster join-projects and sparse matrix multiplications. In: ICDT 2009, pp. 121–126. ACM (2009)

    Google Scholar 

  6. Bender, M., Brodal, G., Fagerberg, R., Jacob, R., Vicari, E.: Optimal sparse matrix dense vector multiplication in the I/O-model. TCS 47(4), 934–962 (2010)

    MATH  MathSciNet  Google Scholar 

  7. Boyer, R.S., Moore, J.S.: MJRTY - A fast majority vote algorithm. Technical Report AI81-32 (February 1, 1981)

    Google Scholar 

  8. Cohen, E.: Estimating the size of the transitive closure in linear time. In: SFCS 1994, pp. 190–200. IEEE Computer Society, Washington, DC (1994)

    Google Scholar 

  9. Cohen, E.: Structure prediction and computation of sparse matrix products. Journal of Combinatorial Optimization 2(4), 307–332 (1998)

    CrossRef  Google Scholar 

  10. Demaine, E.D.: Cache-oblivious algorithms and data structures. In: Lecture Notes from the EEF Summer School on Massive Data Sets, June 27-July 1 (2002)

    Google Scholar 

  11. Flajolet, P., Martin, G.N.: Probabilistic Counting Algorithms for Data Base Applications. Journal of Computer and System Sciences

    Google Scholar 

  12. Greiner, G., Jacob, R.: The I/O complexity of sparse matrix dense matrix multiplication. In: López-Ortiz, A. (ed.) LATIN 2010. LNCS, vol. 6034, pp. 143–156. Springer, Heidelberg (2010)

    Google Scholar 

  13. Irony, D., Toledo, S., Tiskin, A.: Communication lower bounds for distributed-memory matrix multiplication. JPDC 64(9), 1017–1026 (2004)

    MATH  Google Scholar 

  14. Jia-Wei, H., Kung, H.T.: I/O complexity: The red-blue pebble game. In: STOC 1981, pp. 326–333. ACM (1981)

    Google Scholar 

  15. Kane, D.M., Nelson, J., Woodruff, D.P.: An optimal algorithm for the distinct elements problem. In: PODS 2010, pp. 41–52. ACM (2010)

    Google Scholar 

  16. McGregor, A.: Algorithms for Signals, book draft (2013)

    Google Scholar 

  17. Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, New York (1995)

    CrossRef  MATH  Google Scholar 

  18. Mulmuley, K., Vazirani, U., Vazirani, V.: Matching is as easy as matrix inversion. Combinatorica 7(1), 105–113 (1987)

    CrossRef  MATH  MathSciNet  Google Scholar 

  19. Pagh, R.: Compressed matrix multiplication. ACM Trans. Comput. Theory 5(3), 9:1–9:17 (2013)

    Google Scholar 

  20. Pagh, R., Silvestri, F.: The input/output complexity of triangle enumeration. arXiv preprint arXiv:1312.0723 (2013)

    Google Scholar 

  21. Pietracaprina, A., Pucci, G., Riondato, M., Silvestri, F., Upfal, E.: Space-round tradeoffs for mapreduce computations. In: ICS, pp. 235–244. ACM (2012)

    Google Scholar 

  22. Rabin, M.O., Vazirani, V.V.: Maximum matchings in general graphs through randomization. J. Algorithms 10(4), 557–567 (1989)

    CrossRef  MATH  MathSciNet  Google Scholar 

  23. Strassen, V.: Gaussian elimination is not optimal. Numerische Mathematik 13(4), 354–356 (1969)

    CrossRef  MATH  MathSciNet  Google Scholar 

  24. van Dongen, S.: Graph Clustering by Flow Simulation. PhD thesis, University of Utrecht (2000)

    Google Scholar 

  25. Williams, R., Yu, H.: Finding orthogonal vectors in discrete structures. In: SODA 2014, ch. 135, pp. 1867–1877 (2014)

    Google Scholar 

  26. Williams, V.V.: Multiplying matrices faster than coppersmith-winograd. In: STOC 2012, pp. 887–898. ACM, New York (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Pagh, R., Stöckel, M. (2014). The Input/Output Complexity of Sparse Matrix Multiplication. In: Schulz, A.S., Wagner, D. (eds) Algorithms - ESA 2014. ESA 2014. Lecture Notes in Computer Science, vol 8737. Springer, Berlin, Heidelberg.

Download citation

  • DOI:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44776-5

  • Online ISBN: 978-3-662-44777-2

  • eBook Packages: Computer ScienceComputer Science (R0)