Theory of Computing Systems

, Volume 46, Issue 2, pp 301–310 | Cite as

Entropy of Operators or why Matrix Multiplication is Hard for Depth-Two Circuits



We consider unbounded fanin depth-2 circuits with arbitrary boolean functions as gates. We define the entropy of an operator f:{0,1} n →{0,1} m as the logarithm of the maximum number of vectors distinguishable by at least one special subfunction of f.

Our main result is that every depth-2 circuit for f requires at least entropy(f) wires. This is reminiscent of a classical lower bound of Nechiporuk on the formula size, and gives an information-theoretic explanation of why some operators require many wires. We use this to prove a tight estimate Θ(n 3) of the smallest number of wires in any depth-2 circuit computing the product of two n by n matrices over any finite field. Previously known lower bound for this operator was Ω(n 2log n).


Boolean circuits Bilinear forms Matrix multiplication Entropy 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alon, N., Karchmer, M., Wigderson, A.: Linear circuits over GF(2). SIAM. J. Comput. 19(6), 1064–1067 (1990) MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Alon, N., Pudlák, P.: Superconcentrators of depth 2 and 3; odd levels help (rarely). J. Comput. Syst. Sci. 48, 194–202 (1994) MATHCrossRefGoogle Scholar
  3. 3.
    Beigel, R., Tarui, J.: On ACC. Comput. Complex. 4, 350–366 (1994) MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Bshouty, N.H.: A lower bound for matrix multiplication. SIAM J. Comput. 18, 759–765 (1982) CrossRefMathSciNetGoogle Scholar
  5. 5.
    Bürgisser, P., Lotz, M.: Lower bounds on the bounded coefficient complexity of bilinear maps. J. ACM 51(3), 464–482 (2004) MathSciNetGoogle Scholar
  6. 6.
    Cherukhin, D.Y.: The lower estimate of complexity in the class of schemes of depth 2 without restrictions on a basis. Moscow University Math. Bull. 60(4), 42–44 (2005) MathSciNetGoogle Scholar
  7. 7.
    Coppersmith, D., Winograd, S.: Matrix multiplications via arithmetic progressions. J. Symb. Comput. 9, 251–280 (1990) MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Dolev, D., Dwork, C., Pippenger, N., Wigderson, A.: Superconcentrators, generalizer and generalized connectors with limited depth. In: Proc. of the 15th STOC, pp. 42–51 (1983) Google Scholar
  9. 9.
    Friedman, J.: A note on matrix rigidity. Combinatorica 13, 235–239 (1993) MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Jukna, S.: On set intersection representations of graphs. J. Graph Theory (submitted) Google Scholar
  11. 11.
    Morgenstern, J.: Note on a lower bound on the linear complexity of fast Fourier transform. J. ACM 20(2), 305–306 (1973) MATHMathSciNetGoogle Scholar
  12. 12.
    Morgenstern, J.: The linear complexity of computation. J. ACM 22(2), 184–194 (1975) MATHGoogle Scholar
  13. 13.
    Nechiporuk, E.I.: On a Boolean function. Sov. Math. Dokl. 7(4), 999–1000 (1966) MATHGoogle Scholar
  14. 14.
    Pippenger, N.: Superconcentrators. SIAM J. Comput. 6, 298–304 (1977) MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Pippenger, N.: Superconcentrators of depth 2. J. Comput. Syst. Sci. 24, 82–90 (1982) MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Pudlák, P.: Communication in bounded depth circuits. Combinatorica 14(2), 203–216 (1994) MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Pudlák, P.: A note on the use of determinant for proving lower bounds on the size of linear circuits. Inf. Process. Lett. 74, 197–201 (2000) CrossRefGoogle Scholar
  18. 18.
    Pudlák, P., Rödl, V.: Some combinatorial-algebraic problems from complexity theory. Discrete Math. 136, 253–279 (1994) MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Pudlák, P., Rödl, V., Sgall, J.: Boolean circuits, tensor ranks, and communication complexity. SIAM J. Comput. 26(3), 605–633 (1997) MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Pudlák, P., Savický, P.: On shifting networks. Theor. Comput. Sci. 116, 415–419 (1993) MATHCrossRefGoogle Scholar
  21. 21.
    Radhakrishnan, J., Ta-Shma, A.: Bounds for dispersers, extractors, and depth-two superconcentrators. SIAM J. Discrete Math. 13(1), 2–24 (2000) MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Raz, R., Shpilka, A.: Lower bounds for matrix product in bounded depth circuits with arbitrary gates. SIAM J. Comput. 32(2), 488–513 (2003) MATHCrossRefMathSciNetGoogle Scholar
  23. 23.
    Shoup, V., Smolensky, R.: Lower bounds for polynomial evaluation and interpolation problems. Comput. Complex. 6(4), 301–311 (1997) CrossRefMathSciNetGoogle Scholar
  24. 24.
    Strassen, V.: Die Berechnungskomplexität von elementarsymmetrischen Funktionen und von Interpoliationskoefizienten. Numer. Math. 20, 238–251 (1973) MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Valiant, L.: Graph-theoretic methods in low-level complexity. In: Proc. of the 6th MFCS. Lecture Notes in Computer Science, vol. 53, pp. 162–176. Springer, Berlin (1977) Google Scholar
  26. 26.
    Yao, A.C.: On ACC and threshold circuits. In: Proc. of the 31th FOCS, pp. 619–627 (1990) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Institute of Mathematics and InformaticsVilniusLithuania

Personalised recommendations