Abstract
The study of threshold circuits is interesting for several reasons. First, constant-depth threshold circuits are closely related to feed-forward neural networks, a widely studied model. The main difference is that while a threshold gate has Boolean inputs and output, a “neuron” in a neural network can have real numbers as inputs and typically outputs a continuous approximation of a step-function. However, Maass, Schnitger, and Sontag show in [MSS91] that when one considers the computation of Boolean functions the two models are equally powerful (within polynomial factors) (see also Chapter 4).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
N. Alon and J. Bruck. Explicit constructions of depth-2 majority circuits for comparison and addition. Technical Report RJ 8300 (75661), IBM Research Division, August 1991. To appear in SIAM J. Disc. Math.
A. E Andreev. On a method for obtaining more than quadratic effective lower bounds for the complexity of π-schemes. Moscow Univ. Math. Bull., 42(1):63–66, 1987.
N. Alon, J. Spencer, and P. ErdHos. The Probabilistic Method. John Wiley & Sons, Inc., 1992.
L. Babai, P. Franki, and J. Simon. Complexity classes in communication complexity theory. In Proc. 27th IEEE Symposium on Foundations of Computer Science, pages 337–347, 1986.
L. Babai, N. Nisan, and M. Szegedy. Multiparty protocols and logspace-hard pseudorandom sequences. In Proc. 21st ACM Symposium on Theory of Computing, pages 1–11, 1989.
J. Bruck. Harmonic analysis of polynomial threshold functions. SIAM J. Disc. Math., 3(2):168–177, May 1990.
R. Beigel and J. Tarui. On ACC. In Proc. 32nd IEEE Symposium on Foundations of Computer Science, pages 783–792, 1991.
B. Chor and O. Goldreich. Unbiased bits from sources of weak randomness and probabilistic communication complexity. In Proc. 26th IEEE Symposium on Foundations of Computer Science, pages 429–442, 1985.
A. Chandra, L. Stockmeyer, and U. Vishkin. Constant depth reducibility. SIAM J. Computing, 13:423–439, 1984.
U. Feige, P. Peleg, P. Raghavan, and E. Upfal. Computing with unreliable information. In Proc. 22nd ACM Symposium on Theory of Computing, pages 128–137, 1990.
Y. Freund. Boosting a weak learning algorithm by majority. In Workshop on Computational Learning Theory, pages 202–216, 1990.
M. Furst, J. Saxe, and M. Sipser. Parity, circuits, and the polynomial time hierarchy. Math. System Theory, 17:13–27, 1984.
M. Goldmann and J. Håst ad. A simple lower bound for monotone clique using a communication game. Information Processing Letters, 41(4):221–226, 1992.
M. Goldmann, J. Håstad, and A. Razborov. Majority gates vs. general weighted threshold gates. In Proc. 7th Annual Structure in Complexity Theory Conference, pages 2–13, 1992. To appear in Computational Complexity.
M. Goldmann and M. Karpinski. Simulating threshold circuits by majority circuits. In Proc. 25th ACM Symposium on Theory of Computing, pages 551–560, 1993.
M. Goldmann. On the power of a threshold gate at the top. Manuscript, 1993.
F. Green. An oracle separating ⊕P from PP PH. In Proc. 5th Annual Structure in Complexity Theory Conference, pages 295–298, 1990.
H. D. Gröger and G. Turán. On linear decision trees computing Boolean functions. In Proc. 18th International Colloquium on Automata, Languages and Programming, pages 707–718. Springer-Verlag, 1991.
J. Håstad. Computational Limitations of Small-Depth Circuits. The MIT Press, 1986.
J. Håstad. On the size of weights for threshold gates. To appear in SIAM. J Disc. Math., 1992.
J. Håstad and M. Goldmann. On the power of small-depth threshold circuits. Computational Complexity, 1(2): 113–129, 1991.
A. Hajnal, W. Maass, P. Pudlák, M. Szegedy, and G. Turán. Threshold circuits of bounded depth. In Proc. 28th IEEE Symposium on Foundations of Computer Science, pages 99–110, 1987.
M. Krause and P. Pudlak. On the computational power of threshold circuits with MOD-gates. Manuscript, 1993.
B. Kalyanasundaram and G. Schnitger. The probabilistic communication complexity of set intersection. In Proc. 2nd Annual Structure in Complexity Theory Conference, pages 41–49, 1987.
M. Karchmer and A. Wigderson. Monotone circuits for connectivity require super-logarithmic depth. SIAM J. Disc. Math., 3(2):255–265, 1990.
M. Krause and S. Waack. Variation ranks of communication matrices and lower bounds for depth two circuits having symmetric gates with unbounded fan-in. In Proc. 32nd IEEE Symposium on Foundations of Computer Science, pages 777–782, 1991.
L. Lovász. Communication complexity: A survey. In B Korte, L. Lovász, H. J. Prömel, and A. Schrijver, editors, Paths, Flows, and VLSI-Layout Springer-Verlag, 1990.
W. Maass, G. Schnitger, and E. Sontag. On the computational power of sigmoid versus boolean threshold circuits. In Proc. 32nd IEEE Symposium on Foundations of Computer Science, pages 767–776, 1991.
S. Muroga. Threshold Logic and its Applications. Wiley-Interscience, 1971.
J. von Neumann. Zur Theorie der Gesellschaftspiele. Math. Ann., 100:295–320, 1928.
I. Newman. Private vs. common random bits in communication complexity. Information Processing Letters, 39(2):67–71, 1991.
N Nisan. The communication complexity of threshold gates. In D. Miklos, V. T. Sos, and T. Szonyi, editors, Combinatorics, Paul ErdHos is Eighty, Volume I, pages 301–315, 1993. Published by Janos Bolyai Mathematical Society, Budapest.
G. Owen. Game theory. Academic press, second edition, 1982.
A. A. Razborov. Lower bounds on the size of bounded-depth networks over a complete basis with logical addition. Mat. Zametki, 41(4):598–607, 1987. (in Russian); English translation in Math. Notes of the Academy of Sciences of the USSR 41(4):333-338, 1987.
A. A. Razborov. On the distributional complexity of disjointness. In Proc. 17th International Colloquium on Automata, Languages and Programming, pages 249–253, 1990. To appear in Theoretical Computer Science.
A. A. Razborov. On small depth threshold circuits. In Proc. 3rd Scandinavian Workshop on Algorithm Theory, pages 42–52. Springer-Verlag, 1992. Lecture Notes in Computer Science 621.
R. Raz and A. Wigderson. Probabilistic communication complexity of boolean relations. InProc. 30th IEEE Symposium on Foundations of Computer Science, pages 562–567, 1989.
R. Raz and A. Wigderson. Monotone circuits for matching require linear depth. In Proc. 22nd ACM Symposium on Theory of Computing, pages 287–292, 1990.
A. Razborov and A. Wigderson. nΩ(log n) lower bounds on the size of depth 3 threshold circuits with AND gates at the bottom. Information Processing Letters, 45(6):303–307, 1993.
V. Roychowdhury, K.-Y. Siu, and A. Orlitsky. Lower bounds on threshold and related circuits via communication complexity. To appear in IEEE Trans. Information Theory, 1993.
K.-Y. Siu and J. Bruck. On the power of threshold circuits with small weights. SIAM J. Disc. Math., 4:423–435, 1991.
K.-Y. Siu, J. Bruck, T. Kailath, and T. Hofmeister. Depth-efficient neural networks for division and related problems. IEEE Trans. Information Theory. Vol. 39, No. 3, pp. 946–956, May 1993.
D. R. Smith. Bounds on the number of threshold functions. IEEE Trans. Electronic Computers, EC-15:368–369, 1966.
R. Smolensky. Algebraic methods in the theory of lower bounds for boolean circuit complexity. In Proc. 19th ACM Symposium on Theory of Computing, pages 77–82, 1987.
R. Smolensky. On interpolation by analytic functions with special properties and some weak lower bounds on the size of circuits with symmetric gates. In Proc. 31st IEEE Symposium on Foundations of Computer Science, pages 628–631, 1990.
K. Y. Siu and V. Roychowdhury. On optimal depth threshold circuits for multiplication and related problems. To appear in SIAM J. Disc. Math., May 1994.
C. D. Thompson. Area-time complexity for VLSI. In Proc. 11th ACM Symposium on Theory of Computing, pages 81–88, 1979.
A. C. Yao. Some complexity questions related to distributive computing. In Proc. 11th ACM Symposium on Theory of Computing, pages 209–213, 1979.
A. C. Yao. Lower bounds by probabilistic arguments. In Proc. 24th IEEE Symposium on Foundations of Computer Science, pages 420–428, 1983.
A. C. Yao. Separating the polynomial-time hierarchy by oracles. In Proc. 26th IEEE Symposium on Foundations of Computer Science, pages 1–10, 1985.
A. C. Yao. On ACC and threshold circuits. In Proc. 31st IEEE Symposium on Foundations of Computer Science, pages 619–627, 1990.
S. Yajima and T. Ibaraki. A lower bound on the number of threshold functions. IEEE Trans. Electronic Computers, EC-14:926–929, 1965.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer Science+Business Media New York
About this chapter
Cite this chapter
Goldmann, M. (1994). Communication Complexity and Lower Bounds for Threshold Circuits. In: Roychowdhury, V., Siu, KY., Orlitsky, A. (eds) Theoretical Advances in Neural Computation and Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4615-2696-4_3
Download citation
DOI: https://doi.org/10.1007/978-1-4615-2696-4_3
Publisher Name: Springer, Boston, MA
Print ISBN: 978-1-4613-6160-2
Online ISBN: 978-1-4615-2696-4
eBook Packages: Springer Book Archive