Mathematical Notes

, Volume 87, Issue 5–6, pp 860–873

# A small decrease in the degree of a polynomial with a given sign function can exponentially increase its weight and length

• V. V. Podolskii
• A. A. Sherstov
Article

## Abstract

A Boolean function f: {−1, +1} n → {−1, +1} is called the sign function of an integer-valued polynomial p(x) if f(x) = sgn(p(x)) for all x ∈ {−1, +1} n . In this case, the polynomial p(x) is called a perceptron for the Boolean function f. The weight of a perceptron is the sum of absolute values of the coefficients of p. We prove that, for a given function, a small change in the degree of a perceptron can strongly affect the value of the required weight. More precisely, for each d = 1, 2, ..., n − 1, we explicitly construct a function f: {−1, +1} n → {−1, +1} that requires a weight of the form exp{Θ(n)} when it is represented by a degree d perceptron, and that can be represented by a degree d + 1 perceptron with weight equal to only O(n 2). The lower bound exp{Θ(n)} for the degree d also holds for the size of the depth 2 Boolean circuit with a majority function at the top and arbitrary gates of input degree d at the bottom. This gap in the weight values is exponentially larger than those that have been previously found. A similar result is proved for the perceptron length, i.e., for the number of monomials contained in it.

## Key words

Boolean function integer-valued polynomial sign function perceptron Boolean circuit complexity theory discrete Fourier transform exponential gap

## References

1. 1.
R. Beigel, “The polynomial method in circuit complexity,” in Proceedings of the Eighth Annual Structure in Complexity Theory Conference, San Diego, CA, 1993 (IEEE Comput. Soc. Press, Los Alamitos, CA, 1993), pp. 82–95.
2. 2.
M. E. Saks, “Slicing the hypercube,” in Surveys in Combinatorics, 1993 (Keele), London Math. Soc. Lecture Note Ser. (Cambridge Univ. Press, Cambridge, 1993), Vol. 187, pp. 211–255.
3. 3.
H. Buhrman and R. de Wolf, “Complexity measures and decision tree complexity: a survey,” Theoret. Comput. Sci. 288(1), 21–43 (2002).
4. 4.
A. A. Sherstov, “Communication lower bounds using dual polynomials,” Bull. Eur. Assoc. Theor. Comput. Sci. EATCS 95, 59–93 (2008).
5. 5.
M. Minsky and S. Papert, Perceptrons (MIT Press, Cambridge, MA., 1969; Mir,Moscow, 1971).
6. 6.
R. Paturi and M. E. Saks, “Approximating threshold circuits by rational functions,” Inform. and Comput. 112(2), 257–272 (1994).
7. 7.
K.-Y. Siu, V. P. Roychowdhury, and T. Kailath, “Rational approximation techniques for analysis of neural networks,” IEEE Trans. Inform. Theory 40(2), 455–466 (1994).
8. 8.
J. Aspnes, R. Beigel, M. Furst, and S. Rudich, “The expressive power of voting polynomials,” Combinatorica 14(2), 135–148 (1994).
9. 9.
R. Beigel, N. Reingold, and D. Spielman, “PP is closed under intersection,” J. Comput. System Sci. 50(2), 191–202 (1995).
10. 10.
M. Krause and P. Pudlák, “On the computational power of depth-2 circuits with threshold and modulo gates,” Theoret. Comput. Sci. 174(1–2), 137–156 (1997).
11. 11.
A. A. Sherstov, “Separating AC0 from depth-2 majority circuits,” in STOC’07 — Proceedings of the 39th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2007), pp. 294–301.Google Scholar
12. 12.
A. A. Sherstov, “The pattern matrix method for lower bounds on quantum communication,” in STOC’08 — Proceedings of the 40th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2008), pp. 85–94.Google Scholar
13. 13.
A. A. Sherstov, “The unbounded-error communication complexity of symmetric functions,” in Proceedings of the 49th Symposiumon Foundations of Computer Science (FOCS) (IEEE Comput. Soc. Press, 2008), pp. 384–393.Google Scholar
14. 14.
A. A. Razborov and A. A. Sherstov, “The sign-rank of AC0,” in Proceedings of the 49th Symposium on Foundations of Computer Science (FOCS) (IEEE Comput. Soc. Press, 2008), pp. 57–66.Google Scholar
15. 15.
A. R. Klivans and R. A. Servedio, “Learning DNF in time $$2^{\tilde O(n^{{1 \mathord{\left/ {\vphantom {1 3}} \right. \kern-\nulldelimiterspace} 3}} )}$$,” J. Comput. System Sci. 68(2), 303–318 (2004).
16. 16.
R. O’Donnell and R. A. Servedio, “New degree bounds for polynomial threshold functions,” in Proceedings of the 35th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2003), pp. 325–334.Google Scholar
17. 17.
A. R. Klivans and A. A. Sherstov, “Unconditional lower bounds for learning intersections of halfspaces,” Mach. Learn. 69(2–3), 97–114 (2007).
18. 18.
J. C. Jackson, The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice, PhD thesis (Carnegie Mellon Univ., Pittsburgh, PA, 1995).Google Scholar
19. 19.
A. R. Klivans, R. O’Donnell, and R. A. Servedio, “Learning intersections and thresholds of halfspaces,” J. Comput. System Sci. 68(4), 808–840 (2004).
20. 20.
A. R. Klivans and R. A. Servedio, “Toward attribute efficient learning of decision lists and parities,” J.Mach. Learn. Res. 7, 587–602 (2006).
21. 21.
R. Beigel, “Perceptrons, PP, and the polynomial hierarchy,” Comput. Complexity 4(4), 339–349 (1994).
22. 22.
N. K. Vereshchagin, “Lower bounds for perceptrons solving some separation problems and oracle separation of AM from PP,” in Third Israel Symposium on the Theory of Computing and Systems, Tel Aviv, 1995 (IEEE Comput. Soc. Press, Los Alamitos, CA, 1995), pp. 46–51.
23. 23.
H. Buhrman, N. K. Vereshchagin, and R. de Wolf, “On computation and communication with small bias,” in Proceedings of the 22nd Annual IEEE Conference on Computational Complexity (CCC) (IEEE Comput. Soc. Press, 2007), pp. 24–32.Google Scholar
24. 24.
J. Bruck, “Harmonic analysis of polynomial threshold functions,” SIAM J. Discrete Math. 3(2), 168–177 (1990).
25. 25.
K.-Y. Siu and J. Bruck, “On the power of threshold circuits with small weights,” SIAM J. DiscreteMath. 4(3), 423–435 (1991).
26. 26.
J. Bruck and R. Smolensky, “Polynomial threshold functions, AC0 functions, and spectral norms,” SIAM J. Comput. 21(1), 33–42 (1992.).
27. 27.
M. Goldmann, J. Håstad, and A. A. Razborov, “Majority gates vs. general weighted threshold gates,” Comput. Complexity 2(4), 277–300 (1992).
28. 28.
M. Krause and P. Pudlák, “Computing Boolean functions by polynomials and threshold circuits,” Comput. Complexity 7(4), 346–370 (1998).
29. 29.
M. Krause, “On the computational power of Boolean decision lists,” Comput. Complexity 14(4), 362–375 (2005).
30. 30.
M. Goldmann, “On the power of a threshold gate at the top,” Inform. Process. Lett. 63(6), 287–293 (1997).
31. 31.
J. Håstad, “On the size of weights for threshold gates,” SIAMJ. DiscreteMath. 7(3), 484–492 (1994).
32. 32.
V. V. Podolskii, “Perceptrons of large weight,” Probl. Peredachi Inf. 45(1), 51–59 (2009) [Probl. Inf. Transm. 45 (1), 46–53 (2009)].
33. 33.
V. V. Podolskii, “A uniform lower bound on weights of perceptrons,” in Computer Science — Theory and Applications, Lecture Notes in Comput. Sci. (Springer-Verlag, Berlin, 2008), Vol. 5010, pp. 261–272.
34. 34.
J. Myhill and W. H. Kautz, “On the size of weights required for linear-input switching functions,” IEEE Trans. Electron. Comput. 10(2), 288–290 (1961).
35. 35.
A. Hajnal, W. Maass, P. Pudlák, M. Szegedy, and G. Turán, “Threshold circuits of bounded depth,” J. Comput. System Sci. 46(2), 129–154 (1993).