Advertisement

Mathematical Notes

, Volume 87, Issue 5–6, pp 860–873 | Cite as

A small decrease in the degree of a polynomial with a given sign function can exponentially increase its weight and length

  • V. V. Podolskii
  • A. A. Sherstov
Article

Abstract

A Boolean function f: {−1, +1} n → {−1, +1} is called the sign function of an integer-valued polynomial p(x) if f(x) = sgn(p(x)) for all x ∈ {−1, +1} n . In this case, the polynomial p(x) is called a perceptron for the Boolean function f. The weight of a perceptron is the sum of absolute values of the coefficients of p. We prove that, for a given function, a small change in the degree of a perceptron can strongly affect the value of the required weight. More precisely, for each d = 1, 2, ..., n − 1, we explicitly construct a function f: {−1, +1} n → {−1, +1} that requires a weight of the form exp{Θ(n)} when it is represented by a degree d perceptron, and that can be represented by a degree d + 1 perceptron with weight equal to only O(n 2). The lower bound exp{Θ(n)} for the degree d also holds for the size of the depth 2 Boolean circuit with a majority function at the top and arbitrary gates of input degree d at the bottom. This gap in the weight values is exponentially larger than those that have been previously found. A similar result is proved for the perceptron length, i.e., for the number of monomials contained in it.

Key words

Boolean function integer-valued polynomial sign function perceptron Boolean circuit complexity theory discrete Fourier transform exponential gap 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    R. Beigel, “The polynomial method in circuit complexity,” in Proceedings of the Eighth Annual Structure in Complexity Theory Conference, San Diego, CA, 1993 (IEEE Comput. Soc. Press, Los Alamitos, CA, 1993), pp. 82–95.CrossRefGoogle Scholar
  2. 2.
    M. E. Saks, “Slicing the hypercube,” in Surveys in Combinatorics, 1993 (Keele), London Math. Soc. Lecture Note Ser. (Cambridge Univ. Press, Cambridge, 1993), Vol. 187, pp. 211–255.CrossRefGoogle Scholar
  3. 3.
    H. Buhrman and R. de Wolf, “Complexity measures and decision tree complexity: a survey,” Theoret. Comput. Sci. 288(1), 21–43 (2002).zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    A. A. Sherstov, “Communication lower bounds using dual polynomials,” Bull. Eur. Assoc. Theor. Comput. Sci. EATCS 95, 59–93 (2008).zbMATHMathSciNetGoogle Scholar
  5. 5.
    M. Minsky and S. Papert, Perceptrons (MIT Press, Cambridge, MA., 1969; Mir,Moscow, 1971).zbMATHGoogle Scholar
  6. 6.
    R. Paturi and M. E. Saks, “Approximating threshold circuits by rational functions,” Inform. and Comput. 112(2), 257–272 (1994).zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    K.-Y. Siu, V. P. Roychowdhury, and T. Kailath, “Rational approximation techniques for analysis of neural networks,” IEEE Trans. Inform. Theory 40(2), 455–466 (1994).zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    J. Aspnes, R. Beigel, M. Furst, and S. Rudich, “The expressive power of voting polynomials,” Combinatorica 14(2), 135–148 (1994).zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    R. Beigel, N. Reingold, and D. Spielman, “PP is closed under intersection,” J. Comput. System Sci. 50(2), 191–202 (1995).zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    M. Krause and P. Pudlák, “On the computational power of depth-2 circuits with threshold and modulo gates,” Theoret. Comput. Sci. 174(1–2), 137–156 (1997).zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    A. A. Sherstov, “Separating AC0 from depth-2 majority circuits,” in STOC’07 — Proceedings of the 39th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2007), pp. 294–301.Google Scholar
  12. 12.
    A. A. Sherstov, “The pattern matrix method for lower bounds on quantum communication,” in STOC’08 — Proceedings of the 40th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2008), pp. 85–94.Google Scholar
  13. 13.
    A. A. Sherstov, “The unbounded-error communication complexity of symmetric functions,” in Proceedings of the 49th Symposiumon Foundations of Computer Science (FOCS) (IEEE Comput. Soc. Press, 2008), pp. 384–393.Google Scholar
  14. 14.
    A. A. Razborov and A. A. Sherstov, “The sign-rank of AC0,” in Proceedings of the 49th Symposium on Foundations of Computer Science (FOCS) (IEEE Comput. Soc. Press, 2008), pp. 57–66.Google Scholar
  15. 15.
    A. R. Klivans and R. A. Servedio, “Learning DNF in time \( 2^{\tilde O(n^{{1 \mathord{\left/ {\vphantom {1 3}} \right. \kern-\nulldelimiterspace} 3}} )} \),” J. Comput. System Sci. 68(2), 303–318 (2004).zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    R. O’Donnell and R. A. Servedio, “New degree bounds for polynomial threshold functions,” in Proceedings of the 35th Annual ACM Symposium on Theory of Computing (ACM, New York, NY, 2003), pp. 325–334.Google Scholar
  17. 17.
    A. R. Klivans and A. A. Sherstov, “Unconditional lower bounds for learning intersections of halfspaces,” Mach. Learn. 69(2–3), 97–114 (2007).CrossRefGoogle Scholar
  18. 18.
    J. C. Jackson, The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice, PhD thesis (Carnegie Mellon Univ., Pittsburgh, PA, 1995).Google Scholar
  19. 19.
    A. R. Klivans, R. O’Donnell, and R. A. Servedio, “Learning intersections and thresholds of halfspaces,” J. Comput. System Sci. 68(4), 808–840 (2004).zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    A. R. Klivans and R. A. Servedio, “Toward attribute efficient learning of decision lists and parities,” J.Mach. Learn. Res. 7, 587–602 (2006).MathSciNetGoogle Scholar
  21. 21.
    R. Beigel, “Perceptrons, PP, and the polynomial hierarchy,” Comput. Complexity 4(4), 339–349 (1994).zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    N. K. Vereshchagin, “Lower bounds for perceptrons solving some separation problems and oracle separation of AM from PP,” in Third Israel Symposium on the Theory of Computing and Systems, Tel Aviv, 1995 (IEEE Comput. Soc. Press, Los Alamitos, CA, 1995), pp. 46–51.CrossRefGoogle Scholar
  23. 23.
    H. Buhrman, N. K. Vereshchagin, and R. de Wolf, “On computation and communication with small bias,” in Proceedings of the 22nd Annual IEEE Conference on Computational Complexity (CCC) (IEEE Comput. Soc. Press, 2007), pp. 24–32.Google Scholar
  24. 24.
    J. Bruck, “Harmonic analysis of polynomial threshold functions,” SIAM J. Discrete Math. 3(2), 168–177 (1990).zbMATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    K.-Y. Siu and J. Bruck, “On the power of threshold circuits with small weights,” SIAM J. DiscreteMath. 4(3), 423–435 (1991).zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    J. Bruck and R. Smolensky, “Polynomial threshold functions, AC0 functions, and spectral norms,” SIAM J. Comput. 21(1), 33–42 (1992.).zbMATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    M. Goldmann, J. Håstad, and A. A. Razborov, “Majority gates vs. general weighted threshold gates,” Comput. Complexity 2(4), 277–300 (1992).zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    M. Krause and P. Pudlák, “Computing Boolean functions by polynomials and threshold circuits,” Comput. Complexity 7(4), 346–370 (1998).zbMATHCrossRefMathSciNetGoogle Scholar
  29. 29.
    M. Krause, “On the computational power of Boolean decision lists,” Comput. Complexity 14(4), 362–375 (2005).zbMATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    M. Goldmann, “On the power of a threshold gate at the top,” Inform. Process. Lett. 63(6), 287–293 (1997).CrossRefMathSciNetGoogle Scholar
  31. 31.
    J. Håstad, “On the size of weights for threshold gates,” SIAMJ. DiscreteMath. 7(3), 484–492 (1994).zbMATHCrossRefGoogle Scholar
  32. 32.
    V. V. Podolskii, “Perceptrons of large weight,” Probl. Peredachi Inf. 45(1), 51–59 (2009) [Probl. Inf. Transm. 45 (1), 46–53 (2009)].MathSciNetGoogle Scholar
  33. 33.
    V. V. Podolskii, “A uniform lower bound on weights of perceptrons,” in Computer Science — Theory and Applications, Lecture Notes in Comput. Sci. (Springer-Verlag, Berlin, 2008), Vol. 5010, pp. 261–272.CrossRefGoogle Scholar
  34. 34.
    J. Myhill and W. H. Kautz, “On the size of weights required for linear-input switching functions,” IEEE Trans. Electron. Comput. 10(2), 288–290 (1961).CrossRefGoogle Scholar
  35. 35.
    A. Hajnal, W. Maass, P. Pudlák, M. Szegedy, and G. Turán, “Threshold circuits of bounded depth,” J. Comput. System Sci. 46(2), 129–154 (1993).zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Pleiades Publishing, Ltd. 2010

Authors and Affiliations

  1. 1.Moscow State UniversityMoscowRussia
  2. 2.The University of Texas at AustinTexasUSA

Personalised recommendations