Optimal bounds for sign-representing the intersection of two halfspaces by polynomials

Abstract

The threshold degree of a function f: {0,1}n → {−1,+1} is the least degree of a real polynomial p with f(x) ≡ sgnp(x). We prove that the intersection of two halfspaces on {0,1}n has threshold degree Ω(n), which matches the trivial upper bound and solves an open problem due to Klivans (2002). The best previous lower bound was Ω({ie73-1}). Our result shows that the intersection of two halfspaces on {0,1}n only admits a trivial {ie73-2}-time learning algorithm based on sign-representation by polynomials, unlike the advances achieved in PAC learning DNF formulas and read-once Boolean formulas.

This is a preview of subscription content, access via your institution.

References

  1. [1]

    M. Alekhnovich, M. Braverman, V. Feldman, A. R. Klivans and T. Pitassi: The complexity of properly learning simple concept classes, J. Comput. Syst. Sci. 74(1) (2008), 16–34.

    MathSciNet  MATH  Article  Google Scholar 

  2. [2]

    A. Ambainis, A. M. Childs, B. Reichardt, R. Špalek and S. Zhang: Any AND-OR formula of size N can be evaluated in time N 1/2+o(1) on a quantum computer, In: Proceedings of the Forty-Eighth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 363–372, 2007.

    Google Scholar 

  3. [3]

    R. I. Arriaga and S. Vempala: An algorithmic theory of learning: Robust concepts and random projection, Mach. Learn. 63(2) (2006), 161–182.

    MATH  Article  Google Scholar 

  4. [4]

    J. Aspnes, R. Beigel, M. L. Furst and S. Rudich: The expressive power of voting polynomials, Combinatorica 14(2) (1994), 135–148.

    MathSciNet  MATH  Article  Google Scholar 

  5. [5]

    R. Beigel: Perceptrons, PP, and the polynomial hierarchy, Computational Complexity 4 (1994), 339–349.

    MathSciNet  MATH  Article  Google Scholar 

  6. [6]

    R. Beigel, N. Reingold and D. A. Spielman: PP is closed under intersection, J. Comput. Syst. Sci. 50(2) (1995), 191–202.

    MathSciNet  MATH  Article  Google Scholar 

  7. [7]

    A. Blum and R. Kannan: Learning an intersection of a constant number of halfspaces over a uniform distribution, J. Comput. Syst. Sci. 54(2) (1997), 371–380.

    MathSciNet  MATH  Article  Google Scholar 

  8. [8]

    A. L. Blum and R. L. Rivest: Training a 3-node neural network is NP-complete, Neural Networks 5 (1992), 117–127.

    Article  Google Scholar 

  9. [9]

    H. Buhrman, N. K. Vereshchagin and R. de Wolf: On computation and communication with small bias, In: Proceedings of the Twenty-Second Annual IEEE Conference on Computational Complexity (CCC), 24–32, 2007.

    Google Scholar 

  10. [10]

    E. Farhi, J. Goldstone and S. Gutmann: A quantum algorithm for the Hamiltonian NAND tree, Theory of Computing 4(1) (2008), 169–190.

    MathSciNet  Article  Google Scholar 

  11. [11]

    S. A. Gershgorin: Uber die Abgrenzung der Eigenwerte einer Matrix, Izv. Akad. Nauk. U.S.S.R. Otd. Fiz.-Mat. Nauk 7 (1931), 749–754.

    Google Scholar 

  12. [12]

    J. C. Jackson: An efficient membership-query algorithm for learning DNF with respect to the uniform distribution, J. Comput. Syst. Sci. 55(3) (1997), 414–440.

    MATH  Article  Google Scholar 

  13. [13]

    S. Jukna: Extremal Combinatorics with Applications in Computer Science, Springer-Verlag, Berlin, 2001.

    MATH  Book  Google Scholar 

  14. [14]

    S. Khot and R. Saket: On hardness of learning intersection of two halfspaces, In: Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing (STOC), 345–354, 2008.

    Google Scholar 

  15. [15]

    A. R. Klivans: A Complexity-Theoretic Approach to Learning, PhD thesis, MIT, 2002.

    Google Scholar 

  16. [16]

    A. R. Klivans, P. M. Long and A. K. Tang: Baum’s algorithm learns intersections of halfspaces with respect to log-concave distributions, In: Proceedings of the Thirteenth International Workshop on Randomization and Computation (RANDOM), 588–600, 2009.

    Google Scholar 

  17. [17]

    A. R. Klivans, R. O’Donnell and R. A. Servedio: Learning intersections and thresholds of halfspaces, J. Comput. Syst. Sci. 68(4) (2004), 808–840.

    MathSciNet  MATH  Article  Google Scholar 

  18. [18]

    A. R. Klivans and R. A. Servedio: Learning DNF in time {ie95-1}, J. Comput. Syst. Sci. 68(2) (2004), 303–318.

    MathSciNet  MATH  Article  Google Scholar 

  19. [19]

    A. R. Klivans and R. A. Servedio: Learning intersections of halfspaces with a margin, J. Comput. Syst. Sci. 74(1) (2008), 35–48.

    MathSciNet  MATH  Article  Google Scholar 

  20. [20]

    A. R. Klivans and A. A. Sherstov: Cryptographic hardness for learning intersections of halfspaces, J. Comput. Syst. Sci. 75(1) (2009), 2–12. Preliminary version in Proceedings of the Forty-Seventh Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2006.

    MathSciNet  MATH  Article  Google Scholar 

  21. [21]

    M. Krause and P. Pudlák: On the computational power of depth-2 circuits with threshold and modulo gates, Theor. Comput. Sci. 174(1–2) (1997), 137–156.

    MATH  Article  Google Scholar 

  22. [22]

    M. Krause and P. Pudlák: Computing Boolean functions by polynomials and threshold circuits, Comput. Complex. 7(4) (1998), 346–370.

    MATH  Article  Google Scholar 

  23. [23]

    S. Kwek and L. Pitt: PAC learning intersections of halfspaces with membership queries, Algorithmica 22(1/2) (1998), 53–75.

    MathSciNet  MATH  Article  Google Scholar 

  24. [24]

    T. Lee: A note on the sign degree of formulas, 2009. Available at http://arxiv.org/abs/0909.4607.

    Google Scholar 

  25. [25]

    M. L. Minsky and S. A. Papert: Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge, Mass., 1969.

    MATH  Google Scholar 

  26. [26]

    S. Muroga: Threshold Logic and Its Applications, John Wiley & Sons, New York, 1971.

    MATH  Google Scholar 

  27. [27]

    J. Myhill and W. H. Kautz: On the size of weights required for linear-input switching functions, IRE Trans. on Electronic Computers 10(2) (1961), 288–290.

    Article  Google Scholar 

  28. [28]

    D. J. Newman: Rational approximation to |x|, Michigan Math. J. 11(1) (1964), 11–14.

    MathSciNet  MATH  Article  Google Scholar 

  29. [29]

    R. O’Donnell and R. A. Servedio: New degree bounds for polynomial threshold functions, In: Proceedings of the Thirty-Fifth Annual ACM Symposium on Theory of Computing (STOC), 325–334, 2003.

    Google Scholar 

  30. [30]

    R. Paturi and M. E. Saks: Approximating threshold circuits by rational functions, Inf. Comput. 112(2) (1994), 257–272.

    MathSciNet  MATH  Article  Google Scholar 

  31. [31]

    A. A. Razborov and A. A. Sherstov: The sign-rank of AC0, SIAM J. Comput. 39(5) (2010), 1833–1855. Preliminary version in Proceedings of the Forty-Ninth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008.

    MathSciNet  MATH  Article  Google Scholar 

  32. [32]

    T. J. Rivlin: An Introduction to the Approximation of Functions, Dover Publications, New York, 1981.

    MATH  Google Scholar 

  33. [33]

    M. E. Saks: Slicing the hypercube, Surveys in Combinatorics, 211–255, 1993.

    Google Scholar 

  34. [34]

    A. A. Sherstov: The intersection of two halfspaces has high threshold degree, In: Proceedings of the Fiftieth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 343–362, 2009.

    Google Scholar 

  35. [35]

    A. A. Sherstov: Separating AC0 from depth-2 majority circuits, SIAM J. Comput. 38(6) (2009), 2113–2129. Preliminary version in Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing (STOC), 2007.

    MathSciNet  MATH  Article  Google Scholar 

  36. [36]

    A. A. Sherstov: The pattern matrix method, SIAM J. Comput. 40(6) (2011), 1969–2000. Preliminary version in Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing (STOC), 2008.

    MathSciNet  MATH  Article  Google Scholar 

  37. [37]

    A. A. Sherstov: The unbounded-error communication complexity of symmetric functions, Combinatorica 31(5) (2011), 583–614. Preliminary version in Proceedings of the Forty-Ninth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008.

    MathSciNet  MATH  Article  Google Scholar 

  38. [38]

    K.-Y. Siu and J. Bruck: On the power of threshold circuits with small weights, SIAM J. Discrete Math. 4(3) (1991), 423–435.

    MathSciNet  MATH  Article  Google Scholar 

  39. [39]

    K.-Y. Siu, V. P. Roychowdhury and T. Kailath: Rational approximation techniques for analysis of neural networks, IEEE Transactions on Information Theory 40(2) (1994), 455–466.

    MathSciNet  MATH  Article  Google Scholar 

  40. [40]

    L. G. Valiant: A theory of the learnable, Commun. ACM 27(11) (1984), 1134–1142.

    MATH  Article  Google Scholar 

  41. [41]

    S. Vempala: A random sampling based algorithm for learning the intersection of halfspaces, In: Proceedings of the Thirty-Eighth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 508–513, 1997.

    Google Scholar 

  42. [42]

    E. I. Zolotarev: Application of elliptic functions to questions of functions deviating least and most from zero, Izvestiya Imp. Akad. Nauk 30(5), 1877.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Alexander A. Sherstov.

Additional information

A preliminary version of this paper appeared in Proceedings of the Forty-Second Annual ACM Symposium on Theory of Computing (STOC), 2010.

This work was done while the author was a postdoctoral researcher at Microsoft Research, Cambridge, MA.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Sherstov, A.A. Optimal bounds for sign-representing the intersection of two halfspaces by polynomials. Combinatorica 33, 73–96 (2013). https://doi.org/10.1007/s00493-013-2759-7

Download citation

Mathematics Subject Classification (2010)

  • 68Q32
  • 68Q17