Skip to main content
SpringerLink
Log in

Optimal bounds for sign-representing the intersection of two halfspaces by polynomials

  • Original Paper
  • Published:
Combinatorica Aims and scope Submit manuscript

Abstract

The threshold degree of a function f: {0,1}n → {−1,+1} is the least degree of a real polynomial p with f(x) ≡ sgnp(x). We prove that the intersection of two halfspaces on {0,1}n has threshold degree Ω(n), which matches the trivial upper bound and solves an open problem due to Klivans (2002). The best previous lower bound was Ω({ie73-1}). Our result shows that the intersection of two halfspaces on {0,1}n only admits a trivial {ie73-2}-time learning algorithm based on sign-representation by polynomials, unlike the advances achieved in PAC learning DNF formulas and read-once Boolean formulas.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. M. Alekhnovich, M. Braverman, V. Feldman, A. R. Klivans and T. Pitassi: The complexity of properly learning simple concept classes, J. Comput. Syst. Sci. 74(1) (2008), 16–34.

    Article  MathSciNet  MATH  Google Scholar 

  2. A. Ambainis, A. M. Childs, B. Reichardt, R. Špalek and S. Zhang: Any AND-OR formula of size N can be evaluated in time N 1/2+o(1) on a quantum computer, In: Proceedings of the Forty-Eighth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 363–372, 2007.

    Google Scholar 

  3. R. I. Arriaga and S. Vempala: An algorithmic theory of learning: Robust concepts and random projection, Mach. Learn. 63(2) (2006), 161–182.

    Article  MATH  Google Scholar 

  4. J. Aspnes, R. Beigel, M. L. Furst and S. Rudich: The expressive power of voting polynomials, Combinatorica 14(2) (1994), 135–148.

    Article  MathSciNet  MATH  Google Scholar 

  5. R. Beigel: Perceptrons, PP, and the polynomial hierarchy, Computational Complexity 4 (1994), 339–349.

    Article  MathSciNet  MATH  Google Scholar 

  6. R. Beigel, N. Reingold and D. A. Spielman: PP is closed under intersection, J. Comput. Syst. Sci. 50(2) (1995), 191–202.

    Article  MathSciNet  MATH  Google Scholar 

  7. A. Blum and R. Kannan: Learning an intersection of a constant number of halfspaces over a uniform distribution, J. Comput. Syst. Sci. 54(2) (1997), 371–380.

    Article  MathSciNet  MATH  Google Scholar 

  8. A. L. Blum and R. L. Rivest: Training a 3-node neural network is NP-complete, Neural Networks 5 (1992), 117–127.

    Article  Google Scholar 

  9. H. Buhrman, N. K. Vereshchagin and R. de Wolf: On computation and communication with small bias, In: Proceedings of the Twenty-Second Annual IEEE Conference on Computational Complexity (CCC), 24–32, 2007.

    Google Scholar 

  10. E. Farhi, J. Goldstone and S. Gutmann: A quantum algorithm for the Hamiltonian NAND tree, Theory of Computing 4(1) (2008), 169–190.

    Article  MathSciNet  Google Scholar 

  11. S. A. Gershgorin: Uber die Abgrenzung der Eigenwerte einer Matrix, Izv. Akad. Nauk. U.S.S.R. Otd. Fiz.-Mat. Nauk 7 (1931), 749–754.

    Google Scholar 

  12. J. C. Jackson: An efficient membership-query algorithm for learning DNF with respect to the uniform distribution, J. Comput. Syst. Sci. 55(3) (1997), 414–440.

    Article  MATH  Google Scholar 

  13. S. Jukna: Extremal Combinatorics with Applications in Computer Science, Springer-Verlag, Berlin, 2001.

    Book  MATH  Google Scholar 

  14. S. Khot and R. Saket: On hardness of learning intersection of two halfspaces, In: Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing (STOC), 345–354, 2008.

    Chapter  Google Scholar 

  15. A. R. Klivans: A Complexity-Theoretic Approach to Learning, PhD thesis, MIT, 2002.

    Google Scholar 

  16. A. R. Klivans, P. M. Long and A. K. Tang: Baum’s algorithm learns intersections of halfspaces with respect to log-concave distributions, In: Proceedings of the Thirteenth International Workshop on Randomization and Computation (RANDOM), 588–600, 2009.

    Google Scholar 

  17. A. R. Klivans, R. O’Donnell and R. A. Servedio: Learning intersections and thresholds of halfspaces, J. Comput. Syst. Sci. 68(4) (2004), 808–840.

    Article  MathSciNet  MATH  Google Scholar 

  18. A. R. Klivans and R. A. Servedio: Learning DNF in time {ie95-1}, J. Comput. Syst. Sci. 68(2) (2004), 303–318.

    Article  MathSciNet  MATH  Google Scholar 

  19. A. R. Klivans and R. A. Servedio: Learning intersections of halfspaces with a margin, J. Comput. Syst. Sci. 74(1) (2008), 35–48.

    Article  MathSciNet  MATH  Google Scholar 

  20. A. R. Klivans and A. A. Sherstov: Cryptographic hardness for learning intersections of halfspaces, J. Comput. Syst. Sci. 75(1) (2009), 2–12. Preliminary version in Proceedings of the Forty-Seventh Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2006.

    Article  MathSciNet  MATH  Google Scholar 

  21. M. Krause and P. Pudlák: On the computational power of depth-2 circuits with threshold and modulo gates, Theor. Comput. Sci. 174(1–2) (1997), 137–156.

    Article  MATH  Google Scholar 

  22. M. Krause and P. Pudlák: Computing Boolean functions by polynomials and threshold circuits, Comput. Complex. 7(4) (1998), 346–370.

    Article  MATH  Google Scholar 

  23. S. Kwek and L. Pitt: PAC learning intersections of halfspaces with membership queries, Algorithmica 22(1/2) (1998), 53–75.

    Article  MathSciNet  MATH  Google Scholar 

  24. T. Lee: A note on the sign degree of formulas, 2009. Available at http://arxiv.org/abs/0909.4607.

    Google Scholar 

  25. M. L. Minsky and S. A. Papert: Perceptrons: An Introduction to Computational Geometry, MIT Press, Cambridge, Mass., 1969.

    MATH  Google Scholar 

  26. S. Muroga: Threshold Logic and Its Applications, John Wiley & Sons, New York, 1971.

    MATH  Google Scholar 

  27. J. Myhill and W. H. Kautz: On the size of weights required for linear-input switching functions, IRE Trans. on Electronic Computers 10(2) (1961), 288–290.

    Article  Google Scholar 

  28. D. J. Newman: Rational approximation to |x|, Michigan Math. J. 11(1) (1964), 11–14.

    Article  MathSciNet  MATH  Google Scholar 

  29. R. O’Donnell and R. A. Servedio: New degree bounds for polynomial threshold functions, In: Proceedings of the Thirty-Fifth Annual ACM Symposium on Theory of Computing (STOC), 325–334, 2003.

    Google Scholar 

  30. R. Paturi and M. E. Saks: Approximating threshold circuits by rational functions, Inf. Comput. 112(2) (1994), 257–272.

    Article  MathSciNet  MATH  Google Scholar 

  31. A. A. Razborov and A. A. Sherstov: The sign-rank of AC0, SIAM J. Comput. 39(5) (2010), 1833–1855. Preliminary version in Proceedings of the Forty-Ninth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008.

    Article  MathSciNet  MATH  Google Scholar 

  32. T. J. Rivlin: An Introduction to the Approximation of Functions, Dover Publications, New York, 1981.

    MATH  Google Scholar 

  33. M. E. Saks: Slicing the hypercube, Surveys in Combinatorics, 211–255, 1993.

    Google Scholar 

  34. A. A. Sherstov: The intersection of two halfspaces has high threshold degree, In: Proceedings of the Fiftieth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 343–362, 2009.

    Google Scholar 

  35. A. A. Sherstov: Separating AC0 from depth-2 majority circuits, SIAM J. Comput. 38(6) (2009), 2113–2129. Preliminary version in Proceedings of the Thirty-Ninth Annual ACM Symposium on Theory of Computing (STOC), 2007.

    Article  MathSciNet  MATH  Google Scholar 

  36. A. A. Sherstov: The pattern matrix method, SIAM J. Comput. 40(6) (2011), 1969–2000. Preliminary version in Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing (STOC), 2008.

    Article  MathSciNet  MATH  Google Scholar 

  37. A. A. Sherstov: The unbounded-error communication complexity of symmetric functions, Combinatorica 31(5) (2011), 583–614. Preliminary version in Proceedings of the Forty-Ninth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008.

    Article  MathSciNet  MATH  Google Scholar 

  38. K.-Y. Siu and J. Bruck: On the power of threshold circuits with small weights, SIAM J. Discrete Math. 4(3) (1991), 423–435.

    Article  MathSciNet  MATH  Google Scholar 

  39. K.-Y. Siu, V. P. Roychowdhury and T. Kailath: Rational approximation techniques for analysis of neural networks, IEEE Transactions on Information Theory 40(2) (1994), 455–466.

    Article  MathSciNet  MATH  Google Scholar 

  40. L. G. Valiant: A theory of the learnable, Commun. ACM 27(11) (1984), 1134–1142.

    Article  MATH  Google Scholar 

  41. S. Vempala: A random sampling based algorithm for learning the intersection of halfspaces, In: Proceedings of the Thirty-Eighth Annual IEEE Symposium on Foundations of Computer Science (FOCS), 508–513, 1997.

    Google Scholar 

  42. E. I. Zolotarev: Application of elliptic functions to questions of functions deviating least and most from zero, Izvestiya Imp. Akad. Nauk 30(5), 1877.

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, UCLA, Los Angeles, California, 90095, USA

    Alexander A. Sherstov

Authors
  1. Alexander A. Sherstov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander A. Sherstov.

Additional information

A preliminary version of this paper appeared in Proceedings of the Forty-Second Annual ACM Symposium on Theory of Computing (STOC), 2010.

This work was done while the author was a postdoctoral researcher at Microsoft Research, Cambridge, MA.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sherstov, A.A. Optimal bounds for sign-representing the intersection of two halfspaces by polynomials. Combinatorica 33, 73–96 (2013). https://doi.org/10.1007/s00493-013-2759-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00493-013-2759-7

Mathematics Subject Classification (2010)

Search

Navigation