Abstract
We study the approximation of halfspaces \(h:\{0,1\}^n\to\{0,1\}\) in the infinity norm by polynomials and rational functions of any given degree. Our main result is an explicit construction of the “hardest” halfspace, for which we prove polynomial and rational approximation lower bounds that match the trivial upper bounds achievable for all halfspaces. This completes a lengthy line of work started by Myhill and Kautz (1961). As an application, we construct a communication problem that achieves essentially the largest possible separation, of O(n) versus \(2^{-\Omega(n)}\), between the sign-rank and discrepancy. Equivalently, our problem exhibits a gap of log n versus \(\Omega(n)\) between the communication complexity with unbounded versus weakly unbounded error, improving quadratically on previous constructions and completing a line of work started by Babai, Frankl, and Simon (FOCS 1986). Our results further generalize to the k-party number-on-the-forehead model, where we obtain an explicit separation of log n versus \(\Omega(n/4^{n})\) for communication with unbounded versus weakly unbounded error.
Article PDF
Similar content being viewed by others
References
Scott Aaronson & Yaoyun Shi: Quantum lower bounds for the collision and the element distinctness problems. J. ACM 51(4), 595–605 (2004)
Ajtai, Miklós, Iwaniec, Henryk, Komlós, János., Pintz, János., Szemerédi, Endre: Construction of a thin set with small Fourier coefficients. Bulletin of the London Mathematical Society 22(6), 583–590 (1990)
Alekhnovich, Michael, Braverman, Mark, Feldman, Vitaly, Klivans, Adam R., Pitassi, Toniann: The complexity of properly learning simple concept classes. J. Comput. Syst. Sci. 74(1), 16–34 (2008)
Alon, Noga: Eigenvalues, geometric expanders, sorting in rounds, and Ramsey theory. Combinatorica 6(3), 207–219 (1986)
Noga Alon & Yuval Roichman: Random Cayley graphs and expanders. Random Struct. Algorithms 5(2), 271–285 (1994)
Noga Alon & Joel Spencer: The Probabilistic Method. John Wiley & Sons, 3rd edn. (2008)
Ambainis, Andris: Polynomial degree and lower bounds in quantum complexity: Collision and element distinctness with small range. Theory of Computing 1(1), 37–46 (2005)
Ambainis, Andris, Childs, Andrew M., Reichardt, Ben, Špalek, Robert, Zhang, Shengyu: Any AND-OR formula of size \(N\) can be evaluated in time \(N^{1/2+o(1)}\) on a quantum computer. SIAM J. Comput. 39(6), 2513–2530 (2010)
Arriaga, Rosa I., Vempala, Santosh: An algorithmic theory of learning: Robust concepts and random projection. Mach. Learn. 63(2), 161–182 (2006)
Aspnes, James, Beigel, Richard, Furst, Merrick L., Rudich, Steven: The expressive power of voting polynomials. Combinatorica 14(2), 135–148 (1994)
László Babai, Peter Frankl & Janos Simon (1986). Complexity classes in communication complexity theory. In Proceedings of the Twenty-Seventh Annual IEEE Symposium on Foundations of Computer Science (FOCS), 337–347
Babai, László., Nisan, Noam, Szegedy, Mario: Multiparty protocols, pseudorandom generators for logspace, and time-space trade-offs. J. Comput. Syst. Sci. 45(2), 204–232 (1992)
Beals, Robert, Buhrman, Harry, Cleve, Richard, Mosca, Michele, de Wolf, Ronald: Quantum lower bounds by polynomials. J. ACM 48(4), 778–797 (2001)
Paul Beame, Matei David, Toniann Pitassi & Philipp Woelfel (2007). Separating deterministic from nondeterministic NOF multiparty communication complexity. In Proceedings of the Thirty-Fourth International Colloquium on Automata, Languages and Programming (ICALP), 134–145
Paul Beame & Trinh Huynh: Multiparty communication complexity and threshold circuit size of AC\(^0\). SIAM J. Comput. 41(3), 484–518 (2012)
Beigel, Richard: Perceptrons, PP, and the polynomial hierarchy. Computational Complexity 4, 339–349 (1994)
Beigel, Richard, Reingold, Nick, Spielman, Daniel A.: PP is closed under intersection. J. Comput. Syst. Sci. 50(2), 191–202 (1995)
Avrim Blum & Ravindran Kannan: Learning an intersection of a constant number of halfspaces over a uniform distribution. J. Comput. Syst. Sci. 54(2), 371–380 (1997)
Blum, Avrim, Rivest, Ronald L.: Training a 3-node neural network is NP-complete. Neural Networks 5(1), 117–127 (1992)
Buhrman, Harry, Newman, Ilan, Röhrig, Hein, de Wolf, Ronald: Robust polynomials and quantum algorithms. Theory Comput. Syst. 40(4), 379–395 (2007a)
Harry Buhrman, Nikolai K. Vereshchagin & Ronald de Wolf (2007b). On computation and communication with small bias. In Proceedings of the Twenty-Second Annual IEEE Conference on Computational Complexity (CCC), 24–32
Harry Buhrman & Ronald de Wolf (2001). Communication complexity lower bounds by polynomials. In Proceedings of the Sixteenth Annual IEEE Conference on Computational Complexity (CCC), 120–130
Bun, Mark, Kothari, Robin, Thaler, Justin: The polynomial method strikes back: Tight quantum query bounds via dual polynomials. Theory Comput. 16, 1–71 (2020)
Chandra, Ashok K., Furst, Merrick L., Lipton, Richard J.: Multi-party protocols. Presented at the (1983)
Karthekeyan Chandrasekaran, Justin Thaler, Jonathan Ullman & Andrew Wan (2014). Faster private release of marginals on small databases. In Proceedings of the Fifth Conference on Innovations in Theoretical Computer Science (ITCS), 387–402
Arkadev Chattopadhyay & Anil Ada: Multiparty communication complexity of disjointness. In Electronic Colloquium on Computational Complexity (ECCC). Report TR08-002.
Chattopadhyay, Arkadev, Mande, Nikhil S.: Separation of unbounded error models in multi-party communication complexity. Theory of Computing 14(21), 1–23 (2018)
Benny Chor & Oded Goldreich: Unbiased bits from sources of weak randomness and probabilistic communication complexity. SIAM J. Comput. 17(2), 230–261 (1988)
Friedman, Joel, Murty, Ram, Tillich, Jean-Pierre.: Spectral estimates for Abelian Cayley graphs. Journal of Combinatorial Theory, Series B 96(1), 111–121 (2006)
Galil, Zvi, Kannan, Ravi, Szemerédi, Endre: On nontrivial separators for \(k\)-page graphs and simulations by nondeterministic one-tape Turing machines. J. Comput. Syst. Sci. 38(1), 134–149 (1989)
Gill, John: Computational complexity of probabilistic Turing machines. SIAM J. Comput. 6(4), 675–695 (1977)
Gould, Henry W.: Combinatorial Identities: A Standardized Set of Tables Listing 500 Binomial Coefficient Summations. Morgantown Printing and Binding Co. (1972)
Håstad, Johan: On the size of weights for threshold gates. SIAM J. Discret. Math. 7(3), 484–492 (1994)
Hatami, Hamed, Hosseini, Kaave, Lovett, Shachar (2020). Sign rank vs discrepancy. In Proceedings of the Thirty-Fifth Annual IEEE Conference on Computational Complexity (CCC), volume 169 of LIPIcs, 18:1–18:14.
Hoeffding, Wassily: Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58(301), 13–30 (1963)
Jukna, Stasys: Extremal Combinatorics with Applications in Computer Science. Springer-Verlag, Berlin (2001)
Kahn, Jeff, Linial, Nathan, Samorodnitsky, Alex: Inclusion-exclusion: Exact and approximate. Combinatorica 16(4), 465–477 (1996)
Kalai, Adam Tauman, Klivans, Adam R., Mansour, Yishay, Servedio, Rocco A.: Agnostically learning halfspaces. SIAM J. Comput. 37(6), 1777–1805 (2008)
Katz, Nicholas M.: An estimate for character sums. Journal of the American Mathematical Society 2(2), 197–200 (1989)
Subhash Khot & Rishi Saket: On the hardness of learning intersections of two halfspaces. J. Comput. Syst. Sci. 77(1), 129–141 (2011)
Klauck, Hartmut: Lower bounds for quantum communication complexity. SIAM J. Comput. 37(1), 20–46 (2007)
Klivans, Adam R.: A Complexity-Theoretic Approach to Learning. MIT (2002).. (Ph.D. thesis)
Klivans, Adam R., Long, Philip M., Tang, Alex K (2009). Baum’s algorithm learns intersections of halfspaces with respect to log-concave distributions. In Proceedings of the Thirteenth International Workshop on Randomization and Computation (RANDOM), 588-600.
Klivans, Adam R., O’Donnell, Ryan, Servedio, Rocco A.: Learning intersections and thresholds of halfspaces. J. Comput. Syst. Sci. 68(4), 808–840 (2004)
Klivans, Adam R., Servedio, Rocco A.: Learning DNF in time \(2^{\tilde{O}(n^{1/3})}\). J. Comput. Syst. Sci. 68(2), 303–318 (2004)
Klivans, Adam R., Servedio, Rocco A.: Learning intersections of halfspaces with a margin. J. Comput. Syst. Sci. 74(1), 35–48 (2008)
Klivans, Adam R., Sherstov, Alexander A.: Cryptographic hardness for learning intersections of halfspaces. J. Comput. Syst. Sci. 75(1), 2–12 (2009)
Matthias Krause & Pavel Pudlák: On the computational power of depth-\(2\) circuits with threshold and modulo gates. Theor. Comput. Sci. 174(1–2), 137–156 (1997)
Matthias Krause & Pavel Pudlák: Computing Boolean functions by polynomials and threshold circuits. Comput. Complex. 7(4), 346–370 (1998)
Eyal Kushilevitz & Noam Nisan: Communication Complexity. Cambridge University Press (1997)
Stephen Kwek & Leonard Pitt: PAC learning intersections of halfspaces with membership queries. Algorithmica 22(1/2), 53–75 (1998)
Troy Lee & Adi Shraibman: Disjointness is hard in the multiparty number-on-the-forehead model. Computational Complexity 18(2), 309–336 (2009)
Leung, Ka Hin, Nguyen, Vinh, So, Wasin: Nonexistence of a circulant expander family. Bulletin of the Australian Mathematical Society 83(1), 87–95 (2011)
Nathan Linial & Noam Nisan: Approximate inclusion-exclusion. Combinatorica 10(4), 349–365 (1990)
Nathan Linial & Adi Shraibman: Learning complexity vs communication complexity. Combinatorics, Probability & Computing 18(1–2), 227–245 (2009)
Linial, Nati, Mendelson, Shahar, Schechtman, Gideon, Shraibman, Adi: Complexity measures of sign matrices. Combinatorica 27(4), 439–463 (2007)
Minsky, Marvin L., Papert, Seymour A.: Perceptrons: An Introduction to Computational Geometry. MIT Press, Cambridge, Mass (1969)
Muroga, Saburo: Threshold Logic and Its Applications. John Wiley & Sons, New York (1971)
Myhill, J., Kautz, W.H.: On the size of weights required for linear-input switching functions. IRE Trans. on Electronic Computers 10(2), 288–290 (1961)
Newman, D.J.: Rational approximation to \(|x|\). Michigan Math. J. 11(1), 11–14 (1964)
O’Donnell, Ryan, Servedio, Rocco A.: New degree bounds for polynomial threshold functions. Combinatorica 30(3), 327–358 (2010)
Paturi, Ramamohan (1992). On the degree of polynomials that approximate symmetric Boolean functions. In Proceedings of the Twenty- Fourth Annual ACM Symposium on Theory of Computing (STOC), 468-474.
Ramamohan Paturi & Janos Simon: Probabilistic communication complexity. J. Comput. Syst. Sci. 33(1), 106–123 (1986)
Razborov, A., Szemerédi, E., Wigderson, A.: Constructing small sets that are uniform in arithmetic progressions. Combinatorics, Probability and Computing 2(4), 513–518 (1993)
Razborov, Alexander A.: Quantum communication complexity of symmetric predicates. Izvestiya of the Russian Academy of Sciences, Mathematics 67, 145–159 (2002)
Razborov, Alexander A., Sherstov, Alexander A.: The sign-rank of AC\(^0\). SIAM J. Comput. 39(5), 1833–1855 (2010)
Rivlin, Theodore J.: An Introduction to the Approximation of Functions. Dover Publications, New York (1981)
Rosser, Barkley: Explicit bounds for some functions of prime numbers. American Journal of Mathematics 63(1), 211–232 (1941)
Ruzsa, Imre Z.: Essential components. Proceedings of the London Mathematical Society 53–54(1), 38–56 (1987)
Sherstov, Alexander A.: Communication lower bounds using dual polynomials. Bulletin of the EATCS 95, 59–93 (2008a)
Sherstov, Alexander A.: Halfspace matrices. Computational Complexity 17(2), 149–178 (2008b)
Sherstov, Alexander A.: Approximate inclusion-exclusion for arbitrary symmetric functions. Computational Complexity 18(2), 219–247 (2009a)
Sherstov, Alexander A.: Separating AC\(^0\) from depth-\(2\) majority circuits. SIAM J. Comput. 38(6), 2113–2129 (2009b)
Sherstov, Alexander A.: Communication complexity under product and nonproduct distributions. Computational Complexity 19(1), 135–150 (2010)
Sherstov, Alexander A.: The pattern matrix method. SIAM J. Comput. 40(6), 1969–2000 (2011)
Alexander A. Sherstov (2013a). The intersection of two halfspaces has high threshold degree. SIAM J. Comput. 42(6), 2329–2374
Sherstov, Alexander A.: Optimal bounds for sign-representing the intersection of two halfspaces by polynomials. Combinatorica 33(1), 73–96 (2013b)
Sherstov, Alexander A.: Communication lower bounds using directional derivatives. J. ACM 61(6), 1–71 (2014)
Sherstov, Alexander A.: The multiparty communication complexity of set disjointness. SIAM J. Comput. 45(4), 1450–1489 (2016)
Sherstov, Alexander A.: On multiparty communication with large versus unbounded error. Theory of Computing 14(22), 1–17 (2018)
Kai-Yeung Siu & Jehoshua Bruck: On the power of threshold circuits with small weights. SIAM J. Discrete Math. 4(3), 423–435 (1991)
Jun Tarui & Tatsuie Tsukiji (1999). Learning DNF by approximating inclusion-exclusion formulae. In Proceedings of the Fourteenth Annual IEEE Conference on Computational Complexity (CCC), 215-221.
Thaler, Justin (2016). Lower bounds for the approximate degree of block-composed functions. In Proceedings of the Forty-Third International Colloquium on Automata, Languages and Programming (ICALP), 17:1–17:15.
Thaler, Justin, Ullman, Jonathan, Vadhan, Salil P (2012). Faster algorithms for privately releasing marginals. In Proceedings of the Thirty-Ninth International Colloquium on Automata, Languages and Programming (ICALP), 810-821.
Jayram S. Thathachar (1998). On separating the read-\(k\)-times branching program hierarchy. ECCC Report TR98-002. https://eccc.weizmann.ac.il/report/1998/002. Extended abstract in Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing (STOC), 1998
Valiant, Leslie G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)
Vempala, Santosh: A random-sampling-based algorithm for learning intersections of halfspaces. J. ACM 57(6), 32 (2010)
Andrew Chi-Chih Yao (1979). Some complexity questions related to distributive computing. In Proceedings of the Eleventh Annual ACM Symposium on Theory of Computing (STOC), 209-213.
Zolotarev, E.I (1877). Application of elliptic functions to questions of functions deviating least and most from zero. Izvestiya Imp. Akad. Nauk 30(5),(1877)
Acknowledgements
This work was supported by NSF CAREER award CCF-1149018 and an Alfred P. Sloan Foundation Research Fellowship. I am thankful to Mark Bun, T. S. Jayram, Ryan O'Donnell, Rocco Servedio, and Justin Thaler for valuable comments on this work. Special thanks to T. S. Jayram for allowing me to include his short and elegant proof of Fact 4.1.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Sherstov, A.A. The hardest halfspace. comput. complex. 30, 11 (2021). https://doi.org/10.1007/s00037-021-00211-4
Received:
Published:
DOI: https://doi.org/10.1007/s00037-021-00211-4
keywords
- Small-bias communication
- Multiparty communication
- UPP
- PP
- Halfspaces
- Approximation by polynomials and rational functions