Efficient Learning Algorithms Yield Circuit Lower Bounds

  • Lance Fortnow
  • Adam R. Klivans
Conference paper

DOI: 10.1007/11776420_27

Part of the Lecture Notes in Computer Science book series (LNCS, volume 4005)
Cite this paper as:
Fortnow L., Klivans A.R. (2006) Efficient Learning Algorithms Yield Circuit Lower Bounds. In: Lugosi G., Simon H.U. (eds) Learning Theory. COLT 2006. Lecture Notes in Computer Science, vol 4005. Springer, Berlin, Heidelberg

Abstract

We describe a new approach for understanding the difficulty of designing efficient learning algorithms. We prove that the existence of an efficient learning algorithm for a circuit class C in Angluin’s model of exact learning from membership and equivalence queries or in Valiant’s PAC model yields a lower bound against C. More specifically, we prove that any subexponential time, determinstic exact learning algorithm for C (from membership and equivalence queries) implies the existence of a function f in EXPNP such that \(f \not\in C\). If C is PAC learnable with membership queries under the uniform distribution or Exact learnable in randomized polynomial time, we prove that there exists a function fBPEXP (the exponential time analog of BPP) such that \(f {\not\in} C\).

For C equal to polynomial-size, depth-two threshold circuits (i.e., neural networks with a polynomial number of hidden nodes), our result shows that efficient learning algorithms for this class would solve one of the most challenging open problems in computational complexity theory: proving the existence of a function in EXPNP or BPEXP that cannot be computed by circuits from C. We are not aware of any representation-independent hardness results for learning polynomial-size depth-2 neural networks.

Our approach uses the framework of the breakthrough result due to Kabanets and Impagliazzo showing that derandomizing BPP yields non-trivial circuit lower bounds.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Lance Fortnow
    • 1
    • 2
  • Adam R. Klivans
    • 1
    • 2
  1. 1.U. Chicago Comp. Sci.Chicago
  2. 2.UT-Austin Comp. Sci.Austin

Personalised recommendations