Exact Learning Algorithms, Betting Games, and Circuit Lower Bounds

  • Ryan C. Harkins
  • John M. Hitchcock
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6755)


This paper extends and improves work of Fortnow and Klivans [5], who showed that if a circuit class \(\mathcal{C}\) has an efficient learning algorithm in Angluin’s model of exact learning via equivalence and membership queries [2], then we have the lower bound EXP NP \(\not\subseteq \mathcal{C}\). We use entirely different techniques involving betting games [4] to remove the NP oracle and improve the lower bound to EXP \(\not\subseteq \mathcal{C}\). This shows that it is even more difficult to design a learning algorithm for \(\mathcal{C}\) than the results of Fortnow and Klivans indicated.


Learning Algorithm Boolean Function Target Concept Membership Query Equivalence Query 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aizenstein, H., Hegedüs, T., Hellerstein, L., Pitt, L.: Complexity theoretic hardness results for query learning. Computational Complexity 7, 19–53 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Angluin, D.: Queries and concept learning. Machine Learning 2(4), 319–342 (1988)MathSciNetGoogle Scholar
  3. 3.
    Buhrman, H., Homer, S.: Superpolynomial circuits, almost sparse oracles and the exponential hierarchy. In: Shyamasundar, R.K. (ed.) FSTTCS 1992. LNCS, vol. 652, pp. 116–127. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  4. 4.
    Buhrman, H., van Melkebeek, D., Regan, K.W., Sivakumar, D., Strauss, M.: A generalization of resource-bounded measure, with application to the BPP vs. EXP problem. SIAM Journal on Computing 30(2), 576–601 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Fortnow, L., Klivans, A.R.: Efficient learning algorithms yield circuit lower bounds. Journal of Computer and System Sciences 75(1), 27–36 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Hitchcock, J.M.: Online learning and resource-bounded dimension: Winnow yields new lower bounds for hard sets. SIAM Journal on Computing 36(6), 1696–1708 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Karp, R.M., Lipton, R.J.: Some connections between nonuniform and uniform complexity classes. In: Proceedings of the 12th Annual ACM Symposium on Theory of Computing, pp. 302–309 (1980)Google Scholar
  8. 8.
    Kearns, M., Valiant, L.: Cryptographic limitations on learning Boolean formulae and finite automata. J. ACM 41(1), 67–95 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Kharitonov, M.: Cryptographic lower bounds for learnability of Boolean functions on the uniform distribution. J. of Comput. Syst. Sci. 50(3), 600–610 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Lindner, W., Schuler, R., Watanabe, O.: Resource-bounded measure and learnability. Theory of Computing Systems 33(2), 151–170 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2(4), 285–318 (1988)Google Scholar
  12. 12.
    Lutz, J.H.: Almost everywhere high nonuniform complexity. Journal of Computer and System Sciences 44(2), 220–258 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Merkle, W., Miller, J., Nies, A., Reimann, J., Stephan, F.: Kolmogorov-Loveland Randomness and Stochasticity. Annals of Pure and Applied Logic 138(1-3), 183–210 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Muchnik, A.A., Semenov, A.L., Uspensky, V.A.: Mathematical metaphysics of randomness. Theoretical Computer Science 207(2), 263–317 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Toda, S.: On the computational power of PP and ⊕P. SIAM Journal on Computing 20(5), 865–877 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Valiant, L.G.: A theory of the learnable. Communications of the ACM 27(11), 1134–1142 (1984)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ryan C. Harkins
    • 1
  • John M. Hitchcock
    • 1
  1. 1.Department of Computer ScienceUniversity of WyomingUSA

Personalised recommendations