Advertisement

Machine Learning

, Volume 69, Issue 2–3, pp 79–96 | Cite as

DNF are teachable in the average case

  • Homin K. Lee
  • Rocco A. Servedio
  • Andrew Wan
Article

Abstract

We study the average number of well-chosen labeled examples that are required for a helpful teacher to uniquely specify a target function within a concept class. This “average teaching dimension” has been studied in learning theory and combinatorics and is an attractive alternative to the “worst-case” teaching dimension of Goldman and Kearns which is exponential for many interesting concept classes. Recently Balbach showed that the classes of 1-decision lists and 2-term DNF each have linear average teaching dimension.

As our main result, we extend Balbach’s teaching result for 2-term DNF by showing that for any 1≤s≤2Θ(n), the well-studied concept classes of at-most-s-term DNF and at-most-s-term monotone DNF each have average teaching dimension O(ns). The proofs use detailed analyses of the combinatorial structure of “most” DNF formulas and monotone DNF formulas. We also establish asymptotic separations between the worst-case and average teaching dimension for various other interesting Boolean concept classes such as juntas and sparse GF 2 polynomials.

Keywords

DNF formulas Teaching dimension 

References

  1. Alekhnovich, M., Braverman, M., Feldman, V., Klivans, A., & Pitassi, T. (2004). Learnability and automatizability. In Proceedings of the 45th IEEE symposium on foundations of computer science (pp. 621–630). Google Scholar
  2. Anthony, M., Brightwell, G., & Shawe-Taylor, J. (1995). On specifying Boolean functions by labelled examples. Discrete Applied Mathematics, 61(1), 1–25. zbMATHCrossRefMathSciNetGoogle Scholar
  3. Balbach, F. (2005). Teaching classes with high teaching dimension using few examples. In Proceedings of the 18th annual COLT (pp. 637–651). Google Scholar
  4. Blum, A. (2003). Learning a function of r relevant variables (open problem). In Proceedings of the 16th annual COLT (pp. 731–733). Google Scholar
  5. Bshouty, N., & Mansour, Y. (2002). Simple learning algorithms for decision trees and multivariate polynomials. SIAM Journal Computer, 31(6), 1909–1925. zbMATHCrossRefMathSciNetGoogle Scholar
  6. Cherniavsky, J., & Statman, R. (1998). Testing: An abstract approach. In Proceedings of the 2nd workshop on software testing. Google Scholar
  7. Goldman, S., & Kearns, M. (1992). On the complexity of teaching. Journal of Computer and System Sciences, 50(1), 20–31. CrossRefMathSciNetGoogle Scholar
  8. Goldman, S., Rivest, R., & Schapire, R. (1993). Learning binary relations and total orders. SIAM Journal on Computing, 22(5), 1006–1034. zbMATHCrossRefMathSciNetGoogle Scholar
  9. Kuhlmann, C. (1999). On teaching and learning intersection-closed concept classes. In Proceedings of the 4th EUROCOLT (pp. 168–182). Google Scholar
  10. Kushilevitz, E., Linial, N., Rabinovich, Y., & Saks, M. (1996). Witness sets for families of binary vectors. Journal Combinatorial Theory, 73(2), 376–380. zbMATHMathSciNetGoogle Scholar
  11. Mossel, E., O’Donnell, R., & Servedio, R. (2004). Learning functions of k relevant variables. Journal of Computer and System Sciences, 69(3), 421–434. zbMATHCrossRefMathSciNetGoogle Scholar
  12. Roth, R., & Benedek, G. (1991). Interpolation and approximation of sparse multivariate polynomials over GF(2). SIAM Journal on Computing, 20(2), 291–314. zbMATHCrossRefMathSciNetGoogle Scholar
  13. Schapire, R., & Sellie, L. (1996). Learning sparse multivariate polynomials over a field with queries and counterexamples. Journal of Computer and System Sciences, 52(2), 201–213. CrossRefMathSciNetGoogle Scholar
  14. Seroussi, G., & Bshouty, N. (1988). Vector sets for exhaustive testing of logic circuits. IEEE Transactions on Information Theory, 34(3), 513–522. zbMATHCrossRefMathSciNetGoogle Scholar
  15. Shinohara, A., & Miyano, S. (1990). Teachability in computational learning. In Proceedings of the first international workshop on algorithmic learning theory (pp. 247–255). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Computer ScienceColumbia UniversityNew YorkUSA

Personalised recommendations