ALT 2012: Algorithmic Learning Theory pp 96-110 | Cite as
Sauer’s Bound for a Notion of Teaching Complexity
Abstract
This paper establishes an upper bound on the size of a concept class with given recursive teaching dimension (RTD, a teaching complexity parameter.) The upper bound coincides with Sauer’s well-known bound on classes with a fixed VC-dimension. Our result thus supports the recently emerging conjecture that the combinatorics of VC-dimension and those of teaching complexity are intrinsically interlinked.
We further introduce and study RTD-maximum classes (whose size meets the upper bound) and RTD-maximal classes (whose RTD increases if a concept is added to them), showing similarities but also differences to the corresponding notions for VC-dimension.
Another contribution is a set of new results on maximal classes of a given VC-dimension.
Methodologically, our contribution is the successful application of algebraic techniques, which we use to obtain a purely algebraic characterization of teaching sets (sample sets that uniquely identify a concept in a given concept class) and to prove our analog of Sauer’s bound for RTD.
Keywords
VC-dimension teaching Sauer’s bound maximum classesPreview
Unable to display preview. Download preview PDF.
References
- 1.Doliwa, T.: Personal communication (2011)Google Scholar
- 2.Doliwa, T., Simon, H.U., Zilles, S.: Recursive Teaching Dimension, Learning Complexity, and Maximum Classes. In: Hutter, M., Stephan, F., Vovk, V., Zeugmann, T. (eds.) ALT 2010. LNCS, vol. 6331, pp. 209–223. Springer, Heidelberg (2010)CrossRefGoogle Scholar
- 3.Floyd, S., Warmuth, M.K.: Sample compression, learnability, and the Vapnik-Chervonenkis dimension. Machine Learning 21(3), 269–304 (1995)Google Scholar
- 4.Goldman, S.A., Kearns, M.J.: On the complexity of teaching. Journal of Computer and System Sciences 50, 20–31 (1995)MathSciNetMATHCrossRefGoogle Scholar
- 5.Kuzmin, D., Warmuth, M.K.: Unlabeled compression schemes for maximum classes. J. Mach. Learn. Res. 8, 2047–2081 (2007)MathSciNetMATHGoogle Scholar
- 6.Rubinstein, B.I.P., Bartlett, P.L., Rubinstein, J.H.: Shifting: One-inclusion mistake bounds and sample compression. J. Comput. Syst. Sci. 75(1), 37–59 (2009)MathSciNetMATHCrossRefGoogle Scholar
- 7.Sauer, N.: On the density of families of sets. J. Comb. Theory, Ser. A 13(1), 145–147 (1972)MathSciNetMATHCrossRefGoogle Scholar
- 8.Shelah, S.: A combinatorial problem: Stability and order for models and theories in infinitary languages. Pac. J. Math. 4, 247–261 (1972)Google Scholar
- 9.Shinohara, A., Miyano, S.: Teachability in computational learning. New Generation Comput. 8(4), 337–347 (1991)MATHCrossRefGoogle Scholar
- 10.Smolensky, R.: Well-known bound for the VC-dimension made easy. Computational Complexity 6(4), 299–300 (1997)MathSciNetMATHCrossRefGoogle Scholar
- 11.Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16, 264–280 (1971)MATHCrossRefGoogle Scholar
- 12.Welzl, E.: Complete range spaces. Unpublished notes (1987)Google Scholar
- 13.Zilles, S., Lange, S., Holte, R., Zinkevich, M.: Models of cooperative teaching and learning. Journal of Machine Learning Research 12, 349–384 (2011)MathSciNetGoogle Scholar