Exploring Learnability between Exact and PAC
We study a model of Probably Exactly Correct (PExact) learning that can be viewed either as the Exact model (learning from Equivalence Queries only) relaxed so that counterexamples to equivalence queries are distributionally drawn rather than adversarially chosen or as the Probably Approximately Correct (PAC) model strengthened to require a perfect hypothesis. We also introduce a model of Probably Almost Exactly Correct (PAExact) learning that requires a hypothesis with negligible error and thus lies between the PExact and PAC models. Unlike the Exact and PExact models, PAExact learning is applicable to classes of functions defined over infinite instance spaces. We obtain a number of separation results between these models. Of particular note are some positive results for efficient parallel learning in the PAExact model, which stand in stark contrast to earlier negative results for efficient parallel Exact learning.
KeywordsExact Model Separation Result Equivalence Query Instance Space Probably Approximately Correct
Unable to display preview. Download preview PDF.
- 1.M. Anthony, A. Biggs. Computational Learning Theory. Cambridge University Press, 1992.Google Scholar
- 2.Dana Angluin. Queries and Concept Learning. Machine Learning, 2:319–342, 1988.Google Scholar
- 3.Dana Angluin. Negative Results for Equivalence Queries. Machine Learning, 5:121–150, 1990.Google Scholar
- 5.Nader H. Bshouty. Towards the Learnability of DNF Formulae. Proceedings of the ACM Annual Symposium on Theory of Computing, 1996.Google Scholar
- 9.Matthias Krause, Pavel Pudlak. On the Computational Power of Depth 2 Circuits with Threshold and Modulo Gates Proceedings of the ACM Annual Symposium on Theory of Computing, pages 48–57, 1994.Google Scholar
- 10.Nick Littlestone. Learning Quickly When Irrelevant Attributes Abound: A New Linear-threshold Algorithm. Machine Learning, 2:285–318, 1988.Google Scholar
- 12.Rajesh Parekh and Vasant Honavar. Simple DFA are polynomially probably exactly learnable from simple examples. Proceedings of the 16th International Conference on Machine Learning, Morgan Kaufmann, San Francisco, CA, 298–306, 1999.Google Scholar