On the Relationship between Models for Learning in Helpful Environments
The PAC and other equivalent learning models are widely accepted models for polynomial learnability of concept classes. However, negative results abound in the PAC learning framework (concept classes such as deterministic finite state automata (DFA) are not efficiently learnable in the PAC model). The PAC model’s requirement of learnability under all conceivable distributions could be considered too stringent a restriction for practical applications. Several models for learning in more helpful environments have been proposed in the literature including: learning from example based queries , online learning allowing a bounded number of mistakes , learning with the help of teaching sets , learning from characteristic sets , and learning from simple examples [12,4]. Several concept classes that are not learnable in the standard PAC model have been shown to be learnable in these models. In this paper we identify the relationships between these different learning models. We also address the issue of unnatural collusion between the teacher and the learner that can potentially trivialize the task of learning in helpful environments.
KeywordsModels of learning Query learning Mistake bounded learning PAC learning teaching sets characteristic samples DFA learning
Unable to display preview. Download preview PDF.
- 2.Angluin, D.: Queries and concept learning. Machine Learning 2(4), 319–342 (1988)Google Scholar
- 3.Castro, J., Guijarro, D.: Query, PACS and simple-PAC learning. Technical Report LSI-98-2-R, Universitat Polytéctica de Catalunya, Spain (1998)Google Scholar
- 4.Denis, F., D’Halluin, C., Gilleron, R.: Pac learning with simple examples. In: STACS 1996 - Proceedings of the 13th Annual Symposium on the Theoretical Aspectsof Computer Science, pp. 231–242 (1996)Google Scholar
- 6.Goldman, S., Mathias, H.: Teaching a smarter learner. In: Proceedings of the Workshop on Computational Learning Theory (COLT 1993), pp. 67–76. ACM Press, New York (1993)Google Scholar
- 11.Kearns, M., Valiant, L.G.: Cryptographic limitations on learning boolean formulae and finite automata. In: Proceedings of the 21st Annual ACM Symposium on Theory of Computing, New York, pp. 433–444 (1989)Google Scholar
- 14.Littlestone, N.: Learning quickly when irrelevant attributes abound: A new linearthreshold algorithm. Machine Learning 2, 285–318 (1988)Google Scholar
- 16.Parekh, R., Honavar, V.: Simple DFA are polynomially probably exactly learnable from simple examples. In: Proceedings of the Sixteenth International Conference on Machine Learning (ICML 1999), Bled, Slovenia, pp. 298–306 (1999)Google Scholar
- 17.Parekh, R.G., Honavar, V.G.: Learning DFA from simple examples. In: Li, M. (ed.) ALT 1997. LNCS (LNAI), vol. 1316, pp. 116–131. Springer, Heidelberg (1997)Google Scholar
- 19.Pitt, L., Warmuth, M.K.: Reductions among prediction problems: on the difficulty of predicting automata. In: Proceedings of the 3rd IEEE Conference on Structure in Complexity Theory, pp. 60–69 (1988)Google Scholar