Abstract
This article studies self-directed learning, a variant of the on-line (or incremental) learning model in which the learner selects the presentation order for the instances. Alternatively, one can view this model as a variation of learning with membership queries in which the learner is only “charged” for membership queries for which it could not predict the outcome. We give tight bounds on the complexity of self-directed learning for the concept classes of monomials, monotone DNF formulas, and axis-parallel rectangles in {0, 1, ...,n − 1}d. These results demonstrate that the number of mistakes under self-directed learning can be surprisingly small. We then show that learning complexity in the model of self-directed learning is less than that of all other commonly studied on-line and query learning models. Next we explore the relationship between the complexity of self-directed learning and the Vapnik-Chervonenkis (VC-)dimension. We show that, in general, the VC-dimension and the self-directed learning complexity are incomparable. However, for some special cases, we show that the VC-dimension gives a lower bound for the self-directed learning complexity. Finally, we explore a relationship between Mitchell's version space algorithm and the existence of self-directed learning algorithms that make few mistakes.
Article PDF
Similar content being viewed by others
References
Angluin, Dana. (1988). Queries and concept learning.Machine Learning, 2(4, 319–342.
Blumer, Anselm, Ehrenfeucht, Andrzeg, Haussler, David, & Warmuth, Manfred K. (1989). Learnability and the Vapnik-Chervonenkis dimension.Journal of the Association for Computing Machinery, 36(4, 929–965.
Bundy, A., Silver, B., & Plummer, D. (1985). An analytical comparison of some rule-learning programs.Artificial Intelligence, 27, 137–181.
Ehrenfeucht, Andrzej, & Haussler, David. (1989). Learning decision trees from random examples.Information and Computation, 82(3, 231–246.
Floyd, Sally. (1989).On space-bounded learning and the Vapnik-Chervonenkis dimension. Ph.D. thesis, International Computer Science Institute, Berkeley, CA.
Goldman, Sally A., Rivest, Ronald L., & Schapire, Robert E. (1993). Learning binary relations and total orders.SIAM Journal on Computing, 22(5, 1006–1034.
Haussler, David. (1987). Learning conjunctive concepts in structural domains (Technical Report USCS-CRL-87-1). Santa Cruz, CA: Computer Research Laboratory, University of Santa Cruz.
Haussler, David. (1988). Quantifying inductive bias: AI learning algorithms and Valiant's learning framework.Artificial Intelligence, 36, 177–221.
Haussler, David, Littlestone, Nick, & Warmuth, Manfred K. (1988). Predicting {0, 1}-functions on randomly drawn points. In29th Annual Symposium on Foundations of Computer Science, White Plains, NY (pp. 100–109), IEEE Computer Society Press: Los Alamitos, CA.
Helmbold, David, Sloan, Robert, & Warmuth, Manfred K. (1990). Learning nested differences of intersection-closed concept classes.Machine Learning, 5, 165–196. Special issue for COLT 89.
Helmbold, David, Sloan, Robert, & Warmuth, Manfred K. (1992). Learning integer lattices.SIAM Journal on Computing, 21(2, 240–266.
Littlestone, Nick. (1988). Learning when irrelevant attributes abound: A new linear-threshold algorithm.Machine Learning, 2, 285–318.
Maass, Wolfgang, & Turán, György. (1989). On the complexity of learning from counterexamples. In30th Annual Symposium on Foundations of Computer Science, Research Triangle Park, NC (pp. 262–267), IEEE Computer Society Press: Los Alamitos, CA.
Maass, Wolfgang, & Turán, György. (1992). Lower bound methods and separation results for on-line learning models.Machine Learning, 9, 107–145.
Mitchell, Thomas M. (1982). Generalization as search.Artificial Intelligence, 18, 203–226.
Rivest, Ronald L., & Sloan, Robert (In press). A formal model of hierarchical concept learning.Information and Computation.
Sauer, A. (1972). On the density of families of sets.Journal of Combinatorial Theory (A), 13, 145–147.
Valiant, Leslie. (1984). A theory of the learnable.Communications of the ACM, 27(11, 1134–1142.
Vapnik, V.N., & Chervonenkis, A. Ya. (1971). On the uniform convergence of relative frequencies of events to their probabilities.Theory of Probability and Its Applications, XVI(2, 264–280.
Welzl, E. (1987). Complete range spaces. Unpublished manuscript.
Welzl, E., & Woeginger, G. (1986). On Vapnik-Chervonenkis dimension one. Unpublished manuscript.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Goldman, S.A., Sloan, R.H. The power of self-directed learning. Mach Learn 14, 271–294 (1994). https://doi.org/10.1007/BF00993977
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00993977