Learning by extended statistical queries and its relation to PAC learning
PAC learning from examples is factored so that (i) the membership queries are used to evaluate empirically “statistical queries” — certain expectations of functionals involving the unknown target. (ii) approximate value of these statistical queries are used to compute an output — an approximation of the target.
Kearns' original formulation of statistical queries [we use the abbreviation SQ], is extended here to include as SQ functionals of arbitrary range and order higher than one — second order being the most useful addition. This enables us to capture more ground for casting efficient PAC learning algorithms in SQ form: The important Kushilevitz-Mansour Fourier - based algorithm has an SQ rendition, as well as its derivatives, e.g. Jackson's recent DNF learning.
Efficient evaluation of extended SQ by membership queries, if possible at all, becomes quite intricate. We show, however, that it is usually robust under classification noise.
Unable to display preview. Download preview PDF.
- [0 ]Javed A. Aslam and Scott E. Decatur, Simulating statistical queries in the presence of classification noise, unpublished manuscript, 1993.Google Scholar
- [1 ]Avrim Blum, Merrick Furst, Jeffrey Jackson, Michael Kearns, Yishay Mansour, Steven Rudich, Weakly learning DNF and characterizing statistical query learning using Fourier analysis. Proceedings of the 26th Annual ACM Symposium on Theory of Computation, 253–262, 1994.Google Scholar
- [2 ]Jeffrey Jackson, An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution, Proceedings of the 35th Annual IEEE Symposium on the Foundation of Computer Science, 42–53, 1994.Google Scholar
- [3 ]Michael J. Kearns, Efficient noise-tolerant learning form statistical queries. In Proceedings of the 25th Annual ACM Symposium on Theory of Computing pages 392–401, 1993.Google Scholar
- [4 ]Eyal Kushilevitz and Yishay Mansour, Learning decision trees using the Fourier transform. SIAM Journal con Computing, 22(6) 1331–1348, 1993.Google Scholar
- [5 ]Thimothy L. H. Watkins, Albrecht Rau and Michael Biehl, The Statistical Mechanics of Learning a Rule, Reviews of Modern Physics 65, p.499, 1993.Google Scholar
- [6 ]Dana Angluin, Queries and Concept Learning, Machine Learning 2, 219–342, 1988.Google Scholar
- [7 ]Leonid Khachiyan, Complexity of Polytope Volume Computation, In: New Trends in Discrete and Computation Geometry, Janos Pach (Ed.), Springer Verlag, 1993.Google Scholar