PAC Learning from Positive Statistical Queries

  • FranÇois Denis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1501)


Learning from positive examples occurs very frequently in natural learning. The PAC learning model of Valiant takes many features of natural learning into account,but in most cases it fails to describe such kind of learning. We show that in order to make the learning from positive data possible, extra-information about the underlying distribution must be provided to the learner. We define a PAC learning model from positive and unlabeled examples. We also define a PAC learning model from positive and unlabeled statistical queries.Relations with PAC model ([Val84]), statistical query model ([Kea93]) and constant-partition classification noise model ([Dec97]) are studied. We show that k DNF and k decision lists are learnable in both models, i.e. with far less information than it is assumed in previously used algorithms.


Learning Algorithm Boolean Function Concept Class Unlabeled Data Underlying Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. AL88.
    D. Angluin and P. Laird. Learning from noisy examples. Machine Learning, 2(4):343–370, 1988.Google Scholar
  2. Ang80.
    D. Angluin. Inductive inference of formal languages from positive data. In-form. Control, 45(2):117–135, May 1980.Google Scholar
  3. BDL97.
    Shai Ben-David and Michael Lindenbaum. Learning distributions by their density levels:A paradigm for learning without a teacher. Journal of Com-puter and System Sciences, 55(1):171–182, August 1997.Google Scholar
  4. Ber86.
    R. Berwick. Learning from positive-only examples. In Machine Learning, Vol. II, pages 625–645. Morgan Kaufmann, 1986.Google Scholar
  5. Dec97.
    S.E. Decatur. Pac learning with constant-partition classification noise and applications to decision tree induction. In Proceedings of the Fourteenth In-ternational Conference on Machine Learning, 1997.Google Scholar
  6. Den98.
    F. Denis. Pac learning from positive statistical queries. Technical report, L.I.F.L., 1998. full version:
  7. Gol67.
    E.M. Gold. Language identification in the limit. Inform. Control, 10:447–474, 1967.CrossRefzbMATHGoogle Scholar
  8. HSW92.
    D. Helmbold, R. Sloan, and M.K. Warmuth. Learning integer lattices. SIAM J. COMPUT., 21(2):240–266, 1992.zbMATHCrossRefMathSciNetGoogle Scholar
  9. Kea93.
    M. Kearns. Efficient noise-tolerant learning from statistical queries. In Pro-ceedings of the 25th ACM Symposium on the Theory of Computing, pages 392–401. ACM Press, New York, NY, 1993.Google Scholar
  10. KV94.
    M.J. Kearns and U.V. Vazirani. An Introduction to Computational Learning Theory. MIT Press, 1994.Google Scholar
  11. Nat87.
    B.K. Natarajan. On learning boolean functions. In Proceedings of the 19th Annual ACM Symposium on Theory of Computing, pages 296–304. ACM Press, 1987.Google Scholar
  12. Nat91.
    B.K. Natarajan. Probably approximate learning of sets and functions. SIAM J. COMPUT., 20(2):328–351, 1991.zbMATHCrossRefMathSciNetGoogle Scholar
  13. Riv87.
    R.L. Rivest. Learning decision lists. Machine Learning, 2(3):229–246, 1987.Google Scholar
  14. Shi90.
    Takeshi Shinohara. Inductive inference from positive data is powerful. In Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 97–110, Rochester, New York, 6–8 August 1990. ACM Press.Google Scholar
  15. Shv90.
    Haim Shvayster. A necessary condition for learning from positive examples. Machine Learning, 5:101–113, 1990.Google Scholar
  16. Val84.
    L.G. Valiant. A theory of the learnable. Commun. ACM 27(11):1134–1142, November 1984.Google Scholar
  17. ZL95.
    T. Zeugmann and S. Lange. A guided tour across the boudaries of learning recursive languages. In Lectures Notes in Artificial Intelligence, editor, Algorithmic learning for knowledge-based systems, volume 961, pages 190–258. 1995.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • FranÇois Denis
    • 1
  1. 1.Bât. M3, LIFL, Université de Lille IVilleneuve d’Ascq CedexFrance

Personalised recommendations