Algorithmica

, Volume 14, Issue 1, pp 70–84 | Cite as

Can PAC learning algorithms tolerate random attribute noise?

  • S. A. Goldman
  • R. H. Sloan
Article

Abstract

This paper studies the robustness of PAC learning algorithms when the instance space is {0,1}n, and the examples are corrupted by purely random noise affecting only the attributes (and not the labels). Foruniform attribute noise, in which each attribute is flipped independently at random with the same probability, we present an algorithm that PAC learns monomials for any (unknown) noise rate less than21. Contrasting this positive result, we show thatproduct random attribute noise, where each attributei is flipped randomly and independently with its own probability pi, is nearly as harmful as malicious noise-no algorithm can tolerate more than a very small amount of such noise.

Key words

PAC learning Attribute noise Computational learning theory 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    D. Angluin and P. Laird. Learning from noisy examples.Machine Learning, 2(4):343–370, 1988.Google Scholar
  2. [2]
    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension.Journal of the ACM, 36(4):929–965, 1989.Google Scholar
  3. [3]
    D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability.Information and Computation, 95(2): 129–161, 1991.Google Scholar
  4. [4]
    W. Hoeffding. Probability inequalities for sums of bounded random variables.Journal of the American Statistical Association, 58(301): 13–30, 1963.Google Scholar
  5. [5]
    M. Kearns and Ming Li. Learning in the presence of malicious errors.SIAM Journal of Computing, 22(4): 807–837, 1993.Google Scholar
  6. [6]
    P. D. Laird.Learning from Good and Bad Data. Kluwer International Series in Engineering and Computer Science. Kluwer Academic, Boston, 1988.Google Scholar
  7. [7]
    N. Littlestone. Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow.Proceedings of the Fourth Workshop on Computational Learning Theory, 1991, pp. 147–156.Google Scholar
  8. [8]
    J. R. Quinlan. The effect of noise on concept learning. InMachine Learning, An Artificial Intelligence Approach, Volume II, Morgan Kaufmann, Los Altos, CA, 1986, Chapter 6, pp. 149–166.Google Scholar
  9. [9]
    G. Shackelford and D. Volper. Learning k-DNF with noise in the attributes.Proceedings of the First Workshop on Computational Learning Theory, Cambridge, MA, August 1988. Morgan Kaufmann, Los Altos, CA, 1988, pp. 97–103.Google Scholar
  10. [10]
    R. H. Sloan. Types of noise in data for concept learning.Proceedings of the First Workshop on Computational Learning Theory, Cambridge, MA, August 1988. Morgan Kaufmann, Los Altos, CA, 1988, pp. 91–96.Google Scholar
  11. [11]
    L. G. Valiant. A theory of the learnable.Communications of the ACM, 27(11):1134–1142, 1984.Google Scholar
  12. [12]
    L. G. Valiant. Learning disjunctions of conjunctions. InProceedings IJCAI 85. Morgan Kaufmann, Los Altos, CA, 1985, pp. 560–566.Google Scholar

Copyright information

© Springer-Verlag New York Inc 1995

Authors and Affiliations

  • S. A. Goldman
    • 1
  • R. H. Sloan
    • 2
  1. 1.Department of Computer ScienceWashington UniversitySt. LouisUSA
  2. 2.Department of Electrical Engineering and Computer ScienceUniversity of Illinois at ChicagoChicagoUSA

Personalised recommendations