Abstract
This paper studies the robustness of PAC learning algorithms when the instance space is {0,1}n, and the examples are corrupted by purely random noise affecting only the attributes (and not the labels). Foruniform attribute noise, in which each attribute is flipped independently at random with the same probability, we present an algorithm that PAC learns monomials for any (unknown) noise rate less than 12 . Contrasting this positive result, we show thatproduct random attribute noise, where each attributei is flipped randomly and independently with its own probability pi, is nearly as harmful as malicious noise-no algorithm can tolerate more than a very small amount of such noise.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D. Angluin and P. Laird. Learning from noisy examples.Machine Learning, 2(4):343–370, 1988.
A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension.Journal of the ACM, 36(4):929–965, 1989.
D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth. Equivalence of models for polynomial learnability.Information and Computation, 95(2): 129–161, 1991.
W. Hoeffding. Probability inequalities for sums of bounded random variables.Journal of the American Statistical Association, 58(301): 13–30, 1963.
M. Kearns and Ming Li. Learning in the presence of malicious errors.SIAM Journal of Computing, 22(4): 807–837, 1993.
P. D. Laird.Learning from Good and Bad Data. Kluwer International Series in Engineering and Computer Science. Kluwer Academic, Boston, 1988.
N. Littlestone. Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow.Proceedings of the Fourth Workshop on Computational Learning Theory, 1991, pp. 147–156.
J. R. Quinlan. The effect of noise on concept learning. InMachine Learning, An Artificial Intelligence Approach, Volume II, Morgan Kaufmann, Los Altos, CA, 1986, Chapter 6, pp. 149–166.
G. Shackelford and D. Volper. Learning k-DNF with noise in the attributes.Proceedings of the First Workshop on Computational Learning Theory, Cambridge, MA, August 1988. Morgan Kaufmann, Los Altos, CA, 1988, pp. 97–103.
R. H. Sloan. Types of noise in data for concept learning.Proceedings of the First Workshop on Computational Learning Theory, Cambridge, MA, August 1988. Morgan Kaufmann, Los Altos, CA, 1988, pp. 91–96.
L. G. Valiant. A theory of the learnable.Communications of the ACM, 27(11):1134–1142, 1984.
L. G. Valiant. Learning disjunctions of conjunctions. InProceedings IJCAI 85. Morgan Kaufmann, Los Altos, CA, 1985, pp. 560–566.
Author information
Authors and Affiliations
Additional information
Communicated by D. T. Lee.
The research of S. A. Goldman was supported in part by a GE Foundation Junior Faculty grant and NSF Grant CCR-9110108. Part of this research was conducted while the author was at the M.I.T. Laboratory for Computer Science and supported by NSF Grant DCR-8607494 and a grant from the Siemens Corporation. The research of R. H. Sloan was supported in part by NSF Grant CCR-9108753. Part of this research was conducted while the author was at Harvard and supported by ARO Grant DAAL 03-86-K-0171.
Rights and permissions
About this article
Cite this article
Goldman, S.A., Sloan, R.H. Can PAC learning algorithms tolerate random attribute noise?. Algorithmica 14, 70–84 (1995). https://doi.org/10.1007/BF01300374
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01300374