Abstract
This paper presents a polynomial-time algorithm for inferring a probabilistic generalization of the class of read-once Boolean formulas over the usual basis {AND, OR, NOT}. The algorithm effectively infers a good approximation of the target formula when provided with random examples which are chosen according to anyproduct distribution, i.e., any distribution in which the setting of each input bit is chosen independently of the settings of the other bits. Since the class of formulas considered includes ordinary read-once Boolean formulas, our result shows that such formulas are PAC learnable (in the sense of Valiant) against any product distribution (for instance, against the uniform distribution). Further, this class of probabilistic formulas includes read-once formulas whose behavior has been corrupted by large amounts of random noise. Such noise may affect the formula's output (“misclassification noise”), the input bits (“attribute noise”), or it may affect the behavior of individual gates of the formula. Thus, in this setting, we show that read-once formula's can be inferred (approximately), despite large amounts of noise affecting the formula's behavior.
Article PDF
Similar content being viewed by others
References
Angluin, D., Hellerstein, L., and Karpinski, M. (1993). Learning read-once formulas with queries.Journal of the Association for Computing Machinery, 40(1):185–210.
Angluin, D. and Laird, P. (1988). Learning from noisy examples.Machine Learning, 2(4):343–370.
Blumer, A., Ehrenfeucht, A., Haussler, D., and Warmuth, M. K. (1987). Occam's razor.Information Processing Letters, 24(6):377–380.
Bshouty, N. H., Hancock, T. R., and Hellerstein, L. (1992). Learning arithmetic read-once formulas. InProceedings of the Twenty-Fourth Annual ACM Syrnposium on the Theory of Computing, pages 370–381.
Furst, M. L., Jackson, J. C., and Smith, S. W. (1991). Improved learning ofAC 0 functions. InProceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 317–325.
Goldman, S. A., Kearns, M. J., and Schapire, R. E. (1990). Exact identification of circuits using fixed points of amplification functions. In31st Annual Symposium on Foundations of Computer Science, pages 193–202. To appear,SIAM Journal on Computing.
Hancock, T. and Hellerstein, L. (1991). Learning read-once formulas over fields and extended bases. InProceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 326–336.
Hancock, T. and Mansour, Y. (1991). Learning monotonekμ DNF formulas on product distributions. InProceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 179–183.
Hancock, T. R. (1990). Identifying μ-formula decision trees with queries. InProceedings of the Third Annual Workshop on Computational Learning Theory, pages 23–37.
Hellerstein, L. and Karpinski, M. (1990). Read-once formulas over different bases. Technical Report 8556-CS, University of Bonn.
Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables.Journal of the American Statistical Association, 58(301):13–30.
Kearns, M., Li, M., Pitt, L., and Valiant, L. (1987). On the learnability of Boolean formulae. InProceedings of the Nineteenth Annual ACM Symposium on Theory of Computing, pages 285–295.
Kearns, M. and Valiant, L. G. (1989). Cryptographic limitations on learning Boolean formulae and finite automata. InProceedings of the Twenty First Annual ACM Symposium on Theory of Computing, pages 433–444. To appear,Journal of the Association for Computing Machinery.
Kearns, M. J. and Schapire, R. E. (1990). Efficient distribution-free learning of probabilistic concepts. In31st Annual Symposium on Foundations of Computer Science, pages 382–391. To appear,Journal of Computer and System Sciences.
Linial, N., Mansour, Y., and Nisan, N. (1989). Constant depth circuits, Fourier transform, and learnability. In30th Annual Symposium on Foundations of Computer Science, pages 574–579.
Pagallo, G. and Haussler, D. (1989). A greedy method for learning μDNF functions under the uniform distribution. Technical Report UCSC-CRL-89-12, University of California Santa Cruz, Computer Research Laboratory.
Sloan, R. H. (1988). Types of noise in data for concept learning. InProceedings of the 1988 Workshop on Computational Learning Theory, pages 91–96.
Valiant, L. G. (1984). A theory of the learnable.Communications of the ACM, 27(11):1134–1142.
Verbeurgt, K. (1990). Learning DNF under the uniform distribution in quasi-polynomial time. InProceedings of the Third Annual Workshop on Computational Learning Theory, pages 314–326.
Yamanishi, K. (1992). A learning criterion for stochastic rules.Machine Learning, 9(2/3):165–203.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Schapire, R.E. Learning probabilistic read-once formulas on product distributions. Mach Learn 14, 47–81 (1994). https://doi.org/10.1007/BF00993162
Issue Date:
DOI: https://doi.org/10.1007/BF00993162