Abstract
This paper presents a polynomial-time algorithm for inferring a probabilistic generalization of the class of read-once Boolean formulas over the usual basis { AND, OR, NOT }. The algorithm effectively infers a good approximation of the target formula when provided with random examples which are chosen according to any product distribution, i.e., any distribution in which the setting of each input bit is chosen independently of the settings of the other bits. Since the class of formulas considered includes ordinary read-once Boolean formulas, our result shows that such formulas are PAC learnable (in the sense of Valiant) against any product distribution (for instance, against the uniform distribution). Further, this class of probabilistic formulas includes read-once formulas whose behavior has been corrupted by large amounts of random noise. Such noise may affect the formula's output (“misclassification noise”), the input bits ( “attribute noise”), or it may affect the behavior of individual gates of the formula. Thus, in this setting, we show that read-once formula's can be inferred (approximately), despite large amounts of noise affecting the formula's behavior.
Article PDF
Similar content being viewed by others
References
Angluin, D., Hellerstein, L., and Karpinski, M. (1993). Learning read-once formulas with queries. Journal of the Association for Computing Machinery, 40 (1):185–210.
Angluin, D. and Laird, P. (1988). Learning from noisy examples. Machine Learning, 2 (4):343–370.
Blumer, A., Ehrenfeucht, A., Haussler, D., and Warmuth, M. K. (1987). Occam's razor. Information Processing Letters, 24 (6):377–380.
Bshouty, N. H., Hancock, T. R., and Hellerstein, L. (1992). Learning arithmetic read-once formulas. In Proceedings of the Twenty-Fourth Annual ACM Symposium on the Theory of Computing, pages 370–381.
Furst, M. L., Jackson, J. C., and Smith, S. W. (1991). Improved learning of AC 0 functions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 317–325.
Goldman, S. A., Kearns, M. J., and Schapire, R. E. (1990). Exact identification of circuits using fixed points of amplification functions. In 31st Annual Symposium on Foundations of Computer Science, pages 193–202. To appear, SIAM Journal on Computing.
Hancock, T. and Hellerstein, L. (1991). Learning read-once formulas over fields and extended bases. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 326–336.
Hancock, T. and Mansour, Y. (1991). Learning monotone κμ DNF formulas on product distributions. In Proceedings of the Fourth Annual Workshop on Computational Learning Theory, pages 179–183.
Hancock, T. R. (1990). Identifying μ-formula decision trees with queries. In Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 23–37.
Hellerstein, L. and Karpinski, M. (1990). Read-once formulas over different bases. Technical Report 8556-CS, University of Bonn.
Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58 (301):13–30.
Kearns, M., Li, M., Pitt, L., and Valiant, L. (1987). On the learnability of Boolean formulae. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing, pages 285–295.
Kearns, M. and Valiant, L. G. (1989). Cryptographic limitations on learning Boolean formulae and finite automata. In Proceedings of the Twenty First Annual ACM Symposium on Theory of Computing, pages 433–444. To appear, Journal of the Association for Computing Machinery.
Kearns, M. J. and Schapire, R. E. (1990). Efficient distribution-free learning of probabilistic concepts. In 31st Annual Symposium on Foundations of Computer Science, pages 382–391. To appear, Journal of Computer and System Sciences.
Linial, N., Mansour, Y., and Nisan, N. (1989). Constant depth circuits, Fourier transform, and learnability. In 30th Annual Symposium on Foundations of Computer Science, pages 574–579.
Pagallo, G. and Haussler, D. (1989). A greedy method for learning μDNF functions under the uniform distribution. Technical Report UCSC-CRL-89-12, University of California Santa Cruz, Computer Research Laboratory.
Sloan, R. H. (1988). Types of noise in data for concept learning. In Proceedings of the 1988 Workshop on Computational Learning Theory, pages 91–96.
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27 (11):1134–1142.
Verbeurgt, K. (1990). Learning DNF under the uniform distribution in quasi-polynomial time. In Proceedings of the Third Annual Workshop on Computational Learning Theory, pages 314–326.
Yamanishi, K. (1992). A learning criterion for stochastic rules. Machine Learning, 9 (2/3):165–203.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Schapire, R.E. Learning Probabilistic Read-once Formulas on Product Distributions. Machine Learning 14, 47–81 (1994). https://doi.org/10.1023/A:1022646704993
Issue Date:
DOI: https://doi.org/10.1023/A:1022646704993