We study the average-case learnability of DNF formulas in the model of learning from uniformly distributed random examples. We define a natural model of random monotone DNF formulas and give an efficient algorithm which with high probability can learn, for any fixed constant γ> 0, a random t-term monotone DNF for any t = O(n 2 − γ). We also define a model of random nonmonotone DNF and give an efficient algorithm which with high probability can learn a random t-term DNF for any t=O(n 3/2 − γ). These are the first known algorithms that can successfully learn a broad class of polynomial-size DNF in a reasonable average-case model of learning from random examples.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aizenstein, H., Pitt, L.: On the learnability of disjunctive normal form formulas. Machine Learning 19, 183–208 (1995)zbMATHGoogle Scholar
  2. 2.
    Angluin, D.: Queries and concept learning. Machine Learning 2, 319–342 (1988)Google Scholar
  3. 3.
    Angluin, D., Kharitonov, M.: When won’t membership queries help? J. Comput. & Syst. Sci. 50, 336–355 (1995)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Blum, A.: Learning a function of r relevant variables (open problem). In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 731–733. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  5. 5.
    Blum, A.: Machine learning: a tour through some favorite results, directions, and open problems. FOCS 2003 tutorial slides (2003), available at
  6. 6.
    Blum, A., Burch, C., Langford, J.: On learning monotone boolean functions. In: Proc. 39th FOCS, pp. 408–415 (1998)Google Scholar
  7. 7.
    Blum, A., Furst, M., Jackson, J., Kearns, M., Mansour, Y., Rudich, S.: Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In: Proc. 26th STOC, pp. 253–262 (1994)Google Scholar
  8. 8.
    Bollobas, B.: Combinatorics: Set Systems, Hypergraphs, Families of Vectors and Combinatorial Probability. Cambridge University Press, Cambridge (1986)zbMATHGoogle Scholar
  9. 9.
    Golea, M., Marchand, M., Hancock, T.: On learning μ-perceptron networks on the uniform distribution. Neural Networks 9, 67–82 (1994)CrossRefGoogle Scholar
  10. 10.
    Hancock, T.: Learning kμ decision trees on the uniform distribution. In: Proc. Sixth COLT, pp. 352–360 (1993)Google Scholar
  11. 11.
    Jackson, J.: An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. J. Comput. & Syst. Sci. 55, 414–440 (1997)zbMATHCrossRefGoogle Scholar
  12. 12.
    Jackson, J., Klivans, A., Servedio, R.: Learnability beyond ACo. In: Proc. 34th STOC (2002)Google Scholar
  13. 13.
    Jackson, J., Servedio, R.: Learning random log-depth decision trees under the uniform distribution. In: Schölkopf, B., Warmuth, M.K. (eds.) COLT/Kernel 2003. LNCS (LNAI), vol. 2777, pp. 610–624. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Jackson, J., Tamon, C.: Fourier analysis in machine learning. ICML/COLT 1997 tutorial slides (1997), available at
  15. 15.
    Kearns, M.: Efficient noise-tolerant learning from statistical queries. Journal of the ACM 45(6), 983–1006 (1998)zbMATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Kearns, M., Li, M., Pitt, L., Valiant, L.: Recent results on Boolean concept learning. In: Proc. Fourth Int. Workshop on Mach. Learning, pp. 337–352 (1987)Google Scholar
  17. 17.
    Kearns, M., Vazirani, U.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)Google Scholar
  18. 18.
    Klivans, A., O’Donnell, R., Servedio, R.: Learning intersections and thresholds of halfspaces. In: Proc. 43rd FOCS, pp. 177–186 (2002)Google Scholar
  19. 19.
    Kucera, L., Marchetti-Spaccamela, A., Protassi, M.: On learning monotone DNF formulae under uniform distributions. Inform. and Comput. 110, 84–95 (1994)zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    McDiarmid, C.: On the method of bounded differences. In: Surveys in Combinatoric 1989. London Mathematical Society Lecture Notes, pp. 148–188 (1989)Google Scholar
  21. 21.
    Servedio, R.: On learning monotone DNF under product distributions. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, pp. 473–489. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  22. 22.
    Valiant, L.: A theory of the learnable. CACM 27(11), 1134–1142 (1984)zbMATHGoogle Scholar
  23. 23.
    Verbeurgt, K.: Learning DNF under the uniform distribution in quasi-polynomial time. In: Proc. Third COLT, pp. 314–326 (1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Jeffrey C. Jackson
    • 1
  • Rocco A. Servedio
    • 2
  1. 1.Dept. of Math. and Computer ScienceDuquesne UniversityPittsburghUSA
  2. 2.Dept. of Computer ScienceColumbia UniversityNew YorkUSA

Personalised recommendations