Abstract
For hyper-rectangles in R d Auer et al. [1] proved a PAC bound of \(O(\frac{1}{\epsilon}(d+{\rm log}\frac{1}{\delta}))\), where ε and δ are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension d in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of \(O(\frac{1}{\epsilon}(d{\rm log}d+\frac{1}{\delta}))\) for arbitrary intersection-closed concept classes complementing the well-known bounds \(O(\frac{1}{\epsilon}({\rm log}\frac{1}{\delta}+d{\rm log}\frac{1}{\epsilon}))\) and \(O(\frac{d}{\epsilon}{\rm log}\frac{1}{\delta})\) of Blumer et al. and Haussler et al. [4,6]. Our bound is established using the closure algorithm, that generates as its hypothesis the smallest concept that is consistent with the positive training examples. On the other hand, we show that maximum intersection-closed concept classes meet the bound of \(O(\frac{1}{\epsilon}(d+{\rm log}\frac{1}{\delta}))\) as well. Moreover, we indicate that our new as well as the conjectured bound cannot hold for arbitrary consistent learning algorithms, giving an example of such an algorithm that needs \(\Omega(\frac{1}{\epsilon}(d+{\rm log}\frac{1}{\epsilon}+{\rm log \frac{1}{\delta}}))\) examples to learn some simple maximum intersection-closed concept class.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Auer, P.: Learning Nested Differences in the Presence of Malicious Noise. Theor. Comput. Sci. 185(1), 159–175 (1997)
Auer, P., Cesa-Bianchi, N.: On-Line Learning with Malicious Noise and the Closure Algorithm. Annals of Mathematics and Artificial Intelligence 23(1-2), 83–99 (1998)
Auer, P., Long, P.M., Srinivasan, A.: Approximating Hyper-Rectangles: Learning and Pseudorandom Sets. J. Comput. Syst. Sci. 57(3), 376–388 (1998)
Blumer, A., Ehrenfeucht, D., Haussler, M.: Learnability and the Vapnik- Chervonenkis Dimension. J. ACM 36(4), 929–965 (1989)
Floyd, S., Warmuth, M.: Sample Compression, Learnability, and the Vapnik- Chervonenkis Dimension. Machine Learning 21(3), 269–304 (1995)
Haussler, D., Littlestone, N., Warmuth, M.: Predicting 0,1Functions on Randomly Drawn Points. Inf. Comput. 115(2), 248–292 (1994)
Helmbold, D., Sloan, R., Warmuth, M.: Learning Nested Differences of Intersection- Closed Concept Classes. Machine Learning 5, 165–196 (1990)
Leighton, F.T., Plaxton, C.G.: Hypercubic Sorting Networks. SIAM J. Comput. 27(1), 1–47 (1998)
Sauer, N.: On the Density of Families of Sets. J. Combinatorial Theory (A) 13, 145–147 (1972)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Auer, P., Ortner, R. (2004). A New PAC Bound for Intersection-Closed Concept Classes. In: Shawe-Taylor, J., Singer, Y. (eds) Learning Theory. COLT 2004. Lecture Notes in Computer Science(), vol 3120. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-27819-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-27819-1_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22282-8
Online ISBN: 978-3-540-27819-1
eBook Packages: Springer Book Archive