Skip to main content
SpringerLink
Log in

On Agnostic Learning with {0, *, 1}-Valued and Real-Valued Hypotheses

  • Conference paper
  • First Online:
Book cover Computational Learning Theory (COLT 2001)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2111))

Included in the following conference series:

Abstract

We consider the problem of classification using a variant of the agnostic learning model in which the algorithm’s hypothesis is evaluated by comparison with hypotheses that do not classify all possible instances. Such hypotheses are formalized as functions from the instance space X to {0, *, 1}, where * is interpreted as “don’t know”. We provide a characterization of the sets of {0, *, 1}-valued functions that are learnable in this setting. Using a similar analysis, we improve on sufficient conditions for a class of real-valued functions to be agnostically learnable with a particular relative accuracy; in particular, we improve by a factor of two the scale at which scale-sensitive dimensions must be finite in order to imply learnability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the Association for Computing Machinery, 44(4):616–631, 1997.

    MathSciNet  Google Scholar 

  2. M. Anthony and P.L. Bartlett. Neural Network Learning: Theoretical Foundations. Cambridge University Press, 1999.

    Google Scholar 

  3. P.L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2):525–536, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  4. P.L. Bartlett and P.M. Long. Prediction, learning, uniform convergence, and scale-sensitive dimensions. Journal of Computer and System Sciences, 56(2):174–190, 1998.

    Article  MATH  MathSciNet  Google Scholar 

  5. P.L. Bartlett, P.M. Long, and R.C. Williamson. Fat-shattering and the learnability of real-valued functions. Journal of Computer and System Sciences, 52(3):434–452, 1996.

    Article  MATH  MathSciNet  Google Scholar 

  6. S. Ben-David and H.U. Simon. Efficient learning of linear perceptrons. Advances in Neural Information Processing Systems 14, 2000.

    Google Scholar 

  7. Shai Ben-David, Nadav Eiron, and Hans U. Simon. The computational complexity of densest region detection. Proceedings of the 2000 Conference on Computational Learning Theory, 2000.

    Google Scholar 

  8. A. Blum, P. Chalasani, S. Goldman, and D.K. Slonim. Learning with unreliable boundary queries. Proceedings of the 1995 Conference on Computational Learning Theory, pages 98–107, 1995.

    Google Scholar 

  9. A. Blumer, A. Ehrenfeucht, D. Haussler, and M.K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. JACM, 36(4):929–965, 1989.

    Article  MATH  MathSciNet  Google Scholar 

  10. K.L. Buescher and P.R. Kumar. Learning stochastic functions by smooth simultaneous estimation. Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 272–279, 1992.

    Google Scholar 

  11. N. Cristianini and J. Shawe-Taylor. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, 2000.

    Google Scholar 

  12. Y. Freund, R.E. Schapire, Y. Singer, and M.K. Warmuth. Using and combining predictors that specialize. Proceedings of the Twenty-Ninth Annual ACM Symposium on the Theory of Computing, 1997.

    Google Scholar 

  13. D. Haussler. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation, 100(1):78–150, 1992.

    Article  MATH  MathSciNet  Google Scholar 

  14. D. Haussler, M. Kearns, N. Littlestone, and M.K. Warmuth. Equivalence of models for polynomial learnability. Information and Computation, 95:129–161, 1991.

    Article  MATH  MathSciNet  Google Scholar 

  15. D. Haussler, N. Littlestone, and M. K. Warmuth. Predicting {0, 1}-functions on randomly drawn points. Information and Computation, 115(2):129–161, 1994.

    Article  MathSciNet  Google Scholar 

  16. M.J. Kearns and R.E. Schapire. Efficient distribution-free learning of probabilistic concepts. Journal of Computer and System Sciences, 48(3):464–497, 1994.

    Article  MATH  MathSciNet  Google Scholar 

  17. M.J. Kearns, R.E. Schapire, and L.M. Sellie. Toward efficient agnostic learning. Machine Learning, 17:115–141, 1994.

    MATH  Google Scholar 

  18. V.N. Vapnik. Estimation of Dependencies based on Empirical Data. Springer Verlag, 1982.

    Google Scholar 

  19. V.N. Vapnik and A.Y. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2):264–280, 1971.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Genome Institute of Singapore, National University of Singapore, 1 Research Link IMA Building, Singapore, 117604, Republic of Singapore

    Philip M. Long

Authors
  1. Philip M. Long
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Engineering, Department of Computer Science, University of California, Santa Cruz, Santa Cruz, CA, 95064, USA

    David Helmbold

  2. Research School of Information Sciences and Engineering Department of Telecommunications Engineering, Australian National University, Canberra, 0200, Australia

    Bob Williamson

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Long, P.M. (2001). On Agnostic Learning with {0, *, 1}-Valued and Real-Valued Hypotheses. In: Helmbold, D., Williamson, B. (eds) Computational Learning Theory. COLT 2001. Lecture Notes in Computer Science(), vol 2111. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44581-1_19

Download citation

  • DOI: https://doi.org/10.1007/3-540-44581-1_19

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-42343-0

  • Online ISBN: 978-3-540-44581-4

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation