Skip to main content
SpringerLink
Log in

Sharpening Occam’s Razor

Extended Abstract

  • Conference paper
  • First Online:
Computing and Combinatorics (COCOON 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2387))

Included in the following conference series:

Abstract

We provide a new representation-independent formulation of Occam’s razor theorem, based on Kolmogorov complexity. This new formulation allows us to:

  • Obtain better sample complexity than both length-based [4] and VC-based [3] versions of Occam’s razor theorem, in many applications.

  • Achieve a sharper reverse of Occam’s razor theorem than that of [5]. Specifically, we weaken the assumptions made in [5] and extend the reverse to superpolynomial running times.

Supported in part by the NSERC Operating Grant OGP0046506, ITRC, and NSF-ITR Grant 0085801 at UCSB.

Partially supported by an NSERC International Fellowship and ITRC.

Partially supported by the European Community through NeuroCOLT ESPRIT Working Group Nr. 8556. Affiliated with CWI and the University of Amsterdam.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. M. Anthony and N. Biggs, Computational Learning Theory, Cambridge University Press, 1992.

    Google Scholar 

  2. A. Blum, T. Jiang, M. Li, J. Tromp, M. Yannakakis, Linear approximation of shortest common superstrings. Journal ACM, 41:4 (1994), 630–647.

    Article  MATH  MathSciNet  Google Scholar 

  3. A. Blumer and A. Ehrenfeucht and D. Haussler and M. Warmuth, Learnability and the Vapnik-Chervonenkis Dimension. J. Assoc. Comput. Mach., 35(1989), 929–965.

    MathSciNet  Google Scholar 

  4. A. Blumer and A. Ehrenfeucht and D. Haussler and M. Warmuth, Occam’s Razor. Inform. Process. Lett., 24(1987), 377–380.

    Article  MATH  MathSciNet  Google Scholar 

  5. R. Board and L. Pitt, On the necessity of Occam Algorithms. 1990 STOC, pp. 54–63.

    Google Scholar 

  6. A. Ehrenfeucht, D. Haussler, M. Kearns, L. Valiant. A general lower bound on the number of examples needed for learning. Inform. Computation, 82(1989), 247–261.

    Article  MATH  MathSciNet  Google Scholar 

  7. D. Haussler. Quantifying inductive bias: AI learning algorithms and Valiant’s learning framework. Artificial Intelligence, 36:2(1988), 177–222.

    Article  MATH  MathSciNet  Google Scholar 

  8. D. Haussler, N. Littlestone, and, M. Warmuth. Predicting {0,1}-functions on randomly drawn points. Information and Computation, 115:2(1994), 248–292.

    Article  MATH  MathSciNet  Google Scholar 

  9. T. Jiang and M. Li, DNA sequencing and string learning, Math. Syst. Theory, 29(1996), 387–405.

    MathSciNet  MATH  Google Scholar 

  10. M. Li. Towards a DNA sequencing theory. 31st IEEE Symp. on Foundations of Comp. Sci., 125–134, 1990.

    Google Scholar 

  11. M. Li and P. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. 2nd Edition, Springer-Verlag, 1997.

    Google Scholar 

  12. L. G. Valiant. A Theory of the Learnable. Comm. ACM, 27(11), 1134–1142, 1984.

    Article  MATH  Google Scholar 

  13. M.K. Warmuth. Towards representation independence in PAC-learning. In AII-89, pp. 78–103, 1989.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Univ. California Santa Barbara, CA, 93106, USA

    Ming Li

  2. CWI, Kruislaan 413, 1098 SJ, Amsterdam, Netherlands

    John Tromp & Paul Vitányi

Authors
  1. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Tromp
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paul Vitányi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of California, Santa Barbara, California, 93106, USA

    Oscar H. Ibarra

  2. Department of Mathematics, National University of Singapore, Singapore, Singapore, 117543

    Louxin Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Li, M., Tromp, J., Vitányi, P. (2002). Sharpening Occam’s Razor. In: Ibarra, O.H., Zhang, L. (eds) Computing and Combinatorics. COCOON 2002. Lecture Notes in Computer Science, vol 2387. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45655-4_44

Download citation

  • DOI: https://doi.org/10.1007/3-540-45655-4_44

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43996-7

  • Online ISBN: 978-3-540-45655-1

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation