Skip to main content

The Supervised Learning No-Free-Lunch Theorems

  • Chapter

Abstract

This paper reviews the supervised learning versions of the no-free-lunch theorems in a simplified form. It also discusses the significance of those theorems, and their relation to other aspects of supervised learning.

Keywords

  • Cross Validation
  • Error Function
  • Supervise Learning
  • Generalization Error
  • Misclassification Rate

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-1-4471-0123-9_3
  • Chapter length: 18 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-1-4471-0123-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Buntine, W., and A. Weigend. “Bayesian Back-Propagation.” Complex Systems 5 (1991): 603–643.

    Google Scholar 

  2. Dietterich, T. “Machine Learning.”Ann. Rev. Comp. Sci..4 (1990): 255–306.

    Google Scholar 

  3. Duda, R.O., P.E. Hart, and D.G. Stork.Pattern Classification (2nd ed.). Wiley & Sons, 2000.

    Google Scholar 

  4. Blumer, A., A. Ehrenfeueht, D. Haussler, and M. Warmuth. “Occam’s Razor.”Info. Proc. Lett.24 (1987): 377–380.

    Google Scholar 

  5. Titterington, A., F. Smith, and V. E. Makov.Statistical Analysis of Finite Mixture Distributions. New York: Wiley & Sons, 1985.

    MATH  Google Scholar 

  6. Neal, R. “Priors for Infinite Networks.” Technical Report CRG-TR-94–1, Department of Computer Science, University of Toronto, 1994.

    Google Scholar 

  7. Weiss, S. M., and C. A. Kulikowski.Computer Systems that Learn. San Mateo, CA: Morgan Kauffman, 1991.

    Google Scholar 

  8. Wolpert, D. “Filter Likelihoods and Exhaustive Learning.” To appear in Computational Learning Theory and Natural Learning Systems. Volume II: Natural Learning Systems, edited by S. Hanson et al. Cambridge, MA: MIT Press, 1994.

    Google Scholar 

  9. Wolpert, D. “On the Connection Between In-Sample Testing and Generalization Error.” Complex Systems 6 (1992): 47–94.

    Google Scholar 

  10. Wolpert, D. “Bayesian Back-propagation over I-O Functions Rather than Weights.” in Neural Information Processing Systems 6, edited by S. Hanson et al. San Mateo, CA: Morgan-Kauffman, 1994.

    Google Scholar 

  11. Wolpert, D. “The Relationship Between PAC, the Statistical Physics Framework, the Bayesian Framework, and the VC Framework”, In The Mathematics of Generalization, edited by D. H. Wolpert, Addison-Wesley, Reading MA, 1995.

    Google Scholar 

  12. Wolpert, D. “The Lack of a priori Distinctions between Learning Algorithms and the Existence of a priori Distinctions between Learning Algorithms” Neural Computation 8 (1996): 1341–1390, 1391–1421.

    Google Scholar 

  13. Wolpert, D. The Mathematics of Generalization, edited by D. H. Wolpert Addison-Wesley, Reading MA, 1995.

    Google Scholar 

  14. Wolpert, D. “On Bias Plus Variance” Neural Computation 9 (1996): 1211–1244.

    Google Scholar 

  15. Wolpert, D. “Reconciling Bayesian and Non-Bayesian Analysis”, in Maximum Entropy and Bayesian Methods, edited by G. Heidbreder. Boston: Kluwer, 1994.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2002 Springer-Verlag London

About this chapter

Cite this chapter

Wolpert, D.H. (2002). The Supervised Learning No-Free-Lunch Theorems. In: Roy, R., Köppen, M., Ovaska, S., Furuhashi, T., Hoffmann, F. (eds) Soft Computing and Industry. Springer, London. https://doi.org/10.1007/978-1-4471-0123-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0123-9_3

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-1101-6

  • Online ISBN: 978-1-4471-0123-9

  • eBook Packages: Springer Book Archive