International Journal of Theoretical Physics

, Volume 22, Issue 12, pp 1091–1104 | Cite as

A quantitative occam's razor

  • Rafael Sorkin
Article

Abstract

Interpreting entropy as a prior probability suggests a universal but “purely empirical” measure of “goodness of fit.” This allows statistical techniques to be used in situations where the correct theory- and not just its parameters-is still unknown. As developed illustratively for least-squares nonlinear regression, the measure proves to be a transformation of theR2 statistic. Unlike the latter, however, it diminishes rapidly as the number of fitting parameters increases.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fine, T. L. (1973).Theories of Probabilities. Academic Press, New York.Google Scholar
  2. Hanushek, E. A., and Jackson, J. E. (1977).Statistical Methods for Social Scientists. Academic Press, New York.Google Scholar
  3. Kolmogorov, A. (1968). Logical basis for information theory and probability theory,IEEE Trans. Info. Theory,14, 663.Google Scholar
  4. Kruskal, W., and Tanur, J. M. (eds.) (1978).International Encyclopedia of Statistics. The Free Press, New York. See the articles on hypothesis testing and significance tests.Google Scholar
  5. Kullback, S. (1959).Information Theory and Statistics. Wiley, New York.Google Scholar
  6. Penrose, O. (1970).Foundation of Statistical Mechanics. Pergamon Press, Oxford, p. 227.Google Scholar
  7. Schwarz, G. (1978). Estimating the dimension of a model.Ann. Stat.,6, 461–464.Google Scholar
  8. Yasuhara, A. (1971).Recursive Function Theory and Logic, Academic Press, New York.Google Scholar

Copyright information

© Plenum Publishing Corporation 1983

Authors and Affiliations

  • Rafael Sorkin
    • 1
    • 2
  1. 1.Institute for Advanced StudyPrinceton
  2. 2.Center for Theoretical Physics, Department of Physics and AstronomyUniversity of MarylandCollege Park

Personalised recommendations