Advertisement

Massive Inference and Maximum Entropy

  • John Skilling
Part of the Fundamental Theories of Physics book series (FTPH, volume 98)

Abstract

In data analysis, maximum entropy (MaxEnt) has been used to reconstruct measures i.e. positive, additive distributions) from limited data. The MaxEnt prior was originally derived from the “monkey model” in which quanta of uniform intensity could appear randomly in the field of view. To avoid undue digitisation, the quanta had to be small, and this led to difficulties with the Law of Large Numbers, and to unavoidable approximations in computing the posterior. A better way of avoiding digitisation is to give the quanta variable intensity with an exponential prior, that being the natural MaxEnt assignment. We call this technique “Massive Inference” (MassInf). Although the entropy formula no longer appears in the prior, MassInf results show improved quality. MassInf is also capable of assigning a simple prior for polarized images.

Key words

Maximum entropy infinitely divisible polarization regularizaron 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bretthorst, G.L.: 1990, ‘Bayesian Spectrum Analysis and Parameter Estimation’ in Lecture Notes in Statistics 48, Springer-Verlag, New York.Google Scholar
  2. Feller, W.: 1971, An Introduction to Probability Theory and its Applications, Vol II, Wiley, New York.zbMATHGoogle Scholar
  3. Gull, S.F. and Daniell, G.J.: 1978, ‘Image reconstruction from incomplete and noisy data’, Nature 272, 686–690.CrossRefGoogle Scholar
  4. Gull, S.F.: 1989, ‘Developments in Maximum Entropy data analysis’ in Maximum Entropy and Bayesian Methods, J. Skilling (ed.) Kluwer Academic Publishers, Dordrecht, 53–71.Google Scholar
  5. Jaynes, E.T.: 1957, ‘Information Theory and Statistical Mechanics, I’,Phys. Rev. 106, 620–630.MathSciNetzbMATHCrossRefGoogle Scholar
  6. Lukacs, E.: 1970, Characteristic Functions, 2nd ed., Griffin, LondonzbMATHGoogle Scholar
  7. Shannon, C.E.: 1948, ‘A Mathematical Theory of Communication’, Bell Systems Tech. J. 27, 379, 623.MathSciNetGoogle Scholar
  8. Shore, J.E. and Johnson, R.W.: 1980, ‘Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross—entropy’, IEEE Trans. Info. Theory IT-26, 26–39MathSciNetCrossRefGoogle Scholar
  9. Shore, J.E. and Johnson, R.W.: 1980, ‘Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross—entropy’, IEEE Trans. Info. Theory IT-29, 942–943.MathSciNetGoogle Scholar
  10. Skilling, J.: 1989, ‘Classic Maximum Entropy’ in Maximum Entropy and Bayesian Methods, J. Skilling (ed.) Kluwer Academic Publishers, Dordrecht, 45–52.Google Scholar
  11. Skilling, J.: 1991, ‘On Parameter Estimation and Quantified MaxEnt’ in Maximum Entropy and Bayesian Methods, W.T. Grandy, Jr. and L.H. Schick (ed.) Kluwer Academic Publishers, Dordrecht, 267–273.CrossRefGoogle Scholar
  12. Stone, J.M.: 1963, Radiation and Optics, McGraw-Hill, New York, 313.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1998

Authors and Affiliations

  • John Skilling
    • 1
  1. 1.Department of Applied Mathematics and Theoretical PhysicsUniversity of CambridgeEngland

Personalised recommendations