Abstract
This paper presents a fully Bayesian derivation of maximum entropy image reconstruction. The argument repeatedly goes from the particular to the general, in that if there are general theories then they must apply to special cases. Two such special cases, formalised as the “Cox axioms “, lead to the well-known fact that Bayesian probability theory is the only consistent language of inference. Further cases, formalised as the axioms of maximum entropy, show that the prior probability distribution for any positive, additive distribution must be monotonic in the entropy. Finally, a quantified special case shows that this monotonic function must be the exponential, leaving only a single dimensional scaling factor to be determined a posteriori. Many types of distribution, including probability distributions themselves, are positive and additive, so the entropy exponential is very general.
The following paper (Gull 1989) applies these ideas to image reconstruction, showing how a sophisticated treatment can incorporate prior expectation of spatial correlations.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Cox, R.P. (1946). Probability, Frequency and Reasonable Expectation. Am. Jour. Phys. 17, 1–13.
Frieden, B.R. (1972). Restoring with maximum likelihood and maximum entropy. J. Opt. Soc. Am., 62, 511–518.
Gull, S.F. (1989). Developments in maximum entropy data analysis. In these Proceedings.
Gull, S.F. & Daniell, G.J. (1979). The maximum entropy method. In Image Formation from Coherence Functions in Astronomy, ed. C. van Schooneveld, pp. 219–225, Reidel.
Gull, S.F. & Skilling, J. (1984). The maximum entropy method. In Indirect Imaging, ed. J.A. Roberts. Cambridge University Press.
Jaynes, E.T. (1978). “Where do we stand on maximum entropy? Reprinted in E.T. Jaynes: Papers on Probability, Statistics and Statistical Physics, ed. R. Rosenkrantz, 1983 Dordrecht: Reidel.
Jaynes, E.T. (1986). Monkeys, kangaroos and N. In Maximum Entropy and Bayesian Methods in Applied Statistics, ed. J.H. Justice, pp. 26–58, Cambridge Univ. Press.
Klir, G.J. (1987). Where do we stand on measures of uncertainty, ambiguity, fuzziness and the like? Fuzzy sets and systems, 24, 141–160.
Levine, R.D. (1986). Geometry in classical statistical thermodynamics, J. Chem. Phys., 84, 910–916.
Rodriguez, C. (1989). The metrics induced by the Kullback number. In these Proceedings.
Shore, J.E. & Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans.Info.Theory, IT-26, 26–39
Shore, J.E. & Johnson, R.W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans.Info.Theory, IT-29 942–943.
Skilling, J. (1988). The axioms of maximum entropy. In Maximum Entropy and Bayesian Methods in Science and Engineering, Vol. 1., ed. G.J. Erickson & C.R. Smith, pp. 173–188. Kluwer.
Smith, C.R. & Erickson, G.J. (1989). From rationality and consistency to Bayesian probability. In these Proceedings.
Tikochinsky, Y., Tishby, N.Z. & Levine, R.D. (1984). Consistent inference of probabilities for reproducible experiments. Phys. Rev. Lett., 52, 1357–1360.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1989 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Skilling, J. (1989). Classic Maximum Entropy. In: Skilling, J. (eds) Maximum Entropy and Bayesian Methods. Fundamental Theories of Physics, vol 36. Springer, Dordrecht. https://doi.org/10.1007/978-94-015-7860-8_3
Download citation
DOI: https://doi.org/10.1007/978-94-015-7860-8_3
Publisher Name: Springer, Dordrecht
Print ISBN: 978-90-481-4044-2
Online ISBN: 978-94-015-7860-8
eBook Packages: Springer Book Archive