The Maximum Entropy on the Mean Method, Noise and Sensitivity
In this paper we address the problem of building convenient criteria to solve linear and noisy inverse problems of the form y = Ax + n. Our approach is based on the specification of constraints on the solution x through its belonging to a given convex set C. The solution is chosen as the mean of the distribution which is the closest to a reference measure μ on C with respect to the Kullback divergence, or cross-entropy. This is therefore called the Maximum Entropy on the Mean Method (MEMM). This problem is shown to be equivalent to the convex one x = arg min x F(x) submitted to y = Ax (in the noiseless case). Many classical criteria are found to be particular solutions with different reference measures μ. But except for some measures, these primal criteria have no explicit expression. Nevertheless, taking advantage of a dual formulation of the problem, the MEMM enables us to compute a solution in such cases. This indicates that such criteria could hardly have been derived without the MEMM. In order to integrate the presence of additive noise in the MEMM scheme, the object and noise are searched simultaneously for in an appropriate convex C′. The MEMM then gives a criterion of the form x = arg min x F(x) + G(y - Ax), where F and G are convex, without constraints. The functional G is related to the prior distribution of noise, and may be used to account for specific noise distributions. Using the regularity of the criterion, the sensitivity of the solution to variations of the data is also derived.
KeywordsMaximum Entropy Dual Problem Reference Measure Regularized Criterion Convex Constraint
Unable to display preview. Download preview PDF.
- R. E. Kass and L. Wasserman. Formal Rules for Selecting Prior Distributions: A Review and Annotated Bibliography. Technical report, Department of Statistics, Carnegie Mellon University, 1994. Submitted to Jnl. of American Statistic Association.Google Scholar
- G. Le Besnerais. Méthode du maximum d’entropie sur la moyenne, critères de reconstruction d’image et synthese d’ouverture en radio-astronomie. PhD thesis, University of Paris-Sud, 1993.Google Scholar
- G. Le Besnerais, J.-F. Bercher, and G. Demoment. A new look at the entropy for solving linear inverse problems, submitted to IEEE transactions on Information Theory, 1994.Google Scholar
- J. Navaza. The use of non-local constraints in maximum-entropy electron density reconstruction. Acta Crystallographica, pages 212–223, 1986.Google Scholar