Mathematical Programming

, Volume 61, Issue 1–3, pp 251–261 | Cite as

On the failure of maximum entropy reconstruction for Fredholm equations and other infinite systems

  • Jonathan M. Borwein


We analyse several examples where the maximum entropy solution to a system of equations exists but fails to satisfy the natural (dual) formula. These examples highlight the role that finiteness of the number of constraints has in the efficacy of maximum entropy type estimation and reconstruction. We also provide two regularization processes which repair the problem.

AMS 1991 Subject Classifications

Primary 94A12, 90C48, 45B05 Secondary 94A27, 49J27 

Key words

Boltzmann—Shannon entropy maximum entropy estimates Fenchel duality Fredholm equations Lagrange multipliers dual convex programs duality gaps regularization penalization 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. A. Ben-Tal, J.M. Borwein and M. Teboulle. “A dual approach to multidimensionalL p spectral estimation problems,”SIAM Journal on Control and Optimization 26 (1988) 985–996.Google Scholar
  2. A. Ben-Tal, J.M. Borwein and M. Teboulle, “Spectral estimation via convex programming,” in: F.Y. Phillips and J.J. Rousseau, eds.,Systems and Management by Extremal Methods (Kluwer Academic Publishers, Boston, MA, 1992) Chapter 18, pp. 275–289.Google Scholar
  3. J.M. Borwein, “A Lagrange multiplier theorem and a sandwich theorem for convex relations,”Mathematica Scandinavica 48 (1981) 189–204.Google Scholar
  4. J.M. Borwein and A.S. Lewis, “Duality relationships for entropy-like minimization problems,”SIAM Journal on Control and Optimization 29 (1990) 325–338.Google Scholar
  5. J.M. Borwein and A.S. Lewis, “On the convergence of moment problems,”Transactions of the American Mathematical Society 325 (1991a) 249–271.Google Scholar
  6. J.M. Borwein and A.S. Lewis, “Strong rotundity and optimization,” (1991b), to appear in:SIAM Journal on Optimization. Google Scholar
  7. J.M. Borwein and A.S. Lewis, “Convergence of best entropy estimates,”SIAM Journal on Optimization 1 (1991c) 191–205.Google Scholar
  8. J.M. Borwein and A.S. Lewis, “Partially finite convex programming, Part I and II,”Mathematical Programming (Series B) 57 (1992) 15–48 and 49–84.Google Scholar
  9. J.P. Burg, “Maximum entropy spectral analysis,” paper presented atThe 37th Meeting of the Society of Exploration Geophysicists, Oklahoma City, 1967.Google Scholar
  10. S. Cuilli, N. Mounsif, N. Gorman and T.D. Spearman, “On the application of maximum entropy to the moments problem,”Journal of Mathematical Physics 32 (1991) 1717–1719.Google Scholar
  11. D. Dacunha-Castelle and F. Gamboa, “Maximum d'entropie et probléme des moments,”Annales de l'Institut Henri Poincaré 26 (1990) 567–596.Google Scholar
  12. P.P.B. Eggermont, “Maximum entropy regularization for Fredholm integral equations of the first kind,” preprint, University of Delaware (Newark, DE, 1991).Google Scholar
  13. R.K. Goodrich and A. Steinhardt, “L2 spectral estimation,”SIAM Journal on Applied Mathematics 46 (1986) 417–428.Google Scholar
  14. C.W. Groetsch,The Theory of Tikhonov Regularization for Fredholm Integral Equations of the First Kind (Pitman, Boston, MA, 1984).Google Scholar
  15. R.B. Holmes,Geometric Functional Analysis and its Applications (Springer, New York, 1975).Google Scholar
  16. R.T. Rockafellar,Convex Analysis (Princeton University Press, Princeton, NJ, 1970).Google Scholar
  17. R.T. Rockafellar, “Integrals which are convex functionals, II,”Pacific Journal of Mathematics 39 (1971) 439–469.Google Scholar
  18. R.T. Rockafellar,Conjugate Duality and Optimization (SIAM, Philadelphia, PA, 1974).Google Scholar
  19. M. Teboulle and I. Vajda, “Convergence of bestϕ-entropy estimates,” (1991), to appear in:IEEE Transaction on Information Processing. Google Scholar

Copyright information

© The Mathematical Programming Society, Inc. 1993

Authors and Affiliations

  • Jonathan M. Borwein
    • 1
  1. 1.Department of Combinatorics and OptimizationUniversity of WaterlooWaterlooCanada

Personalised recommendations