Solving Oversampled Data Problems By Maximum Entropy

  • R. K. Bryan
Part of the Fundamental Theories of Physics book series (FTPH, volume 39)


A numerical algorithm for the solution of the Classic Maximum Entropy problem is presented, for use when the data are considerably oversampled, so that the amount of independent information they contain is very much less than the actual number of data points. Examples of problems for which this algorithm is particularly appropriate are dynamic light scattering, solution scattering and fibre diffraction. The application of a general purpose entropy maximisation program is then comparatively inefficient. In the new algorithm the independent variables are in the singular space of the transform between map (or image or spectrum) and data, and much fewer in number than either the data or the reconstruction. This reduction in the dimension allows a direct evaluation of the posterior probability of the solution, and thus enables the ‘Classic Maxent’ problem to be solved completely.


Singular Value Decomposition Maximum Entropy Cholesky Decomposition Singular Space Singular Value Decomposition Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bryan, R. K. (1980). Maximum Entropy Image Processing. PhD Thesis, University of Cambridge.Google Scholar
  2. Golub, G. H. & van Loan, C. F. (1983). Matrix Computations. Johns Hopkins, Baltimore.zbMATHGoogle Scholar
  3. Gull, S. F. (1989). Developments in maximum entropy data analysis. In: J. Skilling (ed.), Maximum Entropy and Bayesian Methods. Kluwer Academic, Dordrecht, pp. 53–71.Google Scholar
  4. Gull, S. F. & Daniell, G. J. (1978). Image reconstruction from incomplete and noisy data. Nature,. 272, 686–690.CrossRefGoogle Scholar
  5. Jaynes, E. T. (1968). Prior probabilities. IEEE Trans.,. SCC-4, 227–241.Google Scholar
  6. Jaynes, E. T. (1987). Bayesian spectrum and chirp analysis. In: C. Ray Smith & Gary J. Erickson (ed.), Maximum Entropy and Bayesian Spectral Analysis and Estimation Problems. Kluwer Academic, Dordrecht, pp. 1–37.Google Scholar
  7. Levenberg, K. (1944). A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math.,. 2, 164–168.MathSciNetzbMATHGoogle Scholar
  8. Livesey, A. K., Licinio, P. & Delaye, M. (1986). Maximum entropy analysis of quasielastic lightscattering from colloidal dispersions. J. Chem. Phys.,. 84, 5102–5107.CrossRefGoogle Scholar
  9. Marquardt, D. W. (1963). An algorithm for least squares estimation of non-linear parameters. SIAM J. Appl. Math.,. 11, 431–441.MathSciNetzbMATHCrossRefGoogle Scholar
  10. Provencher, S. W. (1979). Inverse problems in polymer characterization: Direct analysis ofpolydispersity with photon correlation spectroscopy. Makromol. Chem.,. 180, 201–209.CrossRefGoogle Scholar
  11. Sibisi, S. (1989). Regularization and inverse problems. In: J. Skilling (ed.), Maximum Entropyand Bayesian Methods. Kluwer Academic, Dordrecht, pp. 389–396.Google Scholar
  12. Sibisi, S. (1990). Quantified Maxent: an NMR application. These Proceedings.Google Scholar
  13. Skilling, J. (1989a). Classic maximum entropy. In: J. Skilling (ed.), Maximum Entropy andBayesian Methods. Kluwer Academic, Dordrecht, pp. 45–52.Google Scholar
  14. Skilling, J. (1989b). The eigenvalues of mega-dimensional matrices. In: J. Skilling (ed.), Maximum Entropy and Bayesian Methods. Kluwer Academic, Dordrecht, pp. 455–466.Google Scholar
  15. Skilling, J. (1990). Quantified maximum entropy. These Proceedings.Google Scholar
  16. Skilling, J. & Bryan, R. K. (1984). Maximum entropy image reconstruction: general algorithm. Mon. Not. R. astr. Soc.,. 211, 111–124.zbMATHGoogle Scholar
  17. Slepian, D. (1964). Prolate spheroidal wave functions, Fourier analysis and uncertainty, IV: Extensions to many dimensions; Generalized prolate spheroidal functions. Bell Syst. Tech. J., 43, 3009–3058.MathSciNetzbMATHGoogle Scholar
  18. Slepian, D., Pollak, H. O. (1961). Prolate spheroidal wave functions, Fourier analysis and uncertainty, I. Bell Syst. Tech. J.,. 40, 43–64.MathSciNetzbMATHGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1990

Authors and Affiliations

  • R. K. Bryan
    • 1
  1. 1.European Molecular Biology LaboratoryHeidelbergWest Germany

Personalised recommendations