Risk Analysis pp 357-366 | Cite as

Cross-Entropy Estimation of Distributions Based on Scarce Data

  • Niels C. Lind
  • Vicente Solana
Part of the Advances in Risk Analysis book series (AIRA, volume 8)


A method is described to estimate a random variable using fractiles as constraints. The fractiles are exactly known for random samples, whether small or large. The method minimizes the cross-entropy, or entropy relative to a reference distribution, which may be selected to minimize the cross-entropy of the sample data. The method is simple to implement and avoids several disadvantages of the methods presently in common use.


Cross-entropy relative entropy distribution scarce data estimation information fractile constraints 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    B.W. Silverman, Density Estimation for Statistics and Data Analysis, Chapman and Hall, London (1986).Google Scholar
  2. 2.
    J. Shore and R.W. Johnson, Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-Entropy, IEEE Transactions on Information Theory IT-26(1):26–37 (1980).CrossRefGoogle Scholar
  3. 3.
    J. Shore and R.W. Johnson, Properties of Cross-Entropy Minimization, IEEE Transactions on Information Theory IT-27(4):472–482 (1981).CrossRefGoogle Scholar
  4. 4.
    J.N. Kapur and H.K. Kesavan, The Generalized Maximum Entropy Principle (with Applications), Sandford Educational Press, Waterloo, Ontario (1987).Google Scholar
  5. 5.
    S.N. Karbelkar, On the Axiomatic Approach to the Maximum Entropy Principle of Inference, Promana J. Phys. 26(4):301–310 (1986).CrossRefGoogle Scholar
  6. 6.
    D.M. Titterington, A.F.M. Smith, and U.E. Markov, Statistical Analysis of Finite Mixture Distributions, John Wiley and Sons, Chichester, UK (1985).Google Scholar
  7. 7.
    P. Ofverbeck, Small Sample Control and Structural Safety, Report TVBK-3009, Dept. Struct. Eng., Lund University, Lund, Sweden (1980).Google Scholar
  8. 8.
    E.T. Jaynes, Papers on Probability, Statistics, and Statistical Physics, D. Reidel, Dordrecht, Netherlands (1983).Google Scholar
  9. 9.
    H. Akaike, Prediction and Entropy, A Celebration of Statistics, A.C. Atkinson and S.E. Fienberg, eds., Springer-Verlag, New York, NY (1985).Google Scholar
  10. 10.
    W. Feller, An Introduction to Probability Theory and its Applications, 3rd ed., John Wiley and Sons, Inc., New York, NY (1968).Google Scholar
  11. 11.
    H.O. Madsen, S. Krenk and N.C. Lind, Methods of Structural Safety, Prentice-Hall Book Co., Inc., Englewood Cliffs, NJ (1986).Google Scholar
  12. 12.
    G. Matheron, Estimating and Choosing, Springer-Verlag, Berlin (1989).CrossRefGoogle Scholar
  13. 13.
    N.C. Lind and V. Solana, Estimation of Random Variables With Fractile Constraints, IRR paper No. 11, Institute for Risk Research, University of Waterloo, Waterloo, Ontario, Canada (1988).Google Scholar

Copyright information

© Springer Science+Business Media New York 1991

Authors and Affiliations

  • Niels C. Lind
    • 1
  • Vicente Solana
    • 2
  1. 1.University of WaterlooWaterlooCanada
  2. 2.National Research Council of SpainMadridSpain

Personalised recommendations