Cross-Entropy Estimation of Distributions Based on Scarce Data
A method is described to estimate a random variable using fractiles as constraints. The fractiles are exactly known for random samples, whether small or large. The method minimizes the cross-entropy, or entropy relative to a reference distribution, which may be selected to minimize the cross-entropy of the sample data. The method is simple to implement and avoids several disadvantages of the methods presently in common use.
KeywordsCross-entropy relative entropy distribution scarce data estimation information fractile constraints
Unable to display preview. Download preview PDF.
- 1.B.W. Silverman, Density Estimation for Statistics and Data Analysis, Chapman and Hall, London (1986).Google Scholar
- 4.J.N. Kapur and H.K. Kesavan, The Generalized Maximum Entropy Principle (with Applications), Sandford Educational Press, Waterloo, Ontario (1987).Google Scholar
- 6.D.M. Titterington, A.F.M. Smith, and U.E. Markov, Statistical Analysis of Finite Mixture Distributions, John Wiley and Sons, Chichester, UK (1985).Google Scholar
- 7.P. Ofverbeck, Small Sample Control and Structural Safety, Report TVBK-3009, Dept. Struct. Eng., Lund University, Lund, Sweden (1980).Google Scholar
- 8.E.T. Jaynes, Papers on Probability, Statistics, and Statistical Physics, D. Reidel, Dordrecht, Netherlands (1983).Google Scholar
- 9.H. Akaike, Prediction and Entropy, A Celebration of Statistics, A.C. Atkinson and S.E. Fienberg, eds., Springer-Verlag, New York, NY (1985).Google Scholar
- 10.W. Feller, An Introduction to Probability Theory and its Applications, 3rd ed., John Wiley and Sons, Inc., New York, NY (1968).Google Scholar
- 11.H.O. Madsen, S. Krenk and N.C. Lind, Methods of Structural Safety, Prentice-Hall Book Co., Inc., Englewood Cliffs, NJ (1986).Google Scholar
- 13.N.C. Lind and V. Solana, Estimation of Random Variables With Fractile Constraints, IRR paper No. 11, Institute for Risk Research, University of Waterloo, Waterloo, Ontario, Canada (1988).Google Scholar