Abstract
Before using ONB and maximum entropy to analyze changes in inequality, this chapter develops the formal theory necessary to do so. That is, in this chapter, we develop mathematical and statistical properties of the maximum entropy (ME) method and then relate it to other well known flexible functional form approaches. First we shall explain what we mean by the ME principle and then review the Jaynes’ (1979) concentration theorem to provide some justification of the ME method as a density estimation method. Since there has been little previous research on applying the ME principle to derive economic relationships, 2 we shall begin with the physicists’ view of this principle. The ME principle means that the entropy of the physical universe increases constantly because there is a continuous and irrevocable degradation of order into chaos. As a simple example, we can consider a closed system filled with a large number of interacting particles and leave the system to interact freely for a long time. Then the system will reach a maximum entropy state. Statistical physicists find the ME density function for this equilibrium system which is described by a constant average energy per particle. See, for example, the Maxwell-Bolzmann distribution in Rao (1973).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Notes
This chapter follows Ryu (1990).
An exception is the work done by Georgenscu-Roegen(1971).
Suppose we interpret our problem as a multinomial distribution with cell probabilities π1, ⋯, πn. In T independent trials, we have a sequence of numbers T1, ⋯, Tn corresponding to n outcomes and we can define frequencies f1 = T1/T, ⋯, fn= Tn/T. Then the likelihood function of the multinomial distribution is Therefore in (2.3), we defined W as the allowable number of permutations for any given sequence of numbers T1, ⋯, Tn. 4 Mood, Graybill, and Boes (1974) show the Stirling formula in their Appendix A as T ! = (2π)1/2exp(-T)TT+0.5exp[r(T)/12T] where 1-1/(12T+1)<r(T)<1. Hence, log(T !) =-T + (T + 0.5) log T = T log T-T as T → ∞ where = means approximately equal to.
See Mead and Papanicolaou (1984) for the existence conditions.
We can establish accuracy up to the certain decimal point or up to the certain number of digits.
We can divide the unique convergence problem and an uniqueness problem. In my experience with this algorithm, convergence has not been a problem that we shall emphasize uniqueness problem. Rewrite second round iteration B(2) c(2) = d(1) as c(2) = [B(2)]-1d(1). If Λ is an NxN diagonal matrix with nth element 1/n, then Λ[c(2)-c(l)] = Λ[B(2)]-1d(1) c(1). If we define e(2) ≡ B(2)Λ[[B(2)]-1d(1)-c(1)], then [B(2)] Λ[C(2)-C(1)] ≡ e(2). If we set, then we have a relationship Iteration method based on (1*) is equivalent to the iteration method based on (2.18)–(2.20). Therefore, we shall prove unique convergence of the iteration method of (1*). Since we know B(2) (as well as B(3), B(4), ⋯) is a positive definite matrix, unique convergence of this iteration method can be established if we appeal to Gale and Nikaido theorem (1965). Let us elaborate this in the following. Suppose we have a differentiable mapping g: S → RN where S is a region in RN, and g(s)= (gm(s)) (s∈S, m = 1, ⋯ N), gm(s) being differentiable functions on S with total differenctials. Suppose we choose x ∈ [0, 1] and define, then where we used Therefore. Since we know B(2) (as well as B(3), B(4), ⋯) is a positive definite matrix, unique convergence of this iteration method can be established for s if we appeal to Gale and Nikaido theorem (1965). If x ∈(-∞, + ∞), we can derive a similar expression.
Let y ≡ 1/a and dy =-da/a2. Then we have where we have used definition of the gamma function
If z ≡ (S + x0)/a and dz = (S +x0)(-da / a2), then
To show (2.64), we shall derive several useful relationships, (i) If we apply a formula from Zeller (l971) p.372, we can show (T /2)0.5Γ(v + l/2)/Γ(v / 2) → 1 as T → ∞. (ii). (iii) If A ≡ (μ-x0)2 / vs2, then exp[log(l + A)-T / 2] = exp[-(T / 2) · A] = exp[-(μ-x0)2 / vs2]. Using these relationships, we can derive (2.64).
Let us prove (2.75). Suppose we normalize the density function which is given in (2.21). From Therefore, we have proved (2.75).
We have requested y(x) be positive for all x. However, this requirement can be relaxed easily. If we can assume — A < y(x) for any big positive finite constant A, then we can always transform y(x) so that y(x) = y(x) + A > 0. Therefore, the only restricition which violates our assumption is when y(x) becomes negative infinite, then linear transformation of y(x) can not make y(x) to be positive. As a simple example of this exceptional case, suppose we have a bivariate joint normal distribution f(x, y), then we know the regression function is y(x) = α + αx. Therefore, when x approaches negative infinity, y(x) approaches negative infinity if β > 0. In this case, our required assumption is violated, and we have to estimate a bivariate joint pdf to find a regression function from it.
Gallan’t Fourier flexible form includes quadratic trend term in the expansion. If we impose ∫ xn f (x) dx = vn for n = 1, 2 and ∫ exp[inx] f (x) dx = ξ for n = 0, ± 1, ⋯, ± N, then the ME method will produce both the trend term and Fourier series terms.
Jeffereys (1967) shows an example of the simplicity postulate. A physicist would test first whether the whole variation is random as against the existence of a linear trend; then a linear law against a quadratic one, then proceeding in order of increasing complexity. All we have to say that the simpler laws have the greater prior probabilities. This is what Wrinch and Jefferys called the simplicity postulate.
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ryu, H.K., Slottje, D.J. (1998). Maximum Entropy Estimation Method. In: Measuring Trends in U.S. Income Inequality. Lecture Notes in Economics and Mathematical Systems, vol 459. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-58896-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-58896-9_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64229-9
Online ISBN: 978-3-642-58896-9
eBook Packages: Springer Book Archive