On cumulative entropies in terms of moments of order statistics

In this paper relations among some kinds of cumulative entropies and moments of order statistics are presented. By using some characterizations and the symmetry of a non negative and absolutely continuous random variable X, lower and upper bounds for entropies are obtained and examples are given.


Introduction
In reliability theory, to describe and study the information related to a nonnegative absolutely continuous random variable X we use the Shannon entropy, or differential entropy, of X, defined by (Shannon, 1948) where log is the natural logarithm and f is the probability density function (pdf) of X. In the following we use F and F to indicate the cumulative distribution function (cdf) and the survival function (sf) of X, respectively.
In the literature, there are several different versions of entropy, each one suitable for a specific situation. Rao et al., 2004, introduced the Cumulative Residual Entropy (CRE) of X as Di Crescenzo and Longobardi, 2009, introduced the Cumulative Entropy (CE) of X as This information measure is suitable to measure information when uncertainty is related to the past, a dual concept of the cumulative residual entropy which relates to uncertainty on the future lifetime of a system. Mirali et al., 2016, introduced the Weighted Cumulative Residual Entropy (WCRE) of X as Mirali and Baratpour, 2017, introduced the Weighted Cumulative Entropy (WCE) of X as Recently, various authors have discussed different versions of entropy and their applications (see, for instance, [3], [4], [5], [11]).
The paper is organized as follows. In Section 2, we study relations among some kinds of entropies and moments of order statistics and present various examples. In Section 3, bounds are given by using also some characterizations and properties (as the symmetry) of the random variable X, some examples and bounds for known distributions are given.

A relation among entropies and order statistics
We recall that, if we have n i.i.d. random variables X 1 , X 2 , . . . , X n , we can introduce the order statistics X k:n , k = 1, . . . , n. The k-th order statistic is equal to the k-th smallest value from the sample. We know that the cdf of X k:n can be given in terms of the cdf of the parent distribution; in fact whereas the pdf of X k:n is expressed as Choosing k = 1 and k = n we get the smallest and the largest order statistic, respectively. Their cdf and pdf are given by

Cumulative residual entropy
The Cumulative Residual Entropy (CRE) of X can be written also in terms of order statistics, that is where E(X) is the expectation or mean of X, and µ n+1:n+1 the mean of the largest order statistic in a sample of size n + 1 from F , provided that lim x→+∞ −x(1 − F (x)) log(1 − F (x)) = 0. We note that (2) can be rewritten as Example 1 Consider the standard exponential distribution with pdf f (x) = e −x , x > 0. Then, it is known that E(X) = 1 and E(X n:n ) = 1 + 1 2 + · · · + 1 n .
Then, from (3), we readily have Example 2 Consider the standard uniform distribution with pdf f (x) = 1, 0 < x < 1. Then, it is known that and E(X n:n ) = n n + 1 .

Cumulative entropy
The Cumulative Entropy (CE) of X can be rewritten in terms of the mean of the minimum order statistic; integrating by parts (1) where µ 1:n+1 is the mean of the smallest order statistic from a sample of size n + 1 from F , provided that lim x→+∞ −xF (x) log F (x) = 0.
We note that (4) can be rewritten as Example 3 For the standard exponential distribution, it is known that and so from (5), we readily have by the use of Euler's identity.
Example 4 For the standard uniform distribution, using the fact that we obtain from (4) that by the use of Euler's identity.
Remark 1 If the random variable X has finite mean µ and is symmetrically distributed about µ, then we know and so the symmetry property of CE readily follows.

Weighted cumulative entropies
In the same way the Weighted Cumulative Residual Entropy (WCRE) of X can be expressed as where µ (2) n+1:n+1 is the second moment of the largest order statistic in a sample of size n + 1, provided that lim x→+∞ − Example 5 For the standard uniform distribution, using the fact that µ (2) n+1:n+1 = n + 1 n + 3 and E(X 2 ) = 1 3 , we obtain from (6) that E w (X) = 1 2 +∞ n=1 1 n(n + 1) Moreover, we can derive the Weighted Cumulative Entropy (WCE) of X in terms of the second moment of the minimum order statistic in the following where µ 1:n+1 is the second moment of the smallest order statistic in a sample of size n + 1, provided that lim x→+∞ − x 2 2 F (x) log F (x) = 0.

Bounds
Let us consider a sample with parent distribution X such that E(X) = 0 and E(X 2 ) = 1. Hartley andDavid, 1954, andGumbel, 1954, have shown that We relate µ n:n with the mean of the largest statistic order from the standard distribution. In fact, by normalizing the random variable X with mean µ and variance σ 2 we get Hence, the cdf F Z is given in terms of the cdf F X by Then, cdf and pdf of the largest order statistic in a sample of size n are The mean of X n:n is given by The mean of the largest statistic order from Z is given by Using the Hartley-David-Gumbel bound for a non-negative parent distribution with mean µ and variance σ 2 , we get Theorem 1 Let X be a non-negative random variable with mean µ and variance σ 2 . Then, we obtain an upper bound for the CRE of X Proof From (2) and (8) we get i.e., the upper bound given in (9) Remark 2 Since X is non-negative we have that µ n+1:n+1 ≥ 0, for all n ∈ N. For this reason, using finite series approximations we get lower bounds for E(X): for all m ∈ N.
Remark 3 Since X is non-negative we have that µ 1:n+1 ≥ 0, for all n ∈ N. For this reason, using finite series approximations we get upper bounds for CE(X): for all m ∈ N.
Theorem 2 Let X be DFR (decreasing failure rate). Then, we have the following lower bound for CE(X) Proof Let X be DFR. From Theorem 12 of Rychlik (2001) we know that for a sample of size n, if For j = 1 we have δ 1 = 1 n ≤ 2 for all n ∈ N and we get Then, from (4) we get the following lower bound for CE(X) Remark 4 We note that we can not provide an analogous bound for E(X) because δ n ≤ 2 is not fulfilled for n ≥ 4. David and Nagaraya, 2003, showed that if we have a sample X 1 , . . . , X n with parent distribution X symmetric about 0 with variance 1, then where Using the bound (11) for a non-negative parent distribution symmetric about the mean µ, with bounded support and variance σ 2 , we get µ n:n = σE(Z n:n ) + µ ≤ 1 2 σnc(n) + µ.
Theorem 3 Let X be a symmetric non-negative random variable with bounded support, mean µ and variance σ 2 . Then, we obtain an upper bound for the CRE of X Proof From (2) and (12) we get i.e., the upper bound given in (13). About a symmetric distribution, Arnold and Balakrishnan, 1989, showed that if we have a sample X 1 , . . . , X n with parent distribution X symmetric about 0 with variance 1, then where B(n, n) is the complete beta function.
Using the bound (14) for a non-negative parent distribution symmetric about the mean µ and with variance σ 2 , we get µ n:n = σE(Z n:n ) + µ ≤ σ n √ 2 Theorem 4 Let X be a symmetric non-negative random variable with mean µ and variance σ 2 . Then, we obtain an upper bound for the CRE of X Proof From (2) and (15) we get i.e., the upper bound given in (16).
Example 6 Let us consider a sample with parent distribution X ∼ N (0, 1). From Harter, 1961, we get the values of the mean of the largest order statistic for samples of size less than 100. Hence, we compare the finite series approximation of (2) and (16) and we expect the same relation given in Theorem 4 because truncated terms are negligible. We get the following result From (2) and (4) we get the following expression for the sum of the cumulative residual entropy and the cumulative entropy Cal et al., 2017, showed a connection among (17) and the partition entropy studied by Bowden, 2007.

Theorem 5
We have the following bound for the sum of the CRE and the CE Proof From Theorem 3.24 of Arnold and Balakrishnan, 1989, we know the following bound for the difference between the expectation of the largest and the smallest order statistics from a sample of size n + 1 µ n+1:n+1 − µ 1:n+1 ≤ σ 2(n + 1), and so using (19) in (17) we get the following bound for the sum of the CRE and the CE E(X) + CE(X) ≤ +∞ n=1 σ 2(n + 1) n(n + 1) = +∞ n=1 √ 2 σ n √ n + 1 ≃ 3.09 σ.
About a symmetric distribution, Arnold and Balakrishnan, 1989, showed that if we have a sample X 1 , . . . , X n with parent distribution X symmetric about the mean µ with variance 1, then where B(n, n) is the complete beta function.
Proof From (17) and (21)  i.e., the upper bound given in (22). In Table 1 we present some applications of the bounds obtained in this section to important distributions in the reliability theory.