Application of the empirical characteristic function to compare and estimate densities by pooling information

Summary

Independent measurements are taken from distinct populations which may differ in mean, variance and in shape, for instance in the number of modes and the heaviness of the tails. Our goal is to characterize differences between these different populations. To avoid pre-judging the nature of the heterogeneity, for instance by assuming a parametric form, and to reduce the loss of information by calculating summary statistics, the observations are transformed to the empirical characteristic function (ECF). An eigen decomposition is applied to the ECFs to represent the populations as points in a low dimensional space and the choice of optimal dimension is made by minimising a mean square error. Interpretation of these plots is naturally provided by the corresponding density estimate obtained by inverting the ECF projected on the reduced dimension space. Some simulated examples indicate the promise of the technique and an application to the growth of Mirabilis plants is given.

Appendices

A APPENDIX: The expression for the MSE

Recall from the definitions given in Section 2 above that m is the ECF, the super-script m^c denotes the centered version, the tilde $\tilde{m}$ denotes the version standardized by the variance matrix. We suppose that n_i and n grow at the rate. From (4) and the strong law of large numbers we have the a.s. expansion $\tilde{m}_i(t)=\psi_i(t)+n^{-\frac{1}{2}}e_i$ with error $e_i=O(\sqrt{{\rm{lnln}}n})$.

We can express (7) by

$$\begin{array}{*{20}{c}} {M = \sum\limits_{i - 1}^N {\frac{{{n_i}}}{n}\tilde m_i^c(t)\tilde m_i^c(t)' = \sum\limits_{i = 1}^N {\frac{{{n_i}}}{n}\psi _i^c(t)\psi _i^c(t)'\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;} } } \\ { + \;\;{n^{ - \tfrac{1}{2}}}\sum\limits_{i = 1}^N {\frac{{{n_i}}}{n}\left\{ {\psi _i^c(t)e_i^{c'} + e_i^c\psi _i^c(t)'} \right\} + {n^{ - 1}}\sum\limits_{i = 1}^N {\frac{{{n_i}}}{n}e_i^ce_i^{c'}} } } \\ { = \Psi + {n^{ - \tfrac{1}{2}}}{T_1} + {n^{ - 1}}{T_2},\;say\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;} \end{array}$$

(13)

Equation (13) allows the use of the perturbation theory for linear operators for non-analytic perturbations (for an exposition see Fine, 1987) and leads to expressions for the eigen elements of M as a function of those of Ψ where $\Psi=\sum\nolimits_{i=1}^N\frac{n_i}{n}\psi_i^c(t)\psi_i^c(t)'$. Denote by λ_j the eigen values ofΨ ranged in decreasing order and assumed to be distinct, and by P_j the orthogonal eigen projector associated with λ_j; correspondingly $\widehat{\lambda}_j$ and $\widehat{P}_j$ denote the eigen value and projections of M.

We have the following a.s. expansions: for j = 1 to p,

$$\widehat{\lambda}_j=\lambda_j+n^{-1}{\rm{tr}}(T_1P_j)+n^{-2}{\rm{tr}}(T_2P_j-T_1[\Psi-\lambda_jI_p]^-T_1P_j)+O(n^{-3/2})$$

(14)

and

$$\widehat{P}_j=P_j+n^{-\frac{1}{2}}P_j^1+n^{-1}P_j^2+O(n^{-\frac{3}{2}}), {\rm{say}}$$

(15)

where

$$P_j^1=[\Psi-\lambda_jI_p]^-T_1P_j+P_jT_1[\Psi-\lambda_jI_p]^-,$$

(16)

$$P_j^2=[\Psi-\lambda_jI_p]^-T_2P_j+P_jT_2\Psi-\lambda_jI_p]^--P_jT_1[\Psi-\lambda_jI_p]^{-2}T_1P_j-[\Psi-\lambda_jI_p]^{-2}T_1P_jT_1P_j-P_jT_1P_jT_1[\Psi-\lambda_jI_p]^{-2}+P_jT_1[\Psi-\lambda_jI_p]^-T_1[\Psi-\lambda_jI_p]^-+[\Psi - \lambda_jI_p]^-T_1[\Psi-\lambda_jI_p]^-T_1P_j+[\Psi-\lambda_jI_p]^-T_1P_jT_1[\Psi-\lambda_jI_p]^-,$$

(17)

and where [A]⁻ denotes the Moore Penrose generalized inverse of A.

Consequently

$$\widehat{\psi}_i^l(t)-\psi_i(t) = \sum_{j=l+1}^p P_j\psi_i^c(t)+n^{-\frac{1}{2}}\sum_{k=1}^l(P_k^1\psi_i^c(t)+P_k^1e_i^c)+n^{-1}\sum_{k=1}^lP_k^2\psi_i^c(t)$$

(18)

Now, for l = 1, …, p,

$$MSE_l = E\sum_{i=1}^N\frac{n_i}{n}\{\widehat{\psi}_i^l(t) - \psi_i(t)\}'\{\widehat{\psi}_i^l(t)-\psi_i(t)\}.$$

(19)

At the price of simple, but tedious, calculations using equations (14) to (18)

$$E\sum_{i=1}^N\frac{n_i}{n}\left\{ \sum_{j=l+1}^p P_j\psi_i^c(t) \right\}' \left\{ \sum_{j=l+1}^p P_j\psi_i^c(t) \right\} = {\rm{tr}} \left\{ \sum_{i=1}^N \frac{n_i}{n} \sum_{j=l+1}^p P_j\psi_i^c(t)\psi_i^c(t)' \right\} = \sum_{j=l+1}^p \lambda_j.$$

(20)

Furthermore

$$[\Psi-\lambda_jI_p]^-=\sum_{k=1, k \neq j}^p \frac{1}{\lambda_k-\lambda_j}P_k$$

(21)

and

$$\Psi[\Psi-\lambda_jI_p]^-=[\Psi-\lambda_jI_p]^-\Psi=I-P_j+\lambda_j[\Psi-\lambda_jI_p]^-.$$

(22)

For any Hermitian matrices A and B and for any centred random matrix U,

$$E\{{\rm{tr}}(AU BU)\} = {\rm{tr}}\{(B \otimes A)E{\rm{vec}}(U){\rm{vec}}(U)'\}={\rm{tr}}\{(B \otimes A) {\rm{var}}({\rm{vec}}(U))\}$$

where ⊗ here denotes the Kronecker product of matrices. When expanding MSE_l, this property is applied to A = P_k and B = P_j, for k ≠ j, and to $U=\psi_i^c(t)e_i^c(t)' + e_i^c(t)\psi_i^c(t)'$ . Note also

$$Ee_i^c(t)e_i^c(t)' = (\frac{n}{n_i}-1)I_p, \;\;\;{\rm{and}} \\2\sum_{j=1}^l\sum_{k=1}^p\frac{\lambda_j}{\lambda_k-\lambda_j} = -l(l-1)+2\sum_{j=1}^l\sum_{k=l+1}^p\frac{\lambda_j}{\lambda_k-\lambda_j}.$$

Finally the following result is obtained

$$MSE_l = \sum_{j=l+1}^p\lambda_j+n^{-1}\left( l(n - p+l)+p-l-2\sum_{j=1}^l\sum_{k=l+1}^p\frac{\lambda_j}{\lambda_k-\lambda_j}\right)+O(n^{-3/2}).$$

The estimator the unknown parameters by their estimates and by taking into account the bias of the estimators of the eigen values given by:

$$E\sum_{j=l+1}^p\widehat{\lambda}_j=\sum_{j=l+1}^p\lambda_j \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\ \\ \;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;+n^{-1} \left( np-(p-l)-l(n-p+l)+2\sum_{j=1}^l \sum_{k=l+1}^p\frac{\lambda_j}{\lambda_k-\lambda_j} \right)+O(n^{-3/2}).$$

Finally it follows that

$$E(\widehat{MSE}_l)=MSE_l+O(n^{-3/2}) \;\;\; {\rm{and}}$$

(23)

$${\rm{var}}(\widehat{MSE}_l)=4n^{-1}\sum_{j=l+1}^p \lambda_j+O(n^{-3/2}).$$

(24)

B APPENDIX: The parameters of the mixture example

The mixtures in the simulation example of Section 3.3 have no more than 3 components. They are defined by the 3 means, standard deviations, and mixture probabilities.

Table 2