Estimation of Bergsma’s covariance

Abstract

Bergsma (A new correlation coefficient, its orthogonal decomposition and associated tests of independence, arXiv preprint arXiv:math/0604627, 2006) proposed a covariance $\kappa (X,Y)$ between random variables X and Y, and gave two estimates for it, based on n i.i.d. samples. He derived the asymptotic distributions of these estimates under the assumption of independence between X and Y. Our main focus is on the dependent case. This measure turns out to be same as the distance covariance (dCov) measure for multivariate X and Y, when we specialize to real-valued X and Y. We first derive several alternate expressions for $\kappa$, which are useful to understand the properties of $\kappa$ and its estimates better. One of the alternate expressions for $\kappa$ leads to a very intuitive third estimator of $\kappa$ that is a nice function of four U-statistics. We establish the exact finite sample algebraic relation between the three estimates. This yields the relation between the bias of these estimators. In the dependent case, using the U statistics central limit theorem, it is easy to show that our estimate is asymptotic normal. The relation between the three estimates is then used to show that Bergsma’s two estimates have the same limit distribution in the dependent case. When X and Y are independent, the above limit is degenerate. With a higher scaling, the non-degenerate limit distribution of all three estimators is obtained using the theory of degenerate U-statistics and the above algebraic relations. In particular, the known asymptotic distribution results for the two estimates of Bergsma for the independent case follow. For specific parametric bivariate distributions, the value of $\kappa$ can be derived in terms of the natural dependence parameters of these distributions. In particular, we derive the formula for $\kappa$ when (X, Y) are distributed as Gumbel’s bivariate exponential. We bring out various aspects of these estimators through extensive simulations from several prominent bivariate distributions. In particular, we investigate the empirical relationship between $\kappa$ and the dependence parameters, the distributional properties of the estimators, and the accuracy of these estimators. We also investigate the finite sample powers of these measures for testing independence, compare these among themselves, and with other well known such measures. Based on these exercises, the proposed estimator seems as good or better than its competitors both in terms of power and computing efficiency.

Appendix

1.1 A. Six bivariate distributions

Standard univariate distributions can be extended to multivariate distributions in multiple ways. For illustration we have chosen six distributions. We present the specific bivariate extensions used in our simulations.

Normal: This is simply the bivariate normal distribution with zero means, unit variances and correlation denoted by $\theta$.

Uniform: A pair of correlated uniform random variables (X, Y) is generated by starting with two independent uniform variables U and V. Consider a dependency parameter $\theta \in [-1,\ 1]$. Note that $\theta = \pm 1$ indicate perfect correlation. Thus in that case the uniform pair (X, Y) equals $(U, 1-U)$ or (U, U). In other cases we draw another random number, say W, from Beta$(\alpha , 1)$, where the shape parameter $\alpha$ equals

$$\begin{aligned} \alpha = \frac{1}{2} \left[ \sqrt{\frac{49+\theta }{1+\theta }} -5 \right] . \end{aligned}$$

Then we define the dependent pair (X, Y) by $X=U$ and

$$\begin{aligned} Y = \left\{ \begin{array}{ll} |W-X| &{} \text{ if } \, \, \, V < 1/2 \\ 1-|1-W-X| &{} \text{ if } \, \, \, V \ge 1/2. \end{array} \right. \end{aligned}$$

Exponential: The dependent exponential pair (X, Y) is obtained as follows. Let X be Exponential(1) random variable. Let U be a uniform(0,1) random variable. Then define Y as

$$\begin{aligned} Y = \left\{ \begin{array}{ll} Exponential(1 + \theta X))) &{} \text{ if } \, \, \, E/(E + G)) < U \\ Exponential(2 + \theta X))) &{} \text{ if } \, \, \, E/(E + G)) \ge U, \end{array} \right. \end{aligned}$$

where $E = ((1 - \theta + \theta X)/\exp (X))/(1 + \theta X)$ and $G = ((\theta + \theta ^2 X)/\exp (X))/((1 + \theta X)^2).$

Laplace: $Z = (X, Y)$ is said to be bivariate standard Laplace, if their joint density is as follows.

$$\begin{aligned} f(z) = \frac{2}{2\pi {|\Sigma |}^{1/2}} \left\{ \frac{\pi }{2 \sqrt{2 z^T \Sigma ^{-1} z}}\right\} ^{1/2} \exp \big (\sqrt{-2 z^T \Sigma ^{-1} z} \big ), \end{aligned}$$

where $\Sigma$ is the $2\times 2$ matrix with diagonal and off-diagonal elements as 1 and $\theta$ respectively.

Logistic: Let U, V be two correlated uniform random variables as defined above. Define $X = \log (U) - \log (1-U)$ and $Y = \log (V) - \log (1-V)$. Then (X, Y) has bivariate logistic distribution.

Chi-square: Let U, V be bivariate mean zero normal rvs with unit variance and correlation $\theta$. Define $X = U^{2}$ and $Y = V^{2}$. Clearly, each has $\chi ^{2}$ distribution and have correlation $\theta ^2$ and (X, Y) has bivariate Chi-square distribution.

1.2 B. Discrete approximation

Recall that when X and Y are independent, the asymptotic limits feature eigenvalues that depend on the distribution of X and Y. The eigensystem of the kernel $h_F$ is the solution to the integral equation,

$$\begin{aligned} \lambda g(z) = \mathbb {E}h_F(z,Z)g(Z). \end{aligned}$$

(7.1)

In general, this equation does not admit a closed form solution. For the case of discrete and continuous F, this is reduced to a simpler problem as follows.

The eigensystem in the discrete case: Let X be a discrete random variable taking values $x_1< x_2< \cdots < x_{t}$ with $P(X=x_m)=p_m, 1\le m \le t$. Then we have the following lemma.

Lemma 3

(Bergsma (2006)) The non-zero eigenvalues and eigenvectors of $h_F$ are the solutions to the equations;

$$\begin{aligned} \begin{aligned} p_1g(x_1)&=\lambda c_2[g(x_1)-g(x_2)] \\ p_tg(x_t)&=-\lambda c_t[g(x_{t-1})-g(x_t)]\\ p_mg(x_m)&=-\lambda [c_mg(x_{m-1})-(c_m+c_{m+1})g(x_i)+(c_{m+1})g(x_{m+1})], \hspace{0.5cm} 2\le m \le t-1, \end{aligned} \end{aligned}$$

(7.2)

where $c_m=(x_m-x_{m-1})^{-1}$.

In matrix notation we must solve the generalized eigenvalue problem $D_pg=\lambda C g$, where $D_p$ is a diagonal matrix with $\{p_m\}$ on the main diagonal, g is the eigenvector with corresponding eigenvalue $\lambda$, and C is a matrix of coefficients $c_m$ from the above system, i.e.,

$$\begin{aligned} C=\begin{pmatrix} c_2 &{} -c_2 &{} 0 &{} 0 &{} \\ -c_2 &{} (c_2+c_3) &{} -c_3 &{} 0 &{}\ldots \\ 0 &{} -c_3 &{} (c_3+c_4) &{} -c_4 \\ 0 &{} 0 &{} -c_4 &{} (c_4+c_5)\\ &{} &{} &{} &{} \ddots \\ \vdots &{} &{} &{} &{} &{} &{} (c_{t-1}+c_t) &{} -c_t\\ &{} &{} &{} &{} &{} &{} -c_t &{}c_t \\ \end{pmatrix} \end{aligned}$$

Incidentally, the equation given in Bergsma (2006) appears to have some typographical errors in the signs of the entries of his C matrix and the corresponding difference equations. We have made the required corrections in the difference equations and the C matrix above.

The eigensystem in the continuous case:

Lemma 4

(Bergsma (2006)) Suppose F is strictly increasing on the support of the probability distribution and f is the derivative of F. Let g be the eigenfunction corresponding to the integral (7.1). Let $y(x)=g(F^{-1}(x))$ and suppose y is twice differentiable. Then any eigenvalue $\lambda$ and its corresponding eigenvector $y_k(x)$ are solutions to the equation

$$\begin{aligned} \frac{d}{dx}f[F^{-1}(x)]y'(x)+\lambda ^{-1}y(x)=0, \end{aligned}$$

(7.3)

subject to the condition

$$\begin{aligned} f[F^{-1}(x)]y'(x)\rightarrow 0 \hspace{0.2cm} \text {as} \hspace{0.2cm} x\downarrow 0 \hspace{0.2cm}\text {or} \hspace{0.2cm} x \uparrow 1. \end{aligned}$$

For most distributions, Eq. (7.3) does not have a closed from solution. In such cases we take a cue from the disscrete case and use a discrete approximation as follows. Let X be a continuous random variable with distribution function F. For a (large) positive integer t, define a discrete approximation $X^{(t)}$ to X as

$$\begin{aligned} P(X^{(t)}={x_m}^{(t)})=p_m=\frac{1}{t},\ \ \text {where}\ \ {x_m}^{(t)}=F^{-1}\big (\frac{m-(1/2)}{t}\big ), \ m=1, \ldots , t. \end{aligned}$$

Let $F^{(t)}$ be the distribution function of $X^{(t)}$. Then the eigen pair for the kernel $h_{F^{(t)}}$, obtained by solving (7.2) with the coefficients $c_m={({x_m}^{(t)}-{x_{m-1}}^{(t)})}^{-1}$, serves as an approximate eigen pair for $h_F$.

1.3 C. Empirical distributions of the $\kappa$ estimators

Figure 11 presents the box-plots of the empirical asymptotic distributions of the three estimates with various underlying distributions having varying degrees of dependence.

Fig. 11

Box plots of $\kappa ^*$ (left), ${\tilde{\kappa }}$ (middle), and $\hat{\kappa }$ (right) for the six bivariate distributions.The true values of the dependency parameter $\theta$ are on the horizontal axes. Sample size $n=100$ and 1000 replicates

1.4 D. Empirical power

We now present power computation for tests of independence, which are based on the three estimators of $\kappa$, as well as on other known measures of dependence.

Table 1 Estimated powers for various dependence measures based on data from six bivariate distributions with varying degrees of dependence

1.5 E. Computation time

We now present efficiency data for computation of these measures of dependence mentioned earlier.

Table 2 Estimated mean and standard deviations of time taken (in seconds) over 10 replications to compute 100 values each for the various measures of dependence based on 100 pairs of bivariate normal data with varying degree of dependence

1.6 F. Code for the $\kappa$ estimates