Limiting Spectral Distribution of Large Sample Covariance Matrices Associated with a Class of Stationary Processes

Abstract

In this paper, we derive an extension of the Marc̆enko–Pastur theorem to a large class of weak dependent sequences of real-valued random variables having only moment of order 2. Under a mild dependence condition that is easily verifiable in many situations, we derive that the limiting spectral distribution of the associated sample covariance matrix is characterized by an explicit equation for its Stieltjes transform, depending on the spectral density of the underlying process. Applications to linear processes, functions of linear processes, and ARCH models are given.

Appendix

In this section, we give some upper bounds for the partial derivatives of $f$ defined in (4.35).

Lemma 5.1

Let $x$ be a vector of $\mathbb{R }^{n N}$ with coordinates

$$\begin{aligned} x =\big ( x^{(1)}, \ldots , x^{(n)} \big ) \, \text { where for any } i \in \{1, \ldots , n \}, \ x^{(i)} = \big ( x^{(i)}_k \, , \, k \in \{1, \ldots , N \} \big ) . \end{aligned}$$

Let $z=u+{\sqrt{ -1}}v\in \mathbb C ^+$ and $f:=f_z$ be the function defined in (4.35). Then, for any $i \in \lbrace 1 ,\ldots , n \rbrace$ and any $j,k,\ell ,m \in \lbrace 1 ,\ldots , N \rbrace$, the following inequalities hold true:

$$\begin{aligned}&\left| \frac{\partial ^2 f }{\partial x^{(i)}_{m} \partial x^{(i)}_{j}} (x) \right| \le \frac{8}{v^3 n^2 N} \sum _{r=1}^{N}\big |x^{(i)}_{r}\big |^2 + \frac{2}{v^2 n N} \, ,\\&\quad \left| \frac{\partial ^3 f }{\partial x^{(i)}_{\ell }\partial x^{(i)}_{m} \partial x^{(i)}_{j}} (x) \right| \le \frac{48}{v^4 n^3 N} \, \left( \sum _{r=1}^{N}\big |x^{(i)}_{r}\big |^2 \right) ^{3/2} +\frac{24 }{v^3 n^2 N} \left( \sum _{r=1}^{N}\big |x^{(i)}_{r}\big |^2 \right) ^{1/2} , \end{aligned}$$

and

$$\begin{aligned} \left| \frac{\partial ^4 f }{\partial x^{(i)}_{k}\partial x^{(i)}_{\ell }\partial x^{(i)}_{m} \partial x^{(i)}_{j}} (x) \right| \le \frac{24 \times 16}{v^5 n^4 N} \left( \sum _{r=1}^{N} \big |x^{(i)}_{r}\big |^2\right) ^2 + \frac{36 \times 8}{v^4 n^3 N} \sum _{r=1}^{N} \big |x^{(i)}_{r}\big |^2 + \frac{24}{v^3 n^2 N}. \end{aligned}$$

Proof

Recall that $f(x)=\frac{1}{N}\mathrm{Tr}\big (A(x)-z\mathbf{{I}}\big )^{-1}$ where $A(x) = \frac{1}{n}\sum _{k=1}^{n}( x^{(k)})^Tx^{(k)}$. To prove the lemma, we shall proceed as in Chatterjee [4] (see the proof of its Theorem 1.3) but with some modifications since his computations are made in case where $A(x)$ is a Wigner matrix of order $N$.

Let $i \in \lbrace 1, \ldots , n \rbrace$ and consider for any $j,k \in \lbrace 1 , \ldots , N \rbrace$, the notations $\partial _{j}$ instead of $\partial / \partial x^{(i)}_{j}$, $\partial _{jk}^{2}$ instead of $\partial ^{2}/\partial x^{(i)}_{j} \partial x^{(i)}_{k}$ and so on. We shall also write $A$ instead of $A(x)$, $f$ instead of $f(x)$, and define $G= \big (A-z\mathbf{{I}}\big )^{-1}$.

Note that $\partial _{j}A$ is the matrix with $n^{-1}\big (x^{(i)}_{1}, \ldots ,x^{(i)}_{j-1},2x^{(i)}_{j},x^{(i)}_{j+1},\ldots , x^{(i)}_{N}\big )$ as the $j^{th}$ row, its transpose as the $j^{th}$ column, and zero otherwise. Thus, the Hilbert–Schmidt norm of $\partial _{j}A$ is bounded as follows:

$$\begin{aligned} \Vert \partial _{j}A \Vert _{2} = \frac{1}{n}\Big ( 2 \sum _{k=1 \, , k \ne j}^{N}|x^{(i)}_{k}|^2 + 4 |x^{(i)}_{j}|^2 \Big )^{1/2}\le \frac{2}{n} \Big (\sum _{k=1}^{N}|x^{(i)}_{k}|^2\Big )^{1/2}. \end{aligned}$$

(5.1)

Now, for any $m,j \in \lbrace 1 , \ldots , N \rbrace$ such that $m \ne j$, $\partial _{mj}^{2}A$ has only two non-zero entries which are equal to $1/n$, whereas if $m=j$, it has only one non-zero entry which is equal to $2/n$. Hence,

$$\begin{aligned} \Vert \partial _{mj}^{2}A \Vert _{2} \le \frac{2}{n} . \end{aligned}$$

(5.2)

Finally, note that $\partial _{lmj}^{3}A\equiv 0$ for any $j , m , l \in \lbrace 1 , \ldots , N \rbrace$.

Now, by using (4.36), it follows that, for any $j \in \lbrace 1 , \ldots , N \rbrace$,

$$\begin{aligned} \partial _{j}f=-\frac{1}{N}\mathrm{Tr}(G(\partial _{j}A)G) . \end{aligned}$$

(5.3)

In what follows, the notations $\sum _{\lbrace \!j',m'\rbrace \!=\!\lbrace j,m\rbrace }$, $\sum _{\lbrace j',m',\!\ell '\rbrace \!=\!\lbrace j,m, \ell \rbrace }$ and $\sum _{\lbrace j',m',\!\ell ' , k'\rbrace \!=\!\lbrace j,m, \ell , k \rbrace }$ mean, respectively, the sum over all permutations of $\lbrace j,m\rbrace$, of $\lbrace j,m, \ell \rbrace$, and of $\lbrace j,m, \ell , k\rbrace$. Therefore, the first sum consists of $2$ terms, the second one of $6$ terms, and the last one of $24$ terms. Starting from (5.3) and applying repeatedly (4.36), we then derive the following cumbersome formulas for the partial derivatives up to the order four: for any $j,m,\ell ,k \in \{1, \ldots , N \}$,

$$\begin{aligned} \partial _{mj}^{2}f=\frac{1}{N} \sum _{\lbrace j',m'\rbrace =\lbrace j, m \rbrace } \mathrm{Tr}\big (G(\partial _{j'}A) G (\partial _{m'}A) G \big ) - \frac{1}{N}\mathrm{Tr} \big ( G (\partial _{mj}^{2}A) G \big ), \end{aligned}$$

(5.4)

$$\begin{aligned} \partial _{\ell mj}^{3}f&= -\frac{1}{N} \sum _{\lbrace j',m',\ell '\rbrace =\lbrace j,m, \ell \rbrace } \mathrm{Tr}\big (G(\partial _{j'}A)G(\partial _{m'}A)G(\partial _{\ell '}A)G\big ) \nonumber \\&\quad +\frac{1}{N} \sum _{\lbrace j', m'\rbrace =\lbrace j, m \rbrace } \mathrm{Tr}\Big (G(\partial _{\ell j'}^{2}A)G(\partial _{m'}A)G + G(\partial _{j'}A)G(\partial _{\ell m'}^{2}A)G\Big ) \nonumber \\&\quad + \frac{1}{N} \mathrm{Tr}\big (G(\partial _{\ell }A)G(\partial _{mj}^{2}A)G\big ) + \frac{1}{N} \mathrm{Tr}\big (G(\partial _{mj}^{2}A)G(\partial _{\ell }A)G\big ), \end{aligned}$$

(5.5)

and

$$\begin{aligned} \partial _{ k \ell mj}^{4}f := I_1 +I_2 + I_3 + I_4 + I_5 +I_6 , \end{aligned}$$

(5.6)

where

$$\begin{aligned} I_1 = \frac{1}{N} \sum _{\lbrace j',m',\ell ' , k '\rbrace =\lbrace j,m, \ell ,k\rbrace } \mathrm{Tr}\big (G(\partial _{j'}A)G(\partial _{m'}A)G(\partial _{\ell '}A)G (\partial _{k'}A)G\big ), \end{aligned}$$

$$\begin{aligned} I_2&= -\frac{1}{N} \sum _{\lbrace j',m',\ell '\rbrace =\lbrace j,m, \ell \rbrace } \Big ( \mathrm{Tr}\big (G(\partial ^2_{kj'}A)G(\partial _{m'}A)G(\partial _{\ell '}A)G\big )\\&+ \mathrm{Tr}\big (G(\partial _{j'}A)G(\partial ^2_{km'}A)G(\partial _{\ell '}A)G\big ) + \mathrm{Tr}\big (G(\partial _{j'}A)G(\partial _{m'}A)G(\partial ^2_{k \ell '}A)G\big ) \Big ), \end{aligned}$$

$$\begin{aligned}&I_3=-\frac{1}{N} \sum _{\lbrace j',m'\rbrace =\lbrace j,m \rbrace } \Big ( \mathrm{Tr}\big (G(\partial ^2_{\ell j'}A)G(\partial _{k}A)G(\partial _{m'}A)G\big )\\&\quad \!+\!\mathrm{Tr}\big (G(\partial ^2_{\ell j'}A)G(\partial _{m'}A)G(\partial _{k}A)G\big ) \Big ) \!-\!\frac{1}{N} \sum _{\lbrace j',m'\rbrace \!=\!\lbrace j,m \rbrace } \Big ( \mathrm{Tr}\big (G(\partial _{k}A)G(\partial ^2_{\ell j'}A)G(\partial _{m'}A)G\big )\\&\quad \!+\!\mathrm{Tr}\big (G(\partial _{j'}A)G(\partial ^2_{\ell m'}A)G(\partial _{k}A)G\big ) \Big ) \!-\!\frac{1}{N} \sum _{\lbrace j',m'\rbrace \!=\!\lbrace j,m \rbrace } \Big ( \mathrm{Tr}\big (G(\partial _{k}A)G(\partial _{j'}A)G(\partial ^2_{\ell m'}A)G \big )\\&\quad \!+\!\mathrm{Tr}\big (G(\partial _{j'}A)G(\partial _{k}A)G(\partial ^2_{\ell m'}A)G\big ) \Big ) , \end{aligned}$$

$$\begin{aligned} I_4&= -\frac{1}{N} \sum _{\lbrace k',\ell '\rbrace =\lbrace k,\ell \rbrace } \Big ( \mathrm{Tr}\big (G(\partial ^2_{m j}A)G (\partial _{k'}A)G(\partial _{\ell '}A ) G \big )\\&+\mathrm{Tr}\big (G(\partial _{k'}A) G (\partial ^2_{m j}A) G(\partial _{\ell '}A ) G \big ) +\mathrm{Tr}\big (G(\partial _{k'}A) G(\partial _{\ell '}A ) G (\partial ^2_{m j}A) G \big ) \Big ) , \end{aligned}$$

$$\begin{aligned} I_5 = \frac{1}{N} \sum _{\lbrace k',\ell '\rbrace =\lbrace k,\ell \rbrace } \sum _{\lbrace j',m'\rbrace =\lbrace j,m \rbrace } \mathrm{Tr}\big ( G(\partial ^2_{\ell ' j' }A)G(\partial ^2_{k' m' }A)G\big ), \end{aligned}$$

and

$$\begin{aligned} I_6 = \frac{1}{N} \mathrm{Tr}\big (G(\partial _{m j}^{2}A)G(\partial ^2_{k \ell }A)G\big ) +\frac{1}{N} \mathrm{Tr}\big (G(\partial _{k \ell }^{2}A)G(\partial ^2_{m j}A)G\big ) . \end{aligned}$$

We start by giving an upper bound for $\partial _{mj}^{2}f$. Since the eigenvalues of $G^2$ are all bounded by $v^{-2}$, then so are its entries. Then, as $\mathrm{Tr}(G(\partial _{mj}^{2}A)G) = \mathrm{Tr}((\partial _{mj}^{2}A)G^2)$, it follows that

$$\begin{aligned} |\mathrm{Tr}(G(\partial _{mj}^{2}A)G)|=|\mathrm{Tr}((\partial _{mj}^{2}A)G^2)| \le 2v^{-2}n^{-1} . \end{aligned}$$

(5.7)

Next, to give an upper bound for $| \mathrm{Tr}\big (G(\partial _{j}A) G (\partial _{m}A) G \big ) |$, it is useful to recall some properties of the Hilbert–Schmidt norm: Let $B=(b_{ij})_{1\le i,j\le N}$ and $C=(c_{ij})_{1\le i,j\le N}$ be two $N\times N$ complex matrices in $\mathcal L _{2}$, the set of Hilbert–Schmidt operators. Then

• (a) $|\mathrm{Tr}(BC)|\le \Vert B\Vert _{2} \Vert C\Vert _{2}$.

• (b) If $B$ admits a spectral decomposition with eigenvalues $\lambda _1, \ldots , \lambda _N$, then $\max \lbrace \Vert BC\Vert _{2}, \Vert CB \Vert _{2}\rbrace \le \max _{1 \le i \le N} | \lambda _i|.\Vert C \Vert _{2}$.

(See, e.g., [20] pages 55–58, for a proof of these facts).

Using the properties of the Hilbert–Schmidt norm recalled above, the fact that the eigenvalues of $G$ are all bounded by $v^{-1}$, and (5.1), we then derive that

$$\begin{aligned}&| \mathrm{Tr}(G(\partial _{j}A) G(\partial _{m}A)G)| \le \Vert G(\partial _{j}A)G \Vert _{2} . \Vert (\partial _{m}A)G \Vert _{2}\nonumber \\&\quad \le \Vert G \Vert . \Vert (\partial _{j}A)G \Vert _{2} . \Vert \partial _{m}A \Vert _{2} . \Vert G \Vert \nonumber \\&\quad \le \Vert G \Vert ^{3} . \Vert \partial _{j}A \Vert _{2} . \Vert \partial _{m}A \Vert _{2} \le \frac{4}{v^3 n^2} \sum _{k=1}^{N}\big |x^{(i)}_{k}\big |^2. \end{aligned}$$

(5.8)

Starting from (5.4) and considering (5.7) and (5.8), the first inequality of Lemma 5.1 follows.

Next, using again the above properties (a) and (b), the fact that the eigenvalues of $G$ are all bounded by $v^{-1}$, (5.1) and (5.2), we get that

$$\begin{aligned}&| \mathrm{Tr}(G(\partial _{j} A) G(\partial _{m}A)G(\partial _{\ell }A)G)| \le \Vert G(\partial _{j}A)G (\partial _{m}A)G\Vert _{2} . \Vert (\partial _{\ell }A)G \Vert _{2} \nonumber \\&\quad \le \Vert G(\partial _{j}A)G (\partial _{m}A)\Vert _{2} . \Vert G \Vert ^2 . \Vert \partial _{\ell }A \Vert _{2} \le \Vert G(\partial _{j}A) \Vert _2 . \Vert G (\partial _{m}A)\Vert _{2} . \Vert G \Vert ^2 . \Vert \partial _{\ell }A \Vert _{2} \nonumber \\&\quad \le \Vert G \Vert ^4 . \Vert \partial _{j}A \Vert _2 . \Vert \partial _{m} A \Vert _{2} . \Vert \partial _{\ell }A \Vert _{2} \le \frac{8}{v^4 n^3} \Bigl ( \sum _{k=1}^{N}\big |x^{(i)}_{k}\big |^2 \Bigr )^{3/2}, \end{aligned}$$

(5.9)

and

$$\begin{aligned}&| \mathrm{Tr}(G(\partial ^2_{\ell j} A) G(\partial _{m}A)G)| \le \Vert G(\partial ^2_{\ell j}A)G \Vert _{2} . \Vert (\partial _{m}A)G \Vert _{2}\nonumber \\&\quad \le \Vert G \Vert ^2 \Vert G(\partial ^2_{\ell j}A) \Vert _{2} . \Vert \partial _{m} A\Vert _{2} \nonumber \\&\quad \le \Vert G \Vert ^3 . \Vert \partial ^2_{\ell j}A \Vert _2 . \Vert \partial _{m} A \Vert _{2} \le \frac{4}{v^3 n^2} \Bigl ( \sum _{k=1}^{N}\big |x^{(i)}_{k}\big |^2 \Bigr )^{1/2} . \end{aligned}$$

(5.10)

The same last bound is obviously valid for $| \mathrm{Tr}(G(\partial _{m}A)G(\partial ^2_{\ell j} A) G)|$. Hence, starting from (5.5) and considering (5.9) and (5.10), the second inequality of Lemma 5.1 follows.

It remains to prove the third inequality of Lemma 5.1. Using again the above properties (a) and (b), the fact that the eigenvalues of $G$ are all bounded by $v^{-1}$, (5.1) and (5.2), we infer that

$$\begin{aligned}&| \mathrm{Tr}(G(\partial _{j} A) G(\partial _{m}A)G(\partial _{\ell }A)G (\partial _{k}A)G)| \le \frac{16}{v^5 n^4} \Bigl ( \sum _{k=1}^{N}\big |x^{(i)}_{k}\big |^2 \Bigr )^{2},\end{aligned}$$

(5.11)

$$\begin{aligned}&| \mathrm{Tr}(G(\partial ^2_{\ell j} A) G(\partial _{m}A)G (\partial _{k}A)G)| \le \frac{8}{v^4 n^3} \sum _{k=1}^{N}\big |x^{(i)}_{k}\big |^2 , \end{aligned}$$

(5.12)

and

$$\begin{aligned} | \mathrm{Tr}(G(\partial ^2_{\ell j} A) G(\partial ^2_{mk}A)G )| \le \frac{4}{v^3 n^2} . \end{aligned}$$

(5.13)

Clearly, the bound (5.12) is also valid for the quantities $| \mathrm{Tr}( G(\partial _{m}A)\!G(\partial ^2_{\ell j} A)\!G (\partial _{k}A)\!G)|$ and $| \mathrm{Tr}( G(\partial _{m}A)G (\partial _{k}A)G(\partial ^2_{\ell j} A)G)|$. So, overall, starting from (5.6) and considering (5.11), (5.12), and (5.13), the third inequality of Lemma 5.1 follows. $\square$