Block average quantile regression for massive dataset

Abstract

Nowadays, researchers are frequently confronted with challenges from large-scale data computing. Quantile regression on massive dataset is challenging due to the limitations of computer primary memory. Our proposed block average quantile regression provides a simple and efficient way to implement quantile regression on massive dataset. The major novelty of this method is splitting the entire data into a few blocks, applying the convectional quantile regression onto the data within each block, and deriving final results through aggregating these quantile regression results via simple average approach. While our approach can significantly reduce the storage volume needed for estimation, the resulting estimator is theoretically as efficient as the traditional quantile regression on entire dataset. On the statistical side, asymptotic properties of the resulting estimator are investigated. We verify and illustrate our proposed method via extensive Monte Carlo simulation studies as well as a real-world application.

Appendix

Proof of Theorem 1

Note that

$$\begin{aligned} \hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau )=\frac{1}{K}\sum \limits ^K_{k=1}\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )=\frac{1}{K}\sum \limits ^K_{k=1}\left( \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) . \end{aligned}$$

(18)

From Proposition 1 and Proposition 2 of El Bantli and Hallin (1999), $\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\xrightarrow {p}\varvec{0}$ as $n\rightarrow \infty$. According to property of convergence in probability, we have

$$\begin{aligned} \begin{array}{lll} \hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau )\xrightarrow {p}\varvec{0}&as&n\rightarrow \infty . \end{array} \end{aligned}$$

(19)

Consequently, the consistency holds.

By Knight (1998) and Koenker (2005), $\Vert \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\Vert _{2} = O_p(1/\sqrt{n})$. Then, from (18), we may derive

$$\begin{aligned} \begin{array}{lll} \Vert \hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau )\Vert _{2} &{}=&{} \Vert \frac{1}{K}\sum \limits ^K_{k=1}\left( \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) \Vert _2\\ &{}\le &{} \frac{1}{K}\sum \limits ^K_{k=1}\Vert \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\Vert _2\\ &{}=&{} O_p(1/\sqrt{n}). \end{array} \end{aligned}$$

(20)

Thus, the estimator $\hat{\varvec{\beta }}(\tau )$ is said to converge with rate $O_p(1/\sqrt{n})$.

Proof of Theorem 2

From Theorem 4.1 of Koenker (2005), the joint asymptotic distribution of the quantile regression estimator $\hat{\varvec{\beta }}^{(k)}(\tau )$ takes the form:

$$\begin{aligned} \sqrt{n}(\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau ))\xrightarrow {d} N(\varvec{0},\varvec{\Sigma }^{(k)}), \end{aligned}$$

(21)

where $\varvec{\Sigma }^{(k)}=\omega ^{2}_{k}[\varvec{D}^{(k)}_{0}]^{-1}$ for i.i.d. errors and $\varvec{\Sigma }^{(k)}=\tau (1-\tau )[\varvec{D}^{(k)}_{1}(\tau )]^{-1}\varvec{D}^{(k)}_{0}$$[\varvec{D}^{(k)}_{1}(\tau )]^{-1}$ for non-i.i.d. errors.

From C3, the Bahadur representation for the estimator $\hat{\varvec{\beta }}^{(k)}(\tau )$ is $\sqrt{n}(\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau ))=\varvec{W}^{(k)}+R^{(k)}_{n}$. By Arcones (1996) and He and Shao (1996), the remainder term meets

$$\begin{aligned} \begin{array}{lll} R^{(k)}_{n}=O(n^{-1/4}(\log \log n)^{3/4})\xrightarrow {p}0&as&n\rightarrow \infty . \end{array} \end{aligned}$$

(22)

Hence, combining (21) with (22), using the Theorem 2.7 of van der Vaart (1998), we can infer that $\varvec{W}^{(k)}= \sqrt{n}(\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )) - R^{(k)}_{n}$ has the same asymptotic normality as $\sqrt{n}(\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau ))$, i.e.

$$\begin{aligned} \varvec{W}^{(k)}\xrightarrow {d} N(\varvec{0},\varvec{\Sigma }^{(k)}). \end{aligned}$$

(23)

First, based on C2, it is easy to show that $\varvec{\Sigma }^{(1)}=\varvec{\Sigma }^{(2)}=\cdots =\varvec{\Sigma }^{(K)}=\varvec{\Sigma }$ for both i.i.d. and non-i.i.d. cases. Using the BAQR estimator, we derive

$$\begin{aligned} \begin{array}{lll} \sqrt{N}(\hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau )) &{}=&{} \sqrt{N}\left( \frac{1}{K}\sum \limits ^K_{k=1}\hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) \\ &{}=&{} \frac{\sqrt{N}}{K} \sum \limits ^K_{k=1}\left( \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) \\ &{}=&{} \frac{\sqrt{N}}{K\sqrt{n}} \sum \limits ^K_{k=1}\sqrt{n}\left( \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) \\ &{}=&{} \frac{1}{\sqrt{K}} \sum \limits ^K_{k=1}\sqrt{n}\left( \hat{\varvec{\beta }}^{(k)}(\tau )-\varvec{\beta }(\tau )\right) \\ &{}=&{} \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}+\frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}R^{(k)}_{n}. \end{array} \end{aligned}$$

(24)

Second, we show that $\frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}$ follows a multivariate normal distribution. Since $(\varvec{x}_{i},y_{i})$ are independent for $i=1,2,\ldots ,N$, $\varvec{W}^{(k)}$ defined in (9) or (10) is therefore independent for $k=1,2,\ldots ,K$. From (23), we know that $\varvec{W}^{(k)}$ follows a multivariate normal distribution. Hence, we conclude that $\frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}$ also follows a multivariate normal distribution.

Third, we calculate the expectation and covariance matrix of $\frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}$. From (23), we have

$$\begin{aligned} \text {E}\left( \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}\right) =\frac{1}{\sqrt{K}} \sum \limits ^K_{k=1} \text {E}(\varvec{W}^{(k)})=\varvec{0}, \end{aligned}$$

(25)

and

$$\begin{aligned} \begin{array}{lll} \text {Var}\left( \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}\right) &{}=&{} \frac{1}{K} \sum \limits ^K_{k=1} \text {Var}(\varvec{W}^{(k)})\\ &{}=&{} \frac{1}{K} (\varvec{\Sigma }^{(1)}+\varvec{\Sigma }^{(2)}+\cdots +\varvec{\Sigma }^{(K)})\\ &{}=&{} \varvec{\Sigma }. \end{array} \end{aligned}$$

(26)

Based on the results of the second and third parts, we can infer that

$$\begin{aligned} \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}\xrightarrow {d} N(\varvec{0},\varvec{\Sigma }). \end{aligned}$$

(27)

Fourth, we show that $\sqrt{N}(\hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau ))$ follows a multivariate normal distribution. If

$$\begin{aligned} \begin{array}{lll} \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}R^{(k)}_{n}=O(\sqrt{K}n^{-1/4}(\log \log n)^{3/4})\xrightarrow {p}0,&as&n\rightarrow \infty , \end{array} \end{aligned}$$

(28)

holds(see Eq. (22)), combining (27) with (28), using the Theorem 2.7 of van der Vaart (1998), we can infer that $\sqrt{N}(\hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau ))$ has the same asymptotic normality as $\frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}\varvec{W}^{(k)}$, i.e.

$$\begin{aligned} \sqrt{N}(\hat{\varvec{\beta }}(\tau )-\varvec{\beta }(\tau ))\xrightarrow {d} N(\varvec{0},\varvec{\Sigma }). \end{aligned}$$

(29)

To ensure (29), we suggest $N=Ke^{K^{1/2}}$. In addition, using $N=nK$ we then get $K=\log ^2n$. Hence, as $n\rightarrow \infty$, we have

$$\begin{aligned} \begin{array}{lll} \frac{1}{\sqrt{K}}\sum \limits ^K_{k=1}R^{(k)}_{n}&{}=&{}O(\sqrt{K}n^{-1/4}(\log \log n)^{3/4})\\ &{}=&{}O(n^{-1/4}(\log \log n)^{3/4}\log n)\xrightarrow {p}0. \end{array} \end{aligned}$$

(30)

Therefore, the result of asymptotic normality in (12) holds when $N=Ke^{K^{1/2}}$, and we complete the whole proof.