A Necessary Bayesian Nonparametric Test for Assessing Multivariate Normality

Abstract

A novel Bayesian nonparametric test for assessing multivariate normal models is presented. Although there are extensive frequentist and graphical methods for testing multivariate normality, it is challenging to find Bayesian counterparts. The approach considered in this paper is based on the Dirichlet process and the squared radii of observations. Specifically, the squared radii are employed to transform the $m$-variate problem into a univariate problem by relying on the fact that if a random sample is coming from a multivariate normal distribution then the square radii follow a particular beta distribution. While the Dirichlet process is used as a prior on the distribution of the square radii, the concentration of the distribution of the Anderson–Darling distance between the posterior process and the beta distribution is compared to that between the prior process and beta distribution via a relative belief ratio. Key results of the approach are derived. The procedure is illustrated through several examples, in which it shows excellent performance.

Appendices

PROOF OF LEMMA 1

Consider $P_{N}(x)$ as

$$P_{N}(x)=\begin{cases}0\quad x<Y_{(1)}\\ P_{N}(Y_{(i)})\quad Y_{(i)}\leq x<Y_{(i+1)}\,(i=1,\ldots,N-1)\\ 1\quad x\geq Y_{(N)}.\end{cases}$$

Let $g(x)=\frac{dG(x)}{dx}$, then

$$d_{\textrm{AD}}(P_{N},G)=\int\limits_{-\infty}^{\infty}\frac{\left(P_{N}(x)-G(x)\right)^{2}}{G(x)\left(1-G(x)\right)}g(x)\,dx=\int\limits_{-\infty}^{Y_{(1)}}\frac{G(x)^{2}}{G(x)\left(1-G(x)\right)}g(x)\,dx$$

$${}+\int\limits_{Y_{(N)}}^{\infty}\frac{\left(1-G(x)\right)^{2}}{G(x)\left(1-G(x)\right)}g(x)\,dx+\sum_{i=1}^{N-1}\int\limits_{Y_{(i)}}^{Y_{(i+1)}}\frac{\left(P_{N}(Y_{(i)})-G(x)\right)^{2}}{G(x)\left(1-G(x)\right)}g(x)\,dx.$$

Substituting $y=G(x)$, $G(Y_{(i)})=U_{(i)}$, $G(-\infty)=0$, and $G(\infty)=1$, gives

$$d_{\textrm{AD}}(P_{N},G)=\sum_{i=1}^{N-1}\int\limits_{U_{(i)}}^{U_{(i+1)}}\frac{\left(P_{N}(Y_{(i)})-y\right)^{2}}{y\left(1-y\right)}\,dy+\int\limits_{0}^{U_{(1)}}\frac{y}{1-y}\,dy+\int\limits_{U_{(N)}}^{1}\frac{1-y}{y}\,dy$$

$${}=\sum_{i=1}^{N-1}\bigg{[}P_{N}^{2}(Y_{(i)})\log(y)-\Big{[}\left(P_{N}(Y_{(i)})-1\right)^{2}\log(1-y)\Big{]}-y\bigg{]}_{U_{(i)}}^{U_{(i+1)}}$$

$${}+\bigg{[}-y-\log(1-y)\bigg{]}_{0}^{U_{(1)}}+\bigg{[}\log(y)-y\bigg{]}_{U_{(N)}}^{1}=I_{1}+I_{2}+I_{3}.$$

Note that,

$$I_{1}=-\sum_{i=1}^{N-1}\left(U_{(i+1)}-U_{(i)}\right)-\sum_{i=1}^{N-1}\left(P_{N}(Y_{(i)})-1\right)^{2}\left(\log(1-U_{(i+1)})-\log(1-U_{(i)})\right)$$

$${}+\sum_{i=1}^{N-1}P_{N}^{2}(Y_{(i)})\left(\log(U_{(i+1)})-\log(U_{(i)})\right)$$

$${}=\sum_{i=1}^{N-1}P_{N}^{2}(Y_{(i)})\log\frac{U_{(i+1)}\left(1-U_{(i)}\right)}{U_{(i)}\left(1-U_{(i+1)}\right)}+\sum_{i=1}^{N-1}\left(2P_{N}(Y_{(i)})-1\right)\log\frac{1-U_{(i+1)}}{1-U_{(i)}}-\left(U_{(N)}-U_{(1)}\right).$$

Also, $I_{2}=-U_{(1)}-\log\left(1-U_{(1)}\right)$ and $I_{3}=-1-\log U_{(N)}+U_{(N)}.$ Therefore, adding $I_{1}$, $I_{2}$, and $I_{3}$, gives

$$d_{\textrm{AD}}(P_{N},G)=\sum_{i=1}^{N-1}P_{N}^{2}(Y_{(i)})\log\frac{U_{(i+1)}\left(1-U_{(i)}\right)}{U_{(i)}\left(1-U_{(i+1)}\right)}+\sum_{i=1}^{N-1}\left(2P_{N}(Y_{(i)})-1\right)\log\frac{1-U_{(i+1)}}{1-U_{(i)}}$$

$${}-1-\log\left(U_{(N)}(1-U_{(1)})\right).$$

(8)

The proof is completed by substituting $P_{N}(Y_{(i)})=\sum_{j=1}^{i}J^{\prime}_{j}$, $P^{2}_{N}(Y_{(i)})=\sum_{j=1}^{i}J^{\prime^{2}}_{j}+2\sum_{j=1}^{i-1}\sum_{k=j+1}^{i}J^{\prime}_{k}J^{\prime}_{j}$ and $U_{(i)}=G(Y_{(i)})$ in terms on the right-hand side of (8).

PROOF OF LEMMA 3

To prove (i), note that, from the property of the Dirichlet process, for any $t\in\mathbb{R}$, $E_{P}(P(t)-$ $H(t))^{2}=\frac{H(t)(1-H(t))}{a+1}$. Then

$$E_{P}\left(d_{\textrm{AD}}(P,H)\right)=\int\limits_{-\infty}^{\infty}\frac{E_{P}\left(P(t)-H(t)\right)^{2}}{H(t)\left(1-H(t)\right)}\,dH(t)=\frac{1}{a+1}.$$

To prove (ii), it is enough to compute $E_{P}\left(d_{\textrm{AD}}(P,H)\right)^{2}$. According to the Corollary 2, we consider $H$ to be the cdf of the Uniform distribution on [0]. Then

$$E_{P}\left(d_{\textrm{AD}}(P,H)\right)^{2}=E_{P}\left(\int\limits_{-\infty}^{\infty}\frac{\left(P(t)-H(t)\right)^{2}}{H(t)\left(1-H(t)\right)}\,dH(t)\right)^{2}$$

$${}=E_{P}\left(\int\limits_{0}^{1}\frac{\left(P(t)-t\right)^{2}}{t(1-t)}\,dt\int\limits_{0}^{1}\frac{\left(P(s)-s\right)^{2}}{s(1-s)}\,ds\right)=E_{P}\Bigg{(}\int\limits_{0}^{1}\int\limits_{0}^{t}\frac{\left(P(t)-t\right)^{2}\left(P(s)-s\right)^{2}}{t(1-t)s(1-s)}\,ds\,dt$$

$${}+\int\limits_{0}^{1}\int\limits_{0}^{s}\frac{\left(P(t)-t\right)^{2}\left(P(s)-s\right)^{2}}{t(1-t)s(1-s)}\,dt\,ds\Bigg{)}=2E_{P}\Bigg{(}\int\limits_{0}^{1}\int\limits_{0}^{t}\frac{\left(P(t)-t\right)^{2}\left(P(s)-s\right)^{2}}{t(1-t)s(1-s)}\,ds\,dt\Bigg{)}$$

$${}=2\int\limits_{0}^{1}\int\limits_{0}^{t}\frac{E_{P}\big{\{}\left(P(s)+P\left((s,t]\right)-t\right)^{2}\left(P(s)-s\right)^{2}\big{\}}}{t(1-t)s(1-s)}\,ds\,dt.$$

Note that, from the property of the Dirichlet process, for any $s<t$ and $i,j\in\mathbb{N}$, $E_{P}(P^{i}(s)P^{j}((s,t]))=\frac{\Gamma(a)}{\Gamma(a+i+j)}\frac{\Gamma(as+i)}{\Gamma(as)}\frac{\Gamma\left(a(t-s)+j\right)}{\Gamma(a(t-s))}$ and $E_{P}\left(P^{i}(s)\right)=\prod_{k=0}^{i-1}\frac{as+k}{a+k}$. Then

$$E_{P}\left(d_{\textrm{AD}}(P,H)\right)^{2}=\int\limits_{0}^{1}\int\limits_{0}^{t}\frac{1}{ts(1-t)(1-s)}\bigg{\{}\frac{2as(a(t-s)+1)(as+1)(t-s)}{(a+3)(a+2)(a+1)}$$

$${}+\frac{4as(as+2)(as+1)(t-s)}{(a+3)(a+2)(a+1)}+\frac{2s(as+3)(as+2)(as+1)}{(a+3)(a+2)(a+1)}$$

$${}-\frac{4as(2s+t)(as+1)(t-s)}{(a+2)(a+1)}-\frac{4as^{2}(a(t-s)+1)(t-s)}{(a+2)(a+1)}$$

$${}-\frac{4s(as+2)(as+1)(s+t)}{(a+2)(a+1)}+\frac{2s(s^{2}+4st+t^{2})(as+1)}{a+1}$$

$${}+\dfrac{4as^{2}(s+2t)(t-s)}{a+1}+\frac{2s^{2}(t-s)(a(t-s)+1)}{a+1}-4s^{2}t(t-s)-4s^{2}t(s+t)+2s^{2}t^{2}\bigg{\}}\,ds\,dt.$$

After simplification, we get

$$E_{P}\left(d_{\textrm{AD}}(P,H)\right)^{2}=\int\limits_{0}^{1}\int\limits_{0}^{t}\frac{2\left((a-6)\left((3t-2)s-t\right)-6\right)}{t(s-1)(a+3)(a+2)(a+1)}\,ds\,dt$$

$${}=\int\limits_{0}^{1}\frac{2}{t(a+3)(a+2)(a+1)}\Big{\{}(a-6)(3t-2)t-2i\pi\left((a-6)t-a+3\right)$$

$${}+2\left(a(t-1)-6t+3\right)\log(t-1)\Big{\}}\,dt=\frac{a(2\pi^{2}-15)-6(\pi^{2}-15)}{3(a+3)(a+2)(a+1)}$$

for Re$(t)<1$ or $t\not\in\mathbb{R}$, where Re$(t)$ denotes the real part of $t$ and $i$ is the imaginary unit. Then, the variance of $d_{\textrm{AD}}(P,H)$ is given by

$$\textrm{Var}_{P}(d_{\textrm{AD}}(P,H))=E_{P}\left(d_{\textrm{AD}}(P,H)\right)^{2}-E_{P}^{2}\left(d_{\textrm{AD}}(P,H)\right)$$

$${}=\dfrac{2\left((\pi^{2}-9)a^{2}+(30-2\pi^{2})a-3\pi^{2}+36\right)}{3(a+1)^{2}(a+2)(a+3)}.$$

Hence, the proof is completed.

PROOF OF LEMMA 5

Assume that $r^{\prime}=(R^{\prime}(\mathbf{y}_{1}),\ldots,R^{\prime}(\mathbf{y}_{n}))$ is the observed sample from $P$ where $P\sim DP(a,F_{\textrm{beta}})$. Note that

$$d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})=\int\limits_{-\infty}^{\infty}\frac{\left(P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)\right)^{2}}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}\,dF_{\textrm{beta}}(t)$$

$${}\leq\left(\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\right)^{2}\int\limits_{-\infty}^{\infty}\frac{1}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}dF_{\textrm{beta}}(t)$$

$${}\leq\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\int\limits_{-\infty}^{\infty}\frac{1}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}dF_{\textrm{beta}}(t)$$

$${}\leq\int\limits_{-\infty}^{\infty}\frac{1}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}dF_{\textrm{beta}}(t)\bigg{\{}\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-H_{r^{\prime}}(t)|+\displaystyle{\sup_{t\in\mathbb{R}}}|H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\bigg{\}},$$

where the third inequality holds since $0\leq|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\leq 1$ and the fourth inequality holds by triangle inequality. To prove (i), as $a\rightarrow\infty$, from James [34], ${\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-H_{r^{\prime}}(t)|\xrightarrow{\mathrm{a.s.}}0$ and by the continuous mapping theorem ${\sup_{t\in\mathbb{R}}}|H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\xrightarrow{\mathrm{a.s.}}0$. To prove (ii), since ${\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-H_{r^{\prime}}(t)|\xrightarrow{\mathrm{a.s.}}0$ as $n\rightarrow\infty$ and $\mathcal{H}_{0}$ is true, the continuous mapping theorem and Polya’s theorem [18] imply ${\sup_{t\in\mathbb{R}}}|H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\xrightarrow{\mathrm{a.s.}}0$. Note that, the final results of part (i) and (ii) are concluded by practical assumptions in probability and measure theory given in Section 3.1 of Capiński and Kopp [15].

To prove (iii), note that, $\left(F_{\textrm{beta}}(t)(1-F_{\textrm{beta}}(t))\right)^{-1}\geq 4$, then

$$d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})\geq 4\int\limits_{-\infty}^{\infty}\left(P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)\right)^{2}\,dF_{\textrm{beta}}(t)\geq 4\,d_{CvM}(P_{r^{\prime}},F_{\textrm{beta}}).$$

From Choi and Bulgren [17], since $d_{CvM}(P_{r^{\prime}},F_{\textrm{beta}})\geq\dfrac{1}{3}\left(\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\right)^{3}$,

$$d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})\geq\frac{4}{3}\left(\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\right)^{3}.$$

Using the triangle inequality gives

$$\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\geq\displaystyle{\sup_{t\in\mathbb{R}}}|H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|-\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-H_{(d)}(t)|.$$

Similar to the proof of part (ii), as $n\rightarrow\infty$, ${\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-H_{(d)}(t)|\xrightarrow{\mathrm{a.s.}}0$ and ${\sup_{t\in\mathbb{R}}}|H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\xrightarrow{\mathrm{a.s.}}\displaystyle{\sup_{t\in\mathbb{R}}}|P_{\mathrm{true}}(t)-F_{\textrm{beta}}(t)|$, where $P_{\mathrm{true}}$ is the true distribution of the sample $d$. Since $\mathcal{H}_{0}$ is not true, $\liminf\displaystyle{\sup_{t\in\mathbb{R}}}|P_{r^{\prime}}(t)-F_{\textrm{beta}}(t)|\displaystyle{\overset{\mathrm{a.s.}}{>}}0$, which implies $\liminf d_{\textrm{AD}}(P_{r^{\prime}},$ $F_{\textrm{beta}})\displaystyle{\overset{\mathrm{a.s.}}{>}}0$.

To prove (iv), since for any $t\in\mathbb{R}$, $E_{P_{r^{\prime}}}\left(P_{r^{\prime}}(t)\right)=H_{r^{\prime}}(t)$ and $E_{P_{r^{\prime}}}\left(P_{r^{\prime}}(t)-H_{r^{\prime}}(t)\right)^{2}=\frac{H_{r^{\prime}}(t)\left(1-H_{r^{\prime}}(t)\right)}{a+n+1}$, then, as $n\rightarrow\infty$

$$\liminf E_{P_{r^{\prime}}}\left(d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})\right)\geq\liminf\int\limits_{-\infty}^{\infty}\frac{H_{r^{\prime}}(t)\left(1-H_{r^{\prime}}(t)\right)}{(a+n+1)F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}dF_{\textrm{beta}}(t)$$

$${}+\liminf\int\limits_{-\infty}^{\infty}\frac{\left(H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)\right)^{2}}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}dF_{\textrm{beta}}(t).$$

Applying Fatou’s lemma gives

$$\liminf E_{P_{r^{\prime}}}\left(d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})\right)\geq\int\limits_{-\infty}^{\infty}\liminf\left(\frac{H_{r^{\prime}}(t)\left(1-H_{r^{\prime}}(t)\right)}{(a+n+1)F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}\right)dF_{\textrm{beta}}(t)$$

$${}+\int\limits_{-\infty}^{\infty}\liminf\left(\frac{\left(H_{r^{\prime}}(t)-F_{\textrm{beta}}(t)\right)^{2}}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}\right)dF_{\textrm{beta}}(t)=0+\int\limits_{-\infty}^{\infty}\inf\left(\frac{\left(P_{\mathrm{true}}(t)-F_{\textrm{beta}}(t)\right)^{2}}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}\right)dF_{\textrm{beta}}(t).$$

Since $\mathcal{H}_{0}$ is not true, $\inf\left(\frac{\left(P_{\mathrm{true}}(t)-F_{\textrm{beta}}(t)\right)^{2}}{F_{\textrm{beta}}(t)\left(1-F_{\textrm{beta}}(t)\right)}\right)\displaystyle{\overset{\mathrm{a.s.}}{>}}0$. Hence, by Theorem 15.2 of Billingsley [13], $\liminf E_{P_{r^{\prime}}}\left(d_{\textrm{AD}}(P_{r^{\prime}},F_{\textrm{beta}})\right)\displaystyle{\overset{\mathrm{a.s.}}{>}}0$.