Flexible asymmetric multivariate distributions based on two-piece univariate distributions

Abstract

Classical symmetric distributions like the Gaussian are widely used. However, in reality data often display a lack of symmetry. Multiple distributions, grouped under the name “skewed distributions”, have been developed to specifically cope with asymmetric data. In this paper, we present a broad family of flexible multivariate skewed distributions for which statistical inference is a feasible task. The studied family of multivariate skewed distributions is derived by taking affine combinations of independent univariate distributions. These are members of a flexible family of univariate asymmetric distributions and are an important basis for achieving statistical inference. Besides basic properties of the proposed distributions, also statistical inference based on a maximum likelihood approach is presented. We show that under mild conditions, weak consistency and asymptotic normality of the maximum likelihood estimators hold. These results are supported by a simulation study confirming the developed theoretical results, and some data examples to illustrate practical applicability.

Appendix: Proofs of Propositions 2, 3 and 4, and of Theorem 1

Proof of Proposition 2

Suppose that $f({\mathbf{x}};{\varvec{\theta}})=f({\mathbf{x}};{\varvec{\theta}}^*)$ and that we know ${\mathbf{Z}}$ up to its parameters (e.g., $Z_1$ is of a QBA-logistic type etc.). We first prove that ${\varvec{\mu}}_a$ is identifiable. By construction, $f_{Z_j}$, $j=1,\dots ,d$ is unimodal with mode 0. Together with (5), this implies

$$\begin{aligned} \forall {\mathbf{A}},{\mathbf{A}}^*\in {\mathbb{R}}^{d\times d} \, {\text{non-singular}}{:}\,\, \underset{{\mathbf{x}}\in {\mathbb{R}}^d}{\text{arg}}\, {\text{max}} f({\mathbf{x}};{\varvec{\theta}}) = \underset{{\mathbf{x}}\in {\mathbb{R}}^d}{{\text{arg}}\, {\text{max}}} f({\mathbf{x}};{\varvec{\theta}}^*) = {\varvec{\mu}}_{a} . \end{aligned}$$

Thus, ${\varvec{\mu}}_{a} = {\varvec{\mu}}_{a}^*$ and $\left|\det ({\mathbf{A}})\right|=\left|\det ({\mathbf{A}}^*)\right|$. Hence ${\varvec{\mu}}_a$ is identifiable. Without loss of generality, we can assume that for the remainder of the proof, ${\varvec{\mu}}_a={\mathbf{0}}$.

The identifiability result we are aiming at is commonly referred to as uniqueness in the ICA-literature. In Eriksson and Koivunen (2004), necessary and sufficient conditions are provided for a noiseless ICA model (${\mathbf{X}}={\mathbf{AZ}}$) to be unique. These are

• There are no Gaussian sources. Or,

• If ${\mathbf{A}}$ has full column rank, there is at most one Gaussian source.

Since ${\mathbf{A}}\in {\mathbb{R}}^{d\times d}$ is non-singular, it has full column rank. If condition $( I1 )$ holds, the mixing matrix ${\mathbf{A}}$ is unique, i.e., identifiable up to a possible permutation and rescaling together with the accompanying permutation and rescaling of ${\mathbf{Z}}$. A location difference is not possible as ${\mathbf{Z}}$ does not contain a location parameter.

For, the scale ambiguity note that by (3)

$$\begin{aligned} \exists j=1,\dots ,d: \alpha _j^*=1-\alpha _j \,{\text{and}}\, ({\mathbf{A}}^*)^{-1}_{\cdot ,j}=-({\mathbf{A}}^{-1})_{\cdot ,j} \Rightarrow f({\mathbf{x}};{\varvec{\theta}})=f({\mathbf{x}};{\varvec{\theta}}^*) . \end{aligned}$$

By restricting the sign of a single element of $({\mathbf{A}}^{-1})_{\cdot ,j}$ as in $( I2 )$, this problem can no longer occur. By

$$\begin{aligned} ({\widetilde{\mathbf{I}}}_j{\mathbf{A}})^{-1}={\mathbf{A}}^{-1}({\widetilde{\mathbf{I}}}_j)^{-1}={\mathbf{A}}^{-1}{\widetilde{\mathbf{I}}}_j, \end{aligned}$$

with ${\widetilde{\mathbf{I}}}_j\in {\mathbb{R}}^{d\times d}$ the identity matrix with $-1$ at $(\widetilde{I}_j)_{j,j}$, fixing the signs of the diagonal elements of ${\mathbf{A}}$ also suffices.

Since each of the $Z_j$’s lacks a scaling parameter and none of the other parameters of $Z_j$ affects the scaling in a linear way (otherwise it is considered a scaling parameter), any rescaling of ${\mathbf{A}}$ cannot be compensated by rescaling the parameters of ${\mathbf{Z}}$. Hence, ${\mathbf{A}}$ is identifiable up to a permutation. By the identifiability of each of the $Z_j$, also its parameters are uniquely determined up to the same possible permutation. Thus, ${\varvec{\theta}}={\varvec{\theta}}^*$ up to a possible permutation of ${\mathbf{Z}}$ and ${\mathbf{A}}$. Therefore the model is identifiable. □

Proof of Proposition 3

We employ a proof by induction on the dimension of the matrix. For $d=2$ this is trivial as ${\mathbf{A}}$ is invertible and thus has a non-zero determinant. Suppose the statement holds for any invertible $(d-1)\times (d-1)$-matrix. Consider the matrix

$$\begin{aligned} {\mathbf{A}}=\begin{bmatrix} {\mathbf{B}} &\quad {\mathbf{C}} \\ {\mathbf{D}} &\quad E \end{bmatrix} \in {\mathbb{R}}^{d \times d}, \end{aligned}$$

with ${\mathbf{B}}\in {\mathbb{R}}^{(d-1) \times (d-1)}$, ${\mathbf{C}},{\mathbf{D}}^T\in {\mathbb{R}}^{d-1}$ and $E\in {\mathbb{R}}$. Since ${\mathbf{A}}$ is invertible, it must hold that

$$\begin{aligned} {\text{det}}({\mathbf{A}})={\text{det}}({\mathbf{B}})E+\sum _{j=1}^{d-1}(-1)^{d+j} C_j \,{\text{det}}(({\mathbf{B}}^{*})_j) \not =0, \end{aligned}$$

(25)

where $({\mathbf{B}}^{*})_j=\begin{bmatrix} ({\mathbf{B}})_{-j,.} \\ {\mathbf{D}} \end{bmatrix}$, so the $(d-1) \times (d-1)$-matrix where the j-th row of ${\mathbf{B}}$ is omitted and ${\mathbf{D}}$ is added. Now consider the following two cases.

• 1. ${\text{det}}({\mathbf{B}})\not =0$ and $E \not = 0$. By induction, the statement holds for ${\mathbf{A}}$.

• 2. {${\text{det}}({\mathbf{B}})\not =0$ and $E = 0$} or ${\text{det}}({\mathbf{B}})=0$. In this case, by (25), $\exists j \in \{1,\dots ,d-1\}$ such that ${\text{det}}(({\mathbf{B}}^{*})_j)\not =0$ and $C_j\not =0$ . By swapping the j-th row of ${\mathbf{A}}$ with $({\mathbf{D}}, E)$, the resulting matrix falls into case 1. This holds because the element replacing E is nonzero and the new matrix that takes the place of ${\mathbf{B}}$ is invertible as it is a row permutation of $({\mathbf{B}}^{*})_j$, thus conserving the nonzero determinant. Hence, the statement holds.

This concludes the proof as the above two cases contain all possible configurations of ${\mathbf{A}}$. □

Proof of Proposition 4

The proof is largely based on similar arguments concerning the consistency of the maximum likelihood estimator for the univariate quantile-based asymmetric family of distributions: Theorem 3.3 in Gijbels et al. (2019), which in term uses Theorem 2.5 of Newey and McFadden (1994). The latter theorem states that under the following conditions (i) to (iv) the maximum likelihood estimator is weakly consistent, i.e., $\widehat{\varvec{\theta}}_n^{\text{ML}} \overset{P}{\rightarrow }{\varvec{\theta}}_0$ for $n\rightarrow \infty$.

• (i) If ${\varvec{\theta}} \not = {\varvec{\theta}}_0$ then $f_{\mathbf{X}}({\mathbf{x}};{\varvec{\theta}}) \not = f_{\mathbf{X}}({\mathbf{x}};{\varvec{\theta}}_0)$.

• (ii) The true parameter ${\varvec{\theta}}_0 \in {\varvec{\varTheta}}$, with ${\varvec{\varTheta}}$ a parameter space which is compact.

• (iii) The log-likelihood function $\ell \left( {\varvec{\theta}};{\mathbf{x}}\right)$ is continuous at each ${\varvec{\theta}}\in {\varvec{\varTheta}}$.

• (iv) It holds that $E[{\sup }_{{\varvec{\theta}}\in {\varvec{\varTheta}}}\left\Vert \ell \left( {\varvec{\theta}};{\mathbf{X}}\right) \right\Vert ]<\infty$, where $\left\Vert .\right\Vert$ is the Euclidean norm.

Condition (i) is fulfilled by Proposition 2, in which the identifiability of the parameters is guaranteed by assumption $( C1 )$. Conditions (ii) and (iii) follow from respectively Assumption $( C2 )$ and the continuity of both the natural logarithm and $f_{Z_j}$. So only condition (iv) remains to be checked. From (5) and (12), we have that

$$\begin{aligned} E\left[ \left\Vert \ell \left( {\varvec{\theta}};{\mathbf{X}}\right) \right\Vert \right]&= E\left[ \left| -\ln \left|{\text {det}}({\mathbf{A}})\right| + \displaystyle \sum _{j=1}^d \ln f_{Z_j}(({\mathbf{X}} - {\varvec{\mu}}_a)^T({\mathbf{A}}^{-1})_{\cdot ,j};{\varvec{\eta}}_j) \right| \right] \nonumber \\ &\le E\left[ \left|\ln \left|{\text {det}}({\mathbf{A}})\right|\right| + \displaystyle \sum _{j=1}^d \left|f_{Z_j}(({\mathbf{X}}-{\varvec{\mu}}_a)^T({\mathbf{A}}^{-1})_{\cdot ,j};{\varvec{\eta}}_j)\right| \right] \nonumber \\ &= \left|\ln \left|{\text {det}}({\mathbf{A}})\right|\right| + \displaystyle \sum _{j=1}^d E\left[ \left|f_{Z_j}(({\mathbf{X}}-{\varvec{\mu}}_a)^T({\mathbf{A}}^{-1})_{\cdot ,j};{\varvec{\eta}}_j)\right| \right] \nonumber \\ &< \infty , \end{aligned}$$

where boundedness follows from the invertibility of ${\mathbf{A}}$ and Assumption $( C3 )$, as proven in Theorem 3.3 of Gijbels et al. (2019). Since the inequality holds for all ${\varvec{\theta}}\in {\varvec{\varTheta}}_R$, condition (iv) is satisfied and consistency of the maximum likelihood estimator holds. □

Proof of Theorem 1

The proof is largely based on Theorem 3 in Huber (1967), which handles asymptotic normality of maximum likelihood estimators for non-differentiable likelihood functions when consistency has been established.

Since consistency is shown in Proposition 4, only the following four conditions from Huber (1967) need to be fulfilled for the theorem to hold

• (I) For each fixed ${\varvec{\theta}} \in {\varvec{\varTheta}}$, ${\varvec{\varPsi}}({\mathbf{x}};{\varvec{\theta}})$ is $\varOmega$-measurable and ${\varvec{\varPsi}}({\mathbf{x}};{\varvec{\theta}})$ is separable. [See Assumptions A-1 p. 222 of Huber (1967).]

• (II) There exists a ${\varvec{\theta}}_0\in {\varvec{\varTheta}}$ for which ${\varvec{\lambda}}({\varvec{\theta}}_0)={\mathbf{0}}$.

• (III) There are strictly positive numbers a, b, c, $r_0$ such that

  • (i) $\left\Vert {\varvec{\lambda}}({\varvec{\theta}})\right\Vert \ge a\left\Vert {\varvec{\theta}}-{\varvec{\theta}}_0\right\Vert$ for $\left\Vert {\varvec{\theta}}-{\varvec{\theta}}_0\right\Vert \le r_0$.

  • (ii) $E[u({\mathbf{X}};{\varvec{\theta}},r)]\le br$ for $\left\Vert {\varvec{\theta}}-{\varvec{\theta}}_0\right\Vert +r\le r_0, \quad r\ge 0$.

  • (iii) $E[(u({\mathbf{X}};{\varvec{\theta}},r))^2]\le cr$ for $\left\Vert {\varvec{\theta}}-{\varvec{\theta}}_0\right\Vert +r\le r_0, \quad r\ge 0$.

• (IV) The expectation $E[\left\Vert {\varvec{\varPsi}}({\mathbf{X}};{\varvec{\theta}})\right\Vert ^2]$ is finite.

These conditions are checked in a similar way as in the proof of Theorem 3.4 in Gijbels et al. (2019), which is already quite general. We start with condition (I). By Lemma 2${\varvec{\varPsi}}({\mathbf{x}};{\varvec{\theta}})$ is measurable. That ${\varvec{\varPsi}}({\mathbf{x}};{\varvec{\theta}})$ is separable holds under the stated assumptions. Indeed, each of the component functions ${\varvec{\varPsi}}_j({\mathbf{x}};{\varvec{\theta}})$, for $j=1, \dots , d^2+2d$, is separable, and this is a finite number of functions. That each component function is separable follows from its continuity, except on a set with probability measure zero. Condition (II) is met by Proposition 5, whereas for condition (IV) we have by the definition of the Euclidean norm

$$\begin{aligned} E\left[ \left\Vert {\varvec{\varPsi}}({\mathbf{X}};{\varvec{\theta}})\right\Vert ^2\right] =E\left[ {\text {trace}}\left( {\varvec{\varPsi}}({\mathbf{X}};{\varvec{\theta}}) {\varvec{\varPsi}}({\mathbf{X}};{\varvec{\theta}})^T \right) \right] ={\text {trace}}\left( {\mathbf{I}}({\varvec{\theta}}_0) \right) <\infty , \end{aligned}$$

where the finiteness follows from Proposition 6.

Remains to look into condition (III). The key property in this is continuity of ${\varvec{\lambda}}({\varvec{\theta}})$ in a neighborhood of ${\varvec{\theta}}_0$, which holds by Lemma 3. The proof can be completed similarly as in Gijbels et al. (2019). For details, the reader is referred to that paper. □