Estimating non-simplified vine copulas using penalized splines

Abstract

Vine copulas (or pair-copula constructions) have become an important tool for high-dimensional dependence modeling. Typically, so-called simplified vine copula models are estimated where bivariate conditional copulas are approximated by bivariate unconditional copulas. We present the first nonparametric estimator of a non-simplified vine copula that allows for varying conditional copulas using penalized hierarchical B-splines. Throughout the vine copula, we test for the simplifying assumption in each edge, establishing a data-driven non-simplified vine copula estimator. To overcome the curse of dimensionality, we approximate conditional copulas with more than one conditioning argument by a conditional copula with the first principal component as conditioning argument. An extensive simulation study is conducted, showing a substantial improvement in the out-of-sample Kullback–Leibler divergence if the null hypothesis of a simplified vine copula can be rejected. We apply our method to the famous uranium data and present a classification of an eye state data set, demonstrating the potential benefit that can be achieved when conditional copulas are modeled.

Appendices

Appendix 1: Construction of sparse B-spline density basis

In the following, objects signed with superscript $\tilde{\ }$ and $^{(d)}$ are associated with hierarchical B-spline basis functions, whereas objects signed with superscript $\tilde{\ }$ and $^{(d,D)}$ are linked to sparse B-spline basis functions. In order to transform $B^{(\tau (d))}(\mathbf {u}_j)$ in (6) into its hierarchical representation (see Forsey and Bartels 1988, 1995), we define the hierarchical index sets $\mathcal {I}_0=\{1, 2\}$ and $\mathcal {I}_l = \{2j\mid j\in \mathbb {N},1\le j \le 2^{l-1}\}$, $l=1,\dots ,d$. Let $B^{(\tau (l))}_{\mathcal {I}_l}(\mathbf {u}_j)$ denote the columns $\mathcal {I}_l$ of $B^{\tau (l)}(\mathbf {u}_j)$.⁸

Fig. 2

a B-spline density basis $B^{(\tau (2))}(\mathbf {u}_j)$ and corresponding building blocks of the univariate hierarchical B-spline density basis $B^{(\tau (0))}_{\mathcal {I}_0}(\mathbf {u}_j), B^{(\tau (1))}_{\mathcal {I}_1}(\mathbf {u}_j)$ and $B^{(\tau (2))}_{\mathcal {I}_2}(\mathbf {u}_j)$ as graphics (b), (c) and (d)

The univariate hierarchical B-spline basis of degree d is defined as

$$\begin{aligned} \tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j) =\left( {B}^{(\tau (0))}_{\mathcal {I}_0}(\mathbf {u}_j), {{B}^{(\tau (1))}_{\mathcal {I}_1}(\mathbf {u}_j)}, \dots , \ {B}^{(\tau (d))}_{\mathcal {I}_d}(\mathbf {u}_j)\right) . \end{aligned}$$

(22)

Figure 2 presents the univariate B-spline basis $B^{(\tau (d))}(\mathbf {u}_j)$ and the building parts of the corresponding hierarchical basis $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$ for $d=2$. Let $\tilde{e}$ be a K-dimensional row vector such that its kth element is given by $\tilde{e}_k:= \min \{l=0,\dots ,d: k\le |\tau (l)|\}$, e.g., $\tilde{e}= (0,0,1,2,2)$ if $d=2$. The vector $\tilde{e}$ denotes the hierarchical level of $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$, and its kth element identifies the hierarchical level of the kth column of $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$. By construction, $B^{(\tau (d))}(\mathbf {u}_j)$ and $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$ have full rank, i.e., $B^{(\tau (d))}(\mathbf {u}_j)\tilde{A}=\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$ for some invertible $K \times K$ matrix $\tilde{A}$, so that both univariate bases $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$ and $B^{(\tau (d))}(\mathbf {u}_j)$ span the same space. The three-dimensional hierarchical B-spline basis follows as

$$\begin{aligned} \tilde{\varvec{\varPhi }}^{(d)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3) :=\bigotimes _{j=1}^3 \tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j) = \bigotimes _{j=1}^3 B^{(\tau (d))}(\mathbf {u}_j) \tilde{A} \end{aligned}$$

(23)

and the corresponding approximation of the conditional copula density is given by

$$\begin{aligned} c_{12|3} (\mathbf {u}_1,\mathbf {u}_2|\mathbf {u}_3; \tilde{\mathbf {b}}^{(d)})=\tilde{\varvec{\varPhi }}^{(d)}(\mathbf {u}_1, \mathbf {u}_2,\mathbf {u}_3)\tilde{\mathbf {b}}^{(d)}, \end{aligned}$$

where $\tilde{\mathbf {b}}^{(d)} = (\bigotimes _{j=1}^3\tilde{A})^{-1}\mathbf {b}$.

Fig. 3

${\tilde{\varvec{\Phi }}}^{(2,4)}(u_1,u_2)$ is the full tensor product of two univariate B-spline bases, each margin consists of ${B}^{(\tau (0))}_{\mathcal {I}_0}(\mathbf {u}_j)$, ${B}^{(\tau (1))}_{\mathcal {I}_1}(\mathbf {u}_j)$ and ${B}^{(\tau (2))}_{\mathcal {I}_2}(\mathbf {u}_j)$ for $j=1,2$ The construction principle of the sparse B-spline basis ${\tilde{\varvec{\Phi }}}^{(2,2)}(u_1,u_2)$ is to remove columns from the full tensor product, reducing the number of spline bases from 25 to 17 in this bivariate example for $d=2$

Fig. 4

(left) Full tensor product ${\mathbf {\Phi _5}}(u_1,u_2)$ consists of $5^2=25$ basis functions located at each dot. (right) ${\tilde{\varvec{\Phi }}}^{(2,2)}(u_1,u_2)$ consists of 17 basis functions located at each dot

To overcome the exponential increase in the number of spline coefficients, we use a three-dimensional sparse B-spline basis which reduces the dimension by deleting the columns from the full tensor product basis whose cumulated hierarchy level exceeds D, where $d\le D\le 3d$. The cumulated hierarchy level of the full tensor product basis is defined as follows. For $\alpha \in \mathbb {R}^{1\times n}$ and $\beta \in \mathbb {R}^{1\times q}$, define the lth element of $(\alpha \oplus \beta )\in \mathbb {R}^{1\times nq}$ by $(\alpha \oplus \beta )_{l} = \alpha _{\left\lceil \frac{l}{q}\right\rceil }+ \beta _{l-q(\left\lceil \frac{l}{q}\right\rceil -1)}$, where $\lceil \cdot \rceil$ is the ceil function. Note that the operation $\oplus$ is associative. Recall that the kth element of $\tilde{e}$ identifies the hierarchy level of the kth column of $\tilde{\mathbf {B}}^{(\tau (d))}(\mathbf {u}_j)$, so that the lth element of $\epsilon = \tilde{e}\oplus \tilde{e}\oplus \tilde{e}\in \mathbb {R}^{1\times {K}^3}$ denotes the cumulated hierarchy level of the lth column of the hierarchical B-spline basis $\tilde{\varvec{\mathbf {\Phi }}}^{(d)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3)$. Define $\mathcal{O}_D = \{j=1,\ldots , {K_{}}^3 :\epsilon _j\le D \}$, i.e., $\mathcal{O}_D$ contains the position of the columns of $\tilde{\varvec{\mathbf {\Phi }}}^{(d)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3)$ whose cumulated hierarchy level does not exceed D. Let $\mathcal{O}_D(j)$ be the jth smallest element of $\mathcal{O}_D$ and define the orthogonal matrix $\mathcal {E}(\mathcal {O}_D)\in \mathbb {R}^{{K}^3\times |\mathcal{O}_D|}$ such that its $(\mathcal {O}_D(j),j)$-entry is one for $j=1,\dots ,|{\mathcal{O}}_D|,$ and the other entries are zero. The three-dimensional sparse B-spline basis follows as

$$\begin{aligned} \tilde{\varvec{{\Phi }}}^{(d,D)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3) = \left[ \bigotimes _{j=1}^3 \tilde{\mathbf {B}}^{(\tau (d))} (\mathbf {u}_j)\right] \mathcal {E}(\mathcal{O}_D), \end{aligned}$$

(24)

Fig. 5

Linear and quadratic functions of Kendell’s $\tau$ for the construction of the conditional copula in higher trees

Fig. 6

Box plots of the differences $KL_{non-par}-KL_{par}$ for three-dimensional non-simplified vine copulas. $KL_{non-par}$ is the out-of-sample KL divergence of a nonparametric estimation (“SimpA,” “Cond” or “Test”), and $KL_{par}$ is the out-of-sample KL divergence of the parametric estimation using the VineCopula package. Blue refers to the estimator “Test,” pink corresponds to the estimator “Cond,” and green refers to the estimator “SimpA.” The whiskers cover 95% of the data

where the lower index d is the degree of the univariate hierarchical B-spline basis and the upper index $D, d\le D\le 3d,$ refers to the maximum cumulated hierarchy level. Only the columns in the hierarchical B-spline basis ${\tilde{\varvec{\Phi }}}^{(d)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3)$ whose cumulated hierarchy level does not exceed D constitute the three-dimensional sparse B-spline basis $\tilde{\varvec{{\Phi }}}^{(d,D)}(\mathbf {u}_1,\mathbf {u}_2,\mathbf {u}_3)$. Figure 3 shows the construction principle for a bivariate sparse B-spline basis with hierarchy level $d=2$. Figure 4 presents the placements of the knots for the full tensor product of B-splines and the sparse B-spline basis for $d=2$. The corresponding spline coefficients are given by $\tilde{\mathbf {b}}^{(d,D)} = \mathcal{E(O_D)}^\top \tilde{\mathbf {b}}^{(d)}$.

Fig. 7

Box plots of the differences $KL_{non-par}-KL_{par}$ for five-dimensional non-simplified vine copulas. $KL_{non-par}$ is the out-of-sample KL divergence of a nonparametric estimation (“SimpA,” “Cond” or “Test”), and $KL_{par}$ is the out-of-sample KL divergence of the parametric estimation using the VineCopula package. Blue refers to the estimator “Test,” pink corresponds to the estimator “Cond,” and green refers to the estimator “SimpA.” The whiskers cover 95% of the data

Fig. 8

Box plots of the differences $KL_{non-par}-KL_{par}$ for three- and five-dimensional mixtures of normal distributions. $KL_{non-par}$ is the out-of-sample KL divergence of a nonparametric estimation (“SimpA,” “Cond” or “Test”), and $KL_{par}$ is the out-of-sample KL divergence of the parametric estimation using the VineCopula package. Blue refers to the estimator “Test,” pink corresponds to the estimator “Cond,” and green refers to the estimator “SimpA.” The whiskers cover 95% of the data

Fig. 9

Conditional copula densities of the three-dimensional mixture of normal distributions for conditional arguments $0.01, 0.15, 0.29, 0.43, 0.57, 0.71, 0.85, 0.99$ (top left to bottom right)

Appendix 2: Marginal likelihood

The prior (13) is degenerated, which needs to be corrected as follows. For simplicity, we write $\mathbf {b}:= \tilde{\mathbf {b}^{(d,D)}}$ in this section. We decompose $\mathbf {b}$ into the two components $\mathbf {b}_{\sim }$ and $\mathbf {b}_{\bot }$, respectively, such that $\mathbf {b}_{\sim }$ is a normally distributed random vector with non-degenerated variance and $\mathbf {b}_{\bot }$ contains the remaining components treated as parameters, see also Wand and Ormerod (2008). Applying a singular value decomposition, we have

$$\begin{aligned} \tilde{\mathbf {P}^{(D)}}=\tilde{\mathbf {U}} \tilde{\mathbf {\Lambda }} \tilde{\mathbf {U}}^T \end{aligned}$$

where $\tilde{\mathbf {\Lambda }}$ is a diagonal matrix with positive eigenvalues and $\tilde{\mathbf {U}} \in \mathbb {R}^{(K+1) \times h}$ are the corresponding eigenvectors with $K+1$ being the number of elements in $\mathbf {b}$ and $h=K+1-4$ being the rank of $\tilde{\mathbf {P}^{(D)}}$. Extending $\tilde{\mathbf {U}}$ to an orthogonal basis by $\check{\mathbf {U}}$ gives $\mathbf {b}_{\sim }=\tilde{\mathbf {U}}^T\mathbf {b}$. With the a priori assumption $\mathbf {b}_{\sim } \sim N(0, \lambda ^{-1}\tilde{\mathbf {\Lambda }}^{-1})$ and $\mathbf {U}=(\tilde{\mathbf {U}}, \check{\mathbf {U}})$ as orthogonal basis, we get $\mathbf {b}_{\bot }={\check{\mathbf {U}}}^T \mathbf {b}$. Conditioning on $\mathbf {b}_{\sim }$, we get the mixed model log-likelihood

$$\begin{aligned} l_m(\lambda , \mathbf {b}^{\bot })=\log \int |\lambda \tilde{\mathbf {\Lambda }}|^\frac{1}{2} \exp \left\{ l^{(D)}_p(\mathbf {b}, \lambda )\right\} \text{ d }{\mathbf {b}_{\sim }}. \end{aligned}$$

The integral can be approximated by a Laplace approximation (see also Rue et al. 2009)

$$\begin{aligned} l_m(\lambda ,\mathbf {b}^{\bot })\approx & {} \frac{1}{2} \log |\lambda \tilde{\mathbf {\Lambda }}|+ l_p^{(D)} (\hat{\mathbf {b}}, \lambda )\nonumber \\&-\frac{1}{2} \log |\tilde{\mathbf {U}}^T {\mathbf H}_{pen}^{(d,D)} (\hat{\mathbf {b}},\lambda ) \tilde{\mathbf {U}}| \end{aligned}$$

(25)

where $\hat{\mathbf {b}}$ denotes the penalized maximum likelihood estimate. We can now differentiate (25) with respect to $\lambda$ which gives

$$\begin{aligned}&\frac{\partial l_m(\lambda ,\hat{\mathbf {b}}^{\bot })}{\partial \lambda } = - \frac{1}{2} \hat{\mathbf {b}}^T {\tilde{\mathbf {P}}}^{(D)} (\mathbf \lambda ) \hat{\mathbf {b}}^{\bot } + \frac{1}{2 \lambda } \nonumber \\&\quad \cdot \text{ tr } \underbrace{\left\{ \left( \tilde{\mathbf {U}}^\top \mathbf {H}_{pen}^{(d,D)}(\hat{\mathbf {b}},\lambda =0) \tilde{\mathbf {U}} + \lambda \tilde{\mathbf {\Lambda }}\right) ^{-1} \tilde{\mathbf {U}}^\top \mathbf {H}_{pen}^{(d,D)}(\hat{\mathbf {b}},\lambda =0) \tilde{\mathbf {U}} \right\} }_{:=S(\lambda )}.\nonumber \\ \end{aligned}$$

(26)

and ${\mathbf H}_{pen}^{(d,D)}(\hat{\mathbf {b}},\lambda )$ denotes the second-order partial derivative of (10) with respect to $\hat{\mathbf {b}}$, i.e.,

$$\begin{aligned}&{\mathbf {H}}_{pen}^{(d,D)}(\hat{\mathbf {b}},\mathbf {\lambda })=\\&\quad -\sum _{i=1}^{n} \frac{{\tilde{\varvec{\Phi }}}^{(d,D)} (u_{1,i},u_{2,i},u_{3,i}) {\varvec{\tilde{\varPhi }}^{(d,D)}}^\top (u_{1,i},u_{2,i},u_{3,i})}{c_{12|3}(u_{1,i},u_{2,i}|u_{3,i};\hat{\mathbf {b}})}\\&\quad - \tilde{\mathbf {{{P}}}}^{(D)}(\mathbf \lambda ). \end{aligned}$$

Fig. 10

ROC curve for classification of EEG Eye State Data Set

Table 3 Mean elapsed computing time in seconds (standard deviation) for (i) bivariate unconditional copula densities, (ii) bivariate conditional copula densities with one conditioning argument and (iii) the estimation of the whole simplified and non-simplified vine copulas with “Test” estimator (measured for $N=10$ runs)

Appendix 3: Computing time

We present some computing times, defined as elapsed time on the system, measured by R on a machine with an Intel Core i7-2600 CPU @3.40 Ghz $\times$ 4 using R.3.3.1 on Linux Mint 17.2 (64-bit). To this end, we take the first ten data sets of the three-dimensional non-simplified vine copulas and the five-dimensional non-simplified vine copulas from our simulation study, each for case (b) with $\beta =0.6$ for both sample sizes $n=500$ and $n=2000$. Computing time is measured for (i) bivariate unconditional copula densities $c_{12}$ for the first two marginal arguments from the selected data, (ii) bivariate conditional copula densities $c_{12|3}$ with one conditioning argument and (iii) the estimation of the non-simplified vine copula. The computing times are realized with one starting value of $\lambda$. Choosing three starting values for each copula estimation as in the simulation study increases the computing time linearly. See Table 3 for the results. The five-dimensional non-simplified vine copulas are computed in approximately 3–7 min, depending on the basis size and sample size, while the computing time for a three-dimensional non-simplified vine copula is less than 1 min. It might seem counterintuitive that the computing time decreases if the sample size increases from $n=500$ to $n=2000$ observations. That is because less iteration steps in the quadratic programming are required.