Flexible pair-copula estimation in D-vines using bivariate penalized splines

Abstract

The paper presents a new method for flexible fitting of D-vines. Pair-copulas are estimated semi-parametrically using penalized Bernstein polynomials or constant and linear B-splines, respectively, as spline bases in each knot of the D-vine throughout each level. A penalty induce smoothness of the fit while the high dimensional spline basis guarantees flexibility. To ensure uniform univariate margins of each pair-copula, linear constraints are placed on the spline coefficients and quadratic programming is used to fit the model. The amount of penalizations for each pair-copula is driven by a penalty parameter which is selected in a numerically efficient way. Simulations and practical examples accompany the presentation.

Appendices

Appendix A: Quadratic programming

To estimate the pair-copula we make use of the quadprog package in R which allows to solve the quadratic program. We set v=v ^(i,j|D) and λ=λ ^(i,j|D) in this section for simplicity. Let therefore ${\mathbf{s}}^{p}_{ij|D}(\boldsymbol{v},\lambda)$ and ${\mathbf{H}}^{p}_{ij|D}(\boldsymbol{v},\lambda)$ denote the first and second order derivatives of (16) yielding

$$\begin{aligned} &{{\mathbf{s}}^{p}_{ij|D}( \boldsymbol{v},\lambda)=\sum_{t=1}^{T} \frac {\boldsymbol{\phi}_K(\hat{u}_{it|D})\otimes\boldsymbol{\phi}_K(\hat {u}_{jt|D})}{\tilde{c}_{ij|D}(\hat{u}_{it|D},\hat {u}_{jt|D},{\boldsymbol{v}})} - \lambda\mathbf{P}\boldsymbol{v}.} \end{aligned}$$

(22)

$$\begin{aligned} &{{\mathbf{H}}^p_{ij|D}( \boldsymbol{v},\lambda)} \\ &{\quad=-\sum_{t=1}^{T} \frac{(\boldsymbol{\phi}_K(\hat{u}_{it|D})\otimes \boldsymbol{\phi}_K(\hat{u}_{jt|D}))(\boldsymbol{\phi}_K(\hat {u}_{it|D})\otimes\boldsymbol{\phi}_K(\hat{u}_{jt|D}))^T}{\tilde {c}_{ij|D}(\hat{u}_{it|D},\hat{u}_{jt|D},{\boldsymbol{v}})}} \\ &{\qquad{} - \lambda \mathbf{P}.} \end{aligned}$$

(23)

We approximate the penalized likelihood $l^{p}_{ij|D}$ in (16) through a second order Taylor expansion yielding

$$\begin{aligned} l^p_{ij|D} \bigl(\mathbf{v} + \boldsymbol{ \delta}^{(ij|D)}, \lambda \bigr) &\approx l^p_{ij|D} ( \mathbf{v}, \lambda ) \boldsymbol{\delta }^{(ij|D)^T} \mathbf{s}^p_{ij|D} (\mathbf{v}, \lambda ) \\ &\quad{}+ \frac{1}{2} \boldsymbol{\delta}^{(ij|D)^T} \mathbf{H}^p_{ij|D} (\mathbf{v}, \lambda ) \boldsymbol{\delta}^{(ij|D)}, \end{aligned}$$

(24)

where δ ^(ij|D) is the iteration step selected by maximizing (24) subject to the linear constraints (8), (9) and (12). This optimization is carried out iteratively, by approximating the likelihood as in (24) in each iteration step. To start the algorithm an admissible starting value for v ^(i,j|D) is required. We use a uniform distribution on the cube [0,1]² which defines the starting value in unique way.

Appendix B: Penalty matrix for penalizing second order derivatives

We set v=v ^(i,j|D) and λ=λ ^(i,j|D) in this section for simplicity. For the marginal penalties in u _i and u _j in (15) follows with (7) and transformations

$$\begin{aligned} &{\int \biggl(\frac{\partial^2 \tilde{c}_{ij|D}(u_i,u_j)}{(\partial u_i)^2} \biggr)^2 \mathrm{d}u_i\mathrm{d}u_j} \\ &{\quad=\boldsymbol{v}^T \int \biggl[\frac{\partial^2 \phi _{K}(u_{i|D})}{(\partial u_i)^2} \otimes\phi_{K}(u_{j|D}) \biggr]^T \biggl[ \frac{\partial^2\phi_{K}(u_{i|D})}{(\partial u_i)^2}\otimes\phi _{K}(u_{j|D}) \biggr] \mathrm{d}u_i \mathrm{d}u_j \boldsymbol{v}} \\ &{\quad=\boldsymbol{v}^T \underbrace{\int{ \biggl[ \biggl( \frac{\partial^2 \phi _{K}(u_{i|D})}{(\partial u_i)^2} \biggr)^T \frac{\partial^2\phi _{K}(u_{i|D})}{(\partial u_i)^2} \biggr] \mathrm{d}u_i \otimes \bigl[\bigl(\phi _{K}(u_{j|D}) \bigr)^T \phi_{K}(u_{j|D}) \bigr]}}_{:=P_{u_i}} \boldsymbol {v}.} \end{aligned}$$

The integral of the second order derivatives of Bernstein polynomials are calculated easily. The second order derivative of (10) equals (see Doha et al. 2011)

$$\frac{\partial^2 \phi_{Kk}(u)}{(\partial u)^2}=\frac{(K+1)!}{(K-2)!}\sum_{m=\max(0,k+2-K)}^{\min(k,2)}(-1)^{m+2} \binom{2}{m}\phi_{K-2,k-m}(u). $$

This is rewritten as

$$\frac{\partial^2 \phi_{Kk}(u)}{(\partial u)^2}=\bigl(\phi_{K-2,k}(u) B\bigr) w $$

with

$$B= \left (\begin{array}{c@{\quad}c@{\quad}c@{\quad}c@{\quad}c@{\quad}c} 1 & -2 & 1 & 0 &\cdots& 0\\ 0 & 1 &-2 & 1 &\ddots& \vdots\\ \vdots& \ddots& \ddots&\ddots& \ddots& 0 \\ 0 & \cdots&0 &1&-2&1 \end{array} \right ) ,\quad B \in \mathbb{R}^{(K-2)\times(K+1)} $$

and $w=\frac{(K+1)!}{(K-2)!}$. Therefore, the matrix $P_{z_{i}}$ and $P_{z_{j}}$ are equivalent to

$$\begin{aligned} &{P_{u_i}=\biggl(wB^T\int\phi_{K-2,k}(u_{i|D}) \phi_{K-2,k}(u_{i|D})\mathrm{d}u_i Bw\biggr)} \\ &{\phantom{P_{u_i}=}{}\otimes \bigl[\bigl(\phi_{K}(u_{j|D})\bigr)^T \phi_{K}(u_{j|D}) \bigr]} \\ &{P_{u_j}= \bigl[\bigl(\phi_{K}(u_{i|D}) \bigr)^T \phi_{K}(u_{i|D}) \bigr]} \\ &{\phantom{P_{u_j}=}{}\otimes \biggl(wB^T\int\phi_{K-2,k}(u_{j|D}) \phi_{K-2,k}(u_{j|D})\mathrm{d}u_j Bw\biggr).} \end{aligned}$$

So, the penalty can be written as quadratic form λ v ^T P _int v where λ is the penalty parameter steering the amount of smoothness and $P_{int}:=P_{u_{i}}+P_{u_{j}}$.

Appendix C: Marginal likelihood

The prior (18) is degenerated, which needs to be corrected as follows. We decompose v ^(i,j|D) into the two components ${\boldsymbol{v}}^{(i,j|D)}_{\sim}$ and ${\boldsymbol {v}}^{(i,j|D)}_{\bot}$, respectively, such that ${\boldsymbol {v}}^{(i,j|D)}_{\sim}$ is a normally distributed random vector with non degenerated variance and ${\boldsymbol{v}}^{(i,j|D)}_{\bot}$ are the remaining components treated as parameters, see also Wand and Ormerod (2008). In fact based on a singular value decomposition we have

$$\mathbf{P}=\tilde{\mathbf{U}} \tilde{\boldsymbol{\varLambda}} \tilde{ \mathbf{U}}^T $$

with $\tilde{\boldsymbol{\varLambda}}$ as diagonal matrix with positive eigenvalues and $\tilde{\mathbf{U}} \in\mathbb{R}^{(K+1) \times h}$ with corresponding eigenvectors where K+1 is the number of elements in v ^(i,j|D) and h=K+1−4 is the rank of P. Extending $\tilde{\mathbf{U}}$ to an orthogonal basis by $\check{\mathbf{U}}$ gives ${\boldsymbol{v}^{(i,j|D)}_{\sim}}= \tilde{\mathbf{U}}^{T}\boldsymbol{v}^{(i,j|D)}$ with the a priori assumption ${\boldsymbol{v}^{(i,j|D)}_{\sim}} \sim N(0, \lambda^{-1}\tilde{\boldsymbol{\varLambda}}^{-1})$ and with $\mathbf{U}=(\tilde{\mathbf{U}}, \check{\mathbf{U}})$ as orthogonal basis, we get $\boldsymbol {v}^{(i,j|D)}_{\bot}={\check{\mathbf{U}}}^{T} \boldsymbol{v}^{(i,j|D)}$. Conditioning on ${\boldsymbol{v}^{(i,j|D)}_{\sim}}$, we have x being distributed according to (6) and with (18) we get the mixed model log likelihood

$$\begin{aligned} &{ l^m_{ij|D}\bigl(\lambda, { \boldsymbol{v}^{(i,j|D)}}^{\bot}\bigr)} \\ &{\quad=\log\int |\lambda\tilde{ \boldsymbol{\varLambda}}|^{\frac{1}{2}} \exp \bigl\{ l^p_{ij|D} \bigl(\boldsymbol{v}^{(i,j|D)}, \lambda\bigr) \bigr\} \mathrm {d} { \boldsymbol{v}^{(i,j|D)}_{\sim}}.} \end{aligned}$$

(25)

The integral can be approximated by a Laplace approximation (see also Rue et al. 2009)

$$\begin{aligned} l^m_{ij|D}\bigl(\lambda,\hat{ \boldsymbol{v}}^{(i,j|D)_{\bot}} \bigr) \approx&\frac{1}{2} \log|\lambda\tilde{ \boldsymbol{\varLambda}}|+ l^p_{ij|D}\bigl({\hat { \boldsymbol{v}}^{(i,j|D)}}, \lambda\bigr) \\ &{}- \frac{1}{2} \log\bigl \vert \tilde{ \mathbf{U}}^T H^p_{ij|D}\bigl({\hat{ \boldsymbol{v}}^{(i,j|D)}},\lambda\bigr) \tilde{\mathbf{U}}\bigr \vert \end{aligned}$$

(26)

where $\hat{\boldsymbol{v}}^{(i,j|D)}$ denotes the penalized maximum likelihood estimate. We can now differentiate (26) with respect to λ which gives

$$\begin{aligned} \frac{\partial l^m_{ij|D}(\lambda,{\hat{\boldsymbol{v}}^{(i,j|D)}_{\bot }})}{\partial\lambda} &= - \frac{1}{2} {\hat{\boldsymbol{v}}^{(i,j|D)^T}} P \hat{ \boldsymbol{v}}^{(i,j|D)} \\ &\quad{}+ \frac{1}{2 \lambda} \operatorname{tr} \underbrace{ \bigl\{ \bigl( \tilde{\mathbf{U}}^T \mathbf{H}^p_{ij|D}\bigl({ \hat{\boldsymbol{v}}}^{(i,j|D)},\lambda=0\bigr) \tilde{\mathbf{U}} + \lambda \tilde{\boldsymbol{\varLambda}}\bigr)^{-1} \tilde{\mathbf{U}}^T \mathbf{H}^p_{ij|D}\bigl(\hat{ \boldsymbol{v}}^{(i,j|D)},\lambda =0\bigr) \tilde{\mathbf{U}} \bigr \}}_{:=S(\lambda)}. \end{aligned}$$

(27)

Appendix D: Computational aspects

Solving the travelling salesman problem (TSP) in step 2 of Algorithm 1 is a np-hard problem (see Applegate 2006). But the TSP for e.g. p=10 dimensions, faced with the cAIC of $\binom{10}{2}=45$ pair-copulas, is rapidly solved in R using the R-package TSP (see Hahsler and Hornik 2007). Increasing the dimensionality to p=36, say, the TSP is still rapidly solved, while the R package TSP uses the ‘nearest neighbor and repetitive nearest neighbor algorithms for symmetric and asymmetric TSPs’ (see Rosenkrantz et al. 1977). Calculating the $\binom{p}{2}$ pair-copulas in advanced to the TSP is done rapidly using parallel computing, while each calculation of a pair-copula takes only some seconds. Therefore, the calculation of high-dimensional D-vines is done in a short period of time. Table 8 presents elapsed system.time in R for bivariate, four-dimensional and ten-dimensional data with N=100 and N=500 from a Frank copula with τ=0.5 and K=14 on Intel^® Core^™ 2 Quad CPU Q9550 @ 2.83 GHz.

Table 8 Elapsed system.time for estimation of data from a Frank copula with τ=0.5 and K=14 on Intel^® Core^™ 2 Quad CPU Q9550 @ 2.83 GHz

When calculating the optimal λ of a pair-copula, we are faced with the classical problems of the estimation based on Fisher Scoring. Thus, the initial choice of a penalty λ ₀ influences the number of iterations to determine the optimal penalty λ.