Variational inference with vine copulas: an efficient approach for Bayesian computer model calibration

Abstract

With the advancements of computer architectures, the use of computational models proliferates to solve complex problems in many scientific applications such as nuclear physics and climate research. However, the potential of such models is often hindered because they tend to be computationally expensive and consequently ill-fitting for uncertainty quantification. Furthermore, they are usually not calibrated with real-time observations. We develop a computationally efficient algorithm based on variational Bayes inference (VBI) for calibration of computer models with Gaussian processes. Unfortunately, the standard fast-to-compute gradient estimates based on subsampling are biased under the calibration framework due to the conditionally dependent data which diminishes the efficiency of VBI. In this work, we adopt a pairwise decomposition of the data likelihood using vine copulas that separate the information on dependence structure in data from their marginal distributions and leads to computationally efficient gradient estimates that are unbiased and thus scalable calibration. We provide an empirical evidence for the computational scalability of our methodology together with average case analysis and describe all the necessary details for an efficient implementation of the proposed algorithm. We also demonstrate the opportunities given by our method for practitioners on a real data example through calibration of the Liquid Drop Model of nuclear binding energies.

Appendices

Appendix A Scalable algorithm with truncate C-vine copulas

Here we present the details of the C-vine based versions of Algorithm 1 and Algorithm 2. First, we can decompose the log-likelihood $\log p({\varvec{d}}|{\varvec{\phi }})$ using a C-vine as

$$\begin{aligned} \log p({\varvec{d}}|{\varvec{\phi }}) = \sum _{j = 1}^{N-1} \sum _{i = 1}^{N-j} f^C_{j, j+i}({\varvec{\phi }}), \end{aligned}$$

(50)

where

$$\begin{aligned} \begin{aligned} f^C_{j, j +i}({\varvec{\phi }})&= \log c_{j,(j+i); 1, \dots , (j-1)} \\&\quad +\frac{1}{N-1}\big (\log p(d_j|{\varvec{\phi }}) + \log p(d_{j+i}|{\varvec{\phi }})\big ). \end{aligned} \end{aligned}$$

(51)

This now yields the following expression for the ELBO gradient:

$$\begin{aligned} \begin{aligned} \nabla _\lambda {\mathcal {L}}(\lambda )&= \sum _{j = 1}^{N-1} \sum _{i = 1}^{N-j}{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})( f^C_{j, j+i}({\varvec{\phi }}))\bigg ] \\&\quad -{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})\log \frac{ q({\varvec{\phi }}|{\varvec{\lambda }})}{p({\varvec{\phi }})}\bigg ]. \end{aligned} \end{aligned}$$

(52)

Equivalently to Proposition 1, we have the following proposition that establishes the noisy unbiased estimate of the gradient (52) using the C-vine copula decomposition.

Proposition 4

Let $\tilde{{\mathcal {L}}}_C({\varvec{\lambda }})$ be an estimate of the ELBO gradient $\nabla _{{\varvec{\lambda }}} {\mathcal {L}}({\varvec{\lambda }})$ defined as

$$\begin{aligned} \begin{aligned} \tilde{{\mathcal {L}}}_C({\varvec{\lambda }})&= \frac{N(N-1)}{2}{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}}\log q({\varvec{\phi }}|{\varvec{\lambda }})( f^C_{I_C(K)}({\varvec{\phi }}))\bigg ] \\&\quad -{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})\log \frac{ q({\varvec{\phi }}|{\varvec{\lambda }})}{ p({\varvec{\phi }})}\bigg ], \end{aligned} \end{aligned}$$

where $K \sim U(1, \dots , \frac{N(N-1)}{2})$, and $I_C$ is the bijection

$$\begin{aligned} \begin{aligned}&I_C:\{1, \dots , \frac{N(N-1)}{2}\} \\&\quad \rightarrow \{(j,j+i): i \in \{1, \dots , N-j\} \text { for } j \in \{1, \dots N-1\}\}, \end{aligned} \end{aligned}$$

then $\tilde{{\mathcal {L}}}_C({\varvec{\lambda }})$ is unbiased i.e., ${\mathbb {E}}(\tilde{{\mathcal {L}}}_C({\varvec{\lambda }})) = \nabla _{{\varvec{\lambda }}} {\mathcal {L}}({\varvec{\lambda }})$.

Again, $\tilde{{\mathcal {L}}}_C({\varvec{\lambda }})$ can be relatively costly to compute for large datasets due to the recursive nature of the copula density computations. We now carry out exactly the same development an using l-truncated C-vine as in the case of Proposition 2 and Proposition 3.

Proposition 5

If the copula of $p({\varvec{d}}|{\varvec{\phi }})$ is distributed according to an l-truncated C-vine, we can rewrite

$$\begin{aligned} \log p({\varvec{d}}|{\varvec{\phi }}) = \sum _{j = 1}^{l} \sum _{i = 1}^{N-j} f^{C_l}_{i, i+j}({\varvec{\phi }}), \end{aligned}$$

(53)

where

$$\begin{aligned} \begin{aligned} f^{C_l}_{i, i+j}({\varvec{\phi }})&= \log c_{j,(j+i); 1, \dots , (j-1)}+ \frac{1}{a_j}\log p(d_j|{\varvec{\phi }}) \\&\quad +\frac{1}{b_{j+i}} \log p(d_{j+i}|{\varvec{\phi }}), \end{aligned} \end{aligned}$$

(54)

and

$$\begin{aligned} a_j&=N-1, \\ b_{j+i}&= (N-1-l)\mathbb {1}_{j + i \le l} + l. \end{aligned}$$

Let us now replace the full log-likelihood $\log ({\varvec{d}}|{\varvec{\phi }})$ in the definition of ELBO with the likelihood based on a truncated vine copula. This yields the l-truncated ELBO for the l-truncated C-vine

$$\begin{aligned} {\mathcal {L}}_{C_l}({\varvec{\lambda }}) = {\mathbb {E}}_q\bigg [\sum _{j = 1}^{l} \sum _{i = 1}^{N-j} f^{C_l}_{j, j+i}({\varvec{\phi }})\bigg ] - KL(q({\varvec{\phi }}|{\varvec{\lambda }})||p({\varvec{\phi }}))\nonumber \\ \end{aligned}$$

(55)

with its gradient

$$\begin{aligned} \begin{aligned} \nabla _{{\varvec{\lambda }}} {\mathcal {L}}_{C_l}({\varvec{\lambda }})&= \sum _{j = 1}^{l} \sum _{i = 1}^{N-j}{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})(f^{C_l}_{j, j+i}({\varvec{\phi }}))\bigg ] \\&\quad -{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})\log \frac{q({\varvec{\phi }}|{\varvec{\lambda }})}{p({\varvec{\phi }})}\bigg ]. \end{aligned} \end{aligned}$$

Consequently, we get the following proposition that establishes the noisy unbiased estimate of $\nabla _{{\varvec{\lambda }}} {\mathcal {L}}_{C_l}({\varvec{\lambda }})$.

Proposition 6

Let $\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})$ be an estimate of the ELBO gradient $\nabla _{{\varvec{\lambda }}} {\mathcal {L}}_{C_l}({\varvec{\lambda }})$ defined as

$$\begin{aligned} \begin{aligned}&\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }}) \\&\quad =\frac{l(2N-(l + 1))}{2}{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})( f^{C_l}_{I_{C_l}(K)}({\varvec{\phi }}))\bigg ] \\&\qquad -{\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q(\phi |{\varvec{\lambda }})\log \frac{ q(\phi |{\varvec{\lambda }})}{ p(\phi )}\bigg ], \end{aligned} \end{aligned}$$

where $K \sim U(1, \dots , \frac{l(2N-(l + 1))}{2})$, and $I_{C_l}$ is the bijection

$$\begin{aligned} \begin{aligned}&I_{C_l}:\{1, \dots , \frac{l(2N-(l + 1))}{2}\} \\&\quad \rightarrow \{(j,j+i): i \in \{1, \dots , N-j\} \text { for } j \in \{1, \dots l\}\}, \end{aligned} \end{aligned}$$

then $\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})$ is unbiased i.e., ${\mathbb {E}}(\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})) = \nabla _{{\varvec{\lambda }}} {\mathcal {L}}_{C_l}({\varvec{\lambda }})$.

Algorithm 3 postulates the version of Algorithm 1 based on the truncated C-vine decomposition.

1.1 A.1 Variance reduction

Let us now consider the MC approximation of the gradient estimator $\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})$. The $j^{th}$ component of the estimator with Rao-Blackwellization is

$$\begin{aligned} \begin{aligned}&\frac{1}{S}\sum _{s=1}^{S}\bigg [\frac{l(2N-(l + 1))}{2}\nabla _{{\varvec{\lambda }}_j} \log q(\phi _j[s]|{\varvec{\lambda }}_j)\big ({\tilde{f}}^j_{(K)}({\varvec{\phi }}[s]) \\&\quad -\frac{2\log \frac{ q(\phi _j[s]|{\varvec{\lambda }}_j)}{p(\phi _j[s])}}{l(2N-(l + 1))}\big )\bigg ], \end{aligned} \end{aligned}$$

where ${\tilde{f}}^j_{(K)}({\varvec{\phi }})$ are here the components of $f^{C_l}_{I_{C_l}(K)}({\varvec{\phi }})$ that include $\phi _j$.

We can again use the control variates to reduce the variance of MC approximation of the gradient estimator $\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})$. In particular, we consider the following $j^{th}$ element of the Rao-Blackwellized MC approximation of the gradient estimator $\tilde{{\mathcal {L}}}_{C_l}({\varvec{\lambda }})$ with control variates

$$\begin{aligned} \begin{aligned}&\tilde{{\mathcal {L}}}_{C_l}^{CV}({\varvec{\lambda }})_j \\&\quad =\sum _{s=1}^{S}\bigg [\frac{l(2N-(l + 1))}{2S}\nabla _{{\varvec{\lambda }}_j} \log q(\phi _j[s]|{\varvec{\lambda }}_j)( {\tilde{f}}^j_{(K)}({\varvec{\phi }}[s]) \\&\qquad -\frac{2(\log \frac{ q(\phi _j[s]|{\varvec{\lambda }}_j)}{p(\phi _j[s])} + {\hat{a}}^C_j)}{l(2N-(l + 1))})\bigg ], \end{aligned} \end{aligned}$$

where ${\hat{a}}^C_j$ is the sample estimate of the $j^{th}$ component of the optimal control variate scale $a^*$ based on S (or fever) independent draws from the variational distribution. Namely,

$$\begin{aligned} a^* = \frac{{{\mathbb {C}}ov}_q(\xi ^C({\varvec{\phi }}),\psi ^C({\varvec{\phi }}))}{{{\mathbb {V}}ar}_q(\psi ^C({\varvec{\phi }}))}, \end{aligned}$$

where

$$\begin{aligned} \begin{aligned} \xi ^C({\varvec{\phi }})&= \frac{l(2N-(l + 1))}{2}\nabla _{{\varvec{\lambda }}} \log q(\phi |{\varvec{\lambda }})\bigg (f^{C_l}_{I_{C_l}(K)}({\varvec{\phi }}) \\&\quad - \frac{2 \log q(\phi |{\varvec{\lambda }})}{l(2N-(l + 1)) \log p(\phi )}\bigg ) \end{aligned} \end{aligned}$$

and $\psi ^C({\varvec{\phi }}) = \nabla _{{\varvec{\lambda }}} \log q(\phi |{\varvec{\lambda }})$.

As in the case of the D-vine, we now derive the ultimate Algorithm 4. Again, instead of taking the samples from $q({\varvec{\phi }}| {\varvec{\lambda }})$ to approximate the gradient estimates, we will take samples from an overdispersed distribution $r({\varvec{\phi }}|{\varvec{\lambda }}, \tau )$. Combining the Rao-Blackwellization, control variates, and importance sampling, we have the following $j^{th}$ component of the MC approximation of the gradient estimator $\tilde{{\mathcal {L}}}_{C_l}(\lambda )$

$$\begin{aligned}&\tilde{{\mathcal {L}}}_{C_l}^{OCV}({\varvec{\lambda }})_j\\&\quad =\sum _{s=1}^{S}\bigg [\frac{l(2N-(l + 1))}{2S}\nabla _{{\varvec{\lambda }}_j} \log q(\phi _j[s]|{\varvec{\lambda }}_j)({\tilde{f}}^j_{(K)}({\varvec{\phi }}[s])\\&\qquad -\frac{2(\log \frac{ q(\phi _j[s]|{\varvec{\lambda }}_j)}{p(\phi _j[s])} + {\tilde{a}}^C_j)}{l(2N-(l + 1))})w(\phi _j[s])\bigg ], \end{aligned}$$

where ${\varvec{\phi }}[s] \sim r({\varvec{\phi }}|{\varvec{\lambda }}, \tau )$ and $w({\varvec{\phi }}[s] ) = q({\varvec{\phi }}[s] |{\varvec{\lambda }}) / r({\varvec{\phi }}[s]|{\varvec{\lambda }}, \tau )$ with ${\tilde{a}}^C_j$ being the estimate of the $j^{th}$ component of

$$\begin{aligned} a^*_O=\frac{{\mathbb {C}}ov_r[\xi ^C({\varvec{\phi }})w(\phi ) ,\psi ^C({\varvec{\phi }})w(\phi )]}{{\mathbb {V}}ar_r[\psi ^C({\varvec{\phi }})w(\phi )]}. \end{aligned}$$

Appendix B Proofs

Proof of Proposition 1

Since $P(K = k) = \frac{2}{N(N-1)}$, we have directly from the definition of expectation

$$\begin{aligned}&{\mathbb {E}}(\tilde{{\mathcal {L}}}_D({\varvec{\lambda }})) \\&\quad =\frac{N(N-1)}{2} \sum _{k=1}^{\frac{N(N-1)}{2}}\frac{2}{N(N-1)}{\mathbb {E}}_q\\ {}&\qquad \times \bigg [\nabla _{{\varvec{\lambda }}}\log q({\varvec{\phi }}|{\varvec{\lambda }})( f^D_{I_D(k)}({\varvec{\phi }}))\bigg ] \\&\qquad - {\mathbb {E}}_q\bigg [\nabla _{{\varvec{\lambda }}} \log q({\varvec{\phi }}|{\varvec{\lambda }})\log \frac{ q({\varvec{\phi }}|{\varvec{\lambda }})}{ p({\varvec{\phi }})}\bigg ] = \nabla _{{\varvec{\lambda }}} {\mathcal {L}}({\varvec{\lambda }}). \end{aligned}$$

The final equality is the consequence of the uniqueness of the pairs of variables in the conditioned sets of the copula density $c_{i,(i+j); (i+1), \dots , (i+j-1)}$, and that $\frac{N(N-1)}{2}$ is the number of unordered pairs of N variables. $\square$

Proof of Proposition 2

It is sufficient to show that for $l \in \{1, \dots , N-1\}$ the following equality holds:

$$\begin{aligned} \begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\frac{1}{a_i}\log p(d_i|{\varvec{\phi }}) + \frac{1}{b_{i+j}} \log p(d_{i+j}|{\varvec{\phi }})\bigg ] \\&\quad =\sum _{k = 1}^N \log p({\varvec{d}}_k|{\varvec{\phi }}), \end{aligned} \end{aligned}$$

(56)

where

$$\begin{aligned} a_i&= 2l - \bigg [(l+1-i)\mathbb {1}_{i \le l} + (l - N +i)\mathbb {1}_{i> N - l}\bigg ], \\ b_{i+j}&= 2l - \bigg [(l+1-j-i)\mathbb {1}_{i + j \le l} \\&\quad +(l - N +j+i)\mathbb {1}_{i + j > N - l}\bigg ]. \end{aligned}$$

To show this, let us consider the summation

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_i|{\varvec{\phi }}) + \log p(d_{i+j}|{\varvec{\phi }})\bigg ]\\&\quad =\sum _{j = 1}^{l} \bigg [(\log p(d_1|{\varvec{\phi }}) + \log p(d_{1+j}|{\varvec{\phi }})) + \dots \\&\qquad +(\log p(d_{N-j}|{\varvec{\phi }}) + \log p(d_{N}|{\varvec{\phi }}))\bigg ]. \end{aligned}$$

For $l = 1$, we get

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_i|{\varvec{\phi }}) + \log p(d_{i+j}|{\varvec{\phi }})\bigg ]\\&\quad =(\log p(d_1|{\varvec{\phi }}) + \log p(d_{2}|{\varvec{\phi }})) + \dots \\&\qquad +(\log p(d_{N-1}|{\varvec{\phi }}) + \log p(d_{N}|{\varvec{\phi }})), \end{aligned}$$

and for $l \ge 2$

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_i|{\varvec{\phi }}) + \log p(d_{i+j}|{\varvec{\phi }})\bigg ]\\&\quad =\bigg [(\log p(d_1|{\varvec{\phi }}) + \log p(d_{2}|{\varvec{\phi }})) + \dots \\&\qquad +(\log p(d_{N-1}|{\varvec{\phi }}) + \log p(d_{N}|{\varvec{\phi }}))\bigg ] +\dots \\ {}&\qquad + \bigg [(\log p(d_1|{\varvec{\phi }}) + \log p(d_{1+l}|{\varvec{\phi }})) + \dots \\&\qquad +(\log p(d_{N-l}|{\varvec{\phi }}) + \log p(d_{N}|{\varvec{\phi }}))\bigg ]. \end{aligned}$$

Note that in the case of $l = N-1$, the last summation consists of only one element $\log p_1(d_1|{\varvec{\phi }}) + \log p_{1+l}(d_{1+l}|{\varvec{\phi }})$. By careful examination of the two cases above, we get the following results. For $2l \le N$:

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_i|{\varvec{\phi }}) + \log p(d_{i+j}|{\varvec{\phi }})\bigg ]\\&\quad =\sum _{k = 1}^l (l + k -1)\log p(d_k|{\varvec{\phi }}) + \sum _{k =l + 1}^{N-l} 2l \log p(d_k|{\varvec{\phi }}) \\&\qquad +\sum _{k =N - l + 1}^{N} (N - i + l)\log p(d_k|{\varvec{\phi }}), \end{aligned}$$

where the middle term disappears in the case $2l = N$, and for $2l > N$:

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_i|{\varvec{\phi }}) + \log p(d_{i+j}|{\varvec{\phi }})\bigg ]\\&\quad =\sum _{k = 1}^{N-l} (l + k -1)\log p(d_k|{\varvec{\phi }}) \\&\qquad +\sum _{k = N - l + 1}^{l} (N-1) \log p(d_k|{\varvec{\phi }}) \\&\qquad + \sum _{k =l+1}^{N} (N - i + l)\log p(d_k|{\varvec{\phi }}). \end{aligned}$$

If we now check that $a_i$ equals to the factors in front of the log-likelihoods in the two cases above, the proof of Proposition 2 is complete. Note that once we check the equality for $a_i$, the same directly translates to $b_{i+j}$ since $b_{i+j}$ is $a_i$ with indices set to $i+j$ instead of i. Indeed, for $2l \le N$

$$\begin{aligned} a_{i} = {\left\{ \begin{array}{ll} l +i -1 &{} i\le l\\ 2l &{} l< i \le N -l\\ N - i + l &{} N - l < i \end{array}\right. }, \end{aligned}$$

and for $2l > N$

$$\begin{aligned} a_{i} = {\left\{ \begin{array}{ll} l +i -1 &{} i\le N - l\\ N -1 &{} N - l< i \le l\\ N - i + l &{} l < i \end{array}\right. }. \end{aligned}$$

$\square$

Proof of Proposition 3

By the construction of R-vine (see Cooke and Kurowicka (2006)), each tree ${\mathcal {T}}_i$, for $i = 1, \dots , N-1$ has exactly $N-i$ edges (these are the unique conditioned variable pairs). For any R-vine truncated at level $l \in \{1, \dots , N-1\}$, we get the number of edges to be

$$\begin{aligned} \sum _{i = 1}^l (N - i) = lN - \frac{l(l+1)}{2} = \frac{l(2N - (l+1))}{2} \end{aligned}$$

The rest of the proof is identical with that of Proposition 1 due to the uniqueness of the conditioned variable pairs in the copula density $c_{i,(i+j); (i+1), \dots , (i+j-1)}$, but in this case $P(K = k) = \frac{2}{l(2N - (l+1))}$. $\square$

Proof of Proposition 4

The proof is identical with that of Proposition 1 since each conditioned pair in the copula density $c_{j,(j+i); 1, \dots , (j-1)}$ is unique as well. $\square$

Proof of Proposition 5

It is sufficient to show that for $l \in \{1, \dots , N-1\}$ the following equality holds:

$$\begin{aligned} \begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\frac{1}{a_j}\log p(d_j|{\varvec{\phi }}) + \frac{1}{b_{j+i}} \log p(d_{j+i}|{\varvec{\phi }})\bigg ] \\&\quad =\sum _{k = 1}^N \log p({\varvec{d}}_k|{\varvec{\phi }}), \end{aligned} \end{aligned}$$

(57)

where

$$\begin{aligned} a_j&= N-1, \\ b_{j + i}&= (N-1-l)\mathbb {1}_{j + i \le l} + l. \end{aligned}$$

To show this, let us consider the following summation

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p_j(d_j|{\varvec{\phi }}) + \log p(d_{j+i}|{\varvec{\phi }})\bigg ]\\&\quad = \sum _{j = 1}^{l} \bigg [(N - j)\log p(d_j|{\varvec{\phi }}) + \sum _{i = 1}^{N-j}\log p(d_{j+i}|{\varvec{\phi }})\bigg ] \\&\quad = \sum _{j = 1}^{l}(N - j)\log p(d_j|{\varvec{\phi }}) \\&\qquad +\sum _{j = 1}^{l}\bigg [\log p(d_{j+1}|{\varvec{\phi }})) + \dots + \log p(d_{N}|{\varvec{\phi }})\bigg ]. \end{aligned}$$

Now, for $l = 1$, we have

$$\begin{aligned}&\sum _{j = 1}^{l}\bigg [\log p(d_{j+1}|{\varvec{\phi }})) + \dots + \log p(d_{N}|{\varvec{\phi }})\bigg ] \\&\quad =\log p(d_{2}|{\varvec{\phi }}) + \dots \log p(d_{N}|{\varvec{\phi }}). \end{aligned}$$

For $l \ge 2$, we have

$$\begin{aligned}&\sum _{j = 1}^{l}\bigg [\log p(d_{j+1}|{\varvec{\phi }})) + \dots + \log p(d_{N}|{\varvec{\phi }})\bigg ]\\&\quad = \bigg [\log p(d_{2}|{\varvec{\phi }}) + \dots \log p(d_{N}|{\varvec{\phi }})\bigg ] + \dots \\&\qquad + \bigg [\log p(d_{l+1}|{\varvec{\phi }}) + \dots \log p(d_{N}|{\varvec{\phi }})\bigg ] . \end{aligned}$$

Therefore we can rewrite

$$\begin{aligned}&\sum _{j = 1}^{l}\bigg [\log p(d_{j+1}|{\varvec{\phi }})) + \dots + \log p(d_{N}|{\varvec{\phi }})\bigg ]\\&\quad = \sum _{j = 1}^{l}(j-1) \log p(d_j|{\varvec{\phi }}) + \sum _{j = l+1}^{N}l \log p(d_j|{\varvec{\phi }}) \end{aligned}$$

Overall,

$$\begin{aligned}&\sum _{j = 1}^{l} \sum _{i = 1}^{N-j}\bigg [\log p(d_j|{\varvec{\phi }}) + \log p(d_{j+i}|{\varvec{\phi }})\bigg ] \\&\quad = \sum _{j = 1}^{l}(N - j)\log p(d_j|{\varvec{\phi }}) + \sum _{j = 1}^{l}(j-1) \log p(d_j|{\varvec{\phi }}) \\&\qquad +\sum _{j = l+1}^{N}l \log p(d_j|{\varvec{\phi }})\\&\quad = \sum _{k=1}^{l}(N-1)\log p(d_k|{\varvec{\phi }}) + \sum _{k=l+1}^{N}l \log p(d_k|{\varvec{\phi }}). \end{aligned}$$

Since $j \in \{1, \dots , l\}$ and

$$\begin{aligned} b_{j + i} = {\left\{ \begin{array}{ll} N - 1 &{} j + i\le l \\ l &{} j + i > l \end{array}\right. }, \end{aligned}$$

the equality 57 holds. $\square$

Proof of Proposition 6

The proof is identical with that of Proposition 3 since each conditioned pair in the copula density $c_{j,(j+i); 1, \dots , (j-1)}$ is unique, and a C-vine is a special case of R-vine. $\square$

Appendix C Simulation: truncation level

Figure 6 and Table 4 show the changes in values of variational parameters with the increasing size of truncation level l for the calibration simulation described in Sect. 5.1.1. The differences between $L^2$ norms of variational parameters are provided both for all the variational parameters and for the variational approximations of the calibration parameters only. Figure 6 exhibits a clear elbow shape with minimal change of variational parameters for $l \ge 3$.

Fig. 6

$L^2$ norm values for the difference between variational parameters obtained via Algorithm 2 with the increasing value of truncation level l. The symbol ${\varvec{\lambda }}_\theta$ corresponds for the variational parameters of the variational approximations for calibration parameters

Table 4 The values of $L^2$ norms depicted in Fig. 6

Appendix D Simulation: memory profile

Here we present the memory profiles (Fig. 7) for the MH, the NUTS, and the Algorithm 2 under the simulation scenario studied in Chapter 5. These were recorded during a one hour period of running the algorithms. The MH algorithm and the NUTS were implemented in Python 3.0 using the PyMC3 module version 3.5. The memory profiles were measured using the memory-profiler module version 0.55.0 in Python 3.0. The VC was also implemented in Python 3.0. The code was run on the high performance computing cluster at the Institute for Cyber-Enabled Research at Michigan State University.

Fig. 7

Recorded memory profiles of the Algorithm 2, the MH algorithm, and the NUTS for the duration of 1h under the simulation scenario with $n = 0.5 \times 10 ^ 4$, $n = 1 \times 10 ^ 4$, and $n = 2 \times 10 ^ 4$

Appendix E Application: liquid drop model (LDM)

1.1 E.1 GP specifications

In the case of the LDM $E_B(Z,N)$, we consider the GP prior with mean zero and covariance function

$$\begin{aligned}&\eta _{\gamma } \\&\quad \times \text {exp}\left( -\frac{\Vert Z - Z' \Vert ^2}{2\nu ^2_Z} -\frac{\Vert N - N' \Vert ^2}{2\nu ^2_N} -\frac{\Vert \theta _{\textrm{vol}} - \theta _{\textrm{vol}}' \Vert ^2}{2\nu ^2_1} \right. \\&\left. \quad - \frac{\Vert \theta _{\textrm{surf}} - \theta _{\textrm{surf}}' \Vert ^2}{2\nu ^2_2} -\frac{\Vert \theta _{\textrm{sym}} - \theta _{\textrm{sym}}' \Vert ^2}{2\nu ^2_3} -\frac{\Vert \theta _{\textrm{C}} - \theta _{\textrm{C}}' \Vert ^2}{2\nu ^2_4}\right) . \end{aligned}$$

Similarly, we consider the GP process prior for the systematic discrepancy $\delta (Z,N)$ with mean zero and covariance function

$$\begin{aligned} \eta _\delta \times \exp {\left( -\frac{\Vert Z - Z' \Vert ^2}{2l^2_Z} -\frac{\Vert N - N' \Vert ^2}{2l^2_N}\right) }. \end{aligned}$$

1.2 E.2 Experimental design

Kennedy and O’Hagan (2001) recommend to select the calibration inputs for the model runs so that any plausible value ${\varvec{\theta }}$ of the true calibration parameter is covered. In this context, we consider the space of calibration parameters to be centered at the values of least squares estimates ${\hat{{\varvec{\theta }}}}_{L_2}$ and broad enough to contain the majority of values provided by the nuclear physics literature (Weizsäcker 1935; Bethe and Bacher 1936; Myers and Swiatecki 1966; Kirson 2008; Benzaid et al. 2020). Table 5 gives the lower and upper bounds for the parameter space so that $\text {Lower bound} = {\hat{\theta }}_{L_2} - 15 \times SE({\hat{\theta }}_{L_2})$ and $\text {Upper bound} = {\hat{\theta }}_{L_2} + 15 \times SE({\hat{\theta }}_{L_2})$. Here $SE({\hat{\theta }}_{L_2})$ is given by the standard linear regression theory.

Table 5 The space of calibration parameters used for generating the outputs of semi-empirical mass formula (47)

1.3 E.3 Prior distributions

First, we consider the independent Gaussian distributions centered at the LS estimates ${\hat{{\varvec{\theta }}}}_{L_2}$ (in Table 3) with standard deviations $7.5 \times SE({\hat{{\varvec{\theta }}}}_{L_2})$ so that the calibration parameters used for generating the model runs are covered roughly within two standard deviations of the priors. Namely,

$$\begin{aligned} \theta _{\text {vol}}&\sim \mathcal {N}(15.42, 0.203), \\ \theta _{\text {surf}}&\sim \mathcal {N}(16.91, 0.645), \\ \theta _{\text {sym}}&\sim \mathcal {N}(22.47, 0.525), \\ \theta _{\text {C}}&\sim \mathcal {N}(0.69, 0.015). \end{aligned}$$

The prior distributions for hyperparameters of the GPs were selected as $\text {Gamma}(\alpha , \beta )$ with the shape parameter $\alpha$ and scale parameter $\beta$, so that they represent a vague knowledge about the scale of these parameters given by the literature on nuclear mass models (Weizsäcker 1935; Bethe and Bacher 1936; Myers and Swiatecki 1966; Fayans 1998; Kirson 2008; McDonnell et al. 2015; Kortelainen et al. 2010, 2012, 2014; Benzaid et al. 2020; Kejzlar et al. 2020). In particular, the error scale $\sigma$ is in the majority of nuclear applications within units of MeV, therefore we set

$$\begin{aligned} \sigma \sim \text {Gamma}(2,1), \end{aligned}$$

with the scale of the systematic error being

$$\begin{aligned} \eta _\delta \sim \text {Gamma}(10,1), \end{aligned}$$

to allow for this quantity to range between the units and tens of MeV. It is also reasonable to assume that the mass of a given nucleus is correlated mostly with its neighbors on the nuclear chart. We express this notion through these reasonably wide prior distributions

$$\begin{aligned} l_Z&\sim \text {Gamma}(10,1), \\ l_N&\sim \text {Gamma}(10,1), \\ \nu _Z&\sim \text {Gamma}(10,1), \\ \nu _N&\sim \text {Gamma}(10,1), \\ \nu _i&\sim \text {Gamma}(10,1), \quad i = 1,2,3,4. \end{aligned}$$

Finally, the majority of the masses in the training dataset of 2000 experimental binding energies fall into the range of [1000, 2000] MeV (1165 of masses precisely). We consider the following prior distribution for the parameter $\eta _f$ to reflect on the scale of the experimental binding energies:

$$\begin{aligned} \eta _f \sim \text {Gamma}(110,10). \end{aligned}$$