Estimating the Covariance Matrix of the Maximum Likelihood Estimator Under Linear Cluster-Weighted Models

Abstract

In recent years, the research into cluster-weighted models has been intense. However, estimating the covariance matrix of the maximum likelihood estimator under a cluster-weighted model is still an open issue. Here, an approach is developed in which information-based estimators of such a covariance matrix are obtained from the incomplete data log-likelihood of the multivariate Gaussian linear cluster-weighted model. To this end, analytical expressions for the score vector and Hessian matrix are provided. Three estimators of the asymptotic covariance matrix of the maximum likelihood estimator, based on the score vector and Hessian matrix, are introduced. The performances of these estimators are numerically evaluated using simulated datasets in comparison with a bootstrap-based estimator; their usefulness is illustrated through a study aiming at evaluating the link between tourism flows and attendance at museums and monuments in two Italian regions.

1 Introduction

Cluster-weighted models constitute an approach to regression analysis with random covariates in which supervised (regression) and unsupervised (model-based cluster analysis) learning methods are jointly exploited (Hastie et al., 2009). In this approach, a given random vector is assumed to be composed of an outcome Y (response, dependent variable) and its explanatory variables X (covariates, predictors). Furthermore, sample observations are allowed to come from a population composed of several unknown sub-populations. Finally, the joint distribution of the outcome and the covariates is modelled through a finite mixture model specified so as to account for a different effect of the covariates on the response within each sub-population. Thus, cluster-weighted models are useful to perform model-based cluster analysis in a multivariate regression setting with random covariates and unobserved heterogeneity.

Since the introduction of this approach (Gershenfeld, 1997), the research into cluster-weighted models has been intense, especially in the last 8 years. Models for continuous variables under normal mixture models have been proposed by Ingrassia et al. (2012) and Dang et al. (2017). Robustified solutions have been developed by Ingrassia et al. (2014) and Punzo and McNicholas (2017), based on the use of Student t and contaminated normal mixture distributions, respectively. Punzo and Ingrassia (2013), Punzo and Ingrassia (2016), Ingrassia et al. (2015) and Di Mari et al. (2020) have introduced models for various types of responses. Models able to deal with non-linear relationships or many covariates have been proposed by Punzo (2014, Subedi et al. (2013) and Subedi et al. (2015).

By focusing the attention on Gaussian cluster-weighted models in which the effects of the covariates on the response within each sub-population are linear, model parameters are generally estimated through the maximum likelihood (ML) method by resorting to the expectation-maximisation (EM) algorithm (Dempster et al., 1977), which is widely employed in incomplete-data problems. In these models, the observed data $\mathcal {S}=\{(\mathbf {x}_{i},\mathbf {y}_{i}$), i = 1, … , I} are incomplete because the specific component density that generates the I sample observations is missing. This missing information is modelled through an unobserved variable coming from a pre-specified multinomial distribution and is added to the observed data so as to form the so-called complete data. Then, the ML estimate is computed from the maximisation of the complete data log-likelihood. A description of the EM algorithm for the linear Gaussian cluster-weighted model can be found in Dang et al. (2017). Specific functions implementing such algorithm for models with a univariate response are available in the package flexCWM (Mazza et al., 2018) for the R software environment (R Core Team, 2020).

A by-product of this estimating approach is a set of K estimated posterior probabilities that each sample observation comes from the K Gaussian distributions of the mixture. Thus, a by-product of fitting a linear Gaussian cluster-weighted model is a clustering of the I sample observations, based on a rule that assigns an observation to the distribution of the mixture to which it has the highest posterior probability of belonging. However, an estimating approach based on the use of an EM algorithm has the drawback of not automatically producing any estimate of the covariance matrix of the ML estimator. The assessment of the sample variability of the parameter estimates in a linear Gaussian cluster-weighted model is a necessary step in the subsequent development of inference methods for the model parameters, such as asymptotic confidence intervals, tests for the significance of the effect of any covariate on a given response within any sub-population and tests for the significance of the difference between the effects of the same covariate on a given response in two different sub-populations. Thus, additional computations are necessary to obtain an assessment of the sample variability of model parameter estimates. To the best of the author’s knowledge, the only solution currently available for the linear Gaussian cluster-weighted models is implemented in the flexCWM package, where approximated standard errors are only provided for the intercepts and regression coefficients according to an approach in which a number of separate linear regression analyses with random covariates are carried out (one for each component of the mixture), and the sample observations are weighted with their estimated posterior probabilities of coming from the different components of the mixture. However, this approach only partially exploits the sample information about the parameters under a linear normal cluster-weighted model. Thus, other approaches could be investigated and detected. A solution can be obtained by resorting to bootstrap methods (see, e.g., Newton and Raftery 1994; Basford et al., 1997; McLachlan & Peel 2000). However, the overall computational process associated with the use of bootstrap techniques can become particularly time-consuming and complex because of difficulties typically associated with the fitting of finite mixture models (e.g. label-switching problems, possible convergence failures of the EM algorithm on the bootstrap samples). These inconveniences could be avoided through an approach in which the observed information matrix is obtained from the incomplete data log-likelihood and employed to compute information-based estimators of the covariance matrix of the ML estimator (see e.g. McLachlan & Peel 2000). This task has been successfully carried out under normal mixture models (Boldea & Magnus, 2009) and clusterwise linear regression models (Galimberti et al., 2021).

In order to make it possible to properly assess both the variability of and the covariance between ML estimates of all the parameters under multivariate linear normal cluster-weighted models with a multivariate response, the gradient vector and second-order derivative matrix of the incomplete data log-likelihood for these models are explicitly derived here. Then, these results are used to obtain three estimators of the observed information matrix and the covariance matrix of the ML estimator. Properties of such estimators are numerically investigated using simulated datasets in comparison with the parametric bootstrap and the approach implemented in flexCWM. A numerical evaluation of the relationships between the estimators introduced here and those described by Boldea and Magnus (2009) is also provided. The practical usefulness of the proposed estimators is illustrated in a study aiming at evaluating the link between tourism flows and attendance at museums and monuments in two Italian regions.

The remainder of the paper is organised as follows. Section 2 provides the definition of multivariate Gaussian linear cluster-weighted model together with some quantities employed in the derivation of the score vector and the Hessian matrix. Section 3 describes the estimators of the information matrix and the covariance matrix of the ML estimator. Sections 4 and 5 summarize the main results obtained from the analysis of simulated and real datasets, respectively. The analytical expressions of the score vector and the Hessian matrix are reported in Appendix. Technical details and additional results from the analysis of simulated datasets can be found in a separate document as supplementary materials.

2 Score Vector and Hessian Matrix of Gaussian Linear Cluster-Weighted Models

Let $\textbf {X}=(X_{1},...,X_{p})^{\prime }$ and $\textbf {Y}=(Y_{1},...,Y_{q})^{\prime }$ be two continuous random vectors with joint probability density function (p.d.f.) f(x,y). The response vector Y and the covariate vector X take values in $\mathbb {R}^{q}$ and $\mathbb {R}^{p}$, respectively. Following Dang et al. (2017), $(\textbf {X}^{\prime },\textbf {Y}^{\prime })^{\prime }$ follows a cluster-weighted model of order G if

$$ f(\textbf{x},\textbf{y})=\sum\limits_{g=1}^{G} \pi_{g} f(\textbf{x}|\mathbf{\Omega}_{g}) f(\textbf{y}|\textbf{x},\mathbf{\Omega}_{g}), $$

(1)

where Ω₁, … , Ω_G denote the G unknown sub-populations (Ω_g ∩ Ω_j = ∅ ∀g≠j), $\pi _{g}=\mathbb {P}(\mathbf {\Omega }_{g})$, π_g > 0 ∀g, ${\sum }_{g=1}^{G}\pi _{g}=1$, f(x|Ω_g) is the conditional p.d.f. of X given Ω_g, f(y|x,Ω_g) is the conditional p.d.f. of Y given X and Ω_g. A Gaussian linear cluster-weighted model is obtained from Eq. 1 by additionally assuming that the distributions of X|Ω_g and Y|(X = x,Ω_g) are Gaussian for g = 1, … , G and the effect of X on Y for any Ω_g is linear. By embedding all these assumptions into model (1), f(x,y) becomes

$$ f(\textbf{x},\textbf{y}; \boldsymbol{\vartheta})=\sum\limits_{g=1}^{G} \pi_{g} \phi_{p}(\textbf{x};\boldsymbol{\mu}_{\textbf{X}_{g}},\boldsymbol{\Sigma}_{\textbf{X}_{g}}) \phi_{q}(\textbf{y}|\textbf{x};\textbf{B}^{\prime}_{g}\textbf{x}^{*},\boldsymbol{\Sigma}_{\textbf{Y}_{g}}), $$

(2)

where ϕ_d(⋅;μ,Σ) represents the p.d.f. of a normal d-dimensional random vector with expected value μ and positive definite covariance matrix Σ,

$$ \textbf{B}^{\prime}_{g}\textbf{x}^{*}=\mathbb{E}(\textbf{Y}|\textbf{X}=\textbf{x},\mathbf{\Omega}_{g}), \ g=1, \ldots, G, $$

(3)

with $\textbf {B}_{g}\in \mathbb {R}^{(1+p)\times q}$, $\textbf {x}^{*}=(1,\textbf {x}^{\prime })^{\prime }$, and 𝜗 is the vector of the unknown parameters. It has been proved that linear Gaussian cluster-weighted models of order G define the same family of probability distributions generated by mixtures of G Gaussian models for the random vector $\textbf {Z}=(\textbf {X}^{\prime },\textbf {Y}^{\prime })^{\prime }$ (Ingrassia et al., 2012). However, it is important to stress that this latter type of mixtures cannot be employed to account for local linear dependencies between X and Y.

The score vector and Hessian matrix of model (2) are derived by taking account of the fact that the weights π₁, … , π_G sum to one and the covariance matrices are symmetric. The first constraint is introduced in the maximization of the likelihood function by differentiating with respect to $\boldsymbol {\pi }=\left (\pi _{1},\dots ,\pi _{G-1}\right )^{\prime }$ and by setting π_G = 1 − π₁ −⋯ − π_G− 1. The constraints on the covariance matrices are dealt with by using the operators vec(⋅), v(⋅) and the duplication matrix. Namely, vec(B) is the column vector obtained by stacking the columns of matrix B one underneath the other. v(Σ) denotes the column vector obtained from vec(Σ) by eliminating all supradiagonal elements of a symmetric matrix Σ (thus, v(Σ) contains only the lower triangular part of Σ). The duplication matrix G is the unique matrix which transforms v(Σ) into vec(Σ) (G v(Σ) = vec(Σ)) (see e.g. Magnus and Neudecker 1988). Using this notation, the vector of the unknown parameters in model (2) can be denoted as $\boldsymbol {\vartheta }= (\boldsymbol {\pi }^{\prime },\boldsymbol {\theta }^{\prime }_{1}, \ldots , \boldsymbol {\theta }^{\prime }_{G})^{\prime }$, where $\boldsymbol {\theta }_{g}=\left (\boldsymbol {\mu }^{\prime }_{\textbf {X}_{g}}, \mathrm {v}\left (\boldsymbol {\Sigma }_{\textbf {X}_{g}}\right )^{\prime }, \text {vec}\left (\textbf {B}_{g}\right )^{\prime }, \mathrm {v}\left (\boldsymbol {\Sigma }_{\textbf {Y}_{g}}\right )^{\prime }\right )^{\prime }$.

Suppose that the observed data $\mathcal {S}=\{(\mathbf {x}_{i},\mathbf {y}_{i}$), i = 1, … , I} is composed of I independent and identically distributed observations. Then, the incomplete log-likelihood function under the model (2) is

$$ l(\boldsymbol{\vartheta})=\sum\limits_{i=1}^{I}\ln \biggl(\sum\limits_{g=1}^{G}\pi_{g}\phi_{p}\bigl(\textbf{x}_{i};\boldsymbol{\mu}_{\textbf{X}_{g}},\boldsymbol{\Sigma}_{\textbf{X}_{g}}\bigr)\phi_{q}\bigl(\textbf{y}_{i}|\textbf{x}_{i};\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i},\boldsymbol{\Sigma}_{\textbf{Y}_{g}}\bigr)\biggr). $$

(4)

Each sample observation provides its own contribution to the g th term of the sum that defines the mixture (2). As far as the contribution of the i th observation is concerned, it is given by:

$$ \begin{array}{@{}rcl@{}} p_{gi}&=&\pi_{g} (2\pi)^{-\frac{p}{2}} \det(\boldsymbol{\Sigma}_{\textbf{X}_{g}})^{-\frac{1}{2}}\exp\biggl[-{1 \over 2}(\textbf{x}_{i}-\boldsymbol{\mu}_{\textbf{X}_{g}})^{\prime}\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}(\textbf{x}_{i}-\boldsymbol{\mu}_{\textbf{X}_{g}})\biggr] \\ & & (2\pi)^{-\frac{q}{2}} \det(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})^{-\frac{1}{2}}\exp\biggl[-{1 \over 2}(\textbf{y}_{i}-\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i})^{\prime}\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}(\textbf{y}_{i}-\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i})\biggr]. \end{array} $$

(5)

By exploiting properties of the logarithm and trace, the following equality holds true:

$$ \begin{array}{@{}rcl@{}} \ln p_{gi}&=& \ln \pi_{g} -{p \over 2}\ln (2\pi)-{1 \over 2}\text{ln det}(\boldsymbol{\Sigma}_{\textbf{X}_{g}})-{1 \over 2}\text{tr}\biggl[\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}(\textbf{x}_{i}-\boldsymbol{\mu}_{\textbf{X}_{g}})(\textbf{x}_{i}-\boldsymbol{\mu}_{\textbf{X}_{g}})^{\prime}\biggr] +\\ && -{q \over 2}\ln (2\pi)-{1 \over 2}\text{ln det}(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})-{1 \over 2}\text{tr}\biggl[\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}(\textbf{y}_{i}-\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i})(\textbf{y}_{i}-\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i})^{\prime}\biggr]. \end{array} $$

(6)

The explicit forms of the score vector and Hessian matrix, as developed here, require the introduction of some additional notation. Namely, let

$$ \begin{array}{@{}rcl@{}} \alpha_{gi} & = & \frac{p_{gi}}{{\sum}_{j=1}^{G}p_{ji}}, \end{array} $$

(7)

$$ \begin{array}{@{}rcl@{}} \mathbf{a}_{g} & = & \frac{1}{\pi_{g}}\mathbf{e}_{g},\mspace{30mu} g=1,\dots,G-1,\\ \mathbf{a}_{G} & = & -\frac{1}{\pi_{G}}\boldsymbol{1}_{G-1}, \end{array} $$

(8)

where e_g denotes the g th column of the identity matrix of order G − 1 and 1_G− 1 is the (G − 1) × 1 vector having each element equal to 1. The following quantities are also required:

$$ \begin{array}{@{}rcl@{}} \textbf{o}_{gi}&=&\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}(\textbf{y}_{i}-\textbf{B}^{\prime}_{g}\textbf{x}^{*}_{i}), \end{array} $$

(9)

$$ \begin{array}{@{}rcl@{}} \textbf{O}_{gi}&=&\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}-\textbf{o}_{gi}\textbf{o}^{\prime}_{gi}, \end{array} $$

(10)

$$ \begin{array}{@{}rcl@{}} \textbf{f}_{gi} & = & \boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}(\textbf{x}_{i}-\boldsymbol{\mu}_{\textbf{X}_{g}}), \end{array} $$

(11)

$$ \begin{array}{@{}rcl@{}} \textbf{F}_{gi} & = & \boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}-\textbf{f}_{gi}\textbf{f}^{\prime}_{gi}. \end{array} $$

(12)

The explicit forms of the score vector S(𝜗) and Hessian matrix H(𝜗) for a Gaussian linear cluster-weighted model are provided in Theorems 1 and 2 (see Appendix). Proofs can be found in the document with the supplementary materials.

3 Covariance Matrix Estimation of the ML Estimator

In the ML approach, the information matrix $\mathfrak {I}(\boldsymbol {\vartheta })$ plays a crucial role, as it is used to asymptotically estimate the covariance of the ML estimator of the model parameters. Under regularity conditions and if the model is correctly specified, $\mathfrak {I}(\boldsymbol {\vartheta })$ is given either by the covariance of the score function $\mathbb {E}\left (S(\boldsymbol {\vartheta }) S(\boldsymbol {\vartheta })^{\prime }\right )$ or the negative of the expected value of the Hessian matrix $-\mathbb {E}\left (H(\boldsymbol {\vartheta })\right )$. Clearly, an analytical evaluation of the expectations required to obtain $\mathfrak {I}(\boldsymbol {\vartheta })$ under model (2) is quite complex. By exploiting some asymptotic results concerning ML estimation (White, 1982), it is possible to obtain the following asymptotic estimators of $\mathfrak {I}(\boldsymbol {\vartheta })$:

$$ \mathcal{I}_{1}=\sum\limits_{i=1}^{I}\textbf{S}_{i}(\boldsymbol{\hat{\vartheta}})\textbf{S}_{i}(\boldsymbol{\hat{\vartheta}})^{\prime}, \ \mathcal{I}_{2}=-\sum\limits_{i=1}^{I}\textbf{H}_{i}(\boldsymbol{\hat{\vartheta}}), $$

where $\textbf {S}_{i}(\boldsymbol {\hat {\vartheta }})$ and $\textbf {H}_{i}(\boldsymbol {\hat {\vartheta }})$ denote the contribution of the i th sample observation to the score function and Hessian matrix evaluated at the ML estimate $\hat {\boldsymbol {\vartheta }}$, respectively. They can be used to obtain the following asymptotic estimators of $\text {Cov}(\hat {\boldsymbol {\vartheta }})$, the covariance matrix of $\hat {\boldsymbol {\vartheta }}$:

$$ \begin{array}{@{}rcl@{}} \widehat{\text{Cov}}_{1}(\hat{\boldsymbol{\vartheta}}) & = & \mathcal{I}_{1}^{-1}, \end{array} $$

(13)

$$ \begin{array}{@{}rcl@{}} \widehat{\text{Cov}}_{2}(\hat{\boldsymbol{\vartheta}}) & = & \mathcal{I}_{2}^{-1}. \end{array} $$

(14)

Under suitable conditions (see e.g. Newey & McFadden 1994; Ritter 2015, for a general discussion and some results specifically dealing with finite mixture models, respectively), $\widehat {\text {Cov}}_{1}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ can be considered consistent estimators of $\text {Cov}(\hat {\boldsymbol {\vartheta }})$ when the model is correctly specified. By exploiting the so-called sandwich approach (see e.g. White 1980), the following robust estimator can also be employed:

$$ \widehat{\text{Cov}}_{3}(\hat{\boldsymbol{\vartheta}})=\mathcal{I}^{-1}_{2}\mathcal{I}_{1}\mathcal{I}^{-1}_{2}. $$

(15)

It is worth noting that for the existence of the estimators $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$ it is required that matrix $\mathcal {I}_{2}$ is positive definite and well conditioned. The same requirements have to be fulfilled by matrix $\mathcal {I}_{1}$ in order to guarantee that $\widehat {\text {Cov}}_{1}(\hat {\boldsymbol {\vartheta }})$ exists. With large-scale covariance matrices and small sample sizes, $\mathcal {I}_{1}$ and/or $\mathcal {I}_{2}$ could be ill-conditioned. These situations can be managed by resorting to procedures able to produce improved estimators of $\mathfrak {I}(\boldsymbol {\vartheta })$ from either $\mathcal {I}_{1}$ or $\mathcal {I}_{2}$. For example, the algorithm by Higham (1988) computes the nearest positive definite matrix of a given symmetric matrix by adjusting its eigenvalues. Other approaches which exploit techniques of variance reduction could also be adopted (see e.g. Schäfer and Strimmer 2005).

4 Experimental Results from Simulated Datasets

4.1 Numerical Study of the Properties of the Proposed Estimators

In order to evaluate the properties of $\widehat {\text {Cov}}_{1}(\hat {\boldsymbol {\vartheta }})$, $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$ in comparison with the estimators based on the parametric bootstrap and the approach implemented in flexCWM, five Monte Carlo studies have been performed. In the first study, the artificial datasets have been generated under a model defined by Eqs. 2–3 where G = 2, q = 1 and p = 2. As far as the model parameters are concerned, the following values have been employed: π₁ = 0.7, π₂ = 0.3, $\boldsymbol {\Sigma }_{\textbf {Y}_{1}}=1.5$, $\boldsymbol {\Sigma }_{\textbf {Y}_{2}}=1$, $\textbf {B}^{\prime }_{1}=(5,2,2)$, $\textbf {B}^{\prime }_{2}=(1,-2,-2)$, $\boldsymbol {\mu }^{\prime }_{\textbf {X}_{1}}=(-2,-2)$, $\boldsymbol {\mu }^{\prime }_{\textbf {X}_{2}}=(2,2)$, $\boldsymbol {\Sigma }_{\textbf {X}_{1}}=\left (\begin {smallmatrix} 1.0 & 0.2\\ 0.2 &1.0 \end {smallmatrix}\right )$, $\boldsymbol {\Sigma }_{\textbf {X}_{2}}=\left (\begin {smallmatrix} 1.0 & 0.4\\ 0.4 &1.0 \end {smallmatrix}\right )$. Such values have been chosen so as to produce two well-separated groups of observations (see the upper panel of Fig. 1, with the pairwise scatterplots of the variables X₁, X₂ and Y₁ for a sample of size I = 500 generated as just described). In this way, problems of label switching across simulations are less likely to occur. Furthermore, the ML estimates of 𝜗 are expected to be accurate enough to let the analysis be focused on the different ways of estimating the standard error of $\hat {\boldsymbol {\vartheta }}$. Using these parameter values, R = 10,000 datasets (of size I) have been generated as follows:

• 1. For the r th dataset (r = 1, … , R), a sample of size I is drawn from the standard p-dimensional normal distribution; this gives the vectors 𝜖_1r, … , 𝜖_Ir;

• 2. For the r th dataset (r = 1, … , R), a sample of size I is drawn from the standard q-dimensional normal distribution; this gives the vectors η_1r, … , η_Ir;

• 3. For the r th dataset (r = 1, … , R), a sample of size I is drawn from the Bernoulli distribution with parameter π₁; this produces the 0-1 vector $\mathbf {z}_{r}=(z_{1r}, \ldots , z_{Ir})^{\prime }$;

• 4. For the i th observation (i = 1, … , I) of the r th dataset, vectors x_ir and y_ir are obtained as follows:

  $$ \begin{array}{@{}rcl@{}} \mathbf{x}_{ir} & = & \boldsymbol{\mu}_{\textbf{X}_{1}} +\textbf{A}_{\textbf{X}_{1}} \boldsymbol{\epsilon}_{ir}, \ \ \mathbf{y}_{ir} = \textbf{B}^{\prime}_{1} \mathbf{x}_{ir} + \textbf{A}_{\textbf{Y}_{1}} \boldsymbol{\eta}_{ir} \ \ \text{if \ } z_{ir}=1,\\ \mathbf{x}_{ir} & = & \boldsymbol{\mu}_{\textbf{X}_{2}} +\textbf{A}_{\textbf{X}_{2}} \boldsymbol{\epsilon}_{ir}, \ \ \mathbf{y}_{ir} = \textbf{B}^{\prime}_{2} \mathbf{x}_{ir} + \textbf{A}_{\textbf{Y}_{2}} \boldsymbol{\eta}_{ir} \ \ \text{if } \ z_{ir}=0, \end{array} $$

  where $\textbf {A}_{\textbf {X}_{g}}$ and $\textbf {A}_{\textbf {Y}_{g}}$ are matrices obtained from the spectral decompositions of $\boldsymbol {\Sigma }_{\textbf {X}_{g}}$ and $\boldsymbol {\Sigma }_{\textbf {Y}_{g}}$, respectively. Such matrices are constructed such that $\textbf {A}_{\textbf {X}_{g}} \textbf {A}^{\prime }_{\textbf {X}_{g}}= \boldsymbol {\Sigma }_{\textbf {X}_{g}}$, $\textbf {A}_{\textbf {Y}_{g}} \textbf {A}^{\prime }_{\textbf {Y}_{g}}= \boldsymbol {\Sigma }_{\textbf {Y}_{g}}$, for g = 1,2.

In the second study, the datasets have been obtained through the same procedure used in the first one except from the computation of vectors 𝜖_ir and η_ir. Namely, a sample of size I ⋅ p is drawn from the uniform distribution in the interval (0,1) for the r th dataset (r = 1, … , R); this produces a vector $\boldsymbol {\epsilon }^{*}_{r}$, whose elements are transformed as follows: $\epsilon _{jr}=\sqrt {12}(\epsilon _{jr}^{*}-0.5), \ j=1, \ldots , I \cdot p;$ the vector 𝜖_r resulting from this transformation has zero mean and unit variance; partitioning 𝜖_r into Ip −dimensional vectors leads to 𝜖_1r, … , 𝜖_Ir. The same process has been applied to obtain vectors η_1r, … , η_Ir. The second panel of Fig. 1 provides the pairwise scatterplots of X₁, X₂ and Y₁ for a sample of size I = 500 from this second study.

Fig. 1

Pairwise scatterplots of X₁, X₂ and Y₁ for four samples of size I = 500 generated in the first four studies. Upper and lower panels refer to the first and fourth studies, respectively; intermediate panels refer to the second and third ones. Black circles and red triangles correspond to g = 1 and g = 2, respectively

In the third and fourth studies, the datasets have been generated as in the first and second studies, respectively, but using the following model parameters: $\boldsymbol {\mu }_{\textbf {X}_{1}} = (-1,-1)^{\prime }$, $\boldsymbol {\mu }_{\textbf {X}_{2}} = (1,1)^{\prime }$. This change in the values of $\boldsymbol {\mu }_{\textbf {X}_{1}}$ and $\boldsymbol {\mu }_{\textbf {X}_{2}}$ leads to overlapping groups of observations (see the pairwise scatterplots of X₁, X₂ and Y₁ for samples of size I = 500 in third and fourth panels of Fig. 1). The total number of model parameters in the first four studies is 19.

The fifth study has been carried out with the same settings of the first study but with p = 8 explanatory variables. The model parameters pertaining to X which have been employed to generate the datasets are as follows: $\boldsymbol {\mu }_{\textbf {X}_{1}} = -2 \cdot \textbf {1}_{8}$, $\boldsymbol {\mu }_{\textbf {X}_{2}} = 2 \cdot \textbf {1}_{8}$, V (X_j|Ω_g) = 1 ∀j for g = 1,2, $\text {Cov}(X_{j},X_{h}|\mathbf {\Omega }_{1})=1 - \frac {|j-h|}{8} \ \forall j \neq h$, $\text {Cov}(X_{j},X_{h}|\mathbf {\Omega }_{2})=1 - \frac {|j-h|}{4} \ \forall j \neq h$. As far as the effects of the regressors on Y₁ are concerned, they have been set as follows: $\textbf {B}_{1}=(5,\boldsymbol {\mu }^{\prime }_{\textbf {X}_{2}})^{\prime }$, $\textbf {B}_{2}=(1,\boldsymbol {\mu }^{\prime }_{\textbf {X}_{1}})^{\prime }$. In this latter study, the total number of model parameters is 109.

In all studies, Monte Carlo experiments have been performed with two different sample sizes: I = 250,500 in the first four studies, I = 300,500 in the last study. In all experiments, it has been assumed that the r th dataset {(x_1r,y_1r), … , (x_Ir,y_Ir)} is generated from a model defined by Eqs. 2–3 with G = 2. Thus, the maximum likelihood estimate $\hat {\boldsymbol {\vartheta }}_{r}$ of 𝜗 has been computed for r = 1, … , R under such an assumption. Parameter estimation has been carried out through the EM algorithm as implemented in the function cwm of the package flexCWM. As far as the initialisation of the parameters is concerned, an option has been employed, which executes 5 independent runs of the k-means algorithm and picks the solution maximising the observed-data log-likelihood among these runs. The maximum number of iterations of the EM algorithm has been set equal to 400. A convergence criterion based on the Aitken acceleration has been used, with a threshold 𝜖 = 10^− 6 (for further details, see Mazza et al., 2018).

The R independent estimates of 𝜗 are used to approximate the true distribution of $\hat {\boldsymbol {\vartheta }}$ and, in particular, the true standard errors of all the elements of $\hat {\boldsymbol {\vartheta }}$. Estimates of these standard errors have been computed using the three information-based estimators and the parametric bootstrap for R₁ = 2000 datasets obtained as described above. For each bootstrap estimate, 100 bootstrap samples have been employed. For the ML estimates of the model intercepts and regression coefficients, the standard errors estimated by the function cwm of the package flexCWM using the approach illustrated in the introduction have been included in the comparison. The performances of these strategies have been evaluated on the basis of an estimate of their biases and root mean squared errors (RMSE). A comparative evaluation of such approaches has been carried out also through the coverage probabilities (CP) of 90% and 95% confidence intervals based on the examined standard errors’ estimates and the standard normal quantiles. In this latter comparison, the attention is focused on the expected mean values of the regressors (i.e. $\boldsymbol {\mu }_{\textbf {X}_{1}}$ and $\boldsymbol {\mu }_{\textbf {X}_{2}}$) and the regression coefficients (all the entries in the first column of B₁ and B₂ except the first one).

Tables 1, 2, 3 and 4 contain the biases and RMSEs for the first four Monte Carlo studies with samples of size 250. The same information for the last study and the sample size I = 300 can be found in Table 5. The corresponding values for the CPs are summarised in Tables 6, 7, 8, 9 and 10. In all the tables, the results obtained using the function cwm of the package flexCWM, the bootstrap and the estimators defined in Eqs. 13–15 are denoted as cwm, Boot, C₁, C₂ and C₃, respectively. From now on, B_g[j,k] is used to denote the element on the j-th row and k-th column of matrix B_g; μ_g[j] represents the j-th element of μ_g. Due to the large number of parameters pertaining to the regressors in the fifth study, Table 5 provides the mean values of the biases and RMSEs of the ML estimates over the elements of each of the following vectors of model parameters: $\boldsymbol {\mu }_{\textbf {X}_{g}}$, $\mathrm {v}(\boldsymbol {\Sigma }_{\textbf {X}_{g}})$, B_g[− 1,1] for g = 1,2, where B_g[− 1,1] is the vector obtained by dropping the first element from the first column of B_g. Thus, B_g[− 1,1] comprises the regression coefficients of the p covariates on Y₁ given Ω_g. In a similar way, Table 10 contains the mean values of the CPs of 90% and 95% confidence intervals over $\boldsymbol {\mu }_{\textbf {X}_{g}}$ and B_g[− 1,1] for g = 1, 2.

Table 1 Biases and root mean squared errors of the examined standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the first study, I = 250

Table 2 Biases and root mean squared errors of the examined standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the second study, I = 250

Table 3 Biases and root mean squared errors of the examined standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the third study, I = 250

Table 4 Biases and root mean squared errors of the examined standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the fourth study, I = 250

Table 5 Biases and root mean squared errors of the examined standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the fifth study, I = 300

Table 6 Coverage probability of the 90% and 95% confidence intervals for $\boldsymbol {\mu }_{\textbf {X}_{g}}$, B_g[2,1] and B_g[3,1] based on the standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the first study, I = 250

Table 7 Coverage probability of the 90% and 95% confidence intervals for $\boldsymbol {\mu }_{\textbf {X}_{g}}$, B_g[2,1] and B_g[3,1] based on the standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the second study, I = 250

Table 8 Coverage probability of the 90% and 95% confidence intervals for $\boldsymbol {\mu }_{\textbf {X}_{g}}$, B_g[2,1] and B_g[3,1] based on the standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the third study, I = 250

Table 9 Coverage probability of the 90% and 95% confidence intervals for $\boldsymbol {\mu }_{\textbf {X}_{g}}$, B_g[2,1] and B_g[3,1] based on the standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the fourth study, I = 250

Table 10 Coverage probability of the 90% and 95% confidence intervals for $\boldsymbol {\mu }_{\textbf {X}_{g}}$ and B_g[− 1,1] based on the standard error estimators of $\hat {\boldsymbol {\vartheta }}$ in the fifth study, I = 300

Under the experimental conditions considered in the first Monte Carlo study, biases are generally small for all the estimated standard errors (see Table 1). The overall best performance in terms of accuracy seems to be achieved by means of the estimator $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$. The bootstrap approach appears to provide the most precise estimates of the standard errors of $\hat {\boldsymbol {\Sigma }}_{\textbf {Y}_{g}}$. The sandwich method is slightly more accurate than the bootstrap approach in estimating the standard errors of the ML estimates of the expected values of the regressors; the opposite result holds true when dealing with the estimation of the standard errors of $\hat {\boldsymbol {\Sigma }}_{\textbf {X}_{g}}$. The highest root mean square errors are mostly obtained using either the function cwm of the package flexCWM or the estimator $\widehat {\text {Cov}}_{1}(\hat {\boldsymbol {\vartheta }})$, which are therefore not recommended. These results confirm both the best performance of an estimator based on the Hessian and the poor performance of an estimator based on the gradient vector under correctly specified models registered in a study dealing with multivariate normal mixture models (Boldea & Magnus, 2009). It is also worth noting that the accuracy of the approach implemented in flexCWM sharply deteriorates when the ML estimates of the intercept and regression coefficients of the second group (B₂) are considered. As far as the effective confidence levels for the parameters $\boldsymbol {\mu }_{\textbf {X}_{g}}$, B_g[2,1] and B_g[3,1] are concerned (Table 6), the obtained results are generally similar to one another and quite close to the nominal confidence levels for all the examined methods except the one implemented in flexCWM. With this latter method, the effective confidence levels for the regression coefficients clearly deviate from the nominal ones, especially in the second group of observations. All these results have been employed to run tests for the hypotheses of equality between effective and nominal confidence levels. This task has been carried out through asymptotic two-tailed normal tests for a proportion at a 0.00125 significance level (the Bonferroni correction 0.01/8 has been adopted to account for multiple tests performed for each estimation method and each nominal confidence level). All the effective CPs of the confidence intervals for the model regression coefficients obtained using both the estimator based on the gradient and the approach implemented in flexCWM appear to be significantly different from the corresponding nominal ones (see the entries in italics in Table 6). As far as the results from the bootstrap-based and Hessian-based estimators are concerned, the null hypothesis of equality between effective and nominal confidence levels should be rejected for two regression coefficients at both examined confidence levels; the same null hypothesis has to be rejected for only one regression coefficient when using the the sandwich approach.

In the second Monte Carlo study, a substantial increase in the biases of the estimated standard errors of $\hat {\boldsymbol {\Sigma }}_{\textbf {X}_{g}}$ and $\hat {\boldsymbol {\Sigma }}_{\textbf {Y}_{g}}$ has been registered with all the examined estimators except for the sandwich method (Table 2); this latter method is also the most accurate. Using a Gaussian cluster-weighted model for the analysis of datasets generated under a uniform cluster-weighted model seems to have a little impact on the confidence intervals for both the expected values of the regressors and the regression coefficients (Table 7).

When the data are obtained from two overlapping groups of observations drawn from Gaussian distributions (third study), the resulting biases and RMSEs (see Table 3) are quite similar to the ones from the first study; the main effect of the reduction in the separation of the two groups is a general slight increase in the RMSEs with all the examined methods. The best accuracy is still achieved by the estimator $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ for all model parameters, with the exception of $\hat {\boldsymbol {\Sigma }}_{\textbf {Y}_{g}}$, whose standard errors are more accurately estimated by the bootstrap approach. As far as the effect of this reduction on the effective CPs of the confidence intervals is concerned (Table 8), the most remarkable result is an increase in the gap between nominal and effective CPs associated with the use of the approach implemented in flexCWM for the regression coefficients in the second group of observations.

When the two overlapping groups of observations are generated from the uniform distribution (fourth study), both biases and RMSEs of $\hat {\textbf {B}}_{1}$ results remarkably increased with all the examined methods (see Table 4). However, it is worth noting that the lowest of such increases has always been associated with the use of $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$, which is also the estimator with the best accuracy for the ML estimate of variances and covariances of the regressors in both groups and the majority of the model parameter estimates. Furthermore, the sandwich estimator shows the best performance in terms of effective CPs that are not significantly different from the nominal ones (Table 9).

In the presence of datasets generated under a Gaussian cluster-weighted model with p = 8 regressors (fifth study), the most remarkable effects of a larger number of covariates on the performance of the examined estimators with samples of size I = 300 appear to be (see Table 5 in comparison with Table 1) a sharp decrease in the accuracy of the estimates of the standard errors produced by the method based on the gradient for all the parameters of the second group of observations and a deterioration in the performances of the bootstrap approach in reference to all the parameters of the conditional distribution of Y given X and Ω_g, especially B_g[1,1], for g = 1,2. As far as the methods $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$ are concerned, biases and RMSEs result to be quite similar to the ones from the first study for all parameter estimates except the intercepts for both groups. Thus, the best overall accuracy is still achieved by the estimator $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$.

The results from the five studies with samples of size 500 (see Tables A–J in the separate document with the supplementary materials) are generally in line with those just described. It is worth noting that using a larger sample size leads to a reduction in the RMSEs for all the examined estimators. With datasets containing two separated groups of observations (first and second studies), all the effective CPs of the confidence intervals obtained using the sandwich approach appear to be not significantly different from the corresponding nominal ones. When overlapping groups are considered (third and fourth studies), the estimator $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$ has produced confidence intervals whose effective levels are the closest to the nominal ones. Thus, overall, the obtained results show the robustness of the sandwich method.

4.2 A Comparison with Some Estimators Under Normal Mixtures

As already mentioned in the “Introduction”, three information-based estimators of the covariance matrix of the ML estimator for finite normal mixture models were developed by Boldea and Magnus (2009): two of them are based on the gradient vector and the Hessian matrix of the incomplete log-likelihood under a normal mixture model; the third estimator exploits the sandwich approach. From now on, these three estimators will be denoted as BM₁, BM₂ and BM₃, respectively. Furthermore, it has been already highlighted that finite mixtures of Gaussian distributions and linear Gaussian cluster-weighted models define the same family of probability distributions (Ingrassia et al., 2012). Thus, it could be interesting to obtain an evaluation of the relationships between the estimators described in Section 3 and the estimators developed by Boldea and Magnus (2009). This task has been numerically performed by means of two additional simulation studies. The model employed to generate the simulated datasets is:

$$ f(x,y; \boldsymbol{\psi})=\sum\limits_{g=1}^{G} \pi_{g} \phi_{2}(x,y;\boldsymbol{\mu}_{g},\boldsymbol{\Sigma}_{g}), $$

(16)

where p = q = 1, G = 2, $\boldsymbol {\psi }=(\pi _{1}, \boldsymbol {\psi }^{\prime }_{1}, \boldsymbol {\psi }^{\prime }_{2})^{\prime }$, $\boldsymbol {\psi }_{1}=(\boldsymbol {\mu }^{\prime }_{1}, \mathrm v(\boldsymbol {\Sigma }_{1})^{\prime })^{\prime }$, $\boldsymbol {\psi }_{1}=(\boldsymbol {\mu }^{\prime }_{2}, \mathrm {v}(\boldsymbol {\Sigma }_{2})^{\prime })^{\prime }$, π₁ = 0.7, π₂ = 0.3, $\boldsymbol {\mu }^{\prime }_{1} = (0,0)$, $\boldsymbol {\mu }^{\prime }_{2} = (\epsilon ,\epsilon )$, $\boldsymbol {\Sigma }_{1}=\left (\begin {smallmatrix} 1.0 & 0.0\\ 0.0 &1.0 \end {smallmatrix}\right )$, $\boldsymbol {\Sigma }_{2}=\left (\begin {smallmatrix} 2.0 & 1.0\\ 1.0 &2.0 \end {smallmatrix}\right )$. The two studies have been carried out using 5 and 10 as values of 𝜖 so as to obtain two different levels of separation between μ₁ and μ₂. In each study, 100 datasets have been generated for each of two sample sizes: I = 100,250. The R packages mclust (Scrucca et al., 2016) and flexCWM have been employed to compute the ML estimates of ψ in model (16) with G = 2 and the ML estimates of 𝜗 in model (2) with G = 2, respectively. Furthermore, the standard errors of $\hat {\boldsymbol {\vartheta }}$ have been estimated using Eqs. 13–15. As far as $\hat {\boldsymbol {\psi }}$ is concerned, estimated standard errors have been computed according to the solutions described in Boldea and Magnus (2009).

Models (16) and (2) are characterised by the same value of π₁. Furthermore, as illustrated by Ingrassia et al. (2012), some elements in 𝜗 coincide with some elements in ψ; namely, $\boldsymbol {\mu }_{\textbf {X}_{1}}[1]=\boldsymbol {\mu }_{1}[1]$, $\boldsymbol {\Sigma }_{\textbf {X}_{1}}[1,1]=\boldsymbol {\Sigma }_{1}[1,1]$, $\boldsymbol {\mu }_{\textbf {X}_{2}}[1]=\boldsymbol {\mu }_{2}[1]$, $\boldsymbol {\Sigma }_{\textbf {X}_{2}}[1,1]=\boldsymbol {\Sigma }_{2}[1,1]$. Thus, the comparison between the estimators described in Section 3 and the estimators developed by Boldea and Magnus (2009) has been focused on the following two subvectors of model parameters: $\bar {\boldsymbol {\vartheta }}=(\pi _{1}, \boldsymbol {\mu }_{\textbf {X}_{1}}[1], \boldsymbol {\Sigma }_{\textbf {X}_{1}}[1,1], \boldsymbol {\mu }_{\textbf {X}_{2}}[1], \boldsymbol {\Sigma }_{\textbf {X}_{2}}[1,1])$, $\bar {\boldsymbol {\psi }}=(\pi _{1}, \boldsymbol {\mu }_{1}[1], \boldsymbol {\Sigma }_{1}[1,1], \boldsymbol {\mu }_{2}[1], \boldsymbol {\Sigma }_{2}[1,1])$. Let $se_{m}(\hat {\bar {\boldsymbol {\vartheta }}}_{r}[j])$ be the standard error of the j-th element of $\hat {\bar {\boldsymbol {\vartheta }}}$ computed using the estimator C_m and the r-th dataset. Furthermore, let $se_{m}(\hat {\bar {\boldsymbol {\psi }}}_{r}[j])$ be the standard error of the j-th element of $\hat {\bar {\boldsymbol {\psi }}}$ obtained from the estimator BM_m. In order to compare such estimated standard errors, the following differences have been computed: $d_{rm}(j)= se_{m}(\hat {\bar {\boldsymbol {\psi }}}_{r}[j])- se_{m}(\hat {\bar {\boldsymbol {\vartheta }}}_{r}[j])$, j = 1, … , 5, m = 1,2,3, r = 1, … , 100. The results obtained for 𝜖 = 5 and 𝜖 = 10 with samples of size I = 100 are graphically represented in Figs. 2 and 3, respectively. The distributions of the differences d_rm(j) for almost all the examined parameters appear to be centred around 0 and highly homogeneous, thus highlighting a general equivalence between the standard errors resulting from the two models. This result holds true especially for the estimators based on the Hessian matrix and the sandwich approach (m = 2, 3) when the separation between the two groups is larger (𝜖 = 10), and for the estimator based on the gradient vector (m = 1) when the separation is low (𝜖 = 5). However, it is also worth noting that with both levels of separation the differences in the standard errors of $\hat {\pi }_{1}$ computed using the two estimators based on the gradient vector show a median value slightly below 0 and a distribution with negative skewness. Similar results have been obtained with samples of size I = 250 (see Figures A and B in the supplementary material).

Fig. 2

Boxplots of the differences d_rm(j) with samples of size I = 100 and 𝜖 = 5

Fig. 3

Boxplots of the differences d_rm(j) with samples of size I = 100 and 𝜖 = 10

5 Analysing Regional Tourism Data in Italy

Similar to other studies (see e.g. Cellini & Cuccia 2013), the analysis summarised here aims at evaluating the link between tourism flows and attendance at museums and monuments, with a focus on two Italian regions: Emilia Romagna (ER) and Veneto (Ve). For both regions, three variables have been examined: tourist arrivals (denoted Arriv), tourist overnights (Overn) and visits to State museums, monuments and museum networks (Visit). Measurements for such variables are available with a monthly frequency over the period January 1999 to December 2017. The source for Visit is the website of the Italian Ministry of Cultural Heritage (http://www.statistica.beniculturali.it). Data on Arriv and Overn have been obtained from the websites of Emilia-Romagna¹ and Veneto² regional governments. Overall, the analysed dataset is composed of I = 228 monthly observations for six variables. Because of the goal of the analysis, these variables have been partitioned as follows: Y = (Visit ER, Visit Ve$)^{\prime }$, X = (Arriv ER, Overn ER, Arriv Ve, Overn Ve$)^{\prime }$. The analysed data are expressed in thousands.

Models obtained from Eqs. 2–3 have been fitted to the dataset for G from 1 to 8. To this end, a function written for the R software environment which implements an EM algorithm for the ML estimation of multivariate Gaussian cluster-weighted linear models has been employed. In order to prevent problems due to the presence of singular or nearly singular matrices during the iterations, all covariance matrices have been required to have eigenvalues greater than 10^− 20; furthermore, the ratio between the smallest and the largest eigenvalues of such matrices is required to be not lower than 10^− 10. Model parameters are initialised according to a two-step strategy. In the first step, the joint distribution of the covariates and responses is estimated using a mixture of G Gaussian models through the mclust package. This produces the required starting values for the G weights, mean vectors and covariance matrices for the predictors. In the second step, the initialisation of B_g and $\boldsymbol {\Sigma }_{\textbf {Y}_{g}}$ is obtained from the fitting of a Gaussian linear regression model to the sample of observations that have been assigned to the g th component of the mixture model estimated in the first step. The R package systemfit (Henningsen & Hamann, 2007) has been exploited to perform this task. The maximum number of iterations of the EM algorithm has been set equal to 500. A convergence criterion based on the Aitken acceleration has been used, with a threshold 𝜖 = 10^− 8.

Table 11 shows the values of the maximised log-likelihood ($l(\boldsymbol {\hat {\vartheta }})$) and the Bayesian information criterion (BIC) (Schwarz, 1978) for the eight fitted models, where $BIC = 2 l(\hat {\boldsymbol {\vartheta }}) - npar \ln (I)$, and npar denotes the number of model parameters. According to this criterion, the model with the best trade-off between the fit and complexity seems to be the one with G = 5 clusters of months. With this model, there is a perfect correspondence between some clusters and some months (see Table 12): cluster 2 only contains observations in April and May; cluster 4 only contains observations in June, July and August; cluster 5 only contains observations in January, February, November and December. As far as the remaining months are concerned, observations in September for the years 2005–2017 have been assigned to cluster 1; cluster 3 comprises the remaining observations in September together with all the observations in March and October. The obtained cluster structure clearly reflects seasonal patterns characterising tourism flows. Observations in cluster 4 (June, July and August) are characterized by the highest mean values of tourist arrivals and overnights in both regions, followed by those in cluster 1 (September 2005–2017), cluster 2 (April and May), cluster 3 (March, October and September 1999–2004) and cluster 4 (from November to February) (see the $\hat {\boldsymbol {\mu }}_{g}[j]$ in Table 13). In all clusters, Veneto is characterised by mean values of both regressors which are always higher than those of Emilia-Romagna. During the examined period of time, there seems to be heterogeneity also on the effects of the tourist arrivals and overnights on the number of visits in both regions (see the $\hat {\mathbf {B}}_{g}[j,k]$ in Table 13). Such effects do not result to be always positive. Furthermore, an increase in the tourist arrivals and overnights in one region does not necessarily have a positive impact on the number of visits to State museums, monuments and museum networks of the other region.

Table 11 Maximised log-likelihood and Bayesian information criterion of eight cluster-weighted models fitted to the tourism dataset

Table 12 Cross-classification of the observations from the tourism dataset, based on their variable time identified by month and maximum posterior probability estimated from the cluster-weighted model with G = 5

Table 13 Estimated π_g, $\boldsymbol {\mu }_{\mathbf {X}_{g}}$ and B_g of the cluster-weighted model with G = 5 fitted to the tourism dataset

Estimates of the standard errors for the parameter estimates of the selected cluster-weighted model have been computed by the boostrap approach, using 100 bootstrap samples generated from the selected model. Furthermore, $\text {Cov}(\hat {\boldsymbol {\vartheta }})$ has been estimated by resorting to Eqs. 13–15. The algorithm by Higham (1988), as implemented in the R package corpcor, has been employed to adjust the eigenvalues of $\mathcal {I}_{1}$ and $\mathcal {I}_{2}$ so as to obtain their nearest positive definite matrices. Then, the estimated standard errors have been employed to run tests for the hypotheses H₀ : B_g[j,k] = 0 for j = 2, … , p, k = 1, … , q, g = 1, … , G. Such tests have been run under an asymptotic normal distribution for the z_gjk statistics, where $z_{gjk}=\frac {\hat {\mathbf {B}}_{g}[j,k]}{se(\hat {\mathbf {B}}_{g}[j,k])}$, with $se(\hat {\mathbf {B}}_{g}[j,k])$ denoting the estimated standard error of $\hat {\mathbf {B}}_{g}[j,k]$. Table 14 summarises the results obtained using the parametric bootstrap-based estimates and the score-based estimates; the results derived from the use of the other two estimators are reported in Table 15. According to all the examined methods and using α = 0.05, the four examined regressors show a significant linear effect on the visits to State museums, monuments and museum networks in Emilia Romagna over the period 1999 to 2017 in March, October and over the period 1999 to 2004 in September (cluster 3); furthermore, in the central months of the winters from 1999 to 2017 (cluster 5), there are positive significant effects of Arriv on Visit in Emilia Romagna and of Overn on Visit in Veneto. According to the estimator based on $\mathcal {I}_{1}$, five additional regression coefficients (three within cluster 4, two within cluster 5) result to be significantly different from zero; the same conclusion is obtained from the use of $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$. The results obtained using the three methods illustrated in Section 3 suggest that tourist arrivals in Emilia Romagna have a positive significant effect on the visits to State museums, monuments and museum networks both in Emilia Romagna and in Veneto in June, July and August. Arriv ER has a significant and negative effect on Visit Ve in January, February, November and December. Furthermore, Arriv Ve seems to significantly affect Visit Ve, with a positive effect in January, February, November and December and a negative effect in June, July and August. The tests based on the estimators $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$ lead to a rejection of H₀ for further five regression coefficients, three of which concern the effect of tourist arrivals and overnights in Veneto in September for the years 2005–2017 (cluster 1). As far as cluster 2 is concerned, all regression coefficients seem to be not significantly different from 0 according to all approaches except for the effect of Overn Ve on Visit Ve, which results to be significant only when standard errors are estimated from the Hessian matrix.

Table 14 $\hat {\mathbf {B}}_{g}[j,k]$, estimated standard errors, z_gjk values and p-values obtained using the bootstrap (columns 5–7) and Eq. 13 (columns 8–10)

Table 15 $\hat {\mathbf {B}}_{g}[j,k]$, estimated standard errors, z_gjk values and p-values obtained using Eqs. 14 (columns 5–7) and 15 (columns 8–10)

By exploiting the results contained in the non-diagonal elements of matrices $\widehat {\text {Cov}}_{1}(\hat {\boldsymbol {\vartheta }})$, $\widehat {\text {Cov}}_{2}(\hat {\boldsymbol {\vartheta }})$ and $\widehat {\text {Cov}}_{3}(\hat {\boldsymbol {\vartheta }})$, it is also possible to run tests for the significance of the difference between the effects of two different predictors on the same response in a given cluster or tests for the significance of the difference between the effects of the same predictor on the same response in two different clusters. For any given pair of regression coefficients in the model, the null hypothesis can be expressed as follows: $H_{0}: \mathbf {B}_{g_{1}}[j_{1},k_{1}]=\mathbf {B}_{g_{2}}[j_{2},k_{2}]$. Three illustrative examples of hypotheses for the analysis of the tourism dataset are summarised in Tables 16 and 17, where $\hat {\delta }_{(g_{1},j_{1},k_{1})(g_{2},j_{2},k_{2})}=\hat {\mathbf {B}}_{g_{1}}[j_{1},k_{1}]-\hat {\mathbf {B}}_{g_{2}}[j_{2},k_{2}]$. These examples have been obtained from the following questions: (a) In June, July and August (cluster 4), do tourist arrivals in Emilia Romagna have a different effect on Visit ER and Visit Ve (first example)? (b) In January, February, November and December (cluster 5), is the effect of tourist arrivals in Emilia Romagna on Visit ER different from the effect of tourist arrivals in Veneto on Visit Ve (second example)? (c) Is the effect of tourist arrivals in Emilia Romagna on Visit ER in January, February, November and December (cluster 5) different from the one in March, September 1999–2004 and October (cluster 3) (third example)? For each of these illustrations, Table 17 summarises the results of the z tests run using the three estimated covariance matrices (the non-diagonal elements are not reported). For the first two examples, the compared methods lead to results which are all in favour of the null hypothesis. In the third illustration, the null hypothesis should be rejected (α = 0.05) when variances and covariances are estimated using both the Hessian and sandwich approaches.

Table 16 Values of $\hat {\delta }_{(g_{1},j_{1},k_{1})(g_{2},j_{2},k_{2})}$ for testing three examples of $H_{0}: \mathbf {B}_{g_{1}}[j_{1},k_{1}]=\mathbf {B}_{g_{2}}[j_{2},k_{2}]$ in the tourism data

Table 17 Estimated standard errors (se) of $\hat {\delta }_{(g_{1},j_{1},k_{1})(g_{2},j_{2},k_{2})}$, z values and p-values obtained using Eqs. 13 (columns 2–4), 14 (columns 5–7) and 15 (columns 8–10) for testing three examples of $H_{0}: \mathbf {B}_{g_{1}}[j_{1},k_{1}]=\mathbf {B}_{g_{2}}[j_{2},k_{2}]$ in the tourism data

6 Conclusions

Three information-based estimators of the asymptotic covariance matrix of the ML estimator under multivariate Gaussian linear cluster-weighted models have been illustrated. For their computation, formulae for the score vector and Hessian matrix of the incomplete log-likelihood have been derived. Properties of these estimators have been numerically evaluated using simulated samples in comparison with the parametric bootstrap-based estimator. For the ML estimates of the model intercepts and regression coefficients, the comparison has included an approach implemented in the package flexCWM in which estimated standard errors are computed by fitting G separate linear weighted regression models using the estimated posterior probabilities as weights. With correctly specified models, the most accurate estimator of the standard error of the ML estimator is the one based on the Hessian matrix. When Gaussian cluster-weighted models are fitted to datasets generated from a uniform distribution, the best accuracy is achieved with the sandwich estimator. Overall, the obtained results show the robustness of this latter method. Through these information-based estimators, the tasks of computing approximated confidence intervals and running tests concerning pairs of parameters can be easily carried out, as illustrated through a study aiming at evaluating the link between tourism flows and attendance at museums and monuments in two Italian regions. Asymptotic properties of the estimators introduced here could also be studied from a theoretical point of view. For example, suitable regularity conditions can be defined so as to provide a general assessment of their consistency (see for example Galimberti et al. (2020) for a similar study in the context of clusterwise regression models).

Appendix

Theorem 1

The score vector S(𝜗) for a cluster-weighted model of order G contains the following subvectors:

$$ \begin{array}{@{}rcl@{}} {\partial l(\boldsymbol{\vartheta})\over \partial\boldsymbol{\pi}}& = &\sum\limits_{i=1}^{I}\bar{\textbf{a}}_{i}, \ {\partial l(\boldsymbol{\vartheta})\over \partial\boldsymbol{\mu}_{\textbf{X}_{g}}} =\sum\limits_{i=1}^{I}\alpha_{gi}\textbf{f}_{gi} \ \forall g, \ {\partial l(\boldsymbol{\vartheta}) \over \partial \mathrm{v}(\boldsymbol{\Sigma}_{\textbf{X}_{g}})} = {1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\textbf{J}^{\prime}\text{vec}(\textbf{F}_{gi}) \ \forall g,\\ {\partial l(\boldsymbol{\vartheta}) \over \partial\text{vec}(\textbf{B}_{g})}&=&\sum\limits_{i=1}^{I}\alpha_{gi}\text{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi}) \ \forall g, \ {\partial l(\boldsymbol{\vartheta}) \over \partial \mathrm{v}(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})} = -{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\textbf{L}^{\prime}\text{vec}(\textbf{O}_{gi}) \ \forall g, \end{array} $$

with $\bar {\textbf {a}}_{i}={\sum }_{g=1}^{G} \alpha _{gi}\textbf {a}_{g}$, L and J denoting duplication matrices with dimensions $q^{2}\times {q(q+1) \over 2}$ and $p^{2}\times {p(p+1) \over 2}$, respectively.

Theorem 2

The Hessian matrix H(𝜗) for a cluster-weighted model of order G contains the following submatrices:

$$ \begin{array}{@{}rcl@{}} {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\pi}\partial\boldsymbol{\pi}^{\prime}}&=&-\sum\limits_{i=1}^{I}\bar{\textbf{a}}_{i}\bar{\textbf{a}}^{\prime}_{i}, \ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\pi}\partial(\mathrm{vec}(\textbf{B}_{g}))^{\prime}} =\sum\limits_{i=1}^{I}\alpha_{gi}(\textbf{a}_{g}-\bar{\textbf{a}}_{i})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})^{\prime} \ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\pi}\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}}))^{\prime}} &=&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}(\textbf{a}_{g}-\bar{\textbf{a}}_{i})\mathrm{vec}(\textbf{O}_{gi})^{\prime}\textbf{L} \ \forall g, \ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\pi}\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{g}}} \\ &=&\sum\limits_{i=1}^{I}\alpha_{gi}(\textbf{a}_{g}-\bar{\textbf{a}}_{i})\textbf{f}_{gi}^{\prime} \ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\pi}\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}}))^{\prime}} & =&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}(\textbf{a}_{g}-\bar{\textbf{a}}_{i})\mathrm{vec}(\textbf{F}_{gi})^{\prime}\textbf{J} \ \forall g,\\ \partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm{vec}(\textbf{B}_{g}))^{\prime} &=&-\sum\limits_{i=1}^{I}\alpha_{gi}[(\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}\otimes(\textbf{x}^{*}_{i}\textbf{x}^{\prime *}_{i}))\\ &&\left. -(1-\alpha_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})^{\prime}\right] \ \forall g,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm{vec}(\textbf{B}_{j}))^{\prime}}&=&-\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{ji})^{\prime} \ \forall g\neq j,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm{v}(\boldsymbol{\Sigma}_{\textbf{Y}_{g}}))^{\prime}}&=&-\sum\limits_{i=1}^{I}\alpha_{gi}\left[\left( \boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}\otimes(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\right)\right.\\ &&\left. +{1 \over 2}(1-\alpha_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{O}_{gi})^{\prime}\right]\textbf{L} \ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{j}}))^{\prime}}&=&{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{O}_{ji})^{\prime}\textbf{L} \ \forall g\neq j,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{g}}}&=&\sum\limits_{i=1}^{I}\alpha_{gi}(1-\alpha_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\textbf{f}^{\prime}_{gi} \ \forall g, \end{array} $$

$$ \begin{array}{@{}rcl@{}} {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{j}}} & =&-\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\textbf{f}^{\prime}_{ji} \ \forall g\neq j,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}}))^{\prime}}&=&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}(1-\alpha_{gi})\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{F}_{gi})^{\prime}\textbf{J} \ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm{vec}(\textbf{B}_{g})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{j}}))^{\prime}}&=&{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\mathrm{vec}(\textbf{x}^{*}_{i}\textbf{o}^{\prime}_{gi})\mathrm{vec}(\textbf{F}_{ji})^{\prime}\textbf{J} \ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}}))^{\prime}}&=&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\textbf{L}^{\prime}\left[(\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}-2\textbf{O}_{gi})^{\prime}\otimes\boldsymbol{\Sigma}^{-1}_{\textbf{Y}_{g}}\right.\\ &&\left. -{1 \over 2}(1-\alpha_{gi})\mathrm{vec}(\textbf{O}_{gi})\mathrm{vec}(\textbf{O}_{gi})^{\prime}\right]\textbf{L} \ \forall g,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{j}}))^{\prime}}&=&-{1 \over 4}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{L}^{\prime}\mathrm{vec}(\textbf{O}_{gi})\mathrm{vec}(\textbf{O}_{ji})^{\prime}\textbf{L} \ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{g}}}&=&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}(1-\alpha_{gi})\textbf{L}^{\prime}\mathrm{vec}(\textbf{O}_{gi})\textbf{f}^{\prime}_{gi} \ \forall g,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{j}}}&=&{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{L}^{\prime}\mathrm{vec}(\textbf{O}_{gi})\textbf{f}^{\prime}_{ji} \ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}}))^{\prime}}&=&{1 \over 4}\sum\limits_{i=1}^{I}\alpha_{gi}(1-\alpha_{gi})\textbf{L}^{\prime}\mathrm{vec}(\textbf{O}_{gi})\mathrm{vec}(\textbf{F}_{gi})^{\prime}\textbf{J}\ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{Y}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{j}}))^{\prime}}&=&-{1 \over 4}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{L}^{\prime}\mathrm{vec}(\textbf{O}_{gi})\mathrm{vec}(\textbf{F}_{ji})^{\prime}\textbf{J}\ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\mu}_{\textbf{X}_{g}}\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{g}}}&=&-\sum\limits_{i=1}^{I}\alpha_{gi}\left[\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}-(1-\alpha_{gi})\textbf{f}_{gi}\textbf{f}^{\prime}_{gi}\right]\ \forall g,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\mu}_{\textbf{X}_{g}}\partial\boldsymbol{\mu}^{\prime}_{\textbf{X}_{j}}}&=&-\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{f}_{gi}\textbf{f}^{\prime}_{ji}\ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\mu}_{\textbf{X}_{g}}\partial(\mathrm{v}(\boldsymbol{\Sigma}_{\textbf{X}_{g}}))^{\prime}}&=&-\sum\limits_{i=1}^{I}\alpha_{gi}\left[(\textbf{f}^{\prime}_{gi}\otimes\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}})+{1 \over 2}(1-\alpha_{gi})\textbf{f}_{gi}\mathrm{vec}(\textbf{F}_{gi})^{\prime}\right]\textbf{J}\ \forall g, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\boldsymbol{\mu}_{\textbf{X}_{g}}\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{j}}))^{\prime}}&=&{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{f}_{gi}\mathrm{vec}(\textbf{F}_{ji})^{\prime}\textbf{J}\ \forall g\neq j, \\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}}))^{\prime}}&=&-{1 \over 2}\sum\limits_{i=1}^{I}\alpha_{gi}\textbf{J}^{\prime}\left[(\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}-2\textbf{F}_{gi})^{\prime}\otimes\boldsymbol{\Sigma}^{-1}_{\textbf{X}_{g}}\right.\\ &&\left. -{1 \over 2}(1-\alpha_{gi})\mathrm{vec}(\textbf{F}_{gi})\mathrm{vec}(\textbf{F}_{gi})^{\prime}\right]\textbf{J}\ \forall g,\\ {\partial^{2} l(\boldsymbol{\vartheta}) \over \partial\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{g}})\partial(\mathrm v(\boldsymbol{\Sigma}_{\textbf{X}_{j}}))^{\prime}}&=&-{1 \over 4}\sum\limits_{i=1}^{I}\alpha_{gi}\alpha_{ji}\textbf{J}^{\prime}\mathrm{vec}(\textbf{F}_{gi})\mathrm{vec}(\textbf{F}_{ji})^{\prime}\textbf{J}\ \forall g\neq j. \end{array} $$