Identifiability of Gaussian linear structural equation models with homogeneous and heterogeneous error variances

Abstract

In this work, we consider the identifiability assumption of Gaussian linear structural equation models (SEMs) in which each variable is determined by a linear function of its parents plus normally distributed error. It has been shown that linear Gaussian structural equation models are fully identifiable if all error variances are the same or known. Hence, this work proves the identifiability of Gaussian SEMs with both homogeneous and heterogeneous unknown error variances. Our new identifiability assumption exploits not only error variances, but edge weights; hence, it is strictly milder than prior work on the identifiability result. We further provide a structure learning algorithm that is statistically consistent and computationally feasible, based on our new assumption. The proposed algorithm assumes that all relevant variables are observed, while it does not assume causal minimality and faithfulness. We verify our theoretical findings through simulations and real multivariate data, and compare our algorithm to state-of-the-art PC, GES and GDS algorithms.

Appendix

1.1 Proof for Theorem 2

Proof

Without loss of generality, we assume that the true ordering $\pi = (\pi _1,...,\pi _p)$ is unique. For simplicity, we define $X_{1:j} = (X_{\pi _1},X_{\pi _2},\ldots ,X_{\pi _j})$ and $X_{1:0} = \emptyset$. We restate the identifiability assumption of Gaussian SEMs.

Assumption 5

(Identifiability) For any node $m \in V$, let $j = \pi _{m}$ and $k \in V {\setminus } \text{ Nd }(j)$. The conditional variance of $X_j$ given its parents is smaller than the conditional variance of $X_k$ given the variables before j in ordering $\pi$:

$$\begin{aligned} \sigma _j^2 < \sigma _k^2 + \mathbb {E}( \text{ Var }( \mathbb {E}(X_k \mid X_{\text{ Pa }(k)} ) \mid X_{1:(j-1)} ) ). \end{aligned}$$

Now, we prove identifiability of Gaussian SEMs using mathematical induction.

Step (1) By Assumption 5, for any node $k \in V {\setminus } \{\pi _1\}$, we have

$$\begin{aligned} \text{ Var }(X_{\pi _1}) = \sigma _{\pi _1}^2 < \sigma _k^2 + \text{ Var }( \mathbb {E}(X_k \mid X_{\text{ Pa }(k)} ) ) = \text{ Var }(X_k). \end{aligned}$$

Therefore, $\pi _1$ can be correctly identified.

Step (m−1) For the $(m-1)$th element of the ordering, assume that the first $m-1$ elements of the ordering and their parents are correctly estimated.

Step (m) Now, we consider the mth element of the causal ordering and its parents. By Assumption 5, for $k \in \{ \pi _{m+1},\cdots , \pi _{p} \}$,

$$\begin{aligned} \begin{aligned} \mathbb {E}( \text {Var}( X_{\pi _m} \mid X_{1:(m-1)} ) )=&{} \ \sigma _{\pi _m}^2 < \sigma _k^2 +\mathbb {E}( \text {Var}( \mathbb {E}(X_k \mid X_{\text { Pa }(k)} ) \mid X_{1:(m-1)}) ) \\=&{} \ \mathbb {E}( \text {Var}(X_{k} \mid X_{1:(m-1)}) ). \end{aligned} \end{aligned}$$

Hence, we can choose the true $m^{th}$ element of the ordering $\pi _m$.

In terms of the parents search, it is clear that conditional independence relations naturally encoded by the factorization (1) and imply causal minimality (see details in Pearl 2014; Peters and Bühlmann 2014). In our settings, causal minimality states that for any node $j \in V$ and one of its parents $k \in \text{ Pa }(j)$,

Therefore, we can choose the correct parents of $\pi _m$. By mathematical induction, this completes the proof. $\square$

1.2 Identifiability for three-node chain graph

Consider a Gaussian SEM, $X_1 \rightarrow X_2 \rightarrow X_3$, where $X_1 = \epsilon _1$, $X_2 = \beta _1 X_1 + \epsilon _1$, and $X_3 =\beta _2 X_2 + \epsilon _3$ with $\epsilon _j \sim N(0, \sigma _j^2)$ for all $j \in \{1,2,3\}$. Then the first element of the ordering can be determined by comparing the variances of nodes:

$$\begin{aligned} \text{ Var }( X_2 )&= \mathbb {E}( \text{ Var }( X_2 \mid X_1) ) + \text{ Var }( \mathbb {E}( X_2 \mid X_1) ) = \sigma _2^2 + \beta _1^2 \sigma _1^2> \sigma _1^2 = \text{ Var }(X_1) \\ \text{ Var }( X_3 )&= \mathbb {E}( \text{ Var }( X_3 \mid X_2) ) + \text{ Var }( \mathbb {E}( X_3 \mid X_2) ) \\&= \sigma _3^2 + \beta _2^2 \sigma _2^2 + \beta _2^2 \beta _1^2 \sigma _1^2 > \sigma _1^2 = \text{ Var }(X_1). \end{aligned}$$

as long as $\sigma _2^2/ \sigma _1^2 > (1 - \beta _1^2)$ and $\sigma _3^2/ \sigma _1^2 > (1 - \beta _2^2)$.

The second element of the ordering can also be recovered by comparing the expectation of the conditional variance of the remaining variables given the estimated first element of the ordering:

$$\begin{aligned} \mathbb {E}( \text{ Var }( X_3 \mid X_1) )&= \mathbb {E}( \mathbb {E}( \text{ Var }( X_3 \mid X_2) \mid X_1) ) + \mathbb {E}( \text{ Var }( \mathbb {E}( X_3 \mid X_2) \mid X_1 ) )\\&= \sigma _3^2 + \beta _2^2 \sigma _2^2 > \sigma _2^2 = \mathbb {E}( \text{ Var }( X_2 \mid X_1 ) ). \end{aligned}$$

as long as $\sigma _3^2/ \sigma _2^2 > (1 - \beta _2^2)$.

Under the minimality and the Markov condition, we also have the following (conditional) dependence relations: Therefore, the true graph can be recovered.