Bayesian network structural learning from complex survey data: a resampling based approach

Abstract

Nowadays there is increasing availability of good quality official statistics data. The construction of multivariate statistical models possibly leading to the identification of causal relationships is of interest. In this context Bayesian networks play an important role. A crucial step consists in learning the structure of a Bayesian network. One of the most widely used procedures is the PC algorithm consisting in carrying out several independence tests on the available data set and in building a Bayesian network according to the tests results. The PC algorithm is based on the irremissible assumption that data are independent and identically distributed. Unfortunately, official statistics data are generally collected through complex sampling designs, then the aforementioned assumption is not met. In such a context the PC algorithm fails in learning the structure. To avoid this, the sample selection must be taken into account in the structural learning process. In this paper, a modified version of the PC algorithm is proposed for inferring causal structure from complex survey data. It is based on resampling techniques for finite populations. A simulation experiment showing the robustness with respect to departures from the assumptions and the good performance of the proposed algorithm is carried out.

Appendix

Proof of Proposition 1

Here the main lines showing how Proposition 1 descends from Proposition 1 in Conti et al. (2018) are provided.

Define the cumulative distribution functions (c.d.f.s),

$$\begin{aligned} F^{hk}=\sum _{u=1}^{h}\sum _{v=1}^{k}p^{uv}, \; h=1,\dots , H, \; \; k=1,\dots , K \end{aligned}$$

the empirical c.d.f.s

$$\begin{aligned} \widehat{F}^{hk}=\sum _{u=1}^{h}\sum _{v=1}^{k}\widehat{p}^{uv}, \; h=1,\dots , H, \; \; k=1,\dots , K \end{aligned}$$

where $\widehat{p}^{uv}$ are estimated using the classical Hájek estimators as in (6), and the corresponding random vectors (with elements in lexicographic order)

$$\begin{aligned} \varvec{F}^{HK}=\left[ \begin{array}{c} F^{11} \\ F^{12} \\ \dots \\ F^{HK} \\ \end{array} \right] \;\;\;\;\; \widehat{\varvec{F}}^{HK}=\left[ \begin{array}{c} \widehat{F}^{11} \\ \widehat{F}^{12} \\ \dots \\ \widehat{F}^{HK} \\ \end{array} \right] \end{aligned}$$

and

$$\begin{aligned} \varvec{T}^{HK}=\sqrt{n}\left( \widehat{\varvec{F}}^{HK} - \varvec{F}^{HK} \right) . \end{aligned}$$

Note that the random vector $\varvec{T}^{HK}$ lies on a hyperplane of dimension $HK-1$, due to the relationships $\widehat{F}^{HK} = F^{HK}=1$ (then the last component of $\varvec{T}^{HK}$ is 0).

From Conti et al. (2018) it follows that $\varvec{T}^{HK}$ tends in distribution, as $n, N \rightarrow \infty$, to a degenerate multivariate Normal r.v. with mean vector $\varvec{0}^{HK}$ (with HK components) and covariance matrix $\varvec{\Omega }^{HK}$. Since the limiting distribution is degenerate (it lies in a sub-space of dimension $HK-1$), the matrix $\varvec{\Omega }^{HK}$ is degenerate. However, this does not affect neither its definition, nor its basic properties (cfr. Rao (1973), pp. 184-185). In addition, again from Conti et al. (2018), the relationship

$$\begin{aligned} \varvec{\Omega }^{HK} = \varvec{\Omega }^{HK}_1 + f\varvec{\Omega }^{HK}_2 \end{aligned}$$

(25)

holds, where $\varvec{\Omega }^{HK}_1$ is the part of the total variability due to sampling design, $\varvec{\Omega }^{HK}_2$ is the part of variability due to superpopulation model, and f is the limiting sampling fraction.

Define now

$$\begin{aligned} \varvec{W}^{HK}=\sqrt{n} \left[ \begin{array}{c} \widehat{p}^{11} - p^{11} \\ \widehat{p}^{12} - p^{12} \\ \dots \\ \widehat{p}^{HK} - p^{HK} \\ \end{array} \right] \end{aligned}$$

From

$$\begin{aligned} p^{hk}= & {} F^{hk}-F^{h \; k-1}-F^{h-1 \; k}+F^{h-1 \; k-1} \\ \widehat{p}^{hk}= & {} \widehat{F}^{hk}-\widehat{F}^{h \; k-1}-\widehat{F}^{h-1 \; k}+\widehat{F}^{h-1 \; k-1} \end{aligned}$$

where $h=1,\dots , H$, $k=1,\dots , K$, it is immediate to verify that the map

$$\begin{aligned} \varvec{T}^{HK} \mapsto \varvec{W}^{HK} \end{aligned}$$

(26)

is linear, and hence continuous. From the continuous mapping theorem, $\varvec{W}^{HK}$ tends in distribution to a degenerate multivariate Normal distribution with mean $\varvec{0}^{HK}$ and (singular) covariance matrix $\varvec{\Sigma }^{HK}$. In view of (25), the matrix $\varvec{\Sigma }^{HK}$ can be decomposed as

$$\begin{aligned} \varvec{\Sigma }^{HK} = \varvec{\Sigma }^{HK}_1 + f\varvec{\Sigma }^{HK}_2. \end{aligned}$$

(27)

Next, define

$$\begin{aligned} \varvec{W}^{H}=\sqrt{n} \left[ \begin{array}{c} \widehat{p}^{1.} - p^{1.} \\ \widehat{p}^{2.} - p^{2.} \\ \dots \\ \widehat{p}^{H.} - p^{H.} \\ \end{array} \right] =\sqrt{n} \left[ \begin{array}{c} \sum _{k=1}^{K}(\widehat{p}^{1k} - p^{1k}) \\ \sum _{k=1}^{K}(\widehat{p}^{2k} - p^{2k}) \\ \dots \\ \sum _{k=1}^{K}(\widehat{p}^{Hk} - p^{Hk}) \\ \end{array} \right] . \end{aligned}$$

The map $\varvec{W}^{HK} \mapsto \varvec{W}^{H}$ is linear, and hence continuous. From the continuous mapping theorem, it follows that $\varvec{W}^{H}$ tends in distribution to a (degenerate) multivariate Normal distribution, with mean vector $\varvec{0}^{H}$ and covariance matrix $\varvec{\Sigma }^H$. From (27), it also follows that the following decomposition holds:

$$\begin{aligned} \varvec{\Sigma }^{H} = \varvec{\Sigma }^{H}_1 + f\varvec{\Sigma }^{H}_2. \end{aligned}$$

Finally, using exactly the same arguments as above, it is not difficult to see that the degenerate r.v.

$$\begin{aligned} \varvec{W}^{K}=\sqrt{n} \left[ \begin{array}{c} \widehat{p}^{.1} - p^{.1} \\ \widehat{p}^{.2} - p^{.2} \\ \dots \\ \widehat{p}^{.K} - p^{.K} \\ \end{array} \right] \end{aligned}$$

tends in distribution to a (degenerate) multivariate Normal distribution, with mean vector $\varvec{0}^{K}$ and covariance matrix $\varvec{\Sigma }^K$. Again, the decomposition

$$\begin{aligned} \varvec{\Sigma }^{H} = \varvec{\Sigma }^{H}_1 + f\varvec{\Sigma }^{H}_2 \end{aligned}$$

holds.$\square$

In order to prove Propositions 2, 3, define the vectors $\widetilde{{\varvec{p}}}^{HK}$ and $\overline{{\varvec{p}}}^{HK}$ of length HK

$$\begin{aligned} \widetilde{{\varvec{p}}}^{HK}= \left[ \begin{array}{c} \widehat{p}^{1.}\widehat{p}^{.1}\\ \widehat{p}^{1.}\widehat{p}^{.2}\\ \dots \\ \widehat{p}^{1.}\widehat{p}^{.K}\\ \widehat{p}^{2.}\widehat{p}^{.1}\\ \widehat{p}^{2.}\widehat{p}^{.2}\\ \dots \\ \widehat{p}^{2.}\widehat{p}^{.K}\\ \dots \\ \widehat{p}^{H.}\widehat{p}^{.1}\\ \widehat{p}^{H.}\widehat{p}^{.2}\\ \dots \\ \widehat{p}^{H.}\widehat{p}^{.K}\\ \end{array} \right] \quad \overline{{\varvec{p}}}^{HK}=\left[ \begin{array}{c} p^{1.}p^{.1}\\ p^{1.}p^{.2}\\ \dots \\ p^{1.}p^{.K}\\ p^{2.}p^{.1}\\ p^{2.}p^{.2}\\ \dots \\ p^{2.}p^{.K}\\ \dots \\ p^{H.}p^{.1}\\ p^{H.}p^{.2}\\ \dots \\ p^{H.}p^{.K}\\ \end{array} \right] \end{aligned}$$

and the matrices ($H \times HK$ and $K \times HK$, respectively)

$$\begin{aligned} {\varvec{A}}= & {} \left[ {\varvec{A}}_1,{\varvec{A}}_2,\dots ,{\varvec{A}}_H\right] \\ {\varvec{B}}= & {} \left[ {\varvec{B}}_1,{\varvec{B}}_2, \dots ,{\varvec{B}}_H\right] \end{aligned}$$

where

• i) ${\varvec{A}}_h$ is a matrix of size $H \times K$ with all entries equal to 0 but the entries of the hth row which are equal to 1, for $h=1,\dots ,H$.

• ii) ${\varvec{B}}_h$ is an identity matrix of order K, for $h=1,..,H$.

If we set

$$\begin{aligned} \widehat{{\varvec{p}}}^{H.}= \left[ \begin{array}{c} \widehat{p}^{1.}\\ \widehat{p}^{2.}\\ \dots \\ \widehat{p}^{H.}\\ \end{array} \right] \quad \widehat{{\varvec{p}}}^{.K}=\left[ \begin{array}{c} \widehat{p}^{.1}\\ \widehat{p}^{.2}\\ \dots \\ \widehat{p}^{.K}\\ \end{array} \right] \end{aligned}$$

then the relationships

$$\begin{aligned} \widehat{{\varvec{p}}}^{H.}= & {} \varvec{A}\widehat{{\varvec{p}}}^{HK} \\ \widehat{{\varvec{p}}}^{. K}= & {} \varvec{B}\widehat{{\varvec{p}}}^{HK} \end{aligned}$$

hold. Next, define the matrices ($HK\times H$, $HK\times H$ and $HK \times K$, respectively)

$$\begin{aligned} \varvec{\Pi }= \left[ \begin{array}{c} \varvec{\Pi }_1\\ \varvec{\Pi }_2\\ \dots \\ \varvec{\Pi }_H\\ \end{array} \right] \quad \widehat{\varvec{\Pi }}=\left[ \begin{array}{cc} \widehat{\varvec{\Pi }}_1\\ \widehat{\varvec{\Pi }}_2\\ \dots \\ \widehat{\varvec{\Pi }}_H\\ \end{array} \right] \quad \varvec{\Psi }=\left[ \begin{array}{cc} \varvec{\Psi }_1\\ \varvec{\Psi }_2\\ \dots \\ \varvec{\Psi }_H\\ \end{array} \right] \end{aligned}$$

where

• 1. $\varvec{\Pi }_h$ is a matrix of size $K \times H$ having all entries equal to zero but the entries in the hth column that are equal to $p^{.1},p^{.2},\dots ,p^{.K}$, for $h=1,...,H$.

• 2. $\widehat{\varvec{\Pi }}_h$ is a matrix of order $K \times H$ having all entries equal to zero but the entries in the hth column that are equal to $\widehat{p}^{.1},\widehat{p}^{.2},\dots ,\widehat{p}^{.K}$, for $h=1,...,H$.

• 3. $\varvec{\Psi }_h$ is a diagonal matrix of order $K \times K$, with all entries in the main diagonal equal to $p^{h.}$, for $h=1,...,H$.

With this symbols, we may write

$$\begin{aligned} \sqrt{n} \left[ \begin{array}{c} \widehat{{\varvec{p}}}^{HK} - {\varvec{p}}^{HK} \\ \widehat{{\varvec{p}}}^{H.} - {\varvec{p}}^{H.} \\ \widehat{{\varvec{p}}}^{.K} - {\varvec{p}}^{.K} \\ \end{array} \right] =\left[ \begin{array}{c} \varvec{I}^{HK}\\ {\varvec{A}}\\ {\varvec{B}}\\ \end{array} \right] \sqrt{n}( \widehat{{\varvec{p}}}^{HK}-{\varvec{p}}^{HK}) \end{aligned}$$

(28)

where $\varvec{I}^{HK}$ is the identity matrix of size $HK \times HK$.

Lemma 1

$\widehat{p}^{hk}-p^{hk}$ converges in probability to 0 as, n, N go to infinity, for each h, k.

Proof

Immediate consequence of Proposition 1. $\square$

Note that Proposition 1 actually implies that $\widehat{p}^{hk}-p^{hk}=O_{p}(n^{-1/2})$, for each h, k.

Lemma 2

$\widehat{p}^{h.}-p^{h.}$, $\widehat{p}^{.k}-p^{.k}$ converge in probability to 0 as, n, N go to infinity, for each h, k.

Proof

Consequence of Lemma 1. $\square$

Proof of Proposition 2

It is enough to use the relationship (28). Proposition 2 follows from (28), Proposition 1, and the continuous mapping theorem. $\square$

Lemma 3

Under the independence hypothesis ${\mathcal H_{0}}$, the limiting distribution of $\sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\widetilde{{\varvec{p}}}^{HK})$ coincides with the limiting distribution of

$$\begin{aligned} (\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}}) \left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\overline{{\varvec{p}}}^{HK}) \right\} \end{aligned}$$

that turns out to be (degenerate) multivariate Normal with null mean vector and covariance matrix,

$$\begin{aligned} \varvec{\Gamma }^{HK}=(\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}})\varvec{\Sigma }^{HK} (\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}})^{T} \end{aligned}$$

Proof

From the relationship

$$\begin{aligned} \widehat{p}^{hk}-\widehat{p}^{h.}\widehat{p}^{.k}=(\widehat{p}^{hk}-p^{h.}p^{.k})- \widehat{p}^{h.}(\widehat{p}^{.k}-p^{.k})- \widehat{p}^{.k}(\widehat{p}^{h.}-p^{h.}) \end{aligned}$$

it follows that, in matrix terms,

$$\begin{aligned} \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\widetilde{{\varvec{p}}}^{HK})=\sqrt{n} (\varvec{I}^{HK}-\widehat{\varvec{\Pi }} {\varvec{A}}-\varvec{\Psi } {\varvec{B}})(\widehat{{\varvec{p}}}^{HK}-\overline{{\varvec{p}}}^{HK}). \end{aligned}$$

Next, from Lemma 2, the matrix $\widehat{\Pi }$ tends in probability to $\Pi$, as n, N go to infinity. Using the Slutsky Theorem (Serfling 1980), this implies, in its turns, that the limiting distribution of,

$$\begin{aligned} \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\widetilde{{\varvec{p}}}^{HK})= & {} (\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}}) \left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\overline{{\varvec{p}}}^{HK}) \right\} \\- & {} (\widehat{\varvec{\Pi }}-\varvec{\Pi }){\varvec{A}}\left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\overline{{\varvec{p}}}^{HK}) \right\} \end{aligned}$$

coincides with the limiting distribution of

$$\begin{aligned} (\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}}) \left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\overline{{\varvec{p}}}^{HK}) \right\} \end{aligned}$$

(29)

The linearity of (29) and the continuous mapping theorem complete the proof. $\square$

For the sake of simplicity, from now the notation

$$\begin{aligned} \varvec{C}=\varvec{I}^{HK}-\varvec{\Pi } {\varvec{A}}-\varvec{\Psi } {\varvec{B}}\end{aligned}$$

will be used.

Lemma 4

Define

$$\begin{aligned} \chi ^{2}_{2H}=n\sum _{h=1}^{H}\sum _{k=1}^{K}(\widehat{p}^{hk}-\widehat{p}^{h.}\widehat{p}^{.k})^2 \left( \frac{1}{\widehat{p}^{h.}\widehat{p}^{.k}}-\frac{1}{p^{h.}p^{.k}}\right) \end{aligned}$$

(30)

Under the null hypothesis of independence ${{\mathcal {H}}_{0}}$, $\chi ^{2}_{2H}$ converges in probability to 0 as n, N go to infinity.

Proof

First of all, we have

$$\begin{aligned} |\chi ^{2}_{2H}|\le \max _{h,k}\left| \frac{1}{\widehat{p}^{h.}\widehat{p}^{.k}}- \frac{1}{p^{h.}p^{.k}} \right| \left\{ n \sum _{h=1}^{H}\sum _{k=1}^{K}(\widehat{p}^{hk}-\widehat{p}^{h.}\widehat{p}^{.k})^2\right\} . \end{aligned}$$

Since convergence in probability is preserved under continuous transformations, the term

$$\begin{aligned} \max _{h,k} \left| \frac{1}{\widehat{p}^{h.}\widehat{p}^{.k}}- \frac{1}{p^{h.}p^{.k}} \right| {\mathop {\rightarrow }\limits ^{p}} 0 \;\;\; \mathrm {as} \; n,N \rightarrow \infty \end{aligned}$$

(31)

In addition, from Lemma 3 it follows that

$$\begin{aligned} n\sum _{h=1}^H\sum _{k=1}^K (\widehat{p}^{hk}-\widehat{p}^{h.}\widehat{p}^{.k})^2 & = \left\{ \sqrt{n}(\widehat{\varvec{p}}^{HK}-\widetilde{\varvec{p}}^{HK})\right\} ^T \left\{ \sqrt{n}(\widehat{\varvec{p}}^{HK}-\widetilde{\varvec{p}}^{HK})\right\} \nonumber \\&{\mathop {\rightarrow }\limits ^{d}} \varvec{X}^{T}\varvec{X} \end{aligned}$$

(32)

where $\varvec{X}$ is a singular multivariate (HK) Normal r.v. with null mean vector and covariance matrix $\varvec{\Gamma }^{HK}=\varvec{C}\varvec{\Sigma }^{HK}\varvec{C}^T$. The lemma follows from (31) and (32) and the continuous mapping theorem. $\square$

Lemma 5

Define

$$\begin{aligned} \chi ^{2}_{1H}=n\sum _{h=1}^H\sum _{k=1}^k\frac{(\widehat{p}^{hk}-\widehat{p}^{h.}\widehat{p}^{.k})^2}{\widehat{p}^{h.}\widehat{p}^{.k}}. \end{aligned}$$

(33)

Under the null hypothesis of independence ${{\mathcal {H}}_{0}}$, $\chi ^{2}_{1H}$ tends in distribution to $\varvec{X}^{T}{\varvec{p}}^{HK}({\varvec{p}}^{HK})^{T}\varvec{X}$ where $\varvec{X}$ is a (singular) multivariate HK Normal r.v. with null mean vector and covariance matrix $\varvec{\Gamma }^{HK}=\varvec{C} \varvec{\Sigma }^{HK} \varvec{C}^{T}$.

Proof

It is enough to observe that

$$\begin{aligned} \chi ^{2}_{1H}=\left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\widetilde{{\varvec{p}}}^{HK})\right\} ^T \varvec{p}^{HK}({\varvec{p}}^{HK})^{T}\left\{ \sqrt{n}(\widehat{{\varvec{p}}}^{HK}-\widetilde{{\varvec{p}}}^{HK})\right\} \end{aligned}$$

and apply Lemma 3 and the continuous mapping theorem. $\square$

Proof of Proposition 3

The statistic $\chi ^2_{H}$ can be written as $\chi ^2_{1H}+\chi ^2_{2H}$, where $\chi ^2_{1H}$ and $\chi ^2_{2H}$ are defined in (33) and (30) respectively. The proof is a simple application of Lemma 4, 5.$\square$