Context-specific independence in graphical log-linear models

Abstract

Log-linear models are the popular workhorses of analyzing contingency tables. A log-linear parameterization of an interaction model can be more expressive than a direct parameterization based on probabilities, leading to a powerful way of defining restrictions derived from marginal, conditional and context-specific independence. However, parameter estimation is often simpler under a direct parameterization, provided that the model enjoys certain decomposability properties. Here we introduce a cyclical projection algorithm for obtaining maximum likelihood estimates of log-linear parameters under an arbitrary context-specific graphical log-linear model, which needs not satisfy criteria of decomposability. We illustrate that lifting the restriction of decomposability makes the models more expressive, such that additional context-specific independencies embedded in real data can be identified. It is also shown how a context-specific graphical model can correspond to a non-hierarchical log-linear parameterization with a concise interpretation. This observation can pave way to further development of non-hierarchical log-linear models, which have been largely neglected due to their believed lack of interpretability.

Appendices

Appendix 1

Definition 5

Decomposable SG Let (G, L) constitute an SG with G being chordal. Further, let $E_{L}$ denote the set of all stratified edges, $E_{C}$ the set of all edges in the maximal clique C, and $E_{\mathcal {S}}$ the set of all edges in the separators of G. The SG is defined as decomposable if

$$\begin{aligned} E_{L}\cap E_{\mathcal {S}} = \varnothing , \end{aligned}$$

and

$$\begin{aligned} E_{L} \cap E_{C} = \varnothing \quad \text { or } \bigcap _{\{\delta ,\gamma \}\in E_{L}\cap E_{C}} \{\delta ,\gamma \} \ne \varnothing \quad \text { for all } C \in \mathcal {C}(G). \end{aligned}$$

An SGM where (G, L) constitutes a decomposable SG is termed a decomposable SGM.

Appendix 2

Derivation of the parameters in Eq. (6).

We will here give a more detailed explanation of how $\hat{\theta }_{0,0} = \hat{P}(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }, X_{\delta } = 0, X_{\gamma } = 0)$ is derived. It is generally possible to use the factorization

$$\begin{aligned} P(X_{L_{\{\delta ,\gamma \}}}= & {} x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }, X_{\delta } = 0, X_{\gamma } = 0) \\= & {} P(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }) P(X_{\delta } = 0, X_{\gamma } = 0 \mid X_{L_{\{\delta , \gamma \}}}\\= & {} x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }). \end{aligned}$$

When considering a probability distribution where $\delta$ and $\gamma$ can be dependent, it is generally not true that $P(X_{\delta }, X_{\gamma }) = P(X_{\delta }) P(X_{\gamma })$. A standard result, see e.g. Whittaker (1990), states that for a distribution where two variables are dependent the ML projection to the set of distributions where the variables are independent is obtained by calculating the product of the marginal probabilities of the two variables. This implies, in our case, creating a new distribution $\hat{P}$ according to

$$\begin{aligned}&\hat{P}(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }, X_{\delta } = 0, X_{\gamma } = 0) \\&\quad = P(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }) P(X_{\delta } = 0 \mid X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta , \gamma \}}}, X_{\varOmega } = x_{\varOmega }) \\&P(X_{\gamma } = 0 \mid X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta , \gamma \}}}, X_{\varOmega } = x_{\varOmega }). \end{aligned}$$

Using the earlier introduced notations this corresponds to setting

$$\begin{aligned} \hat{\theta }_{0,0}= & {} \hat{P}(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega } , X_{\delta } = 0, X_{\gamma } = 0)\\= & {} P(X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }) P(X_{\delta } = 0 \mid X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta , \gamma \}}}, X_{\varOmega } = x_{\varOmega }) \\&P(X_{\gamma } = 0 \mid X_{L_{\{\delta ,\gamma \}}} = x_{L_{\{\delta ,\gamma \}}}, X_{\varOmega } = x_{\varOmega }) \\= & {} (\theta _{0,0}+\theta _{0,1}+\theta _{1,0}+\theta _{1,1}) \cdot (\theta _{0,0}+\theta _{0,1}) / (\theta _{0,0}+\theta _{0,1}+\theta _{1,0}+\theta _{1,1}) \\&\cdot \,(\theta _{0,0}+\theta _{1,0}) / (\theta _{0,0}+\theta _{0,1}+\theta _{1,0}+\theta _{1,1}) \\= & {} (\theta _{0,0}+\theta _{0,1}) \cdot (\theta _{0,0}+\theta _{1,0}) / (\theta _{0,0}+\theta _{0,1}+\theta _{1,0}+\theta _{1,1}). \end{aligned}$$

The other parameters $\hat{\theta }_{0,1}$, $\hat{\theta }_{1,0}$, and $\hat{\theta }_{1,1}$ can be derived in a similar fashion.

Appendix 3

Proposal functions used for model optimization.

The search for the optimal stratified graph is conducted using two separate Markov chains. One Markov chain is used to traverse different underlying graphs. A second chain is used to identify the optimal set of strata given the underlying graph. Combining these two searches will ultimately result in the discovery of the optimal SG.

Using the proposal function defined in Algorithm 1, running a sufficient amount of iterations, we can be assured to find the optimal set of strata for any chordal graph.

Algorithm 1

Proposal function for finding optimal strata for a chordal graph.

Let G denote the underlying graph. By $L_A$ we denote all possible instances that can be added to any stratum of G. If $L_A$ is empty no strata may be added to G and the algorithm is terminated. L denotes the current state with L being empty in the starting state.

• 1. Set the candidate state $L^* = L$.

• 2. Perform one of the following steps.

  • 2.1. If L is empty add a randomly chosen instance from $L_A$ to $L^*$.

  • 2.2. Else if $\{L_A {\setminus } L\}$ is empty remove a randomly chosen instance from $L^*$.

  • 2.3. Else with probability 0.5 add a randomly chosen instance from $\{L_A {\setminus } L\}$ to $L^*$.

  • 2.4. Else remove a randomly chosen instance from $L^*$.

Using this proposal function the optimal set of strata can be found for any underlying graph and we can proceed to the search for the best underlying graph. The proposal function in Algorithm 2 is used for this task.

Algorithm 2

Proposal function used to find the optimal underlying chordal graph.

The starting state is set to be the graph containing no edges. Let G denote the current graph with $G_L = (G, L)$ being the stratified graph with underlying graph G and optimal set of strata L.

• 1. Set the candidate state $G^* = G$.

• 2. Randomly choose a pair of nodes $\delta$ and $\gamma$. If the edge $\{\delta , \gamma \}$ is present in $G^*$ remove it, otherwise add the edge $\{\delta , \gamma \}$ to $G^*$.

• 3. While $G^*$ is non-chordal repeat steps 1 and 2.

The resulting candidate state $G^*$ is used along with the corresponding optimal set of strata $L^*$ to form the stratified graph $G^*_L = (G^*, L^*)$ which is used when calculating the acceptance probability.

Appendix 4

Questions considered in parliament election data.

• 1. Since the mid-1990’s the income differences have grown rapidly in Finland. How should we react to this?

  0—The income differences do not need to be narrowed.

  1—The income differences need to be narrowed.

• 2. Should homosexual couples have the same rights to adopt children as heterosexual couples?

  0—Yes.

  1—No.

• 3. Child benefits are paid for each child under the age of 18 living in Finland, independent of the parents’ income. What should be done about child benefits?

  0—The income of the parents should not affect the child benefits.

  1—Child benefits should be dependent on parents’ income.

• 4. In Finland military service is mandatory for all men. What is your opinion on this?

  0—The current practice should be kept or expanded to also include women.

  1—The military service should be more selective or abandoned altogether.

• 5. Should Finland in its affairs with China and Russia more actively debate issues regarding human rights and the state of democracy in these countries?

  0—Yes.

  1—No.

• 6. Russia has prohibited foreigners from owning land close to the borders. In recent years, Russians have bought thousands of properties in Finland. How should Finland react to this?

  0—Finland should not restrict foreigners from buying property in Finland.

  1—Finland should restrict foreigners’ rights to buy property and land in Finland.

• 7. During recent years municipalities have outsourced many services to privately owned companies. What is your opinion on this?

  0—Outsourcing should be used to an even higher extent.

  1—Outsourcing should be limited to the current extent or decreased.

• 8. Currently, a system is in place where tax income from more wealthy municipalities is transferred to less wealthy municipalities. In practice this means that municipalities in the Helsinki region transfer money to the other parts of the country. What is your opinion of this system?

  0—The current system is good, or even more money should be transferred.

  1—The Helsinki region should be allowed to keep more of its tax income.