A Spatial Logistic Regression Model Based on a Valid Skew-Gaussian Latent Field

Abstract

Logistic regression is commonly used to estimate the association of one (or more) independent variable(s) with a binary- dependent outcome. In many applications latent sources are both spatially dependent and non-Gaussian; thus, it is desirable to exploit both properties jointly. Spatial logistic regression is a well-established technique of including spatial dependence in logistic regression models. In this paper, we develop a spatial logistic regression model based on a valid skew-Gaussian random field. For parameter estimation, we use a Monte Carlo extension of the EM algorithm along with an approximation based on the standard logistic function. A simulation study is applied in order to determine the performance of the proposed model and also to compare the results with a recently introduced model with established efficiency. The identifiability of the parameters is investigated as well. As an illustrative purpose, an application to the Meuse heavy metals dataset is presented.

Supplementary materials accompanying this paper appear online.

Appendix

In what follows, we use the notation ${{{\vartheta }}_i}$ to show ${{\vartheta }}\left( {{s_i}} \right)$. Equation (4) can be written as

$$\begin{aligned} Q\left( {\mathbf{{{\varvec{\eta }}}}\left| {{\mathbf{{{\varvec{\eta }}}}^t}} \right. } \right)= & {} - \sum \limits _i {{E_1}\left[ {\left. {\ln \left\{ {1 + \exp \left( {\mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}} + {\gamma }{W_i} + {\varepsilon _i}} \right) } \right\} } \right| \mathbf{{Z}}} \right] } \nonumber \\&+ \sum \limits _i {{Z_i}{} \mathbf{{x}}_i^\mathrm{T} {\varvec{\beta }}} + \gamma \sum \limits _i {{Z_i}{E_2}\left[ {{W_i}\left| \mathbf{{Z}} \right. } \right] } + \sum \limits _i {{Z_i}{E_3}\left[ {{\varepsilon _i}\left| \mathbf{{Z}} \right. } \right] } \nonumber \\&- \frac{1}{{2{\tau ^2}}}{E_4}\left[ {{\varvec{\varepsilon }}^\mathrm{T}{\varvec{\varepsilon }}\left| \mathbf{{Z}} \right. } \right] - \ln \left| {H + {\delta ^2}{I_n}} \right| - \frac{n}{2}\ln {\pi ^2}{\tau ^2}\nonumber \\&- \frac{1}{2} \mathrm{trace}\left\{ {{{\left( {H + {\delta ^2}{I_n}} \right) }^{ - 1}}{E_5}\left( {\mathbf{{WW^\mathrm{T}}}\left| \mathbf{{Z}} \right. } \right) } \right\} \nonumber \\&- \sqrt{\frac{2}{\pi }} \delta {{\mathbf{{1}}}_n^\mathrm{T}}{\left[ {H + {\delta ^2}{I_n}} \right] ^{ - 1}}{E_6}\left[ {\mathbf{{W}}\left| \mathbf{{Z}} \right. } \right] - \frac{1}{\pi }{\delta ^2}{\mathbf{{1}}_n^\mathrm{T}} {\left[ {H + {\delta ^2}{I_n}} \right] ^{ - 1}}{\mathbf{{1}}_n} \nonumber \\&+ {E_7}\left[ {\left. {\ln {\Phi _n}\left\{ {\delta {{\left( {H + {\delta ^2}{I_n}} \right) }^{ - 1}}(\mathbf{{W}} + \sqrt{\frac{2}{\pi }} \delta {\mathbf{{1}}_n});\mathbf{{0}},\Delta } \right\} } \right| \mathbf{{Z}}} \right] , \end{aligned}$$

(A1)

in which the fourth line has been derived from the properties $x^\mathrm{T}Ax = \mathrm{trace}\left( {x^\mathrm{T}Ax} \right)$ and $\mathrm{trace}\left( {AB} \right) = \mathrm{trace}\left( {BA} \right)$. A closer scrutiny shows, however, that one of the main problematic terms of (A1) is $\sum \nolimits _i {{E_1}\left[ {\left. {\ln \left( {1 + {e^{{Y_i}}}} \right) } \right| \mathbf{{Z}}} \right] }$. Hardouin (2019) proposed a variational method which is based on replacing this term by an initial approximation of the logistic function $\kappa \left( x \right) = {e^x}/ \left( {1 + {e^x}} \right) = 1/ \left( {1 + {e^{ - x}}} \right)$ which had been studied by Jaakkola and Jordan (2000) as

$$\begin{aligned} \ln \kappa \left( x \right) \ge \ln \kappa \left( \theta \right) + \frac{{x - \theta }}{2} - \lambda \left( \theta \right) \left( {{x^2} - {\theta ^2}} \right) , \qquad \lambda \left( \theta \right) = \frac{{\kappa \left( \theta \right) - 1/2}}{{2\theta }}. \end{aligned}$$

This variational lower bound involves the model parameters and the so-called variational parameter $\theta$. Let $\Theta = {\left( {{\theta _1}, \ldots ,{\theta _n}} \right) ^\mathrm{T} }$, we apply this lower bound to $- \sum \nolimits _i {\ln \left( {1 + {e^{{Y_i}}}} \right) } = \sum \nolimits _i {\ln \kappa \left( { - {Y_i}} \right) }$ as the first term of (3). Therefore,

$$\begin{aligned} - \sum \nolimits _i {\ln \left( {1 + {e^{{Y_i}}}} \right) }\ge & {} \sum \nolimits _i[\ln \kappa \left( {\theta _i} \right) - \frac{{\theta _i} }{2} + {{\theta _i^2}}\lambda \left( {\theta _i} \right) ]\nonumber \\&- \frac{1}{2}\left[ {\sum \nolimits _i {\mathbf{{x}}_i^\mathrm{T} {\varvec{\beta }}+ \gamma {W_i} + {\varepsilon _i}} } \right] \nonumber \\&-\sum \nolimits _i \lambda \left( {\theta _i} \right) \left[ \frac{}{} {{\left( {\mathbf{{x}}_i^\mathrm{T} {\varvec{\beta }}} \right) }^2} + {\gamma ^2}W_i^2 + \varepsilon _i^2 \right. \nonumber \\&\left. +2\gamma \mathbf{{x}}_i^\mathrm{T} {\varvec{\beta }}{W_i} + 2\mathbf{{x}}_i^\mathrm{T} {\varvec{\beta }}{\varepsilon _i} + 2\gamma {W_i}{\varepsilon _i}\frac{}{} \right] . \end{aligned}$$

(A2)

The monotonicity of expectation implies that getting the (conditional) expectation of (A2) (given $\mathbf{Z}$) preserves the inequality. Now, we can write $Q\left( {{{{\varvec{\eta }}}}\left| {{{{{\varvec{\eta }}}}^t}} \right. } \right) \ge {{\tilde{Q}}}\left( {{{{\varvec{\eta }}},\Theta }\left| {{{{{\varvec{\eta }}}}^t}},\Theta ^t \right. } \right)$, where ${{\tilde{Q}}}$ has been resulted by replacing the first term of Q with the expectation of the right hand side of (A2) given $\mathbf{Z}$, which eliminates ${E_1}\left[ {\left. {\ln \left( {1 + \exp \left\{ {{Y_i}} \right\} } \right) } \right| \mathbf{{Z}}} \right]$ and incorporates ${E_8}\left[ {{W_i^2}\left| \mathbf{{Z}} \right. } \right]$ and ${E_9}\left[ {{\varepsilon _i^2}\left| \mathbf{{Z}} \right. } \right]$ into inference. We then use a two-stage estimation procedure in the M-step, where the first stage consists of maximizing ${{\tilde{Q}}}\left( {{{{\varvec{\eta }}},\Theta }\left| {{{{{\varvec{\eta }}}}^t}},\Theta ^t \right. } \right)$ with respect to the model parameters for fixed $\Theta$ results in ${{\tilde{Q}}}\left( {{{{\varvec{\eta }}}^{t+1},\Theta }\left| {{{{{\varvec{\eta }}}}^t},\Theta ^t} \right. } \right)$, and in the second stage, updated variational parameters $\Theta ^{t+1}$ is obtained by maximizing ${{\tilde{Q}}}\left( {{{{\varvec{\eta }}}^{t+1},\Theta }\left| {{{{{\varvec{\eta }}}}^t},\Theta ^t} \right. } \right)$ with respect to $\Theta$. The updates of the model parameters are as follows. ${\tau ^{{2^{t + 1}}}} = {n^{ - 1}} {{\mathbb {E}}}_4^t$, ${\varvec{\beta }}^{t+1}$ can be easily obtained as a solution of the systems of linear equations

$$\begin{aligned} \sum \nolimits _i {\lambda \left( {\theta _i^t} \right) \left( {\mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}^{t + 1}}} \right) \mathbf{{x}}_i^\mathrm{T} } = - \sum \nolimits _i {\left[ {\frac{1}{4} + \frac{{{Z_i}}}{2} + \lambda \left( {\theta _i^t} \right) \left( {{\gamma ^t}{\mathbb {E}}_2^t + {{\mathbb {E}}}_3^t} \right) } \right] } \mathbf{{x}}_i^\mathrm{T}, \end{aligned}$$

in which the left-hand side can be rewritten as $\left[ {\sum \nolimits _i {{\mathbf{{x}}_i}{} \mathbf{{x}}_i^\mathrm{T} } } \right] {{\varvec{\beta }}^{t + 1}}$, then,

$$\begin{aligned} {{\varvec{\beta }}^{t + 1}} = - {\left[ {\sum \nolimits _i {\lambda \left( {\theta _i^t} \right) {\mathbf{{x}}_i}{} \mathbf{{x}}_i^\mathrm{T} } } \right] ^{ - 1}}\sum \nolimits _i {\left[ {\frac{1}{4} + \frac{{{Z_i}}}{2} + \lambda \left( {\theta _i^t} \right) \left( {{\gamma ^t}{\mathbb {E}}_2^t + {{\mathbb {E}}}_3^t} \right) {\mathbf{{x}}_i}} \right] }. \end{aligned}$$

Moreover,

$$\begin{aligned} {\gamma ^{t + 1}}= & {} \frac{{\sum \nolimits _i {\left[ {{Z_i} - 0.5 - 2\lambda \left( {\theta _i^t} \right) \left( {\mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}^t} + {{\mathbb {E}}}_3^t} \right) } \right] {{\mathbb {E}}}_2^t} }}{{2\sum \nolimits _i {\lambda \left( {\theta _i^t} \right) {\mathbb {E}}_8^t} }},\\&{\delta ^{t + 1}} = \mathop {\arg \max }\limits _\delta \left\{ { - \ln \left| {{H^t} + {\delta ^2}{I_n}} \right| - \frac{1}{2}tr\{ {{{( {{H^t} + {\delta ^2}{I_n}} )}^{ - 1}} {{\mathbb {E}}}_5^t} \}} \right. \\&- \sqrt{\frac{2}{\pi }} \delta {\mathbf{{1}}_n}^\mathrm{T} {\left( {{H^t} + {\delta ^2}{I_n}} \right) ^{ - 1}} {{\mathbb {E}}}_6^t - \frac{1}{\pi }{\delta ^2}{\mathbf{{1}}_n}^\mathrm{T} {\left( {{H^t} + {\delta ^2}{I_n}} \right) ^{ - 1}}{\mathbf{{1}}_n}\\&\left. + {{\mathbb {E}}}_7^t\{ {{\ln {\Phi _n}( {\delta {{( {{H^t} + {\delta ^2}{I_n}} )}^{ - 1}}(\mathbf{{W}} + \sqrt{\frac{2}{\pi }} \delta {\mathbf{{1}}_n});\mathbf{{0}},{I_n} - {\delta ^2}{{[ {{H^t} + {\delta ^2}{I_n}} ]}^{ - 1}}} )} \left. \right| \mathbf{{Z}}} \}\right\} ,\\&{\psi ^{t + 1}} = \mathop {\arg \max }\limits _\psi \left\{ { - \ln \left| {{H} + {\delta ^{t^2}}{I_n}} \right| - \frac{1}{2}tr\{ {{{( {{H} + {\delta ^{t^2}}{I_n}} )}^{ - 1}} {{\mathbb {E}}}_5^t} \}} \right. \\&- \sqrt{\frac{2}{\pi }} \delta ^t {\mathbf{{1}}_n}^\mathrm{T} {\left( {{H} + {\delta ^{t^2}}{I_n}} \right) ^{ - 1}} {\mathbb {E}}_6^t - \frac{1}{\pi }{\delta ^{t^2}}{\mathbf{{1}}_n}^\mathrm{T} {\left( {{H} + {\delta ^{t^2}}{I_n}} \right) ^{ - 1}}{\mathbf{{1}}_n}\\&\left. + {{\mathbb {E}}}_7^t\{ {{\ln {\Phi _n}( {\delta ^t {{( {{H} + {\delta ^{t^2}}{I_n}} )}^{ - 1}}(\mathbf{{W}} + \sqrt{\frac{2}{\pi }} \delta ^t {\mathbf{{1}}_n});\mathbf{{0}},{I_n} - {\delta ^{t^2}}{{( {{H} + {\delta ^{t^2}}{I_n}} )}^{ - 1}}} )} \left. \right| \mathbf{{Z}}} \}\right\} ,\\&{\left( {{{\theta _i^{t+1}} }} \right) ^2} = {{\left( {\mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}^{t + 1}}} \right) }^2} + {{\left( {{\gamma ^{t + 1}}} \right) }^2}{{\mathbb {E}}}_8^{t+1} + {\mathbb {E}}_9^{t+1} \\&+ 2{\gamma ^{t + 1}}{} \mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}^{t + 1}}{{\mathbb {E}}}_2^{t+1} + 2\mathbf{{x}}_i^\mathrm{T} {{\varvec{\beta }}^{t + 1}}{{\mathbb {E}}}_3^{t+1} + 2{\gamma ^{t + 1}}{{\mathbb {E}}}_2^{t+1} {{\mathbb {E}}}_3^{t+1}. \end{aligned}$$