Robust Mixture Regression Using Mixture of Different Distributions

Abstract

In this paper, we examine the mixture regression model based on mixture of different type of distributions. In particular, we consider two-component mixture of normal-t distributions, and skew t-skew normal distributions. We obtain the maximum likelihood (ML) estimators for the parameters of interest using the expectation maximization (EM) algorithm. We give a simulation study and real data examples to illustrate the performance of the proposed estimators.

Appendix

To get the conditional expectation of the complete data log-likelihood function given in (8), the following conditional expectations should be calculated given $y_j$ and the current parameter estimate $\varvec{\hat{\varTheta }}=({\varvec{\hat{\beta }}}_{1},\hat{\sigma }_{1}^{2},{\varvec{\hat{\beta }}}_{2},\hat{\sigma }_{2}^{2},\hat{\nu })$

$$\begin{aligned} \hat{z}_{j}= & {} E(z_{j}|y_{j},\varvec{\hat{\varTheta }}) =\frac{ \hat{w}\phi ( y_{j};\mathbf {x}_{j}^{{\prime }} \hat{{\varvec{\beta }}}_{1},\hat{\sigma }_{1}^{2}) }{ \hat{w}\phi ( y_{j};\mathbf {x}_{j}^{{\prime }} \hat{{\varvec{\beta }}}_{1},\hat{\sigma }_{1}^{2}) +( 1-\hat{w}) f_{t}( y_{j};\mathbf {x}_{j}^{{\prime }}\hat{{\varvec{\beta }}}_{2},\hat{\sigma }_{2}, \hat{\nu }) } , \end{aligned}$$

(36)

$$\begin{aligned} \hat{u}_{1j}= & {} E( u_{j}|y_{j},\varvec{\hat{\varTheta }}) =\frac{ \hat{\nu }+1}{\hat{\nu }+\left( \left( y_{j}-\mathbf {x} _{j}^{^{\prime }}{\varvec{\hat{\beta }}}_{2}\right) /\hat{\sigma } _{2}\right) ^{2}} , \end{aligned}$$

(37)

$$\begin{aligned} \hat{u}_{2j}= & {} E(\log u_{j}|y_{j},\varvec{\hat{\varTheta }}) =DG\left( \frac{\hat{\nu }+1}{2}\right) -\log \left( \frac{\hat{\nu }}{2} +\frac{\left( y_{j}-\mathbf {x}_{j}^{{\prime }}{\varvec{\hat{\beta }}}_{2}\right) ^{2}}{2\hat{\sigma }_{2}^{2}}\right) . \end{aligned}$$

(38)

These conditional expectations will be used in EM algorithm given in Sect. 2.1.

Similarly, to obtain the conditional expectation of the complete data log-likelihood function given in (21) the following expectations should be computed given $y_j$ and the current parameter estimate $\varvec{\hat{\varTheta }}=({\varvec{\hat{\beta }}}_{1},\hat{\sigma }_{1}^{2},\hat{\lambda }_{1},\hat{\nu },{\varvec{\hat{\beta }}}_{2},\hat{\sigma }_{2}^{2},\hat{\lambda }_{2})$

$$\begin{aligned} \hat{z}_{j}= & {} E(z_{j}|y_{j},\varvec{\hat{\varTheta }}) =\frac{ \hat{w}f_{ST}(y_{j};\mathbf {x}_{j}^{^{\prime }}\hat{{ \varvec{\beta }}}_{1},\hat{\sigma }_{1}^{2},\hat{ \lambda }_{1},\hat{\nu }) }{\hat{w}f_{ST}( y_{j};\mathbf {x} _{j}^{^{\prime }}\hat{{\varvec{\beta }}}_{1},\hat{\sigma } _{1}^{2},\hat{\lambda }_{1},\hat{\nu }) +( 1-\hat{w} )f_{SN}(y_{j};\mathbf {x}_{j}^{^{\prime }}\hat{{\varvec{ \beta }}}_{1},\hat{\sigma }_{2}^{2},\hat{\lambda }_{2}) } , \end{aligned}$$

(39)

$$\begin{aligned} \hat{s}_{1j}= & {} E(z_{j}\tau _{j}|y_{j},\varvec{\hat{\varTheta }}) = \hat{z}_{j}\bigg ( \frac{\hat{\nu }+1}{\hat{\eta }_{1j}^{2}+ \hat{\nu }}\bigg ) \frac{T_{\hat{\nu }+3}\bigg ( \hat{M}_{j}\sqrt{ \frac{\hat{\nu }+3}{\hat{\nu }+1}}\bigg ) }{T_{\hat{\nu } +1}(\hat{M}_{j}) } , \end{aligned}$$

(40)

$$\begin{aligned} \hat{s}_{2j}= & {} E(z_{j}\gamma _{j}\tau _{j}|y_{j},\varvec{\hat{\varTheta }} ) =\frac{\hat{\delta }_{\lambda _{1}}( y_{j}-\mathbf {x}_{j}^{^{\prime }}\hat{{\varvec{\beta }}} _{1}) \hat{s}_{1j}}{\hat{\sigma }_{1}} +\frac{\hat{z}_{j}\sqrt{1-\hat{\delta } _{\lambda _{1}}^{2}}}{\pi \hat{\sigma }_{1}\hat{f}(y_{j})}\bigg ( \frac{\hat{\eta }_{1j}^{2}}{\hat{\nu }(1-\hat{\delta }_{\lambda _{1}}^{2})}+1\bigg ) ^{-(\frac{\hat{\nu }}{2}+1) }, \end{aligned}$$

(41)

$$\begin{aligned} \hat{s}_{3j}= & {} E(z_{j}\gamma _{j}^{2}\tau _{j}|y_{j},\varvec{\hat{\varTheta }}) =\hat{\delta }_{\lambda _{1}}^{2}\bigg ( \frac{y_{j}- \mathbf {x}_{j}^{^{\prime }}\hat{{\varvec{\beta }}}_{1}}{\hat{ \sigma }_{1}}\bigg ) ^{2}\hat{s}_{1j}+\hat{z}_{j}\bigg \{ (1-\hat{ \delta }_{\lambda _{1}}^{2}) \nonumber \\+ & {} \frac{\hat{\delta }_{\lambda _{1}}( y_{j}-\mathbf {x} _{j}^{^{\prime }}\hat{{\varvec{\beta }}}_{1}) \sqrt{1-\hat{ \delta }_{\lambda _{1}}^{2}}}{\pi \hat{\sigma }_{1}^{2}\hat{f}(y_{j}) }\bigg ( \frac{\hat{\eta }_{1j}^{2}}{\hat{\nu }(1-\hat{\delta } _{\lambda _{1}}^{2})}+1\bigg ) ^{-(\frac{\hat{\nu }}{2}+1)}\bigg \} , \end{aligned}$$

(42)

$$\begin{aligned} \hat{s}_{4j}= & {} E( z_{j}\log (\tau _{j})|y_{j},\varvec{\hat{\varTheta }} ) =\hat{z}_{j}\bigg \{ DG\bigg ( \frac{\hat{\nu }+1}{2}\bigg ) -\log \bigg ( \frac{\hat{\eta }_{1j}^{2}+\hat{\nu }}{2}\bigg ) \nonumber \\+ & {} \bigg ( \frac{\hat{\nu }+1}{\hat{\eta }_{1j}^{2}+\hat{\nu }} \bigg ) \left( \frac{T_{\hat{\nu }+3}\bigg ( \hat{\lambda }_{1} \hat{\eta }_{1j}\sqrt{\frac{\hat{\nu }+3}{\hat{\nu }+\hat{ \eta }_{1j}^{2}}}\bigg ) }{T_{\hat{\nu }+1}\bigg ( \hat{\lambda }_{1} \hat{\eta }_{1j}\sqrt{\frac{\hat{\nu }+1}{\hat{\nu }+\hat{ \eta }_{1j}^{2}}}\bigg ) }-1\right) \nonumber \\+ & {} \frac{\hat{\lambda }_{1}\hat{\eta }_{1j}(\hat{\eta } _{1j}^{2}-1)}{\sqrt{(\hat{\nu }+1)(\hat{\nu }+\hat{\eta } _{1j}^{2})^{3}}}\frac{t_{\hat{\nu }+1}\bigg ( \hat{\lambda }_{1} \hat{\eta }_{1j}\sqrt{\frac{\hat{\nu }+1}{\hat{\nu }+\hat{ \eta }_{1j}^{2}}}\bigg ) }{T_{\hat{\nu }+1}\bigg ( \hat{\lambda }_{1} \hat{\eta }_{1j}\sqrt{\frac{\hat{\nu }+1}{\hat{\nu }+\hat{ \eta }_{1j}^{2}}}\bigg ) } \nonumber \\+ & {} \frac{1}{T_{\hat{\nu }+1}\bigg ( \hat{\lambda }_{1}\hat{ \eta }_{1j}\sqrt{\frac{\hat{\nu }+1}{\hat{\nu }+\hat{\eta } _{1j}^{2}}}\bigg ) }\int \limits _{-\infty }^{\hat{M}_{j}}g_{\hat{\nu } }(x)t_{\hat{\nu }+1}(x)dx\bigg \} , \end{aligned}$$

(43)

$$\begin{aligned} \hat{t}_{1j}= & {} E( \gamma _{j}|y_{j},\varvec{\hat{\varTheta }}) = \hat{\delta }_{\lambda _{2}}\hat{\eta }_{2j}+\sqrt{1-\hat{\delta }_{\lambda _{2}}^{2}}\frac{\phi \left( \hat{\lambda }_{2}\hat{\eta } _{2j}\right) }{\Phi \left( \hat{\lambda }_{2}\hat{\eta }_{2j}\right) } , \end{aligned}$$

(44)

$$\begin{aligned} \hat{t}_{2j}= & {} E(\gamma _{j}^{2}|y_{j},\varvec{\hat{\varTheta }}) =1-\hat{\delta }_{\lambda _{2}}^{2}+\hat{\delta }_{\lambda _{2}} \hat{\eta }_{2j}\hat{t}_{1j} , \end{aligned}$$

(45)

where

$$\begin{aligned} \hat{\eta }_{1j}= & {} \frac{(y_{j}-\mathbf {x}_{j}^{{\prime }}\hat{{\varvec{\beta }}}_{1})}{\hat{\sigma }_{1}},\hat{ \delta }_{\lambda _{1}}=\frac{\hat{\lambda }_{1}}{\sqrt{1+\hat{ \lambda }_{1}^{2}}}, \\ \hat{\eta }_{2j}= & {} \frac{(y_{j}-\mathbf {x}_{j}^{{\prime }}\hat{{\varvec{\beta }}}_{2})}{\hat{\sigma }_{2}},\hat{ \delta }_{\lambda _{2}}=\frac{\hat{\lambda }_{2}}{\sqrt{1+\hat{\lambda }_{2}^{2}}},\hat{M}_{j} =\hat{\lambda }_{1}\hat{\eta }_{1j}\sqrt{\frac{\hat{\nu }}{\hat{\nu }+\hat{\eta }_{1j}^{2}}},\\ g_{\hat{\nu }} (x)= & {} DG\bigg (\frac{\hat{\nu }+2}{2}\bigg )-DG\bigg (\frac{\hat{\nu }+1}{2}\bigg )-\log \bigg (1+\frac{x^2}{\hat{\nu }+1}\bigg )+\frac{x^2(\hat{\nu }+1)-\hat{\nu }-1}{(\hat{\nu }+1)(\hat{\nu }+1+x^2)},\\ \hat{f} (y_{j})= & {} \hat{w_{1}}\frac{2}{\hat{\sigma }_{1}}t_{\hat{\nu }}(\hat{ \eta }_{1j})T_{\hat{\nu }+1}(\hat{M}_{j})+(1-\hat{w_{1}})\frac{2}{\hat{\sigma }_{2}}\phi (\hat{\eta }_{2j})\Phi (\hat{\lambda _{2}}\hat{\eta }_{2j}). \end{aligned}$$

These conditional expectations will be used in EM algorithm given in Sect. 2.2.