Background subtraction using finite mixtures of asymmetric Gaussian distributions and shadow detection

Abstract

Foreground segmentation of moving regions in image sequences is a fundamental step in many vision systems including automated video surveillance, human-machine interface, and optical motion capture. Many models have been introduced to deal with the problems of modeling the background and detecting the moving objects in the scene. One of the successful solutions to these problems is the use of the well-known adaptive Gaussian mixture model. However, this method suffers from some drawbacks. Modeling the background using the Gaussian mixture implies the assumption that the background and foreground distributions are Gaussians which is not always the case for most environments. In addition, it is unable to distinguish between moving shadows and moving objects. In this paper, we try to overcome these problem using a mixture of asymmetric Gaussians to enhance the robustness and flexibility of mixture modeling, and a shadow detection scheme to remove unwanted shadows from the scene. Furthermore, we apply this method to real image sequences of both indoor and outdoor scenes. The results of comparing our method to different state of the art background subtraction methods show the efficiency of our model for real-time segmentation.

Appendices

Appendix A

In this Appendix, we calculate the derivative of $\frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \mu _{jk}}, \frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk}}}, \frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk}}}, \frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk}}^{2}}$, and $\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk}}^{2}}$ used in the EM algorithm and background subtraction.

$$\begin{aligned} \frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \mu _{jk}}&= \sum _{i=1,X_{ik}<\mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})}{\sigma _{l_{jk}}^2} \nonumber \\&+\sum _{i=1,X_{ik}\ge \mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})}{\sigma _{r_{jk}}^2}\end{aligned}$$

(40)

$$\begin{aligned} \frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk}}}&= -\sum _{i=1}^{N}\frac{\hat{Z_{ij}}}{\sigma _{l_{jk}}+\sigma _{r_{jk}}} \nonumber \\&+\sum _{i=1,X_{ik}<\mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})^2}{\sigma _{l_{jk}}^3}\end{aligned}$$

(41)

$$\begin{aligned} \frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk}}^{2}}&= \sum _{i=1}^{N}\frac{\hat{Z_{ij}}}{(\sigma _{l_{jk}}+\sigma _{r_{jk}})^2} \nonumber \\&-3\sum _{i=1,X_{ik}<\mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})^2}{\sigma _{l_{jk}}^4}\end{aligned}$$

(42)

$$\begin{aligned} \frac{\partial L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk}}}&= -\sum _{i=1}^{N}\frac{\hat{Z_{ij}}}{\sigma _{l_{jk}}+\sigma _{r_{jk}}} \nonumber \\&+\sum _{i=1,X_{ik}\ge \mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})^2}{\sigma _{r_{jk}}^3}\end{aligned}$$

(43)

$$\begin{aligned} \frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk}}^{2}}&= \sum _{i=1}^{N}\frac{\hat{Z_{ij}}}{(\sigma _{l_{jk}}+\sigma _{r_{jk}})^2} \nonumber \\&-3\sum _{i=1,X_{ik}\ge \mu _{jk}}^{N}\frac{\hat{Z_{ij}}(X_{ik}-\mu _{jk})^2}{\sigma _{r_{jk}}^4} \end{aligned}$$

(44)

Appendix B

In this Appendix, we develop the solutions for Eqs. (24, 25, 26) used in the MML algorithm

$$\begin{aligned}&\!\!\!-\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \mu _{jk}^{2}} = \sum _{i=l,X_{ik}<\mu _{jk}}^{l+n_j-1}\frac{1}{\sigma _{l_{jk}}^2} \nonumber \\&\!\!\! + \sum _{i=l,X_{ik}\ge \mu _{jk}}^{l+n_j-1}\frac{1}{\sigma _{r_{jk}}^2}\end{aligned}$$

(45)

$$\begin{aligned}&\!\!\!\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \mu _{jk_1} \mu _{jk_2}} = 0\end{aligned}$$

(46)

$$\begin{aligned}&\!\!\!\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk}}^{2}} = \sum _{i=l}^{l+n_j-1}\frac{1}{(\sigma _{l_{jk}}+\sigma _{r_{jk}})^2} \nonumber \\&\!\!\!-3\sum _{i=l,X_{ik}<\mu _{jk}}^{l+n_j-1}\frac{(X_{ik}-\mu _{jk})^2}{\sigma _{l_{jk}}^4}\end{aligned}$$

(47)

$$\begin{aligned}&\!\!\!\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{l_{jk_{1}}}\sigma _{l_{jk_{2}}}} = 0\end{aligned}$$

(48)

$$\begin{aligned}&\!\!\!\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk}}^{2}} =\sum _{i=l}^{l+n_j-1}\frac{1}{(\sigma _{l_{jk}}+\sigma _{r_{jk}})^2} \nonumber \\&\!\!\!-3\sum _{i=l,X_{ik}\ge \mu _{jk}}^{l+n_j-1}\frac{(X_{ik}- \mu _{jk})^2}{\sigma _{r_{jk}}^4}\end{aligned}$$

(49)

$$\begin{aligned}&\!\!\!\frac{\partial ^2 L(\varTheta ,Z,{\mathcal {X}})}{\partial \sigma _{r_{jk_{1}}}\sigma _{r_{jk_{2}}}} = 0 \end{aligned}$$

(50)