An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

Abstract

In this paper, we propose a new expectation-maximization (EM) algorithm, named GMM-EM, to blind separation of noisy instantaneous mixtures, in which the non-Gaussianity of independent sources is exploited by modeling their distribution using the Gaussian mixture model (GMM). The compatibility between the incomplete-data structure of the GMM and the hidden variable nature of the source separation problem leads to an efficient hierarchical learning and alternative method for estimating the sources and the mixing matrix. In comparison with conventional blind source separation algorithms, the proposed GMM-EM algorithm has superior performance for the separation of noisy mixtures due to the fact that the covariance matrix of the additive Gaussian noise is treated as a parameter. Furthermore, the GMM-EM algorithm works well in underdetermined cases by incorporating any prior information one may have and jointly estimating the mixing matrix and source signals in a Bayesian framework. Systematic simulations with both synthetic and real speech signals are used to show the advantage of the proposed algorithm over conventional independent component analysis techniques, such as FastICA, especially for noisy and/or underdetermined mixtures. Moreover, it can even achieve similar performance to a recent technique called null space component analysis with less computational complexity.

Appendices

Appendix 1: Proof of Equation (13)

Since

$$\begin{aligned}&f\left( \mathbf{s},Y|\mathbf{x},{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g\right) \nonumber \\&\quad = \prod \limits _{t = 1}^T {f\left( \mathbf{s}(t)|\mathbf{x},{\mathbf{A}}^g ,\mathbf{R}_w^g,\varTheta ^g \right) f\left( y(t)|\mathbf{s}(t),\mathbf{x},{\mathbf{A}}^g ,\mathbf{R}_w^g,\varTheta ^g \right) } \end{aligned}$$

(39)

On the other hand,

$$\begin{aligned}&f(\mathbf{x},\mathbf{s},Y|{\mathbf{A}},\mathbf{R}_w,\varTheta ) \nonumber \\&\quad = \prod \limits _{t = 1}^T {f\left( \mathbf{x}(t)|\mathbf{s}(t),{\mathbf{A}},\mathbf{R}_w \right) f\left( \mathbf{s}(t)|y(t),\varTheta \right) } \end{aligned}$$

(40)

Substituting (39) and (40) in (11), it is straightforward to derive that

$$\begin{aligned} J= & {} \sum \limits _{t = 1}^T {\sum \limits _{m = 1}^M {\int _\mathbf{s} {f(y(t) = m|\mathbf{s}(t),\mathbf{x},{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g )} } } \nonumber \\&\quad \log \omega _m f(\mathbf{s}(t)|y(t) = m,\varTheta )\mathrm{d}{} \mathbf{s} \nonumber \\&\quad + \sum \limits _{t = 1}^T {\int _\mathrm{{s}} {f(\mathbf{s}(t)|\mathbf{x},{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g )\log f(\mathbf{x}(t)|\mathbf{s}(t),{\mathbf{A}},\mathbf{R}_w )} } \mathrm{d}{} \mathbf{s} \end{aligned}$$

Appendix 2: Proof of Equation (14)

Based on the Bayesian theory, it is easy to obtain

$$\begin{aligned}&f\left( \mathbf{s}(t)|\mathbf{x}(t),{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g \right) \nonumber \\&\quad = \sum \limits _{y(t) = 1}^M {f\left( \mathbf{x}(t)|\mathbf{s}(t),{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g \right) }\nonumber \\&\qquad f\left( \mathbf{s}(t)|y(t) = m,{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g \right) \end{aligned}$$

(41)

Hence,

$$\begin{aligned}&f(\mathbf{x}(t),\mathbf{s}(t),{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g ) \nonumber \\&\quad = \frac{1}{{\left| {2\pi \mathbf{R}_w^g } \right| ^{1/2} }}\nonumber \\&\qquad \exp \left\{ { - \frac{1}{2}(\mathbf{x}(t) - {\mathbf{A}}^g \mathbf{s}(t))^{\mathrm{T}} (\mathbf{R}_w^g )^{ - 1} (\mathbf{x}(t) - {\mathbf{A}}^g \mathbf{s}(t))} \right\} \nonumber \\&\qquad \sum _{m = 1}^M {\omega _m^g \frac{1}{{\left| {2\pi \mathbf{C}_m^g } \right| ^{1/2} }}} \nonumber \\&\qquad \quad \exp \left\{ { - \frac{1}{2}(\mathbf{s}(t) - {\varvec{\mu }}_m^g )^{\mathrm{T}} (\mathbf{C}_m^g )^{ - 1} (\mathbf{s}(t) - {\varvec{\mu }}_m^g )} \right\} \nonumber \\&\quad = \sum _{m = 1}^M {\omega _m^g \frac{1}{{\left| {2\pi \mathbf{R}_w^g } \right| ^{1/2} }}\frac{1}{{\left| {2\pi \mathbf{C}_m^g } \right| ^{1/2} }}} \nonumber \\&\qquad \exp \left\{ { - \frac{1}{2}(\mathbf{x}(t) - {\mathbf{A}}^g \mathbf{s}(t))^{\mathrm{T}} (\mathbf{R}_w^g )^{ - 1} (\mathbf{x}(t) - {\mathbf{A}}^g \mathbf{s}(t))} \right\} \nonumber \\&\qquad \exp \left\{ { - \frac{1}{2}(\mathbf{s}(t) - {\varvec{\mu }}_m^g )^{\mathrm{T}} (\mathbf{C}_m^g )^{ - 1} (\mathbf{s}(t) - {\varvec{\mu }}_m^g )} \right\} \end{aligned}$$

(42)

After a series of derivations, (42) can be simplified as

$$\begin{aligned} f(\mathbf{s}(t)|\mathbf{x}(t),{\mathbf{A}}^g,\mathbf{R}_w^g,\varTheta ^g ) = \sum _{m = 1}^M {\tilde{\omega } _{mt}^g \mathcal{N}\left[ {\mathbf{s}(t);{\varvec{\tilde{\mu }}}_{mt}^g,{\tilde{\mathbf{C}}}_{mt}^g } \right] } \end{aligned}$$

where

$$\begin{aligned} \left\{ \begin{array}{l} {\tilde{\mathbf{C}}}_{mt}^g = \left( {({\mathbf{A}}^g )^{\mathrm{T}} (\mathbf{R}_w^g )^{ - 1} {\mathbf{A}}^g + (\mathbf{C}_m^g )^{ - 1} } \right) ^{ - 1} \\ {\varvec{\tilde{\mu }}}_{mt}^g ~= \left( {{\tilde{\mathbf{C}}}_{mt}^g } \right) \left( {({\mathbf{A}}^g )^{\mathrm{T}} (\mathbf{R}_w^g )^{ - 1} \mathbf{x}(t) + (\mathbf{C}_m^g )^{ - 1} {\varvec{\mu }}_m^g } \right) \\ \tilde{\omega } _{mt}^g ~= \omega _m^g \left( {{{\left| {({\tilde{\mathbf{C}}}_{mt}^g )} \right| ^{1/2} } \Big / {\left| {2\pi \mathbf{R}_w^g } \right| ^{1/2} \left| {\mathbf{C}_m^g } \right| ^{1/2} }}} \right) \\ \qquad \quad \mathrm{{ }}\exp \left\{ { - \frac{1}{2}\left[ {\mathbf{x}^{\mathrm{T}} (t)(\mathbf{R}_w^g )^{ - 1} \mathbf{x}(t)} \right. } \right. \\ \qquad \qquad \quad \left. {\left. {\mathrm{{ }} + ({\varvec{\mu }}_m^g )^\mathrm{{T}} (\mathbf{C}_m^g )^{ - 1} {\varvec{\mu }}_m^g - ({\varvec{\tilde{\mu }}}_{mt}^g )^\mathrm{{T}} ({\tilde{\mathbf{C}}}_{mt}^g )^{ - 1} {\varvec{\tilde{\mu }}}_{mt}^g } \right] } \right\} \\ \end{array} \right. \end{aligned}$$

Appendix 3: Definition of Similarity Score

In order to measure the separation performance, the similarity score is introduced to evaluate the separation performance of the proposed algorithm

$$\begin{aligned} \rho _{ii} = {{\sum _{t = 1}^T {s_i (t)\hat{s}_i (t)} } \Big / {\sqrt{\sum _{t = 1}^T {(s_i (t))^2 } \sum _{t = 1}^T {(\hat{s}_i (t))^2 } } }} \end{aligned}$$

(43)

where ${{\hat{s}}_i(t)}$ is the ith recovered source signal. $\rho _{ii}$ depicts the similarity between the ith original source signal and the corresponding recovered source signal. It is clear that the larger the value of $\rho _{ii}$, the higher the degree of similarity between the original sources and the recovered sources.