Performance Study for Complex Independent Component Analysis

Abstract

The goal of independent component analysis (ICA) is to decompose observed signals into components as independent as possible. In linear instantaneous blind source separation, ICA is used to separate linear instantaneous mixtures of source signals into signals that are as close as possible to the original signals. In the estimation of the so-called demixing matrix one has to distinguish two different factors:

• 1. Variance of the estimated inverse mixing matrix in the noiseless case due to randomness of the sources.

• 2. Bias of the demixing matrix from the inverse mixing matrix:

This chapter studies both factors for circular and noncircular complex mixtures. It is important to note that the complex case is not directly equivalent to the real case of twice larger dimension. In the derivations, we aim to clearly show the connections and differences between the complex and real cases. In the first part of the chapter, we derive a closed-form expression for the CRB of the demixing matrix for instantaneous noncircular complex mixtures. We also study the CRB numerically for the family of noncircular complex generalized Gaussian distributions (GGD) and compare it to simulation results of several ICA estimators. In the second part, we consider a linear noisy noncircular complex mixing model and derive an analytic expression for the demixing matrix of ICA based on the Kullback-Leibler divergence (KLD). We show that for a wide range of both the shape parameter and the noncircularity index of the GGD, the signal-to-interference-plus-noise ratio (SINR) of KLD-based ICA is close to that of linear MMSE estimation. Furthermore, we show how to extend our derivations to the overdetermined case ($M>N$) with circular complex noise.

Sections 3.1.1 and 3.2 of this chapter are based on our previous journal publication [35]. ©  2013 IEEE. Reprinted, with permission, from Loesch and Yang [35].

Sections 3.3.1–3.3.3 of this chapter are based on our previous conference publication [34]. First published in the Proceedings of the 20th European Signal Processing Conference (EUSIPCO-2012) in 2012, published by EURASIP.

An erratum to this chapter is available at 10.1007/978-3-642-55016-4_3

An erratum to this chapter can be found at http://dx.doi.org/10.1007/978-3-642-55016-4_20

Appendices

Appendix 1

1.1 Values of $\kappa$, $\xi$, $\beta$, $\eta$ for Complex GGD

The pdf of a noncircular complex GGD with zero mean, variance $\mathrm{E}[|s|^2]=1$ and noncircularity index $\gamma \in [0,1]$ is given by

$$\begin{aligned} p(s,s^*) = \frac{c \alpha \cdot \exp \left( \!-\!\left[ \frac{\alpha /2}{\gamma ^2-1} \left( \gamma s^2 + \gamma {s^*}^2 - 2 s s^*\right) \right] ^c\right) }{\pi \varGamma (1/c)(1-\gamma ^2)^{1/2}} , \end{aligned}$$

(3.84)

where $\alpha = \varGamma (2/c)/\varGamma (1/c)$ and $\varGamma (\cdot )$ is the Gamma function. The function $\varphi (s,s^*) = - \frac{\partial }{\partial s^*} \ln p(s,s^*)$ is then given by

$$\begin{aligned} \varphi (s,s^*) = \frac{2c (\alpha /2)^c}{(\gamma ^2-1)^c} \left( \gamma s^2 + \gamma (s^*)^2 - 2ss^*\right) ^{c-1} (\gamma s^* -s). \end{aligned}$$

(3.85)

By integration in polar coordinates, it can be shown that $\kappa$, $\xi$, $\beta$ and $\eta$ are given by:

$$\begin{aligned} \kappa&= \mathrm{E}\left[ |\varphi (s)|^2\right] = \frac{c^2 \varGamma (2/c)}{(1-\gamma ^2) \varGamma ^2(1/c)}, \end{aligned}$$

(3.86)

$$\begin{aligned} \xi&= \mathrm{E}\left[ (\varphi ^*(s))^2\right] = -\frac{c^2 \gamma \varGamma (2/c)}{(1-\gamma ^2) \varGamma ^2(1/c)} = - \gamma \kappa , \end{aligned}$$

(3.87)

$$\begin{aligned} \eta&= \mathrm{E}\left[ |s|^2 |\varphi (s)|^2\right] = \frac{(c+1) \cdot (2-\gamma ^2)}{2 (1-\gamma ^2)} ,\end{aligned}$$

(3.88)

$$\begin{aligned} \beta&= \mathrm{E}\left[ s^2 (\varphi ^*(s))^2\right] =\frac{(c+1) \cdot (2-3 \gamma ^2)}{2 (1-\gamma ^2)} . \end{aligned}$$

(3.89)

1.2 Induced CRB for Real ICA

Here, we briefly review the iCRB for real ICA [41, 45]. In the following, all real quantities $q$ are denoted as $\mathring{q}$. In the derivation of the iCRB for the real case $\mathring{\varphi }(\mathring{s}) = - \partial \ln p(\mathring{s}) / \partial \mathring{s}$ and the parameters $\mathring{\kappa }=E [ \mathring{\varphi }^2(\mathring{s})]$, $\mathring{\eta }=\mathrm{E}[\mathring{s}^2 \mathring{\varphi }^2(\mathring{s})] = 2+\mathrm{E}\left[ \mathring{s}^2 \frac{\partial \mathring{\varphi }(\mathring{s})}{\partial \mathring{s}}\right]$ are defined using real derivatives. In [41, 45] it was shown that

$$\begin{aligned} {{\mathrm{var}}}(\hat{G}_{ii})&\ge \frac{1}{L (\mathring{\eta }_i-1)},\end{aligned}$$

(3.90)

$$\begin{aligned} {{\mathrm{var}}}(\hat{G}_{ij})&\ge \frac{1}{L} \frac{\mathring{\kappa }_j}{\mathring{\kappa }_i \mathring{\kappa }_j -1}. \end{aligned}$$

(3.91)

Appendix 2

Here we derive an analytic expression for ${\mathbf W}_{\text {ICA}}$ in the presence of noise by using a perturbation analysis. Motivated by ${\mathbf W}_{\text {ICA}} \mathop {=}\limits ^{\sigma ^2=0} {\mathbf A}^{-1}$, we assume that ${\mathbf W}_{\text {ICA}}$ can be written as ${\mathbf W}_{\text {ICA}} = {\mathbf A}^{-1} + \sigma ^2 {\mathbf B} + \fancyscript{O}(\sigma ^4)$ and derive ${\mathbf B}$ by a two-step perturbation analysis:

• 1. Taylor series approximation of $\mathrm{E}({\pmb {\varphi }}^*({\mathbf{y}}) {\mathbf{y}}^T)$ in (3.51) at ${\mathbf{y}}={\hat{\varvec{y}}} = {{\mathbf W}}_{\text {ICA}} {{\mathbf A}} {\mathbf{s}}$,

• 2. Taylor series approximation of the result of the above step by exploiting ${\mathbf W}_{\text {ICA}} = {\mathbf A}^{-1}+\sigma ^2 {\mathbf B} + \fancyscript{O}(\sigma ^4)$ and $\hat{\mathbf{y}} = {\mathbf{s}} + \sigma ^2 {\mathbf B} {\mathbf A} {\mathbf{s}}+\fancyscript{O}(\sigma ^4) = {\mathbf{s}} + \sigma ^2 {\mathbf C} {\mathbf{s}}+\fancyscript{O}(\sigma ^4)={\mathbf{s}}+\sigma ^2 {\mathbf{b}} + \fancyscript{O}(\sigma ^4)$ with ${\mathbf C}={\mathbf B}{\mathbf A}$ and $\mathbf{b} = {\mathbf C} \mathbf{s} = [b_1, \dots , b_N]^T$.

In this way, we determine explicitely the deviation $\sigma ^2{\mathbf B}$ of ${\mathbf W}_{\text {ICA}}$ from the inverse solution ${\mathbf A}^{-1}$.

The general Taylor series expansion of $\varphi ^*(y) \mathop {\widehat{=}} \varphi ^*(y,y^*)$ is given as

$$\begin{aligned} \varphi ^*(y,y^*)&=\varphi ^*(\hat{y},\hat{y}^*) + \frac{\partial \varphi ^*}{\partial y} \Delta y + \frac{\partial \varphi ^*}{\partial y^*} \Delta y^* + \frac{1}{2}\left( \frac{\partial ^2 \varphi ^*}{(\partial y)^2} (\Delta y)^2 + \frac{\partial ^2 \varphi ^*}{(\partial y^*)^2} (\Delta y^*)^2\right) \nonumber \\&\quad + \frac{\partial ^2 \varphi ^*}{\partial y \partial y^*} \Delta y \Delta y^* + \ldots \nonumber \\&= \varphi ^*(\hat{y},\hat{y}^*) + \varpi (y,y^*) \Delta y + \vartheta (y,y^*) \Delta y^* \nonumber \\&\quad +\frac{1}{2}\left( \nu (y,y^*) (\Delta y)^2 + \zeta (y,y^*) (\Delta y^*)^2\right) + \epsilon (y,y^*) \Delta y \Delta y^* + \ldots \end{aligned}$$

(3.92)

with $\varpi (y,y^*) = \frac{\partial \varphi ^*}{\partial y}$, $\vartheta (y,y^*) = \frac{\partial \varphi ^*}{\partial y^*}$, $\nu (y,y^*) = \frac{\partial ^2 \varphi ^*}{(\partial y)^2}$, $\zeta (y,y^*) = \frac{\partial ^2 \varphi ^*}{(\partial y^*)^2}$ and $\epsilon (y,y^*) = \frac{\partial ^2 \varphi ^*}{\partial y \partial y^*}$. To simplify notation, we will drop the dependence of $\varphi ^*(\cdot )$, $\varpi (\cdot )$, $\vartheta (\cdot )$, $\nu (\cdot )$, $\zeta (\cdot )$, $\epsilon (\cdot )$ on $y^*$ and keep only the dependence on $y$ in the following.

Let

$$\begin{aligned} \rho _i&=\mathrm{E}\left[ \varpi _i(s_i) s_i^2\right] , \qquad \delta _i = \mathrm{E}\left[ \vartheta _i(s_i) s_i^* s_i\right] ,\end{aligned}$$

(3.93)

$$\begin{aligned} \kappa _i&=\mathrm{E}\left[ \vartheta _i(s_i)\right] , \qquad \quad \quad \xi _i = \mathrm{E}\left[ \varpi _i(s_i)\right] , \end{aligned}$$

(3.94)

$$\begin{aligned} \omega _i&=\mathrm{E}\left[ \nu _i(s_i) s_i\right] , \qquad \quad \tau _i =\mathrm{E}\left[ \zeta _i(s_i) s_i\right] , \end{aligned}$$

(3.95)

$$\begin{aligned} \lambda _i&= \mathrm{E}\left[ \epsilon _i(s_i) s_i\right] , \qquad \quad \gamma _i = \mathrm{E}[s_i^2]. \end{aligned}$$

(3.96)

As shown in [31, 34], ${\mathbf W}_{\text {ICA}} = {\mathbf A}^{-1} + \sigma ^2 {\mathbf C}$, where the elements of ${\mathbf C}$ can be computed from

$$\begin{aligned} \rho _i C_{ii} + \delta _i C_{ii}^* + C_{ii} = - (\kappa _i + \lambda _i )\left[ {\mathbf R}_{-1}\right] _{ii} - (\xi _i + \frac{1}{2}\omega _i) \left[ \bar{{\mathbf R}}_{-1}\right] _{ii} - \frac{1}{2}\tau _i \left[ \bar{{\mathbf R}}_{-1}\right] _{ii}^*. \end{aligned}$$

(3.97)

and

$$\begin{aligned} \gamma _j \xi _i C_{ij} + \kappa _i C_{ij}^* + C_{ji}&= - \kappa _i \left[ {\mathbf R}_{-1}\right] _{ij}^* - \xi _i \left[ \bar{{\mathbf R}}_{-1}\right] _{ij},\nonumber \\ \gamma _i \xi _j C_{ji} + \kappa _j C_{ji}^* + C_{ij}&= - \kappa _j \left[ {\mathbf R}_{-1}\right] _{ji}^* - \xi _j \left[ \bar{{\mathbf R}}_{-1}\right] _{ji}. \end{aligned}$$

(3.98)

with the transformed noise covariance matrix ${\mathbf R}_{-1} = {\mathbf W} {\mathbf R}_{\mathbf{v}} {\mathbf W}^H ={\mathbf A}^{-1} {\mathbf R}_{\mathbf{v}} {\mathbf A}^{-H} + \fancyscript{O}(\sigma ^2)$ and the transformed noise pseudo-covariance matrix $\bar{{\mathbf R}}_{-1}\!=\! {\mathbf W} \bar{{\mathbf R}}_{\mathbf{v}} {\mathbf W}^T = {\mathbf A}^{-1} \bar{{\mathbf R}}_{\mathbf{v}} {\mathbf A}^{-T} + \fancyscript{O}(\sigma ^2)$. Note that ${\mathbf R}_{-1}^H = {\mathbf R}_{-1}$ and $\bar{{\mathbf R}}_{-1}^T = \bar{{\mathbf R}}_{-1}$.

If $p(s,s^*)$ is symmetric in the real part $\mathfrak {R}s$ or imaginary part $\mathfrak {I}s$ of $s$, i.e., $p(-\mathfrak {R}s, \mathfrak {I}s)=p(\mathfrak {R}s, \mathfrak {I}s)$ or $p(\mathfrak {R}s, -\mathfrak {I}s)=p(\mathfrak {R}s, \mathfrak {I}s)$, the parameters $\kappa _i$, $\rho _i$, $\delta _i$, $\lambda _i$, $\xi _i$, $\omega _i$, $\tau _i$ are real. For $\rho _i + 1 \pm \delta _i \ne 0$, we then get from (3.97)

$$\begin{aligned} \mathfrak {R}{C_{ii}}&= - \frac{(\kappa _i + \lambda _i)\left[ {\mathbf R}_{-1}\right] _{ii} + (\xi _i + \frac{1}{2}(\omega _i + \tau _i) )\left[ \mathfrak {R}\bar{{\mathbf R}}_{-1}\right] _{ii}}{\rho _i + 1 + \delta _i}, \nonumber \\ \mathfrak {I}{C_{ii}}&= - \frac{(\xi _i + \frac{1}{2}(\omega _i - \tau _i) )\left[ \mathfrak {I}\bar{{\mathbf R}}_{-1}\right] _{ii}}{\rho _i + 1 - \delta _i}. \end{aligned}$$

(3.99)

For $(\gamma _j \xi _i+\kappa _i)(\gamma _i \xi _j + \kappa _j) \ne 1$ and $(\gamma _j \xi _i-\kappa _i)(\gamma _i \xi _j - \kappa _j) \ne 1$, we obtain from (3.98)

$$\begin{aligned} \mathfrak {R}{C_{ij}}&=\frac{(\kappa _j - \kappa _i ( \gamma _i \xi _j + \kappa _j)) \left[ \mathfrak {R}{\mathbf R}_{-1}\right] _{ij} + ( \xi _j - \xi _i (\gamma _i \xi _j +\kappa _j)) \left[ \mathfrak {R}\bar{{\mathbf R}}_{-1}\right] _{ij}}{(\gamma _j \xi _i+\kappa _i)(\gamma _i \xi _j + \kappa _j)- 1}, \nonumber \\ \mathfrak {I}{C_{ij}}&=\frac{(\kappa _j + \kappa _i ( \gamma _i \xi _j - \kappa _j) )\left[ \mathfrak {I}{\mathbf R}_{-1}\right] _{ij} + ( \xi _j - \xi _i (\gamma _i \xi _j -\kappa _j)) \left[ \mathfrak {I}\bar{{\mathbf R}}_{-1}\right] _{ij}}{(\gamma _j \xi _i-\kappa _i)(\gamma _i \xi _j - \kappa _j)- 1}. \end{aligned}$$

(3.100)