A Fast Gradient Approximation for Nonlinear Blind Signal Processing

Abstract

When dealing with nonlinear blind processing algorithms (deconvolution or post-nonlinear source separation), complex mathematical estimations must be done giving as a result very slow algorithms. This is the case, for example, in speech processing, spike signals deconvolution or microarray data analysis. In this paper, we propose a simple method to reduce computational time for the inversion of Wiener systems or the separation of post-nonlinear mixtures, by using a linear approximation in a minimum mutual information algorithm. Simulation results demonstrate that linear spline interpolation is fast and accurate, obtaining very good results (similar to those obtained without approximation) while computational time is dramatically decreased. On the other hand, cubic spline interpolation also obtains similar good results, but due to its intrinsic complexity, the global algorithm is much more slow and hence not useful for our purpose.

Appendix: Post-nonlinear Blind Source Separation (PNL-BSS)

The PNL-BSS Model

In [8, 9], Taleb and Jutten have studied a realistic case of nonlinear mixtures, called post-nonlinear (PNL) mixtures which are separable. As it is shown in Fig. 8, this two-stage system consists of a linear mixing matrix A, followed by component-wise nonlinear distortions $f_1(.),{\ldots},f_n(.)$. We assume that the input of the system consists of a set of n independent sources $s_1,{\ldots},s_n$. The square matrix ${\bf A}\in{{{\rm I}\!{\rm R}}^{n\times n}}$ is assumed unknown and invertible, and the nonlinear distortions (memoryless) ${f_i:\mathbb{R}\rightarrow \mathbb{R}}$ ($i=1,2,{\ldots},n$) are also assumed unknown and invertible. Again, we assume that all involved stochastic processes are wide sense stationary and ergodic, so expectations are computed using time samples.

Fig. 8

The PNL-BSS model. The unknown nonlinear mixing system and the separating system are shown in the left and in the right, respectively

As in the blind deconvolution case, we will need to compute the score function of the outputs. Since now we have several outputs, each one with a corresponding score function ψ _i , we also define a vector of score functions, the so-called marginal score function $\Uppsi=[\psi_1, \psi_2,{\ldots},\psi_n]^T$.

The Min-MI Post-Nonlinear Blind Source Separation Algorithm (Min-MI PNL-BSS)

An algorithm for solving this problem was introduced in [6] where the separating system coefficients (matrix B) and the compensation nonlinearities $g_1,{\ldots},g_n$ are chosen such that the mutual information among estimated sources $y_1,{\ldots},y_n$ is minimized. The mixing observations are:

$$ e_i(t) = f_i\left(\sum_{j=1}^n a_{ij}s_j(t)\right), $$

(8)

where $i=1,2,{\ldots},n, s_j(t)$ ($j=1,2,{\ldots},n$) are the independent sources, e _i (t) is the i-th observation, a _ij denotes the entries of the unknown mixing matrix A, and f _i is the unknown nonlinear mapping on the component i.

We highlight that, since the statistical independence among sources is the main assumption, the separation structure is tuned so that the components of its output become statistically independent. Similar to the previous case (NL-BD) where we minimized MI between time simples of the (only) output, in the PNL-BSS we have to minimize the MI between estimated sources.

The gradient of the MI with respect to the parameters of the system can be written as follows (see [8] for a detailed derivation of these equations). Special perturbation signals $\varepsilon (x_i(t))$ are needed to compensate the nonlinear distortions. The vector composed by the perturbation signals is defined as follows:

$$ \varepsilon (x_j(t)) = -\mathbb{E}\left[\left\{ \sum_{l=1}^n \psi_l(s_l(t))b_{lj}\right\} K(y_j(t)-\tau_j) + K^{\prime}(y_j(t)-\tau_j)\right], $$

(9)

additionally, the gradient of the MI with respect to matrix B becomes

$$ \frac{\partial {\bf I}({\bf y})}{\partial {\bf B}}=-\mathbb{E}\left[{\bf \Uppsi}_{{\bf y}}({\bf y}){\bf x}^T\right] - {\bf B}^{-T}, $$

(10)

where K must satisfy an additional condition of positivity in order to insure convergence of the gradient method.

As proposed in the NL-BD case, replacing expectations by their corresponding time average, we can obtain practical equations analogous to Eqs. (5) and (6) for the deconvolution problem. Additionally, we are able to reduce the complexity of the algorithm by using the same strategy as proposed for NL-BD, that is, by computing Eq. (9) only at N points in a regular grid covering the ranges of the variables. In Algorithm 2, the steps of the Min-MI PNL-BSS Algorithm are shown.

Algorithm 2 Min-MI PNL-BSS [8]
INPUT: Maximum number of iterations Niter and output signal e(t)
OUTPUT: Estimated input signal $\widehat{{\bf s}}(t)$
1: i = 1; B = I; Initialization
2: y = B e
3: i ≤ Niter and NOT convergence is reached do
4:  for j = 1:n do
5:     Estimate the score function ψ j (Eq. (4))
6:     Compute the perturbation signal $\varepsilon (x_j(t))$ (Eq. (9))
7:     $x_j(t)\leftarrow x_j(t)+\mu_1 \varepsilon(x_j(t))$; Nonlinear compensationa
8:     $x_j(t)\leftarrow (x_j(t)-m_{x_j})/\sigma_{x_j}$; Normalization
9:   end for
10:   Estimate $E=\frac{\partial {\bf I}({\bf y})}{\partial {\bf B}}$ (Eq. (10))
11:   ${\bf B}\leftarrow {\bf B}+\mu_2 E$; Separating matrix updatea
12:   y = B x; Current Estimation of source signals
13:   for j = 1:n do
14:     $y_j(t)\leftarrow (y_j(t)-m_{y_j})/\sigma_{y_j}$; Normalizationb
15:     ${\bf B}\leftarrow {\bf \Uplambda}^{-1}{\bf B}$; Normalizationc
16:   end for
17:   i = i + 1;
18: end while
19: $\widehat{{\bf s}}(t)={\bf y}(t)$;

1. ^aμ₁ and μ₂ are small scalars, typically 0.1–0.2
2. ^b m _x and σ _x are the mean and the standard deviation of the signal x(t) respectively
3. ^c ${\bf \Uplambda}$ is a diagonal matrix with the standard deviations of each source in its main diagonal, i.e. ${\bf \Uplambda}_{i,i}=\hat{\sigma}_{\hat{s}_i}$