Independent vector analysis based on overlapped cliques of variable width for frequency-domain blind signal separation

Abstract

A novel method is proposed to improve the performance of independent vector analysis (IVA) for blind signal separation of acoustic mixtures. IVA is a frequency-domain approach that successfully resolves the well-known permutation problem by applying a spherical dependency model to all pairs of frequency bins. The dependency model of IVA is equivalent to a single clique in an undirected graph; a clique in graph theory is defined as a subset of vertices in which any pair of vertices is connected by an undirected edge. Therefore, IVA imposes the same amount of statistical dependency on every pair of frequency bins, which may not match the characteristics of real-world signals. The proposed method allows variable amounts of statistical dependencies according to the correlation coefficients observed in real acoustic signals and, hence, enables more accurate modeling of statistical dependencies. A number of cliques constitutes the new dependency graph so that neighboring frequency bins are assigned to the same clique, while distant bins are assigned to different cliques. The permutation ambiguity is resolved by overlapped frequency bins between neighboring cliques. For speech signals, we observed especially strong correlations across neighboring frequency bins and a decrease in these correlations with an increase in the distance between bins. The clique sizes are either fixed, or determined by the reciprocal of the mel-frequency scale to impose a wider dependency on low-frequency components. Experimental results showed improved performances over conventional IVA. The signal-to-interference ratio improved from 15.5 to 18.8 dB on average for seven different source locations. When we varied the clique sizes according to the observed correlations, the stability of the proposed method increased with a large number of cliques.

1 Introduction

When an audio signal is recorded by a microphone in a closed room, it reaches the microphone via not only a direct path, but also infinitely many reverberant paths. The source sound wave is delayed in time and its energy is absorbed by walls when it is delivered by a reverberant path. In order to make the problem practically tractable, the time delay is usually limited to a certain number by which the signal energy may almost disappear through repeated reflections. The signal recorded by a digital microphone can then be modeled by a discrete convolution of a finite impulse response (FIR) filter and the source signal [1–3]. When there are multiple microphones and multiple sources, each microphone recording is expressed by the sum of the convolutions of corresponding transfer functions and source signals [4–6] such that

$$\begin{array}{ll}\hfill x_j\left(t\right)& =\sum _{i=1}^M\sum _{\tau =0}^T{a}_{ji}\left(\tau \right)s_i\left(t-\tau \right)\\ =\sum _{i=0}^M{a}_{ji}\left(t\right)*s_i\left(t\right),j=1,\dots ,N,\end{array}$$

(1)

where the integer numbers j, M, N, and T are, the microphone number, number of sources, number of microphones, and order of the FIR filter, respectively. The time-domain sequences x _j (t) and s _i (t) are the signals recorded by microphone j and generated by source i, respectively, and a _ji (t) is the coefficient at time t of the FIR filter for the transfer function from source i to microphone j; it is affected by the recording environment, including the source and microphone locations. To ensure that the linear transformation is invertible, the number of sources should be equal to the number of microphones, i.e., N = M[4].

This type of problem is often called blind signal separation (BSS) because there is no assumption of the source characteristics. Many studies have been carried out to tackle BSS problems based on independent component analysis (ICA), which minimizes the statistical dependency among the output signals [4–8]. However, direct inversion of the time-domain mixing filter in Equation 1 is difficult and often leads to unstable solutions. To obtain a more stable convergence, the short-time Fourier transform (STFT) is used to convert the convolution in Equation 1 to multiplications in the frequency domain [5]:

$$X_j\left(\omega ,k\right)=A_{ji}\left(\omega \right)S_i\left(\omega ,k\right),j=\mathsf{\text{1}},\dots ,N,$$

(2)

where ω is the center frequency of each component of STFT, and the complex values X _j (ω, k), A _ji (ω), and S _i (ω, k) are STFT components of x _j (t), a _ji (t), and s _i (t), respectively. Note that another discrete time domain exists which is denoted by the dummy variable k. This is different from the real-time variable t, as each value of k corresponds to a frame of the STFT. The value of A _ji (ω) is assumed to be constant over time, so it is not a function of k. Because we use discrete STFT, the center frequency of each discretized frequency bin is expressed as ${\omega }_b=\frac{b}{B}{\omega }_{\text{max}}$, where B is the total number of frequency bins, b denotes the frequency bin number, and ω_max is the maximum frequency equivalent to half of the Nyquist sampling rate. This means that the frequency-domain BSS methods only consider the STFT components at the frequencies in [0 π] [5]. The components at frequencies in [-π 0] can be reconstructed perfectly because a real-valued time-domain signal has a conjugate symmetric Fourier series: $X\left(-\omega \right)=\bar{X}\left(\omega \right)$ for ω ∈ [0 π], where $\overline{X}\left(\omega \right)$ is the complex conjugate of X(ω). For a more compact notation, we rewrite Equation 2 as

$${\mathbf{x}}^b\left[k\right]={\mathbf{A}}^b{\mathbf{s}}^b\left[k\right]b=\mathsf{\text{1}},\mathsf{\text{2}},\dots ,B,$$

(3)

where x^b [k] = [X₁(ω _b , k) . . . X _N (ω _b , k)] ^T , s^b [k] = [S₁(ω _b , k) . . . S _M (ω _b , k)] ^T , and A^b is an N × M matrix whose (j, i)th element is A _ji (ω _b , k). Dealing with the signals in the frequency domain improves the performance since longer filter lengths are better handled in the frequency domain and the convolved mixture problem reduces to an instantaneous mixture problem in each frequency bin; this is expressed as

$${\mathbf{y}}^b\left[k\right]={\mathbf{W}}^b{\mathbf{x}}^b\left[k\right],b=\mathsf{\text{1}},\mathsf{\text{2}},\dots ,B,$$

(4)

where y^b [k] is a vector of M estimated independent sources and W^b is an M × N matrix. Ideally, when W^b = (A^b ) ^-1, we can perfectly reconstruct the original sources by y^b [k] = (A^b ) ^-1A^bs^b [k] = s^b [k]. However, all frequency-domain ICA algorithms inherently suffer from permutation and scaling ambiguity because they assume different frequency components to be independent [4, 9]. The instantaneous ICA may assign individual frequency bins of a single source to different outputs, so grouping the frequency components of individual source signals is required for the success of the frequency-domain BSS [10]. One of the simplest solutions is smoothing the frequency-domain filter [10–12] at the expense of performance because of the lost frequency resolution. There are other methods for colored signals, such as explicitly matching components with larger inter-frequency correlations of signal envelopes [13–15].

Recently, a method called independent vector analysis (IVA) has been developed to overcome the permutation problem by embedding statistical dependency across different frequency components [16–19]. The joint dependency model assumes that the frequency bins of the acoustic sources have radially symmetric distributions [20]. Because speech signals are known to be spherically invariant random processes in the frequency domain [21], such an assumption seems valid and also results in decent separation results. However, when compared to the frequency-domain ICA followed by perfect permutation correction, the separation performance of IVA using spherically symmetric joint densities is slightly inferior [19]. This suggests that such source priors do not exactly match the distribution of speech signals and that the IVA performance for speech separation can be improved by finding better dependency models [22, 23].

We propose a new dependency model for IVA. The single and fully-connected clique is decomposed into many cliques of smaller sizes. A new objective function is derived to account for strong dependency inside the individual cliques and weak dependency across the cliques by retaining a considerable amount of overlap between adjacent cliques. The clique sizes are either fixed or determined by a mel-scale with its frequency index reversed; the latter was proven to be more robust to the increased number of cliques through simulated 2 × 2 speech separation experiments.

This article is organized as follows. Section 2 explains conventional IVA; Section 3 gives a detailed algorithm of the proposed method to contrast with IVA. Section 4 presents the results of the simulated speech separation experiments, and Section 5 summarizes the proposed method and its future extensions.

2 IVA

The key idea behind IVA is that all of the frequency components of a single source are regarded as a single vector, the components of which are dependent on one another. The independence between source vectors is approximated by a multivariate, joint probability density function (pdf) of the components from each source vector, and the joint pdf is maximized rather than the individual independencies between each frequency bin. The IVA model consists of a set of basic ICA models where the univariate sources across different dimensions have some dependency such that they can be grouped and aligned as a multidimensional variable.

Figure 1 illustrates a 2 × 2 IVA mixture model. Let the multidimensional source vector be ${\mathbf{s}}_i={\left[s_i^1,s_i^2,\dots ,s_i^B\right]}^T$ for i = 1, 2. Each component of s₁ is linearly mixed with the component in the same dimension of s₂ by A^b such that

Figure 1

Mixture model of IVA.

$$\begin{array}{l}\hfill \left[\begin{array}{c}\hfill x_1^b\hfill \\ \hfill x_2^b\hfill \end{array}\right]=\left[\begin{array}{c}\hfill a_{11}^b{a}_{12}^b\hfill \\ \hfill a_{21}^b{a}_{22}^b\hfill \end{array}\right]\left[\begin{array}{c}\hfill s_1^b\hfill \\ \hfill s_2^b\hfill \end{array}\right]\\ \hfill =\left[\begin{array}{c}\hfill a_{11}^b{s}_1^b+a_{12}^b{s}_2^b\hfill \\ \hfill a_{21}^b{s}_1^b+a_{22}^b{s}_2^b\hfill \end{array}\right],\end{array}$$

(5)

for b = 1, . . . , B. For microphone j = 1, 2, the observation vector is expressed as

$${\mathbf{x}}_j=\left[\begin{array}{c}\hfill x_j^1\hfill \\ \hfill x_j^2\hfill \\ \hfill ⋮\hfill \\ \hfill x_j^B\hfill \end{array}\right]=\left[\begin{array}{c}\hfill a_{j1}^1{s}_1^1+a_{j2}^1{s}_2^1\hfill \\ \hfill a_{j1}^2{s}_1^2+a_{j2}^2{s}_2^1\hfill \\ \hfill ⋮\hfill \\ \hfill a_{j1}^B{s}_1^B+a_{j2}^B{s}_2^B\hfill \end{array}\right].$$

(6)

The mixing of the multivariate sources is dimensionally constrained so that a linear mixture model is formulated in each layer. The instantaneous ICA is extended to a formulation with multidimensional variables or vectors, where the mixing process is constrained to the sources on the same horizontal layer or on the same dimensions. The joint dependency within the dependent sources is modeled by a multidimensional pdf, and hence, correct permutation is achieved.

To derive the objective function of IVA, a single dimension of the estimated sources in Equation 4 is extracted, and a new vector is constructed by collecting the source coefficients of all the frequency bins. The source estimate y _i is expressed by the following matrix-vector multiplication:

$${\mathbf{y}}_i=\left[\begin{array}{c}\hfill y_i^1\hfill \\ \hfill y_i^2\hfill \\ \hfill ⋮\hfill \\ \hfill y_i^B\hfill \end{array}\right]=\left[\begin{array}{c}\hfill \sum _{j=1}^N{w}_{ij}^1{x}_j^1\hfill \\ \hfill \sum _{j=1}^N{w}_{ij}^2{x}_j^2\hfill \\ \hfill ⋮\hfill \\ \hfill \sum _{j=1}^N{w}_{ij}^B{x}_j^B\hfill \end{array}\right]$$

(7)

$$=\left[\begin{array}{c}\hfill {\mathbf{w}}_i^1\hfill \\ \hfill {\mathbf{w}}_i^2\hfill \\ \hfill ⋮\hfill \\ \hfill {\mathbf{w}}_i^B\hfill \end{array}\right]\left[{\mathbf{x}}_1{\mathbf{x}}_2\dots {\mathbf{x}}_N\right],$$

(8)

where ${\mathbf{w}}_i^b$ is the i th row of matrix W^b and ${\mathbf{w}}_{ij}^b$ is the j th element of ${\mathbf{w}}_i^b$. For a simple derivation of the IVA algorithm, we assume that $y_i^b$ has a unit variance to eliminate the variance terms from the original IVA learning algorithm [19]. This can easily be achieved by scaling ${\mathbf{w}}_i^b$ appropriately such that

$${\mathbf{w}}_i^b\leftarrow {\mathbf{w}}_i^b/\sqrt{E\left[|y_i^b{|}^2\right]}.$$

(9)

In resynthesis, the above normalization is reversed to restore the original scales. The likelihood of y _i is computed by the following multivariate pdf [19, 20]:

$$p\left({\mathbf{y}}_i\right)\propto \text{exp}\left(-\sqrt{\sum _{b=1}^B|y_i^b{|}^2}\right).$$

(10)

The goal of IVA is optimizing {W¹, W², . . . , W^B } to maximize the independence among the separated sources, {y₁, y₂, . . . , y _M }, where the independence is approximated by the sum of the log likelihoods of the given data computed by Equation (10). The detailed learning algorithm can be found in [19, 20].

Figure 2 illustrates the mixing assumption and how the IVA algorithm works. Two sources are mixed at different amounts in different frequency bins. To find y₁ and y₂ for the estimates of s₁ and s₂, IVA instead estimates the unmixing matrices to minimize the dependency between different sources while maintaining strong dependency across frequency bins. There is only a single dependency model in which all the frequency bins distinguished by their center frequencies are connected to one another: that is, the spherical dependency described by Equation (10).

Figure 2

Mixing and separation models of conventional IVA.

3 Proposed dependency models for IVA

For real-sound sources, it is unreasonable for neighboring and distant frequency components to be assigned the same dependency because the dependency of neighboring frequency components is much stronger than that of distant frequency components. This section describes the proposed dependency models in which the single and fully connected statistical dependency of IVA is decomposed into several cliques whose sizes are set to be fixed or mel-scaled. The details of the proposed models are explained in this section.

3.1 Overlapped cliques of a fixed size

The statistical dependency between adjacent frequency components is much larger than that between distant components. For example, the dependency between $y_i^b$ and $y_i^{b+1}$ for an arbitrary b is much stronger than that between $y_i^b$ and $y_i^{b+k}$ when k ≫ 1. We considered the difference in center frequencies of the STFT components in the proposed dependency model. As shown in Figure 3, the clique of the components of the estimated source vectors y _i was broken into several cliques in order to eliminate the direct dependency between distant frequency bins. This segmentation of the spherical model can be visualized as a chain of cliques [23]. The dependency among the source components propagates through chain-like overlaps of spherical dependencies such that the dependency between components weakens as the distance between them grows. The corresponding multivariate pdf is given in the following form:

Figure 3

Illustration of the proposed dependency model.

$$p\left({\mathbf{y}}_i\right)\propto \text{exp}\left(-\sum _{c=1}^C\sqrt{\sum _{b=f_c}^{l_c}|y_i^b{|}^2}\right),$$

(11)

where C is the number of cliques, and f _c and l _c are the first and last indices, respectively, of clique c designed to satisfy the condition

$$f_c<l_{c-1},c=\mathsf{\text{2}},\mathsf{\text{3}},\dots ,C,$$

(12)

so that the series of cliques have chained overlaps. With the proposed source prior in Equation (11), we derive a new learning algorithm to find a set of linear transformation matrices that make the components as statistically independent as possible, such that

$$\left\{{\mathbf{W}}^{b*}\right\}=\underset{\left\{{\mathbf{W}}^b\right\}}{\text{arg}\text{max}}\mathcal{L}\left(\left\{{\mathbf{W}}^b\right\}\right),$$

(13)

where the log likelihood function is defined as

$$\begin{array}{c}\mathcal{L}\left(\left\{{\mathbf{W}}^b\right\}\right)\propto \text{log}\left[\prod _b^B\left|\text{det}{\mathbf{W}}^b\right|\cdot \prod _i^{M}p\left({\mathbf{y}}_i\right)\right]\\ =\sum _b^B\text{log}\left|\text{det}{\mathbf{W}}^b\right|+\sum _i^M\text{log}p\left({\mathbf{y}}_i\right)\\ =\sum _b^B\text{log}\left|\text{det}{\mathbf{W}}^b\right|-\sum _i^M\sum _{c=1}^C\sqrt{\sum _{b=f_c}^{l_c}\left|\underset{b}{\overset{i}{y}}\right|},\end{array}$$

(14)

where M is the number of sources defined in Equation (1). We apply the natural gradient learning rule [24] to W^b at each frequency bin b:

$$\Delta {\mathbf{W}}^b\propto \left[\mathbf{I}-\phi \left({\mathbf{y}}_i^b\right){\mathbf{y}}_i^b\right]{\mathbf{W}}^b,$$

(15)

where I is an M × M identity matrix, (·) ^H is the Hermitian transpose operator, and φ(y^b ) is a vector function whose i^th element is

$$\begin{array}{ll}\hfill {\left[\phi \left({\mathbf{y}}^b\right)\right]}_i& =\frac{\partial \text{log}p\left(y_i^b\right)}{\partial y_i^b}\\ =\sum _{c\in {\mathcal{S}}_b}\frac{y_i^b}{\sqrt{\sum _{b=f_c}^{l_c}|y_i^b{|}^2}},\end{array}$$

(16)

where ${\mathcal{S}}_b$ is a set of cliques that includes bin b. At every adaptation step, W^b is constrained to be orthogonal by the following symmetric decorrelation scheme:

$${\mathbf{W}}^b\leftarrow {\left({\mathbf{W}}^b{\left({\mathbf{W}}^b\right)}^H\right)}^{-\frac{1}{2}}{\mathbf{W}}^b,b=\text{1},\text{2},\dots ,B.$$

(17)

At the end of the learning, the well-known minimal distortion principle [25] is applied to W^b by

$${\mathbf{W}}^b\leftarrow \text{diag}\left({{\left({\mathbf{W}}^b\right)}^{-}}^{\text{1}}\right){\mathbf{W}}^b,b=\text{1},\text{2},\dots ,B.$$

(18)

To select an appropriate set of cliques that is suited to our goal, we constructed a matrix of size B × B whose (i, j)th element is the correlation coefficient between bin i and bin j from a single source. Figure 4A-D shows the computed correlation coefficient matrices obtained from four different speech signals of two females and two males. In all four cases, a strong correlation was observed around the diagonal with a positive slope because they were from closely located frequency pairs. The correlation decreased as it went off-diagonal. Although the low-frequency components had a widespread dependence over the 0-3 kHz region, it was much weaker than that along the positive sloping diagonal. All of the speech signals are from the TIMIT database, and the same observations held true for other speech signals as well. To consider strong correlations among neighboring frequency bins, we adopted a dependency graph consisting of several cliques of the same size and increasing center frequencies. Taking 1,024 frequency bins as an example, the beginning and ending indices of Equation (11) were [f₁l₁] = [1 256], [f₂l₂] = [2 257], [f₃l₃] = [3 258], . . ., [f _C l _C ] = [769 1024], where the number of frequency bins for each clique was fixed to 256. This simple dependency model using overlapped cliques is shown in Figure 5. All of the cliques were of the same size but with varying center frequencies.

Figure 4

Correlation coefficient matrix of (A) female speech; (B) male speech; (C) another female; (D) another male. All speech signals are from the TIMIT database.

Figure 5

Dependency model of fixed clique size.

3.2 Overlapped cliques of variable sizes

Figure 6 shows another model that reflects the spread dependence at low frequencies. The cliques have variable sizes based on the reversed mel-frequency scale. We adopted the mel-scale to prevent being biased to any specific cases; this scale has been proven to be efficient in numerous speech signal-processing applications such as speech recognition and enhancement. General human speech is characterized by rapid changes occurring more often in the lower-frequency regions. Therefore, most auditory frequency scales, including the mel-scale, use a narrow bandwidth for the low-frequency region based on the observation that there is little dependence among neighboring frequencies [26]. In the high-frequency region, there is greater dependence among neighboring frequencies, so a relatively large bandwidth is used. However, in the proposed method, we set the sizes of the bands in the opposite fashion. We assigned larger clique sizes to low frequencies because they have less statistical dependence to one another, and smaller clique sizes to higher frequencies. Since the cliques play the role of joining the same source components distributed in different frequencies, a larger bandwidth is necessary to cover the weak and spread dependence in the low-frequency region. For higher frequencies, a smaller amount of overlap is enough because of the greater dependence among neighboring frequency components, as shown in Figure 4. The overlapped vertices between the adjacent cliques in the dependency graph enables collection of the same source components. Therefore, the clique size is determined by the reversed mel-scale, which is computed by

Figure 6

Dependency model of mel-scale clique sizes.

$$h\left({\omega }_c\right)=A\left[{\text{log}}_{10}\left(1+\frac{{\omega }_c}{700}\right)-{\text{log}}_{10}\left(1+\frac{{\omega }_c-1}{700}\right)\right],$$

(19)

where ω _c is the center frequency of clique c, A is a constant, and h(ω _c ) is the bandwidth of clique c. The beginning and ending indices f _c and l _c in Equation (11) are then obtained by

$$f_c=\text{max}\left(\text{1},b_c-h\left({\omega }_c\right)\right),l_c=\text{min}\left(B,b_c+h\left({\omega }_c\right)\right),$$

(20)

where b _c is the center-bin number of clique c. The max and min operators ensure that the computed bin numbers are within a valid range.

4 Experiments

We compared the performance of the audio source separation using the proposed dependency models with that of the fully connected dependency model of the conventional IVA. Both methods were applied to multiple speech separation problems. The geometric configuration for the simulated room environments is shown in Figure 7. Various 2×2 cases were simulated by combining pairs of source locations from A to J. For example, experiment 1 was a combination of sound source 1 from location A and sound source 2 from location H, experiment 2 was a combination of sources from locations B and G, etc. We set the dimensions of the room to 7 m × 5 m × 2.75 m and the heights of all microphones and source locations to 1.5 m. The reverberation time was 100 ms, and the corresponding reflection coefficient was 0.57 for every wall, floor, and ceiling. Room impulse responses were obtained by an image method [1–3] using the above parameters. The impulse responses of the transfer functions from source locations A and H to the two microphones are shown in Figure 8. The peak location was not at the origin because the direct path had its own delay. The filter length was 100 ms, which was equivalent to 800 tabs at an 8-kHz sampling rate. The amplitude dropped rapidly because of the loss of energy due to the reflection.

Figure 7

Geometric configuration of the simulated room environments.

Figure 8

Impulse responses of the transfer function of a simulated room. The source locations are A and H.

Male and female speech signals chosen from the TIMIT database were synthetically convolved with the impulse responses corresponding to the locations of the sources and microphones in each experiment. When the algorithm was applied to source separation in the STFT domain, a 2048-point FFT, 2048-tab Hanning window, and shift size of 512 samples were used. The separation performance was measured in terms of the signal-to-interference ratio (SIR) which is defined as [19]:

$$\text{SIR}=10\text{log}\left(\frac{{\sum }_{k,b}|{\sum }_i{r}_{iq\left(i\right)}^b{s}_{q\left(i\right)}^b\left[k\right]{|}^2}{{\sum }_{n,b}|{\sum }_{i\ne j}{r}_{iq\left(j\right)}^b{s}_{q\left(j\right)}^b\left[k\right]{|}^2}\right),$$

(21)

where q(i) indicates the separated source index of the i th source and r _iq(j) is the overall impulse response computed by $r_{iq\left(j\right)}={\sum }_m{w}_{im}^b{a}_{mq\left(j\right)}^b$. In order to represent how close the estimated ${\mathbf{W}}_i^b$ was to the inverse of the mixing filters ${\mathbf{A}}_j^b$, the SIR numbers were measured in decibels, because the acoustic signal power ratio is in the log scale [26]. The higher SIR is, the closer the result is to perfect separation.

We compared the single clique model of IVA with the proposed multiple clique models. The multiple clique designs are shown in Figure 9. The numbers of cliques were 2, 4, 8, 12, and 16, and the overlap ratio between neighboring cliques was set to 50%. In A-E, the center frequencies were "linearly" increased, and the sizes were all fixed except for the first and last because they were located at opposite ends. For example, the four cliques in Figure 9B cover the frequency regions of 0-1.5, 0.5-2.5, 1.5-3.5, and 2.5-4 kHz. The neighboring cliques overlap by 50%, so the dependency is well propagated. In contrast, the center frequencies of F-J are on the "reversed mel-scale" in Equation (19): the clique sizes are inversely proportional to the rate of change in the mel-scale. The same four cliques in Figure 9G cover 0-2.2, 1.1-3.1, 2.4-3.7, and 3.3-4 kHz. Their actual bandwidths were 2.2, 2.0, 1.36, and 0.74 kHz, although the bandwidths computed by Equation (19) were 1.47, 1.02, 0.68, and 0.49 kHz. Because the first and last cliques had only one neighbor, their sizes were 1.5 times larger than the expected bandwidths, while the sizes of the second and the third cliques were twice as large to impose a 50% overlap with neighboring cliques.

Figure 9

Various clique designs. (A)-(E) The center frequencies are linearly scaled, and the clique sizes are equal. (F)-(J) The center frequencies and clique sizes are on an inverse mel-scale. The "CR" values are the ratios of the sum of the correlation coefficients included in the cliques to the sum of all of the coefficients.

The "CR" number in each of the clique designs in Figure 9 is the ratio of the sum of correlation coefficients enclosed by the union of all the cliques to the sum of the total correlation coefficients. It approaches unity as the enclosed region approaches the total area. The correlation map is identical to Figure 4A from the speech of female 1, who was one of the input sources of our experiments. The CR number does not account for the separation performance directly but roughly shows how well a clique design models the dependence of the frequency bins.

All of the separation performances were measured for their SIR and are summarized in Table 1. The first "IVA" row represents the SIR numbers obtained by the conventional IVA algorithm [19]. Rows labeled "LIN2," "LIN4," "LIN8," "LIN12," and "LIN16" are the results of the proposed models utilizing the clique designs in Figure 9A-E, and rows labeled "MEL2," ..., "MEL16" are the results with the clique designs in Figure 9F-J. Columns indicate various combinations of source locations, average SIR (denoted by "SIR") of the seven experiments, average number of iterations (denoted by "Iter.") for the solution to converge, and CR number of the corresponding clique design. The average SIRs that were larger than 18 dB are boldfaced. Among the linear scales, the average SIRs of LIN4 and LIN8 were 18.7 and 18.8 dB, and the average numbers of iterations were 397 and 544, respectively. These indicate that LIN4 and LIN8 greatly improved both the separation performance and convergence speed compared to IVA. However, when the number of cliques was more than 8, SIR degraded rapidly (13.9 and 12.0 dB), and the separation performance were even poorer than those of IVA. Among mel-scales, the average SIRs of MEL4 and MEL8 were both 18.7 dB, and their numbers of iterations were 543 and 408, respectively, which were about the same as those of LIN4 and LIN8. The difference from the linear scales was when the number of cliques was more than 8: the separation performance measured by SIR did not degrade as badly as that of LIN12 and LIN16. However, many more number of iterations was required for both MEL12 and MEL16, implying that the broken dependency made the algorithm oscillate around the optimal solution. When comparing LIN and MEL, their best SIRs were almost the same, but the average iterations revealed that the mel-scales were more robust for large numbers of cliques. This can be explained by comparing the amount of correlation captured by the clique designs. Figure 9 shows that the CR numbers of MEL12 and MEL16 were 0.356 and 0.291, and those of LIN12 and LIN16 were 0.333 and 0.272, respectively. For 12 and 16 cliques, mel-scale designs had CR numbers larger by about 0.02 than the linear-scale designs. The difference mostly originated from the low-frequency region: the spread dependence observed at 1-2 kHz was better captured by the mel-scale cliques, and which in turn enabled correct source permutation. In summary, the proposed method was more effective than the original IVA in most clique configurations in terms of separation performance, and the mel-scale clique designs were better than the fixed-size designs in terms of stability.

Table 1 Separation performances in SIR (dB)

5 Conclusions

The totally spherical dependency model of IVA was relaxed by the dependency models of chained cliques. The new clique designs are advantageous because the weak dependency among distant frequencies is modeled by indirect dependency propagation, which helps in finding a better local solution compared to the original IVA, where the same amount of dependency is assigned to any pair of frequency bins. In this article, two types of non-spherical models are proposed. The first uses the same number of frequency bins for all of the cliques, while the other varies the number of frequency bins in reversed mel-scales based on the measured correlation coefficients between different frequency bins. Both dependency models achieved higher source separation performance and faster convergence to correct solutions owing to more accurate modeling of the statistical dependency. For simulated mixtures of male and female speech signals, both models obtained the highest performance when the number of cliques was set to 4 or 8. When the clique size was fixed, the performance degraded drastically for more than eight cliques. However, when the clique size was determined by the mel-scales, the same level of performance was kept at the expense of convergence rate. This implies the presence of up to 16 independent units in speech signals along the mel-scale frequency axis. One of the ongoing research issues is finding more flexible dependency models, such as instantaneously varying the dependency graph based on the correlation coefficients measured from the input signals or on their harmonic structures. Another research issue is finding appropriate dependency models for natural sounds because the dependency among the frequency components may not be related to the mel-scale.