Task-Driven Variability Model for Speaker Verification

Abstract

The total variability model (TVM)/probabilistic linear discriminant analysis (PLDA) framework is one of the most popular methods for speaker verification. In this framework, the i-vector representations are first extracted from utterances via an estimated TVM and then employed to estimate the PLDA parameters for classification. The TVM and PLDA are estimated serially, so the information loss in the TVM is inherited by the i-vectors, and then passed into the PLDA classifier. More seriously, this loss cannot be compensated by the PLDA. To solve this problem, we propose a task-driven variability model (TDVM) to jointly estimate the TVM and PLDA classifier. In this method, the feedback from the PLDA can supervise the optimal solution of the TVM to move toward the space that has the maximum between-class separation and minimum within-class variation. Meanwhile, this space is suitable for open-set test which can deal with unenrolled speakers. Unlike most embedding methods which extract the embedding representations via the stack of network structures, the TDVM contains the assumptions about latent variables, which can enhance the interpretation of speaker representation extraction. The proposed method is evaluated on the King-ASR-010 and VoxCeleb databases, and the experimental results show that the TDVM method can achieve better performance than the traditional TVM/PLDA and VGG-M network with different cost functions.

A Appendix

1.1 A.1 Derivation of Lower-Level Objective Function

The logarithmic likelihood function of parameter $\varvec{T}$ can be expressed as:

$$\begin{aligned} \begin{aligned} L(\varvec{T})&= \sum _{s,h}\log P(\varvec{M}_{s,h};\varvec{T})\\&= \sum _{s,h}\log \sum _{\varvec{w}_{s,h}} P(\varvec{M}_{s,h},\varvec{w}_{s,h};\varvec{T})\\&= \sum _{s,h}\log \sum _{\varvec{w}_{s,h}}Q(\pmb w_{s,h})\frac{P(\varvec{M}_{s,h},\varvec{w}_{s,h};\varvec{T})}{Q(\pmb w_{s,h})} \end{aligned} \end{aligned}$$

(20)

where $Q(\pmb w_{s,h})$ is an auxiliary function. Since logarithmic likelihood function $f(x) = \log (x)$ is a concave function, according to the Jensen’s inequality rule, $\log (\mathbb E[x])\ge {\mathbb {E}}[\log (x)]$ can be obtained. Thus, (20) can be transformed into the following:

$$\begin{aligned} \begin{aligned} L(\varvec{T})&= \sum _{s,h}\log \sum _{\varvec{w}_{s,h}}Q(\varvec{w}_{s,h})\frac{P(\varvec{M}_{s,h},\varvec{w}_{s,h};\varvec{T})}{Q(\varvec{w}_{s,h})}\\&\ge \sum _{s,h}\sum _{\varvec{w}_{s,h}}Q(\varvec{w}_{s,h})\log \frac{P(\varvec{M}_{s,h},\varvec{w}_{s,h};\varvec{T})}{Q(\varvec{w}_{s,h})}\\&=\sum _{s,h}\sum _{\varvec{w}_{s,h}}Q(\varvec{w}_{s,h})\log \frac{P(\varvec{M}_{s,h}|\varvec{w}_{s,h};\varvec{T})P(\varvec{w}_{s,h})}{Q(\varvec{w}_{s,h})}\\&=\sum _{s,h}{\mathbb {E}}_{\varvec{w}}\Big [\log P(\varvec{M}_{s,h}|\varvec{w}_{s,h};\varvec{T})+\log P(\varvec{w}_{s,h})-\log Q(\varvec{w}_{s,h})\Big ]. \end{aligned} \end{aligned}$$

(21)

In (21), $\varvec{M}_{s,h}|\varvec{w}_{s,h}\sim {\mathbb {N}}(\varvec{m} + \varvec{Tw}_{s,h},\varvec{\varPhi })$; ${\mathbb {E}}_{\varvec{w}}$ is short for $\mathbb E_{\varvec{w}_{s,h}\sim Q}$, and the “$\varvec{w}_{s,h}\sim Q$” subscript indicates that the expectation is with respect to $\varvec{w}_{s,h}$ drawn from Q. Then, the items not related to $\varvec{T}$ are removed; therefore, the lower-level function $f_L$ can be written as (9).

1.2 A.2 Convergence Analysis

For the development supervector set, we have:

$$\begin{aligned} \begin{aligned} \prod _{s,h}P(\varvec{M}_{s,h};\theta )&=\frac{\prod _{s,h}P(\varvec{M}_{s,h},\varvec{w}_{s,h};\theta )}{\prod _{s,h}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta )}\\&=\frac{\prod _{s,h}P(\varvec{M}_{s,h},\varvec{w}_{s,h},\varvec{z}_{s};\theta )}{\prod _{s,h}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta )P(\varvec{z}_{s}|\varvec{w}_{s,h},\varvec{M}_{s,h};\theta )} \end{aligned} \end{aligned}$$

(22)

where $\theta$ is the parameter set $\{\varvec{T},\varvec{\varLambda },\varvec{\varPsi }\}$. Then, the logarithm of (22) can be expressed as:

$$\begin{aligned} \begin{aligned} \sum _{s,h}\log P(\varvec{M}_{s,h};\theta )=&\sum _{s,h}\log P(\varvec{M}_{s,h},\varvec{w}_{s,h},\varvec{z}_{s};\theta )\\&-\sum _{s,h}\log P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta )\\&-\sum _{s,h}\log P(\varvec{z}_{s}|\varvec{w}_{s,h},\varvec{M}_{s,h};\theta ). \end{aligned} \end{aligned}$$

(23)

Here, for each iteration index i of the TDVM, we set $L(\theta ,\theta ^{(i)})$, $H(\theta ,\theta ^{(i)})$ and $G(\theta ,\theta ^{(i)})$ as follows:

$$\begin{aligned} \left\{ \begin{aligned} L(\theta ,\theta ^{(i)}) =&\sum _{s,h}\sum _{\varvec{w}_{s,h}}\sum _{\varvec{z}_{s}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})P(\varvec{z}_{s}|\varvec{w}_{s,1},\dots ,\varvec{w}_{s,H_{s}};\theta ^{(i)}) \\&\qquad \cdot \log P(\varvec{M}_{s,h},\varvec{w}_{s,h},\varvec{z}_{s};\theta ) \\ H(\theta ,\theta ^{(i)}) =&\sum _{s,h}\sum _{\varvec{w}_{s,h}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})\log P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ) \\ G(\theta ,\theta ^{(i)}) =&\sum _{s,h}\sum _{\varvec{w}_{s,h}}\sum _{\varvec{z}_{s}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})P(\varvec{z}_{s}|\varvec{w}_{s,1},\dots ,\varvec{w}_{s,H_{s}};\theta ^{(i)}) \\&\qquad \cdot \log P(\varvec{z}_{s}|\varvec{w}_{s,h},\varvec{M}_{s,h};\theta ). \end{aligned} \right. \end{aligned}$$

(24)

Therefore, $\sum _{s,h}\log P(\varvec{M}_{s,h};\theta )$ can be rewritten as follows:

$$\begin{aligned} \sum _{s,h}\log P(\varvec{M}_{s,h};\theta )=L(\theta ,\theta ^{(i)})-H(\theta ,\theta ^{(i)})-G(\theta ,\theta ^{(i)}). \end{aligned}$$

(25)

In (25), $\theta$ is taken as $\theta ^{(i+1)}$ and $\theta ^{(i)}$, respectively, and then they are subtracted to obtain the follows formula:

$$\begin{aligned} \begin{aligned}&\sum _{s,h}\log P(\varvec{M}_{s,h};\theta ^{(i+1)})-\sum _{s,h}\log P(\varvec{M}_{s,h};\theta ^{(i)})\\&\quad =\Big [L(\theta ,\theta ^{(i+1)})-L(\theta ,\theta ^{(i)})\Big ]-\Big [H(\theta ,\theta ^{(i+1)})-H(\theta ,\theta ^{(i)})\Big ]\\&\qquad -\Big [G(\theta ,\theta ^{(i+1)})-G(\theta ,\theta ^{(i)})\Big ]. \end{aligned} \end{aligned}$$

(26)

For the first item, since $\theta ^{(i+1)}$ makes $L(\theta ,\theta ^{(i+1)})$ take the maximum value, the following inequality can be obtained:

$$\begin{aligned} L(\theta ,\theta ^{(i+1)})-L(\theta ,\theta ^{(i)}) \ge 0. \end{aligned}$$

(27)

For the second item, it can be transformed as follows:

$$\begin{aligned} \begin{aligned}&~-\Big [H(\theta ,\theta ^{(i+1)})-H(\theta ,\theta ^{(i)})\Big ]\\&\quad =~-\sum _{s,h}\sum _{\varvec{w}_{s,h}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})\log \frac{P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i+1)})}{P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})}\\&\quad \ge ~\sum _{s,h}\log \Big [\sum _{\varvec{w}_{s,h}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})\frac{P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i+1)})}{P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i)})}\Big ]\\&\quad =~\sum _{s,h}\log \Big [\sum _{\varvec{w}_{s,h}}P(\varvec{w}_{s,h}|\varvec{M}_{s,h};\theta ^{(i+1)})\Big ]=0. \end{aligned} \end{aligned}$$

(28)

The third item is similar to the second one, and therefore, the following inequality can be obtained:

$$\begin{aligned} -[G(\theta ,\theta ^{(i+1)})-G(\theta ,\theta ^{(i)})] \ge 0. \end{aligned}$$

(29)

According to (27)–(29), we have:

$$\begin{aligned} \begin{aligned}&\sum _{s,h}\log P(\varvec{M}_{s,h};\theta ^{(i+1)})-\sum _{s,h}\log P(\varvec{M}_{s,h};\theta ^{(i)})\ge 0. \end{aligned} \end{aligned}$$

(30)

Thus, the value of the logarithmic likelihood function has been increasing during the iterative process, and the algorithm for the TDVM is convergent.