Metric Learning with Biometric Applications

Abstract

Learning a desired distance metric from given training samples plays a significant role in the field of machine learning. In this chapter, we first present two novel metric learning methods based on a support vector machine (SVM). We then present a kernel classification framework for metric learning that can be implemented efficiently by using the standard SVM solvers. Some novel kernel metric learning method s, such as the double-SVM and the triplet-SVM , are also introduced in this chapter.

Appendices

Appendix 1: The Dual of PCML

The original problem of PCML is formulated as

$$ \begin{aligned} & \mathop {\hbox{min} }\limits_{{{\mathbf{M}},b,{\xi }}} \quad \frac{1}{2}\left\| {\mathbf{M}} \right\|_{F}^{2} + C\sum\limits_{i,j} {\xi_{ij} } \hfill \\ & {\text{s}} . {\text{t}} .\quad h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right) \ge 1 - \xi_{ij} ,\xi_{ij} \ge 0,\;\forall i,j,{\mathbf{M}} \, \succcurlyeq \, 0. \hfill \\ \end{aligned} $$

(2.60)

Its Lagrangian is

$$ \begin{aligned} L\left( {{\lambda },{\kappa },{\mathbf{Y}},{\mathbf{M}},b,{\xi }} \right) & = \frac{1}{2}\left\| {\mathbf{M}} \right\|_{F}^{2} + C\sum\limits_{i,j} {\xi_{ij} } - \sum\limits_{i,j} {\lambda_{ij} \left[ {h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right) - 1 + \xi_{ij} } \right]} \\ & \quad - \sum\limits_{i,j} {\kappa_{ij} \xi_{ij} } - \left\langle {{\mathbf{Y}},{\mathbf{M}}} \right\rangle, \\ \end{aligned} $$

(2.61)

where λ, κ, and Y are the Lagrange multiplier s that satisfy λ _ij  ≥ 0, κ _ij  ≥ 0, ∀i, j, and ${\mathbf{Y}} \, \succcurlyeq \, 0$. Based on the KKT conditions, the original problem can be converted into the dual problem . KKT conditions are defined as follows:

$$ \frac{{\partial L\left( {{\lambda },{\kappa },{\mathbf{Y}},{\mathbf{M}},b,{\xi }} \right)}}{{\partial {\mathbf{M}}}} = {0} \Rightarrow {\mathbf{M}} - \sum\limits_{i,j} {\lambda_{ij} h_{ij} {\mathbf{X}}_{ij} } - {\mathbf{Y}} = {0}, $$

(2.62)

$$ \frac{{\partial L\left( {{\lambda },{\kappa },{\mathbf{Y}},{\mathbf{M}},b,{\xi }} \right)}}{\partial b} = 0 \Rightarrow \sum\limits_{i,j} {\lambda_{ij} h_{ij} } = 0, $$

(2.63)

$$ \frac{{\partial L\left( {{\lambda },{\kappa },{\mathbf{Y}},{\mathbf{M}},b,{\xi }} \right)}}{{\partial \xi_{ij} }} = C - \lambda_{ij} - \kappa_{ij} = 0 \Rightarrow 0 \le \lambda_{ij} \le C,\;\forall i,j, $$

(2.64)

$$ h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right) - 1 + \xi_{ij} \ge 0,\quad \xi_{ij} \ge 0, $$

(2.65)

$$ \lambda_{ij} \ge 0,\quad \kappa_{ij} \ge 0,\quad {\mathbf{Y}} \, \succcurlyeq \, 0, $$

(2.66)

$$ \lambda_{ij} \left[ {h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right) - 1 + \xi_{ij} } \right] = 0,\;\kappa_{ij} \xi_{ij} = 0. $$

(2.67)

Equation (2.62) implies the relationship between ${\lambda }$, Y and M as follows:

$$ {\mathbf{M}} = \sum\limits_{i,j} {\lambda_{ij} h_{ij} {\mathbf{X}}_{ij} } + {\mathbf{Y}}. $$

(2.68)

Substituting Eqs. (2.62)–(2.69) back into the Lagrangian , we get the following Lagrange dual problem of PCML:

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{{{\lambda },{\mathbf{Y}}}} \quad - \frac{1}{2}\left\| {\sum\limits_{i,j} {\lambda_{ij} h_{ij} {\mathbf{X}}_{ij} } + {\mathbf{Y}}} \right\|_{F}^{2} + \sum\limits_{i,j} {\lambda_{ij} } \\ & {\text{s}} . {\text{t}} .\quad \sum\limits_{i,j} {\lambda_{ij} h_{ij} } = 0,0 \le \lambda_{ij} \le C,{\mathbf{Y}} \, \succcurlyeq \, 0. \\ \end{aligned} $$

(2.69)

From Eqs. (2.68) and (2.69), we can see that matrix M is explicitly determined by the training procedure, and b is not. Nevertheless, b can be easily obtained by using the KKT complementarity condition in Eqs. (2.64) and (2.67), which shows that ξ _ij  = 0 if λ _ij  < C, and $h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right) - 1 + \xi_{ij} = 0$ if λ _ij  > 0. Thus, we can simply take any training point for which 0 < λ _ij  < C to calculate b by

$$ b = \frac{1}{{h_{ij} }} - \left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle ,\quad {\text{for}}\;{\text{all}}\;\;0 < \lambda_{ij} < C s. $$

(2.70)

Note that it is reasonable to take the average of all such training points. After we obtained b, we can calculate ξ _ij by

$$ \xi_{ij} = \left\{ {\begin{array}{*{20}l} {0\quad {\text{for}}\;{\text{all}}\;\lambda_{ij} < C} \hfill \\ {\left[ {1 - h_{ij} \left( {\left\langle {{\mathbf{M}},{\mathbf{X}}_{ij} } \right\rangle + b} \right)} \right]_{ + } \quad {\text{for}}\;{\text{all}}\;\lambda_{ij} = C}, \hfill \\ \end{array} } \right. $$

(2.71)

where term [z]₊= max (z, 0) denotes the standard hinge loss .

Appendix 2: The Dual of NCML

The primal problem of NCML is as follows:

$$ \begin{aligned} & \mathop {\hbox{min} }\limits_{{{\alpha },b,{\xi }}} \quad \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\alpha_{ij} \alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + C\sum\limits_{i,j} {\xi_{ij} } \\ & {\text{s}} . {\text{t}} .\quad h_{ij} \left( {\sum\limits_{k,l} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right) \ge 1 - \xi_{ij} ,\quad \xi_{ij} \ge 0, \;\;\alpha_{ij} \ge 0,\;\;\forall i,j. \\ \end{aligned} $$

(2.72)

Its Lagrangian can be defined as

$$ \begin{aligned} L\left( {{\beta },{\sigma },{\nu },{\alpha },b,{\xi }} \right) & = \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\alpha_{ij} \alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + C\sum\limits_{i,j} {\xi_{ij} } - \sum\limits_{i,j} {\sigma_{ij} \alpha_{ij} } \\ & \quad - \sum\limits_{i,j} {\beta_{ij} \left[ {h_{ij} \left( {\sum\nolimits_{kl} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right) - 1 + \xi_{ij} } \right]} - \sum\limits_{i,j} {\nu_{ij} \xi_{ij} }, \\ \end{aligned} $$

(2.73)

where β, σ, and ν are the Lagrange multiplier s that satisfy β _ij  ≥ 0, σ _ij  ≥ 0 and ν _ij  ≥ 0, ∀i, j. Converting the original problem to its dual problem needs the following KKT conditions:

$$ \frac{{\partial L\left( {{\beta },{\sigma },{\nu },{\alpha },b,{\xi }} \right)}}{{\partial \alpha_{ij} }} = 0 \Rightarrow \sum\limits_{k,l} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } - \sum\limits_{k,l} {\beta_{kl} h_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } - \sigma_{ij} = 0, $$

(2.74)

$$ \frac{{\partial L\left( {{\beta },{\sigma },{\nu },{\alpha },b,{\xi }} \right)}}{\partial b} = 0 \Rightarrow \sum\limits_{i,j} {\beta_{ij} h_{ij} } = 0, $$

(2.75)

$$ \frac{{\partial L\left( {{\beta },{\sigma },{\nu },{\alpha },b,{\xi }} \right)}}{{\partial \xi_{ij} }} = 0 \Rightarrow C - \beta_{ij} - \nu_{ij} = 0 \Rightarrow 0 \le \beta_{ij} \le C, $$

(2.76)

$$ h_{ij} \left( {\sum\limits_{k,l} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right) - 1 + \xi_{ij} \ge 0,\quad \xi_{ij} \ge 0,\;\;\alpha_{ij} \ge 0,\;\;\forall i,j, $$

(2.77)

$$ \beta_{ij} \ge 0,\;\sigma_{ij} \ge 0,\;\nu_{ij} \ge 0,\quad \forall i,j, $$

(2.78)

$$ \beta_{ij} \left[ {h_{ij} \left( {\sum\limits_{kl} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right) - 1 + \xi_{ij} } \right] = 0,\quad \nu_{ij} \xi_{ij} = 0,\;\;\sigma_{ij} \alpha_{ij} = 0,\;\; \forall i,j. $$

(2.79)

Here, we introduce a coefficient vector η, which satisfies $\sigma_{ij} = \sum\limits_{k,l} {\eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle }$, where $\left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle$ denotes a positive definite kernel . Thus, we can guarantee that every η has a unique corresponding $\varvec{\sigma}$, and vice versa. According to Eq. (2.74), the relationship between α, β, and η is

$$ \alpha_{ij} = \beta_{ij} h_{ij} + \eta_{ij} ,\quad \forall i,j. $$

(2.80)

Substituting Eqs. (2.74)–(2.76) back into the Lagrangian , the Lagrange dual problem of NCML can be rewritten as follows:

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{{{\eta },{\beta }}} \quad - \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\left( {\beta_{ij} h_{ij} + \eta_{ij} } \right)\left( {\beta_{kl} h_{kl} + \eta_{kl} } \right)\left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + \sum\limits_{i,j} {\beta_{ij} } \\ & {\text{s}} . {\text{t}} .\quad \sum\limits_{k,l} {\eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } \ge 0, \quad 0 \le \beta_{ij} \le C, \;\; \forall i,j, \quad \sum\limits_{i,j} {\beta_{ij} h_{ij} } = 0. \\ \end{aligned} $$

(2.81)

Analogous to PCML, we can use the KKT complementarity condition in Eq. (2.75) to compute b and ξ _ij in NCML. Eqs. (2.76) and (2.79) show that ξ _ij  = 0 if β _ij < C, and $h_{ij} \left( {\sum\nolimits_{kl} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right) - 1 + \xi_{ij} = 0$ if β _ij > 0. With any training point for which 0 < β _ij < C, b can be obtained by

$$ b = \frac{1}{{h_{ij} }} - \sum\limits_{kl} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle }. $$

(2.82)

Therefore, β _ij can also be obtained by

$$ \xi_{ij} = \left\{ {\begin{array}{*{20}l} {0\quad {\text{for}}\;{\text{all}}\;\beta_{ij} < C} \hfill \\ {\left[ {1 - h_{ij} \left( {\sum\limits_{k,l} {\alpha_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + b} \right)} \right]_{ + } \quad {\text{for}}\;{\text{all}}\;\beta_{ij} = C}, \hfill \\ \end{array} } \right. $$

(2.83)

where term [z]₊ = max (z, 0) denotes the standard hinge loss .

Appendix 3: The Dual of the Subproblem of η in NCML

The subproblem of η is defined as follows:

$$ \begin{aligned} & \mathop {\hbox{min} }\limits_{\eta } \quad \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\eta_{ij} \eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + \sum\limits_{i,j} {\eta_{ij} \gamma_{ij} } \\ & {\text{s}} . {\text{t}} .\quad \sum\limits_{k,l} {\eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } \ge 0,\quad \forall i,j, \\ \end{aligned} $$

(2.84)

where $\gamma_{ij} = \sum\limits_{k,l} {\beta_{kl} h_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle }$. Its Lagrangian is

$$ L\left( {{\mu },{\eta }} \right) = \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\eta_{ij} \eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + \sum\limits_{i,j} {\eta_{ij} \gamma_{ij} } - \sum\limits_{i,j} {\mu_{ij} \sum\limits_{k,l} {\eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } }, $$

(2.85)

where μ is the Lagrange multiplier that satisfies μ _ij  ≥ 0, ∀i, j. Converting the original problem to its dual problem needs the following KKT condition:

$$ \frac{{\partial L\left( {{\mu },{\eta }} \right)}}{{\partial \eta_{ij} }} = 0 \Rightarrow \sum\limits_{k,l} {\eta_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } + \gamma_{ij} - \sum\limits_{k,l} {\mu_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } = 0. $$

(2.86)

According to Eq. (2.86), the relationship between μ, η and β is

$$ \eta_{ij} = \mu_{ij} - h_{ij} \beta_{ij} ,\quad \forall i,j. $$

(2.87)

Substituting Eqs. (2.86) and (2.87) back into the Lagrangian , we get the following Lagrange dual problem of the subproblem of η

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{\mu } \quad - \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\mu_{ij} \mu_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + \sum\limits_{i,j} {\gamma_{ij} \mu_{ij} } \\ & \quad - \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\beta_{ij} \beta_{kl} h_{ij} h_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } \\ & \quad {\text{s}} . {\text{t}} .\quad \mu_{ij} \ge 0,\forall i,j. \\ \end{aligned} $$

(2.88)

Since β is fixed in this subproblem, $\sum\nolimits_{i,j} {\sum\nolimits_{k,l} {\beta_{ij} \beta_{kl} h_{ij} h_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } }$ remains constant in Eq. (2.88). Thus, we can omit this term and derive the simplified Lagrange dual problem as follows:

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{\mu } \quad - \frac{1}{2}\sum\limits_{i,j} {\sum\limits_{k,l} {\mu_{ij} \mu_{kl} \left\langle {{\mathbf{X}}_{ij} ,{\mathbf{X}}_{kl} } \right\rangle } } + \sum\limits_{i,j} {\gamma_{ij} \mu_{ij} } \\ & {\text{s}} . {\text{t}} .\quad \mu_{ij} \ge 0, \quad \forall i,j. \\ \end{aligned} $$

(2.89)

Appendix 4: The Dual of the Doublet-SVM

According to the original problem of the doublet-SVM defined in Eq. (2.56), its Lagrange function can be defined as follows

$$ \begin{aligned} L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right) & = \frac{1}{2}\left\| {\mathbf{M}} \right\|_{F}^{2} + C\sum\limits_{l} {\xi_{l} } \\ & \quad - \sum\limits_{l} {\alpha_{l} \left[ {h_{l} \left( {({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} )^{\text{T}} {\mathbf{M}}({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} ) + b} \right) - 1 + \xi_{l} } \right]} - \sum\limits_{l} {\beta_{l} \xi_{l} }, \\ \end{aligned} $$

(2.90)

where α and β are the Lagrange multiplier s that satisfy α _l  ≥ 0 and β _l  ≥ 0, ∀ l. To convert the original problem to its dual needs the following KKT conditions:

$$ \frac{{\partial L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right)}}{{\partial {\mathbf{M}}}} = {0} \Rightarrow {\mathbf{M}} - \sum\limits_{l} {\alpha_{l} h_{l} \left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)^{\text{T}} } = {0}, $$

(2.91)

$$ \frac{{\partial L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right)}}{\partial b} = 0 \Rightarrow \sum\limits_{l} {\alpha_{l} h_{l} } = 0, $$

(2.92)

$$ \frac{{\partial L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right)}}{{\partial \xi_{l} }} = 0 \Rightarrow C - \alpha_{l} - \beta_{l} = 0 \Rightarrow 0 < \alpha_{l} < C,\quad \forall l. $$

(2.93)

According to Eq. (2.91), the relationship between M and α is

$$ {\mathbf{M}} = \sum\limits_{l} {\alpha_{l} h_{l} \left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)^{\text{T}} }. $$

(2.94)

Substituting Eqs. (2.91)–(2.93) back into the Lagrangian function, we have

$$ L\left( {\alpha } \right) = - \frac{1}{2}\sum\limits_{i,j} {\alpha_{i} \alpha_{j} h_{i} h_{j} K_{p} \left( {{\mathbf{z}}_{i} ,{\mathbf{z}}_{j} } \right)} + \sum\limits_{i} {\alpha_{i} }. $$

(2.95)

Thus, the dual problem of the doublet-SVM can be formulated as follows:

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{\alpha } \, - \frac{1}{2}\sum\limits_{i,j} {\alpha_{i} \alpha_{j} h_{i} h_{j} K_{p} \left( {{\mathbf{z}}_{i} ,{\mathbf{z}}_{j} } \right)} + \sum\limits_{i} {\alpha_{i} } \\ & {\text{s}} . {\text{t}} .\quad 0 \le \alpha_{l} \le C, \quad \sum\limits_{l} {\alpha_{l} h_{l} } = 0,\quad \forall l. \\ \end{aligned} $$

(2.96)

Appendix 5: The Dual of the Triplet-SVM

According to the original problem of the triplet-SVM in Eq. (2.58), its Lagrange function can be defined as follows:

$$ \begin{aligned} L\left( {{\mathbf{M}},{\xi },{\alpha },{\beta }} \right) & = \frac{1}{2}\left\| {\mathbf{M}} \right\|_{F}^{2} + C\sum\limits_{l} {\xi_{l} } - \sum\limits_{l} {\alpha_{l} [} ({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} )^{\text{T}} {\mathbf{M}}({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} ) \\ & \quad - ({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} )^{\text{T}} {\mathbf{M}}({\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} )] + \sum\limits_{l} {\alpha_{l} } - \sum\limits_{l} {\alpha_{l} \xi_{l} } - \sum\limits_{l} {\beta_{l} \xi_{l} }, \\ \end{aligned} $$

(2.97)

where α and β are the Lagrange multiplier s. To convert the original problem to its dual, we let the derivative of the Lagrangian function requires the following KKT conditions:

$$ \begin{aligned} & \frac{{\partial L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right)}}{{\partial {\mathbf{M}}}} = {0} \Rightarrow \\ & {\mathbf{M}} - \sum\limits_{l} {\alpha_{l} \left[ {\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} } \right)^{\text{T}} - \left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)^{\text{T}} } \right]} = {0}, \\ \end{aligned} $$

(2.98)

$$ \frac{{\partial L\left( {{\mathbf{M}},b,{\xi },{\alpha },{\beta }} \right)}}{{\partial \xi_{l} }} = 0 \Rightarrow C - \alpha_{l} - \beta_{l} = 0,\;\forall l. $$

(2.99)

According to Eq. (2.98), the relationship between M and ${\alpha }$ is:

$$ {\mathbf{M}} = \sum\limits_{l} {\alpha_{l} \left[ {\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,3} } \right)^{\text{T}} - \left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)\left( {{\mathbf{x}}_{l,1} - {\mathbf{x}}_{l,2} } \right)^{\text{T}} } \right]}. $$

(2.100)

Substituting Eqs. (2.98) and (2.99) back into the Lagrangian , we get

$$ L\left( {\alpha } \right) = - \frac{1}{2}\sum\limits_{i,j} {\alpha_{i} \alpha_{j} K_{p} \left( {{\mathbf{t}}_{i} ,{\mathbf{t}}_{j} } \right)} + \sum\limits_{i} {\alpha_{i} }. $$

(2.101)

Thus, the dual problem of the triplet-SVM can be rewritten as follows:

$$ \begin{aligned} & \mathop {\hbox{max} }\limits_{\alpha } \, - \frac{1}{2}\sum\limits_{i,j} {\alpha_{i} \alpha_{j} K_{p} \left( {{\mathbf{t}}_{i} ,{\mathbf{t}}_{j} } \right)} + \sum\limits_{i} {\alpha_{i} } \\ & {\text{s}} . {\text{t}} .\quad 0 \le \alpha_{l} \le C,\quad \forall l. \\ \end{aligned} $$

(2.102)