L3AM: Linear Adaptive Additive Angular Margin Loss for Video-Based Hand Gesture Authentication

Abstract

Feature extractors significantly impact the performance of biometric systems. In the field of hand gesture authentication, existing studies focus on improving the model architectures and behavioral characteristic representation methods to enhance their feature extractors. However, loss functions, which can guide extractors to produce more discriminative identity features, are neglected. In this paper, we improve the margin-based Softmax loss functions, which are mainly designed for face authentication, in two aspects to form a new loss function for hand gesture authentication. First, we propose to replace the commonly used cosine function in the margin-based Softmax losses with a linear function to measure the similarity between identity features and proxies (the weight matrix of Softmax, which can be viewed as class centers). With the linear function, the main gradient magnitude decreases monotonically as the quality of the model improves during training, thus allowing the model to be quickly optimized in the early stage and precisely fine-tuned in the late stage. Second, we design an adaptive margin scheme to assign margin penalties to different samples according to their separability and the model quality in each iteration. Our adaptive margin scheme constrains the gradient magnitude. It can reduce radical (excessively large) gradient magnitudes and provide moderate (not too small) gradient magnitudes for model optimization, contributing to more stable training. The linear function and the adaptive margin scheme are complementary. Combining them, we obtain the proposed linear adaptive additive angular margin (L3AM) loss. To demonstrate the effectiveness of L3AM loss, we conduct extensive experiments on seven hand-related authentication datasets, compare it with 25 state-of-the-art (SOTA) loss functions, and apply it to eight SOTA hand gesture authentication models. The experimental results show that L3AM loss further improves the performance of the eight authentication models and outperforms the 25 losses. The code is available at https://github.com/SCUT-BIP-Lab/L3AM.

Proofs

1.1 Proof of Gradient Magnitudes

The gradients of ${\mathscr {L}}$ with respect to ${\varvec{W}}$ and ${\varvec{x}}$ in Eq. 2 are

$$\begin{aligned}{} & {} \begin{array}{l} \frac{\partial {\mathscr {L}}}{\partial {\varvec{W}}_{l}}= \frac{\partial {\mathscr {L}}}{\partial y_{l}}\frac{\partial y_{l}}{\partial {\varvec{W}}_{l}} = \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial f(\theta _{l})} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}} \frac{\partial cos\theta _{l}}{\partial {\varvec{W}}_{l}}, \end{array} \end{aligned}$$

(A1)

$$\begin{aligned}{} & {} \begin{array}{l} \frac{\partial {\mathscr {L}}}{\partial {\varvec{W}}_o}= \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial {\varvec{W}}_o} = \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial cos\theta _o} \frac{\partial cos\theta _o}{\partial {\varvec{W}}_o},o \ne l, \end{array} \end{aligned}$$

(A2)

$$\begin{aligned}{} & {} \begin{array}{l} \frac{\partial {\mathscr {L}}}{\partial {\varvec{x}}} = \frac{\partial {\mathscr {L}}}{\partial y_{l}}\frac{\partial y_{l}}{\partial {\varvec{x}}} = \frac{\partial {\mathscr {L}}}{\partial y_{l}}( \frac{\partial y_{l}}{\partial f(\theta _{l})} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}} \frac{\partial cos\theta _{l}}{\partial {\varvec{x}}} \\ \qquad \qquad + \sum _{c\ne l}^{C} \frac{\partial y_{l}}{\partial cos\theta _c} \frac{\partial cos\theta _c}{\partial {\varvec{x}}} ), \end{array} \end{aligned}$$

(A3)

where

$$\begin{aligned}{} & {} \frac{\partial {\mathscr {L}}}{\partial y_{l}} =-\frac{1}{y_{l}}, \end{aligned}$$

(A4)

$$\begin{aligned}{} & {} \frac{\partial y_{l}}{\partial f(\theta _{l})}=sy_{l}(1-y_{l}), \end{aligned}$$

(A5)

$$\begin{aligned}{} & {} \frac{\partial y_{l}}{\partial cos\theta _o}= sy_{l}(0-y_o), y_{o}=\frac{e^{s\cos \theta _{o}}}{e^{sf(\theta _{l})}+\sum _{c \ne l}^{C}e^{s\cos \theta _{c}}}, \nonumber \\ \end{aligned}$$

(A6)

$$\begin{aligned}{} & {} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}}= \frac{\partial f(\theta _{l})}{\partial \theta _{l}} \frac{\partial \theta _{l}}{\partial cos\theta _{l}}=m_0m_1\frac{sin\left( m_1\theta _{l}+m_2\right) }{sin\theta _{l}}. \nonumber \\ \end{aligned}$$

(A7)

$\partial cos\theta _j / \partial {\varvec{W}}_j$ and $\partial cos\theta _j / \partial {\varvec{x}}$ $(j=1,2,\ldots ,C)$ contain a portion of the gradient magnitude and the main gradient direction, which can be further calculated as

$$\begin{aligned}{} & {} \frac{\partial {cos\theta _j}}{\partial {{\varvec{W}}_j}}=\frac{1}{\Vert {{\varvec{W}}_j} \Vert _2}\left( \hat{{\varvec{x}}}^{T}-{cos\theta _{j}}\hat{{\varvec{W}}}_j\right) , \end{aligned}$$

(A8)

$$\begin{aligned}{} & {} {\frac{\partial c o s\theta _{j}}{\partial {\varvec{x}}}}={\frac{1}{\Vert {\varvec{x}}\Vert _{2}}}\left( {\hat{{\varvec{W}}}}_{j}^{T}-c o s\theta _{j}{\hat{{\varvec{x}}}}\right) . \end{aligned}$$

(A9)

The gradient magnitudes of Eqs. A8 and A9 are

$$\begin{aligned}{} & {} \left\| \frac{\partial c o s\theta _{j}}{\partial {\varvec{W}}_{j}}\right\| _{2}=\frac{\left| s i n\theta _{j}\right| }{\Vert {\varvec{W}}_{j}\Vert _{2}}, \end{aligned}$$

(A10)

$$\begin{aligned}{} & {} \left\| \frac{\partial c o s\theta _{j}}{\partial {\varvec{x}}}\right\| _{2}=\frac{\left| s i n\theta _{j}\right| }{\Vert {\varvec{x}}\Vert _{2}}. \end{aligned}$$

(A11)

Hence, we can obtain the magnitudes of C gradients with respect to proxy $\varvec{W_j}$ ($j=1,2,\ldots ,C$) and the magnitudes of C gradient components (associating with proxy $\varvec{W_j}$) with respect to ${\varvec{x}}$. These magnitudes can be divided into two types, associating with the corresponding proxy ${\varvec{W}}_{l}$ ($cos\theta _{l}=\hat{{\varvec{W}}} _{l}\hat{{\varvec{x}}}$) and non-corresponding proxies ${\varvec{W}}_{o}$ ($cos\theta _{o}=\hat{{\varvec{W}}} _o\hat{{\varvec{x}}}$, $o \ne l$). The two type magnitudes of the gradients with respect to ${\varvec{W}}$ are

$$\begin{aligned} \left\| \nabla _{{\varvec{W}}}^{l}\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial f(\theta _{l})} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}} \frac{\partial cos\theta _{l}}{\partial {\varvec{W}}_{l}}\right\| _2\nonumber \\= & {} \frac{s}{\Vert {\varvec{W}}_{l}\Vert _2}\left| (y_{l}-1)m_0m_1sin\left( m_1\theta _{l}+m_2\right) \right| , \nonumber \\ \end{aligned}$$

(A12)

$$\begin{aligned} \left\| \nabla _{{\varvec{W}}}^o\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial cos\theta _o} \frac{\partial cos\theta _o}{\partial {\varvec{W}}_o} \right\| _2 \nonumber \\{} & {} = \frac{s}{\Vert {\varvec{W}}_o\Vert _2}\left| (y_o-0)sin\theta _o\right| . \end{aligned}$$

(A13)

The two type magnitudes of the gradient components with respect to ${\varvec{x}}$ are

$$\begin{aligned} \left\| \nabla _{{\varvec{x}}}^{l}\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}}\frac{\partial y_{l}}{\partial f(\theta _{l})} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}} \frac{\partial cos\theta _{l}}{\partial {\varvec{x}}} \right\| _2 \nonumber \\= & {} \frac{s}{\Vert {\varvec{x}}\Vert _2}\left| (y_{l}-1)m_0m_1sin\left( m_1\theta _{l}+m_2\right) \right| , \end{aligned}$$

(A14)

$$\begin{aligned} \left\| \nabla _{{\varvec{x}}}^o\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}}\frac{\partial y_{l}}{\partial cos\theta _o} \frac{\partial cos\theta _o}{\partial {\varvec{x}}} \right\| _2 = \frac{s}{\Vert {\varvec{x}}\Vert _2}\left| (y_o-0)sin\theta _o\right| . \nonumber \\ \end{aligned}$$

(A15)

1.2 Proof of Linear Similarity Measurement Function

If we take $\frac{\partial f(\theta _j)}{\partial cos\theta _j}=\frac{1}{sin\theta _j} (j=1,2,\ldots ,C$), we get

$$\begin{aligned} \left\| \nabla _{{\varvec{W}}}^{l}\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial f(\theta _{l})} \frac{\partial f(\theta _{l})}{\partial cos\theta _{l}} \frac{\partial cos\theta _{l}}{\partial {\varvec{W}}_{l}}\right\| _2\nonumber \\= & {} \left| -\frac{1}{y_{l}} sy_{l}(1-y_{l}) \frac{1}{sin\theta _{l}} \frac{s i n\theta _{l}}{\Vert {\varvec{W}}_{l}\Vert _{2}} \right| \nonumber \\= & {} \frac{s}{\Vert {\varvec{W}}_{l}\Vert _{2}}\left| y_{l}-1\right| \nonumber \\= & {} \frac{s}{\Vert {\varvec{W}}_{l}\Vert _{2}} B_{l}\cdot 1, \end{aligned}$$

(A16)

$$\begin{aligned} \left\| \nabla _{{\varvec{W}}}^{o}\right\| _2= & {} \left\| \frac{\partial {\mathscr {L}}}{\partial y_{l}} \frac{\partial y_{l}}{\partial f(\theta _o)} \frac{\partial f(\theta _o)}{\partial cos\theta _o} \frac{\partial cos\theta _o}{\partial {\varvec{W}}_o}\right\| _2\nonumber \\= & {} \left| -\frac{1}{y_{l}} sy_{l}(0-y_o) \frac{1}{sin\theta _o} \frac{s i n\theta _o}{\Vert {\varvec{W}}_o\Vert _{2}} \right| \nonumber \\= & {} \frac{s}{\Vert {\varvec{W}}_o\Vert _{2}}\left| y_o-0\right| \nonumber \\= & {} \frac{s}{\Vert {\varvec{W}}_o\Vert _{2}} B_o\cdot 1. \end{aligned}$$

(A17)

Note that $cos \theta _o$ is a spacial case of $f(\theta _o)$ ($m_0=m_1=1$ and $m_2=m_3=0$, see Eq. 2). Thus, the modulation gradient magnitudes of the proposed L3AM loss (Eq. 15) are equal to one ($M_{l}$=$M_{o}=1$).