A sparse neighborhood preserving non-negative tensor factorization algorithm for facial expression recognition

Abstract

In this paper, a novel sparse neighborhood preserving non-negative tensor factorization (SNPNTF) algorithm is proposed for facial expression recognition. It is derived from non-negative tensor factorization (NTF), and it works in the rank-one tensor space. A sparse constraint is adopted into the objective function, which takes the optimization step in the direction of the negative gradient, and then projects onto the sparse constrained space. To consider the spatial neighborhood structure and the class-based discriminant information, a neighborhood preserving constraint is adopted based on the manifold learning and graph preserving theory. The Laplacian graph which encodes the spatial information in the face samples and the penalty graph which considers the pre-defined class information are considered in this constraint. By using it, the obtained parts-based representations of SNPNTF vary smoothly along the geodesics of the data manifold and they are more discriminant for recognition. SNPNTF is a quadratic convex function in the tensor space, and it could converge to the optimal solution. The gradient descent method is used for the optimization of SNPNTF to ensure the convergence property. Experiments are conducted on the JAFFE database, the Cohn–Kanade database and the AR database. The results demonstrate that SNPNTF provides effective facial representations and achieves better recognition performance, compared with non-negative matrix factorization, NTF and some variant algorithms. Also, the convergence property of SNPNTF is well guaranteed.

Appendix

The calculations about Eq. (25)

Firstly, we discuss the calculation of $\nabla f_{{{\mathbf{V,Z}}}} ({\mathbf{U}}^{(t)} )$ and $\nabla f_{{{\mathbf{U,Z}}}} ({\mathbf{V}}^{(t)} )$. The objective function of SNPNTF is now written as:

$$\begin{aligned} {\text{f}} & = \frac{1}{2}\left\| {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } } \right\|_{F}^{2} + \frac{1}{2}\varepsilon ||\varvec{u}||_{1} + \frac{1}{2}\gamma ||\varvec{v}||_{1} + \frac{1}{2}\lambda {\text{tr}}({\mathbf{Z}}^{T} {\mathbf{LZ}}){ - }\frac{1}{2}\sigma {\text{tr}}({\mathbf{Z}}^{T} {\mathbf{L}}^{p} {\mathbf{Z}}) \\ & = \frac{1}{2}\left\langle {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,{\mathbf{A}}{ - }\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } } \right\rangle + \frac{1}{2}\varepsilon ||\varvec{u}||_{1} + \frac{1}{2}\gamma ||\varvec{v}||_{1} + \frac{1}{2}\lambda \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {{\mathbf{z}}_{\text{r}}^{T} {\mathbf{Lz}}_{\text{r}} } { - }\frac{1}{2}\sigma \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {{\mathbf{z}}_{\text{r}}^{T} {\mathbf{L}}^{p} {\mathbf{z}}_{\text{r}} } \\ \end{aligned}$$

(36)

The differential of f is

$$\begin{aligned} d\left( {\text{f}} \right) & = {\frac{1}{2}d {\left\langle {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } } \right\rangle } + \frac{1}{2}d(\varepsilon ||\varvec{u}||_{1} ) + \frac{1}{2}d(\gamma ||\varvec{v}||_{1} )} \\&\quad + { \frac{1}{2}d(\lambda \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{z}_{\text{r}}^{T} {\mathbf{L}}\varvec{z}_{\text{r}} } ) - \frac{1}{2}d(\sigma \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{z}_{\text{r}}^{T} {\mathbf{L}}^{p} \varvec{z}_{\text{r}} } )} \\ & = {\left\langle {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,d({\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } )} \right\rangle + \frac{1}{2}d(\varepsilon ||\varvec{u}||_{1} ) + \frac{1}{2}d(\gamma ||\varvec{v}||_{1} )} \\&\quad + { \frac{1}{2}d(\lambda \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{z}_{\text{r}}^{T} {\mathbf{L}}\varvec{z}_{\text{r}} } ) - \frac{1}{2}d(\sigma \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{z}_{\text{r}}^{T} {\mathbf{L}}^{p} \varvec{z}_{\text{r}} } )}\\ \end{aligned}$$

(37)

To calculate $\nabla f_{{{\mathbf{V,Z}}}} ({\mathbf{U}}^{(t)} )$, the differential of f along $\varvec{u}_{s} (\forall s,1 \le s \le R)$ is

$$\begin{aligned} d({\text{f}}_{{\varvec{u}_{\text{s}} }} ) &= \left\langle {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } , - d(\varvec{u}_{\text{s}} ) \otimes \varvec{v}_{\text{s}} \otimes \varvec{z}_{\text{s}} } \right\rangle + \varepsilon + 0 + 0- 0\\ &= \left\langle {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,d(\varvec{u}_{\text{s}} ) \otimes \varvec{v}_{\text{s}} \otimes \varvec{z}_{\text{s}} } \right\rangle - \left\langle {{\mathbf{A}},d(\varvec{u}_{\text{s}} ) \otimes \varvec{v}_{\text{s}} \otimes \varvec{z}_{\text{s}} } \right\rangle + \varepsilon \\ \end{aligned}$$

(38)

In Eq. (38), the sparseness constraint acts as $\varepsilon$. It means the value of the coefficient $\varepsilon$ controls the sparse degree, which proves the analysis in Sect. 3.2 mathematically.

Similarly, the partial differential for $\varvec{u}_{s}^{p} (1 \le p \le m_{1} )$ is

$$\frac{{\partial {\text{f}}}}{{\partial {\text{u}}_{\text{s}}^{\text{p}} }} = \left\langle {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\varvec{u}_{\text{r}} \otimes \varvec{v}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,\varvec{e}^{\text{p}} \otimes \varvec{v}_{\text{s}} \otimes \varvec{z}_{\text{s}} } \right\rangle - \left\langle {{\mathbf{A}},\varvec{e}^{\text{p}} \otimes \varvec{v}_{\text{s}} \otimes \varvec{z}_{\text{s}} } \right\rangle + \varepsilon$$

(39)

where the pth element in $\varvec{e}^{\text{p}} \in {\mathbb{R}}^{{{\text{m}}_{ 1} }}$ is 1, others are 0 s. That is $(\varvec{e}^{\text{p}} )_{\text{p}} = 1$ and $(\varvec{e}^{\text{p}} )_{{{\text{k}} \ne {\text{p}}}} = 0$. According to Definition 2.1, for any order tensors ${\mathbf{A}}_{ 1} ,{\mathbf{A}}_{ 2} \in {\mathbb{R}}^{{{\text{a}}_{1} \times {\text{a}}_{2} \cdots \times {\text{a}}_{\text{n}} }}$,${\mathbf{B}}_{ 1} ,{\mathbf{B}}_{ 2} \in {\mathbb{R}}^{{{\text{b}}_{1} \times {\text{b}}_{2} \cdots \times {\text{b}}_{\text{n}} }}$, there is $\left\langle {{\mathbf{A}}_{ 1} \otimes {\mathbf{B}}_{ 1} ,{\mathbf{A}}_{ 2} \otimes {\mathbf{B}}_{ 2} } \right\rangle = \left\langle {{\mathbf{A}}_{ 1} ,{\mathbf{A}}_{ 2} } \right\rangle \left\langle {{\mathbf{B}}_{ 1} ,{\mathbf{B}}_{ 2} } \right\rangle$. Then, Eq. (39) could be written as:

$$\begin{aligned} \frac{{\partial {\text{f}}}}{{\partial {\text{u}}_{\text{s}}^{\text{p}} }} & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {\left\langle {\varvec{u}_{\text{r}} \varvec{,e}^{\text{p}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} \otimes {\mathbf{z}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} \otimes {\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \left( {\sum\nolimits_{{{\text{k}} = 1,{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{1} , {\text{m}}_{2} ,{\text{N}}}} {{\text{A}}_{\text{kqi}} (\varvec{e}^{\text{p}} )_{\text{k}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } } \right) + \varepsilon \\ & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {\left\langle {\varvec{u}_{\text{r}} \varvec{,e}^{\text{p}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} \otimes {\mathbf{z}}_{\text{r}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{s}} \otimes {\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \left( {\sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} ,{\text{N}}}} {{\text{A}}_{\text{pqi}} (\varvec{e}^{\text{p}} )_{\text{p}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } + \sum\nolimits_{{{\text{k}} \ne {\text{p}}}} {{\text{A}}_{\text{kqi}} (\varvec{e}^{\text{p}} )_{\text{k}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } } \right) + \varepsilon \\ & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {\left\langle {\varvec{u}_{\text{r}} \varvec{,e}^{\text{p}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \left( {\sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} ,{\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} + 0} } \right) + \varepsilon \\ & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {\left( {{\text{u}}_{\text{r}}^{\text{p}} (\varvec{e}^{\text{p}} )_{\text{p}} + \sum\nolimits_{{{\text{k}} \ne {\text{p}}}} {{\text{u}}_{\text{r}}^{\text{k}} (\varvec{e}^{\text{p}} )_{\text{k}} } } \right)\left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} ,{\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } + \varepsilon \\ & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} ,{\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } + \varepsilon \\ \end{aligned}$$

(40)

According to Eq. (25), the update rule for ${\text{u}}_{\text{s}}^{\text{p}}$ is

$$\begin{aligned} {\text{u}}_{\text{s}}^{\text{p}} & = {\text{u}}_{\text{s}}^{\text{p}} - \mu ({\text{u}}_{\text{s}}^{\text{p}} )\frac{{\partial {\text{f}}}}{{\partial {\text{u}}_{\text{s}}^{\text{p}} }} \\ & = {\text{u}}_{\text{s}}^{\text{p}} - \mu ({\text{u}}_{\text{s}}^{\text{p}} )\left( {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} - \sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} , {\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} } + \varepsilon } \right) \\ \end{aligned}$$

(41)

To confirm the non-negative of ${\text{u}}_{\text{s}}^{\text{p}}$, the update step $\mu ({\text{u}}_{\text{s}}^{\text{p}} )$ is set as:

$$\mu ({\text{u}}_{\text{s}}^{\text{p}} ) = {{{\text{u}}_{\text{s}}^{\text{p}} } \mathord{\left/ {\vphantom {{{\text{u}}_{\text{s}}^{\text{p}} } {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }}} \right. \kern-0pt} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }}$$

(42)

If the denominator is close to 0, Eq. (42) would lead to instable results. Therefore, an extra positive additive value is adopted in the denominator which is set to be 0.01. In the following part of this paper, the additive values are used in all denominators.

Now the update equation for ${\text{u}}_{\text{s}}^{\text{p}}$ is

$$\begin{aligned} {\text{u}}_{\text{s}}^{\text{p}} & = {\text{u}}_{\text{s}}^{\text{p}} {{(\sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} , {\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} - \varepsilon } )} \mathord{\left/ {\vphantom {{(\sum\nolimits_{{{\text{q}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{2} , {\text{N}}}} {{\text{A}}_{\text{pqi}} {\text{v}}_{\text{s}}^{\text{q}} {\text{z}}_{\text{s}}^{\text{i}} - \varepsilon } )} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }}} \right. \kern-0pt} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{u}}_{\text{r}}^{\text{p}} \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }} \\ & = {{{\text{u}}_{\text{s}}^{\text{p}} (\varvec{v}_{\text{s}}^{T} A_{\text{p;;}} {\mathbf{z}}_{\text{s}} - \varepsilon )} \mathord{\left/ {\vphantom {{{\text{u}}_{\text{s}}^{\text{p}} (\varvec{v}_{\text{s}}^{T} A_{\text{p;;}} {\mathbf{z}}_{\text{s}} - \varepsilon )} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {({\text{u}}_{\text{r}}^{\text{p}} ({\mathbf{v}}_{\text{r}}^{T} {\mathbf{v}}_{\text{s}} )({\mathbf{z}}_{\text{r}}^{T} {\mathbf{z}}_{\text{s}} ))} }}} \right. \kern-0pt} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {({\text{u}}_{\text{r}}^{\text{p}} ({\mathbf{v}}_{\text{r}}^{T} {\mathbf{v}}_{\text{s}} )({\mathbf{z}}_{\text{r}}^{T} {\mathbf{z}}_{\text{s}} ))} }} \\ & = {{{\text{u}}_{\text{s}}^{\text{p}} (\varvec{v}_{\text{s}}^{T} A_{\text{p;;}} {\mathbf{z}}_{\text{s}} - \varepsilon )} \mathord{\left/ {\vphantom {{{\text{u}}_{\text{s}}^{\text{p}} (\varvec{v}_{\text{s}}^{T} A_{\text{p;;}} {\mathbf{z}}_{\text{s}} - \varepsilon )} {({\mathbf{U}}_{{{\text{p}};}} (({\mathbf{V}}^{T} {\mathbf{v}}_{\text{s}} ){ \odot }({\mathbf{Z}}^{T} {\mathbf{z}}_{\text{s}} )))}}} \right. \kern-0pt} {({\mathbf{U}}_{{{\text{p}};}} (({\mathbf{V}}^{T} {\mathbf{v}}_{\text{s}} ){ \odot }({\mathbf{Z}}^{T} {\mathbf{z}}_{\text{s}} )))}} \\ \end{aligned}$$

(43)

where ${\mathbf{U}}_{{{\text{p}};}} \in {\mathbb{R}}^{{1 \times {\text{R}}}}$ represents the pth row of ${\mathbf{U}} = [\varvec{u}_{ 1} ,\varvec{u}_{ 2} , \ldots ,\varvec{u}_{\text{R}} ]$,$\odot$ is the matrix Hadamard product (e.g., $(X{ \odot }Y)_{\text{ij}} = X_{\text{ij}} Y_{\text{ij}}$), ${\mathbf{A}}_{\text{p;;}} \in {\mathbb{R}}^{{{\text{m}}_{2} \times {\text{N}}}}$ represents the matrix which fixes the first mode of A, and traversals the other two modes. It is defined as:

$$({\mathbf{A}}_{\text{p;;}} )_{\text{qi}} = {\mathbf{A}}_{\text{pqi}} ,\text{ }\quad 1 \le \forall {\text{q}} \le {\text{m}}_{2} ,\quad 1 \le \forall {\text{i}} \le {\text{N}}$$

(44)

According to the analysis above, similarly, the update equation of the qth element of $\varvec{v}_{\text{s}}$ (${\text{v}}_{\text{s}}^{\text{q}}$, $1 \le {\text{s}} \le {\text{R}}$, $1 \le {\text{q}} \le {\text{m}}_{2}$) can be written as:

$$\begin{aligned} {\text{v}}_{\text{s}}^{\text{q}} & = {{{\text{v}}_{\text{s}}^{\text{q}} \sum\nolimits_{{{\text{p}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{1} , {\text{N}}}} { ( {\text{A}}_{\text{pqi}} {\text{u}}_{\text{s}}^{\text{p}} {\text{z}}_{\text{s}}^{\text{i}} - \gamma )} } \mathord{\left/ {\vphantom {{{\text{v}}_{\text{s}}^{\text{q}} \sum\nolimits_{{{\text{p}} = 1,{\text{i}} = 1}}^{{{\text{m}}_{1} , {\text{N}}}} { ( {\text{A}}_{\text{pqi}} {\text{u}}_{\text{s}}^{\text{p}} {\text{z}}_{\text{s}}^{\text{i}} - \gamma )} } {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{v}}_{\text{r}}^{\text{q}} \left\langle {{\mathbf{u}}_{\text{r}} \varvec{,}{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }}} \right. \kern-0pt} {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left( {{\text{v}}_{\text{r}}^{\text{q}} \left\langle {{\mathbf{u}}_{\text{r}} \varvec{,}{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{z}}_{\text{r}} ,{\mathbf{z}}_{\text{s}} } \right\rangle } \right)} }} \\ & = {{{\text{v}}_{\text{s}}^{\text{q}} ({\mathbf{u}}_{\text{s}}^{T} A_{{ ; {\text{q;}}}} {\mathbf{z}}_{\text{s}} - \gamma )} \mathord{\left/ {\vphantom {{{\text{v}}_{\text{s}}^{\text{q}} ({\mathbf{u}}_{\text{s}}^{T} A_{{ ; {\text{q;}}}} {\mathbf{z}}_{\text{s}} - \gamma )} {({\mathbf{V}}_{{{\text{q}};}} (({\mathbf{U}}^{T} {\mathbf{u}}_{\text{s}} ){ \odot }({\mathbf{Z}}^{T} {\mathbf{z}}_{\text{s}} )))}}} \right. \kern-0pt} {({\mathbf{V}}_{{{\text{q}};}} (({\mathbf{U}}^{T} {\mathbf{u}}_{\text{s}} ){ \odot }({\mathbf{Z}}^{T} {\mathbf{z}}_{\text{s}} )))}} \\ \end{aligned}$$

(45)

where ${\mathbf{V}}_{{{\text{q}};}} \in {\mathbb{R}}^{{1 \times {\text{R}}}}$ represents the pth row of ${\mathbf{V}} = [\varvec{v}_{ 1} ,\varvec{v}_{ 2} , \ldots ,\varvec{v}_{\text{R}} ] \in {\mathbb{R}}^{{{\text{m}}_{2} \times {\text{R}}}}$, $\odot$ is the matrix Hadamard product (e.g., $(X{ \odot }Y)_{\text{ij}} = X_{\text{ij}} Y_{\text{ij}}$), ${\mathbf{A}}_{{ ; {\text{q;}}}} \in {\mathbb{R}}^{{{\text{m}}_{ 1} \times {\text{N}}}}$ represents the matrix which fixes the second mode of A, and traversals the other two modes. It is defined as:

$$({\mathbf{A}}_{{ ; {\text{q;}}}} )_{\text{pi}} = {\mathbf{A}}_{\text{pqi}} ,\text{ }1 \le \forall {\text{p}} \le {\text{m}}_{1} ,1 \le \forall {\text{i}} \le {\text{N}}$$

(46)

Now, $\varvec{v}_{\text{r}}$ and $\varvec{u}_{\text{r}}$ are calculated.

Then, we discuss the calculation of $\nabla f_{{{\mathbf{U,V}}}} ({\mathbf{Z}}^{(t)} )$. The differential of f along ${\mathbf{z}}_{\text{s}} (\forall {\text{s}},1 \le {\text{s}} \le {\text{R}})$ is

$$\begin{aligned} d({\text{f}}_{{\varvec{z}_{\text{s}} }} ) & = \left\langle {{\mathbf{A}} - \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {{\mathbf{u}}_{\text{r}} \otimes {\mathbf{v}}_{\text{r}} \otimes {\mathbf{z}}_{\text{r}} } , - d(\varvec{z}_{\text{s}} ) \otimes {\mathbf{u}}_{\text{s}} \otimes {\mathbf{v}}_{\text{s}} } \right\rangle + \frac{1}{2}d\left( {\lambda {\mathbf{z}}_{\text{s}}^{T} L{\mathbf{z}}_{\text{s}} } \right) - \frac{1}{2}d\left( {\sigma {\mathbf{z}}_{\text{s}}^{T} L^{p} {\mathbf{z}}_{\text{s}} } \right) \\ & = \left\langle {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {{\mathbf{u}}_{\text{r}} \otimes {\mathbf{v}}_{\text{r}} \otimes \varvec{z}_{\text{r}} } ,d({\mathbf{z}}_{\text{s}} ) \otimes {\mathbf{u}}_{\text{s}} \otimes {\mathbf{v}}_{\text{s}} } \right\rangle - \left\langle {{\mathbf{A}},d({\mathbf{z}}_{\text{s}} ) \otimes {\mathbf{u}}_{\text{s}} \otimes {\mathbf{v}}_{\text{s}} } \right\rangle + d\left( {\frac{1}{2}\lambda {\mathbf{z}}_{\text{s}}^{T} (D - S){\mathbf{z}}_{\text{s}} } \right) - d\left( {\frac{1}{2}\sigma {\mathbf{z}}_{\text{s}}^{T} (D^{p} - S^{p} ){\mathbf{z}}_{\text{s}} } \right) \\ \end{aligned}$$

(47)

For ${{\left( {\lambda {\mathbf{z}}_{\text{s}}^{T} (D - S){\mathbf{z}}_{\text{s}} } \right)} \mathord{\left/ {\vphantom {{\left( {\lambda {\mathbf{z}}_{\text{s}}^{T} (D - S){\mathbf{z}}_{\text{s}} } \right)} 2}} \right. \kern-0pt} 2}$, the partial differential for $\varvec{z}_{\text{s}}^{\text{i}}$ is

$$\begin{aligned} \frac{{\partial \left( {\frac{1}{2}\lambda {\mathbf{z}}_{\text{s}}^{T} (D - S){\mathbf{z}}_{\text{s}} } \right)}}{{\partial {\text{z}}_{\text{s}}^{\text{i}} }} & = \frac{{\partial ({\mathbf{z}}_{\text{s}} )}}{{\partial {\text{z}}_{\text{s}}^{\text{i}} }}\frac{{\partial \left( {\frac{1}{2}\lambda {\mathbf{z}}_{\text{s}}^{T} (D - S){\mathbf{z}}_{\text{s}} } \right)}}{{\partial {\mathbf{z}}_{\text{s}} }} \\ & = (\varvec{e}^{\text{i}} )^{T} \lambda (D - S){\mathbf{z}}_{\text{s}} \\ \end{aligned}$$

(48)

where the i ^h element in $\varvec{e}^{\text{i}} \in {\mathbb{R}}^{\text{N}}$ is 1, others are 0 s. That is $(\varvec{e}^{\text{i}} )_{\text{i}} = 1$ and $(\varvec{e}^{\text{i}} )_{{{\text{k}} \ne {\text{i}}}} = 0$. Then, the partial differential for ${\text{z}}_{\text{s}}^{\text{i}}$ is:

$$\begin{aligned} \frac{{\partial {\text{f}}}}{{\partial {\text{z}}_{\text{s}}^{\text{i}} }} & = \left\langle {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {{\mathbf{u}}_{\text{r}} \otimes {\mathbf{v}}_{\text{r}} \otimes {\mathbf{z}}_{\text{r}} } ,{\mathbf{u}}_{\text{s}} \otimes {\mathbf{v}}_{\text{s}} \otimes \varvec{e}^{\text{i}} } \right\rangle - \left\langle {{\mathbf{A}},{\mathbf{u}}_{\text{s}} \otimes {\mathbf{v}}_{\text{s}} \otimes \varvec{e}^{\text{i}} } \right\rangle + \lambda (\varvec{e}^{\text{i}} )^{T} (D - S){\mathbf{z}}_{\text{s}} - \sigma (\varvec{e}^{\text{i}} )^{T} (D^{p} - S^{p} ){\mathbf{z}}_{\text{s}} \\ & = \sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left\langle {{\mathbf{u}}_{\text{r}} ,{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle {\text{z}}_{\text{r}}^{\text{i}} } - \sum\nolimits_{{{\text{p}} = 1,{\text{q}} = 1}}^{{{\text{m}}_{1} , {\text{m}}_{2} }} {{\text{A}}_{\text{pqi}} {\text{u}}_{\text{s}}^{\text{p}} {\text{v}}_{\text{s}}^{\text{q}} } + (\lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} - \lambda \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}} {\text{z}}_{\text{s}}^{\text{k}} } ) - (\sigma D_{\text{ii}}^{p} {\text{z}}_{\text{s}}^{\text{i}} - \sigma \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}}^{p} {\text{z}}_{\text{s}}^{\text{k}} } ) \\ \end{aligned}$$

(49)

According to Eq. (25), the update rule for ${\text{z}}_{\text{s}}^{\text{i}}$ is

$$\begin{aligned} {\text{z}}_{\text{s}}^{\text{i}} & = {\text{z}}_{\text{s}}^{\text{i}} - \mu ({\text{z}}_{\text{s}}^{\text{i}} )\frac{{\partial {\text{f}}}}{{\partial {\text{z}}_{\text{s}}^{\text{i}} }} \\ & = {\text{z}}_{\text{s}}^{\text{i}} - \mu ({\text{z}}_{\text{s}}^{\text{i}} )\left( {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left\langle {{\mathbf{u}}_{\text{r}} ,{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle {\text{z}}_{\text{r}}^{\text{i}} } + \lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} + \sigma \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}}^{p} {\text{z}}_{\text{s}}^{\text{k}} } - \sum\nolimits_{{{\text{p}} = 1,{\text{q}} = 1}}^{{{\text{m}}_{1} , {\text{m}}_{2} }} {{\text{A}}_{\text{pqi}} {\text{u}}_{\text{s}}^{\text{p}} {\text{v}}_{\text{s}}^{\text{q}} } - \lambda \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}} {\text{z}}_{\text{s}}^{\text{k}} } - \sigma D_{\text{ii}}^{p} {\text{z}}_{\text{s}}^{\text{i}} } \right) \\ \end{aligned}$$

(50)

To ensure the non-negative, the update step $\mu ({\text{z}}_{\text{s}}^{\text{i}} )$ is set as:

$$\mu ({\text{z}}_{\text{s}}^{\text{i}} ) = {{{\text{z}}_{\text{s}}^{\text{i}} } \mathord{\left/ {\vphantom {{{\text{z}}_{\text{s}}^{\text{i}} } {\left( {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left\langle {{\mathbf{u}}_{\text{r}} ,{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle {\text{z}}_{\text{r}}^{\text{i}} } + \lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} + \sigma \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}}^{p} {\text{z}}_{\text{s}}^{\text{k}} } } \right)}}} \right. \kern-0pt} {\left( {\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left\langle {{\mathbf{u}}_{\text{r}} ,{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle {\text{z}}_{\text{r}}^{\text{i}} } + \lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} + \sigma \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}}^{p} {\text{z}}_{\text{s}}^{\text{k}} } } \right)}}$$

(51)

And the final update equation of ${\text{z}}_{\text{s}}^{\text{i}}$ is

$$\begin{aligned} {\text{z}}_{\text{s}}^{\text{i}} & = {\text{z}}_{\text{s}}^{\text{i}} \frac{{\sum\nolimits_{{{\text{p}} = 1,{\text{q}} = 1}}^{{{\text{m}}_{1} , {\text{m}}_{2} }} {{\text{A}}_{\text{pqi}} {\text{u}}_{\text{s}}^{\text{p}} {\text{v}}_{\text{s}}^{\text{q}} } + \lambda \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}} {\text{z}}_{\text{s}}^{\text{k}} } + \sigma D_{\text{ii}}^{p} {\text{z}}_{\text{s}}^{\text{i}} }}{{\sum\nolimits_{{{\text{r}} = 1}}^{\text{R}} {\left\langle {{\mathbf{u}}_{\text{r}} ,{\mathbf{u}}_{\text{s}} } \right\rangle \left\langle {{\mathbf{v}}_{\text{r}} ,{\mathbf{v}}_{\text{s}} } \right\rangle {\text{z}}_{\text{r}}^{\text{i}} } + \lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} + \sigma \sum\nolimits_{{{\text{k}} = 1}}^{\text{N}} {S_{\text{ik}}^{p} {\text{z}}_{\text{s}}^{\text{k}} } }} \\ & = {\text{z}}_{\text{s}}^{\text{i}} \frac{{{\mathbf{u}}_{\text{s}}^{T} A_{{ ; ; {\text{i}}}} {\mathbf{v}}_{\text{s}} + \lambda S_{\text{i;}} \varvec{z}_{\text{s}} + \sigma D_{\text{ii}}^{p} {\text{z}}_{\text{s}}^{\text{i}} }}{{{\mathbf{Z}}_{{{\text{i}};}} \left( {\left( {{\mathbf{U}}^{T} {\mathbf{u}}_{\text{s}} } \right){ \odot }\left( {{\mathbf{V}}^{T} {\mathbf{v}}_{\text{s}} } \right)} \right) + \lambda D_{\text{ii}} {\text{z}}_{\text{s}}^{\text{i}} + \sigma \lambda S_{\text{i;}}^{p} \varvec{z}_{\text{s}} }} \\ \end{aligned}$$

(52)

where ${\mathbf{Z}}_{{{\text{i}};}} \in {\mathbb{R}}^{{1 \times {\text{R}}}}$ represents the i ^th row of ${\mathbf{Z}} = [{\mathbf{z}}_{ 1} ,{\mathbf{z}}_{ 2} , \ldots ,{\mathbf{z}}_{\text{R}} ]$;$S_{\text{i;}} \in {\mathbb{R}}^{{1 \times {\text{N}}}}$ and $S_{\text{i;}}^{p} \in {\mathbb{R}}^{{1 \times {\text{N}}}}$ represent the ith row of $S$ and $S^{p}$, respectively; $\odot$ is Hadamard product,${\mathbf{A}}_{{ ; ; {\text{i}}}} \in {\mathbb{R}}^{{{\text{m}}_{ 1} \times {\text{m}}_{ 2} }}$ and is defined as:

$$({\mathbf{A}}_{{ ; ; {\text{i}}}} )_{\text{pq}} = {\mathbf{A}}_{\text{pqi}} ,\text{ }1 \le \forall {\text{p}} \le {\text{m}}_{1} ,1 \le \forall {\text{q}} \le {\text{m}}_{2}$$

(53)

Now $\varvec{u}_{\text{r}}$, $\varvec{v}_{\text{r}}$ and $\varvec{z}_{\text{r}}$ in the objective function are all solved. The illustrations about $A_{\text{p;;}}$, ${\text{A}}_{{ ; {\text{q;}}}}$ and $A_{{ ; ; {\text{i}}}}$ are given in Fig. 11.

Fig. 11

The matrices corresponding to different transverses of a tensor