An adaptive neural networks formulation for the two-dimensional principal component analysis

Abstract

This study, for the first time, developed an adaptive neural networks (NNs) formulation for the two-dimensional principal component analysis (2DPCA), whose space complexity is far lower than that of its statistical version. Unlike the NNs formulation of principal component analysis (PCA, i.e., 1DPCA), the solution with lower iteration in nature aims to directly deal with original image matrices. We also put forward the consistence in the conceptions of ‘eigenfaces’ or ‘eigengaits’ in both 1DPCA and 2DPCA neural networks. To evaluate the performance of the proposed NN, the experiments were carried out on AR face database and on 64 × 64 pixels gait energy images on CASIA(B) gait database. The less reconstruction error was exploited using the proposed NN in the condition of a large sample set compared to adaptive estimation of learning algorithms for NNs of PCA. On the contrary, if the sample set was small, the proposed NN could achieve a higher residue error than PCA NNs. The amount of calculation for the proposed NN here could be smaller than that for the PCA NNs on the feature extraction of the same image matrix, which represented an efficient solution to the problem of training images directly. On face and gait recognition tasks, a simple nearest neighbor classifier test indicated a particular benefit of the neural network developed here which serves as an efficient alternative to conventional PCA NNs.

Appendix

Here, we provide a proof of Eq. (29).

$$ \begin{aligned} \varvec{s}(t + 1) & = \varvec{s}(t) + \gamma [\varvec{y}_{r} (t)\varvec{Y}^{\text{T}} (t) - \varvec{y}_{r} (t)\varvec{y}_{r}^{\text{T}} (t)\varvec{s}(t)] \\ & = s(t) + \gamma [\varvec{w}_{r} (t)\varvec{X}(t)\varvec{X}^{\text{T}} (t)\varvec{W}^{\text{T}} (t) \\ \quad + \varvec{s}(t)\varvec{W}(t)\varvec{X}(t)\varvec{X}^{\text{T}} (t)\varvec{W}^{\text{T}} (t) - \varvec{y}_{r} (t)\varvec{y}_{r}^{\text{T}} (t)\varvec{s}(t)] \\ \end{aligned} $$

The statistical average of $\varvec{s}(t + 1)$ can be written as

$$ \begin{aligned} \varvec{s}(t + 1) & = \varvec{s}(t) + \gamma [\varvec{w}_{r} (t)\varvec{RW}^{\text{T}} (t) \\ \quad + \varvec{s}(t)\varvec{W}(t)\varvec{RW}^{\text{T}} (t) - \sigma (t)\varvec{s}(t)] \\ \end{aligned} $$

Therefore,

$$ \begin{aligned} \varvec{s}_{i} (t + 1) & = \varvec{s}_{i} (t) + \gamma \theta_{i} (t)\lambda_{i} + \gamma \varvec{s}_{i} (t)\lambda_{i} - \gamma \sigma (t)\varvec{s}_{i} (t) \\ & = \gamma\varvec{\theta}_{i} (t)\lambda_{i} + (1 + \gamma \lambda_{i} - \gamma \sigma (t))\varvec{s}_{i} (t) \\ \end{aligned} $$