Subspace Discrimination Method for Images Using Singular Value Decomposition

Abstract

In this paper, we introduce the linear subspace method for the second-order tensor. The subspace method based on the principal component analysis for vector data is a conventional and established method for pattern recognition and classification. A pair of orthonormal vector sets derived by the singular value decomposition of matrices provides a pair of linear spaces to express images. The application of the subspace method to a pair of linear subspaces provides recognition methodologies for images through tensor analysis.

Appendix

Let $\boldsymbol{P}_1$ and $\boldsymbol{P}_2$ be the orthogonal projections to the linear subspaces $\mathbf{\Pi }_1$ and $\mathbf{\Pi }_2$, respectively. Setting $\sigma _i$ to be the singular values of $\boldsymbol{P}_1\boldsymbol{P}_2$ and $\boldsymbol{P}_2\boldsymbol{P}_1$ such that $0\le \sigma _1<\sigma _2<\cdots <\sigma _k$, the principal angles between $\mathbf{\Pi }_1$ and $\mathbf{\Pi }_2$ are $\cos ^{-1}\sigma _1>\cos ^{-1}\sigma _2>\cdots > \cos ^{-1}\sigma _k$, and the Grassmann distance along the geodesic between $\mathbf{\Pi }_1$ and $\mathbf{\Pi }_2$ is

$$ d(\mathbf{\Pi }_1,\mathbf{\Pi }_2)=\sqrt{\sum _{i=1}^k \theta _i^2} $$

for $\theta _i=\cos ^{-1}\sigma _i$. For orthonormal bases $\{\boldsymbol{u}_{1i}\}_{i=1}^m$ and $\{\boldsymbol{u}_{2j}\}_{j=1}^n$,

$$ \boldsymbol{P}_1=\sum _{i=1}^m \boldsymbol{u}_{1i}\boldsymbol{u}_{1i}^{*}, \, \, \, \boldsymbol{P}_2=\sum _{i=1}^n \boldsymbol{u}_{2i}\boldsymbol{u}_{2i}^{*} $$

are the orthogonal projections to linear subspaces $\mathbf{\Pi }_1=\mathcal {L}(\{\boldsymbol{u}_{1i}\}_{i=1}^m)$ and $\mathbf{\Pi }_2=\mathcal {L}(\{\boldsymbol{u}_{2i}\}_{j=1}^n)$, respectively. Since

$$ \boldsymbol{P}_1\boldsymbol{P}_2\boldsymbol{u}_{2i}=(\boldsymbol{u}_{1i}^{*}\boldsymbol{u}_{2i}) \boldsymbol{u}_{1i},\, \, \, \boldsymbol{P}_2\boldsymbol{P}_1\boldsymbol{u}_{1i}=(\boldsymbol{u}_{2i}^{*}\boldsymbol{u}_{1i}) \boldsymbol{u}_{2i}, $$

that is,

$$ \boldsymbol{P}_1\boldsymbol{P}_2\boldsymbol{P}_1\boldsymbol{u}_{1i} =|(\boldsymbol{u}_{1i}^{*}\boldsymbol{u}_{1i})|^2 \boldsymbol{u}_{1i},\, \, \, \boldsymbol{P}_2\boldsymbol{P}_1\boldsymbol{P}_2\boldsymbol{u}_{2i} =|(\boldsymbol{u}_{2i}^{*}\boldsymbol{u}_{1i})|^2 \boldsymbol{u}_{2i}, $$

the principal angles between $\mathbf{\Pi }_1$ and $\mathbf{\Pi }_2$ are $\angle [\boldsymbol{u}_{1i}, \boldsymbol{u}_{2i}] =\cos ^{-1}\sqrt{|(\boldsymbol{u}_{1i}^{*}\boldsymbol{u}_{2i})|^2}$.

Let $\{\boldsymbol{u}_{ji}\}_{i=1}^{k(j)}$ be orthogonal base sets of linear subspaces $\mathbf{\Pi }_j$ and $\boldsymbol{P}_j$ be the orthogonal projection to $\mathbf{\Pi }_j$ fro $j=1,2,\cdots , m$. If $\boldsymbol{w}\in \mathbf{\Pi }_j$, $\boldsymbol{P}_j\boldsymbol{w}=\boldsymbol{w}$ for $j=1,2,\cdots ,m$. Therefore, if $|\boldsymbol{P}_j\boldsymbol{w}-\boldsymbol{w}|^2\ll \epsilon ^2$, then $\boldsymbol{w}$ lies in a space close to all of $\{\mathbf{\Pi }_j\}_{j=1}^m$. These geometric conditions imply that the minimiser of

$$ J(\boldsymbol{w})=\sum _{j=1}^m|\boldsymbol{P}_j\boldsymbol{w}-\boldsymbol{w}|^2 $$

is close to all $\{\mathbf{\Pi }_j\}_{j=1}^m$. The minimiser of $J(\boldsymbol{w})$ with respect to $|\boldsymbol{w}|=1$ derives the eigenvalue equation

$$ \sum _{j=1}^m\boldsymbol{P}_j^*\boldsymbol{P}_j\boldsymbol{w}=\lambda \boldsymbol{w}, \, \, \, \, \boldsymbol{P}_j=\sum _{i=1}^{k(j)}\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*}. $$

Since

$$ \boldsymbol{P}_j^*=\sum _{i=1}^{k(j)}(\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*})^*=\sum _{i=1}^{k(j)}\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*}, $$

the relations

$$ \boldsymbol{P}_j^*=\boldsymbol{P}_j, \, \, \, \, \boldsymbol{P}_i^*\boldsymbol{P}_i= \left( \sum _{i=1}^{k(j)}(\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*})^*\right) \left( \sum _{i=1}^{k(j)}\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*}\right) =\sum _{i=1}^{k(j)}\boldsymbol{u}_{ji}\boldsymbol{u}_{ji}^{*} $$

are satisfied. Therefore, the eigenvectors of $\boldsymbol{A}=\sum _{j=1}^m\boldsymbol{P}_j$ for different eigenvalues are mutually orthogonal and all eigenvalues are nonnegative. Therefore, setting

$$ 0=\lambda _1\le \cdots \le \lambda _k<\lambda _{k+1}<\cdots <\lambda _n, $$

for $k+1=\min \{ k(1),k(2),\cdots ,k(m)\}$,

$$ \boldsymbol{P}_{\wedge }=\sum _{k=1}^l\boldsymbol{w}_k\boldsymbol{w}_k^*$$

is the orthogonal projection to the linear subspace

$$ \mathbf{\Pi }_{\wedge }= \{ \boldsymbol{x}\, |\, |\boldsymbol{P}_j\boldsymbol{x}-\boldsymbol{x}|\ll \epsilon ^2, \, \, j=1,2,\cdots ,m\} $$

for $0<\epsilon \ll 1$. The orthogonal projection $\boldsymbol{P}_{\wedge }$ is the canonical projection to the common linear subspace of $\{\mathbf{\Pi }_j\}_{j=1}^m$. Therefore, for $\boldsymbol{x}\in \mathbf{\Pi }_j$, $\mathbf{\Pi }_{\wedge }$ excludes $\boldsymbol{y}=(\boldsymbol{I}-\boldsymbol{P}_{\wedge })\boldsymbol{x}$.

Let $\mathbf{\Pi }$ and $\mathbf{\Omega }$ be a pair of linear subspace in $\mathbf{C}^n$ with the condition $\mathbf{\Pi }\cap \mathbf{\Omega }=\{\boldsymbol{o}\}$. For $\forall \boldsymbol{p}\in \mathbf{\Pi }$ such that $\boldsymbol{p}\ne \boldsymbol{o}$ and $\forall \boldsymbol{q}\in \mathbf{\Omega }$ such that $\boldsymbol{q}\ne \boldsymbol{o}$, if $\boldsymbol{p}^*\boldsymbol{q}=0$, $\mathbf{\Pi }$ and $\mathbf{\Omega }$ are orthogonal complement each other. Therefore, the relations $\boldsymbol{p}\ne \mathbf{\Omega }$ and $\boldsymbol{q}\ne \mathbf{\Pi }$ are satisfied if $\boldsymbol{p}\in \mathbf{\Pi }$ and $\boldsymbol{q}\in \mathbf{\Omega }$. We set $\mathbf{\Omega }^\perp :=\mathbf{\Pi }$ and $\mathbf{\Pi }^\perp :=\mathbf{\Omega }$. Assuming $\mathbf{\Pi }\cup \mathbf{\Omega }=\mathbf{C}^n$, $\boldsymbol{x}\in \mathbf{R}^n$ is uniquely expressed as $\boldsymbol{x}=\boldsymbol{p}+\boldsymbol{q}$ for $\boldsymbol{p}\in \mathbf{\Pi }$ and $\boldsymbol{q}\in \mathbf{\Omega }$. Therefore, using the orthogonal projections $\boldsymbol{P}_\mathbf{\Pi }$ and $\boldsymbol{P}_\mathbf{\Omega }=\boldsymbol{I}-\boldsymbol{P}_\mathbf{\Pi }=\boldsymbol{P}_\mathbf{\Pi }^\perp$ to $\mathbf{\Pi }$ and $\mathbf{\Omega }$, respectively, $\boldsymbol{x}\in \mathbf{C}^n$ is uniquely decomposed into $\boldsymbol{P}_\mathbf{\Pi }\boldsymbol{x}$ and $\boldsymbol{P}_\mathbf{\Omega }\boldsymbol{x}$.