Robust Periocular Recognition by Fusing Sparse Representations of Color and Geometry Information

Abstract

In this paper, we propose a re-weighted elastic net (REN) model for biometric recognition. The new model is applied to data separated into geometric and color spatial components. The geometric information is extracted using a fast cartoon - texture decomposition model based on a dual formulation of the total variation norm allowing us to carry information about the overall geometry of images. Color components are defined using linear and nonlinear color spaces, namely the red-green-blue (RGB), chromaticity-brightness (CB) and hue-saturation-value (HSV). Next, according to a Bayesian fusion-scheme, sparse representations for classification purposes are obtained. The scheme is numerically solved using a gradient projection (GP) algorithm. In the empirical validation of the proposed model, we have chosen the periocular region, which is an emerging trait known for its robustness against low quality data. Our results were obtained in the publicly available FRGC and UBIRIS.v2 data sets and show consistent improvements in recognition effectiveness when compared to related state-of-the-art techniques.

Appendices

Appendix A: Existence of Solution

We state necessary and sufficient conditions for the existence of a solution for the proposed model (7). We follow the notations and similar arguments to those used in [19, 48]. Suppose that $A_{i}=(A_{1i},\dots ,A_{mi})^{T}$, i=1,⋯ ,n are the linear independent predictors and y=(y ₁,⋯ ,y _m )^T is the response vector. Let A=[A ₁,⋯ ,A _n ] be the predictor matrix. In terms of ℓ ¹ and ℓ ² norms, we rewrite the minimization problem in Eq. (7) as,

$$ \min_{\mathbf{x}}\left\{m\|W\mathbf{x}\|_{1}+\frac{m}{2}\|(1-W)\mathbf{x}\|_{2}^{2}+\frac{1}{2}\|\mathbf{y}-A\mathbf{x}\|_{2}^{2}\right\}. $$

(13)

Let us denote by x ^∗ and $\hat {\mathbf {x}}$ the real and estimated solution of Eq. 13 respectively. Given $\mathcal {I}=supp(\mathbf {x}^{\ast })=\{i:\,x^{\ast }_{i}\neq 0\}$, we define the block-wise form matrix

$$ A_{\mathcal{I},\mathcal{I}^{c}}=\frac{1}{m} \left( \begin{array}{cc} A^{T}_{\mathcal{I}}A_{\mathcal{I}} & A^{T}_{\mathcal{I}}A_{\mathcal{I}^{c}} \\ \\ A^{T}_{\mathcal{I}^{c}}A_{\mathcal{I}} & A^{T}_{\mathcal{I}^{c}}A_{\mathcal{I}^{c}} \end{array} \right), $$

where $A_{\mathcal {I}}$ ($A_{\mathcal {I}^{c}}$) is a $m\times \#\mathcal {I}$ ($m\times \#\mathcal {I}^{c}$) matrix formed by concatenating the columns $\{A_{i}:\,i\in \mathcal {I}\}$ ($\{A_{i}:\,i\in \mathcal {I}^{c}\}$) and $A^{T}_{\mathcal {I}}A_{\mathcal {I}}$ is assumed to be invertible.

First we assume that there exist $v\mathbf {x}\in \mathbb {R}^{n}$ satisfying (13) and $sign(\hat {\mathbf {x}})=sign(\mathbf {x}^{\ast })$. Lets define $\mathbf {b}=W_{\mathcal {I}}sign(\mathbf {x}_{\mathcal {I}}^{\ast })$ together with the set,

$$ \mathcal{D}=\left\{\mathbf{d}\in\mathbb{R}^{n}:\,\left\{\begin{array}{ll} d_{i}=b_{i}, &\text{for}\,\,\hat{x}_{i}\neq 0 \\ |d_{i}|\leq w_{i}, & \text{otherwise}\end{array}\right.\,\,\right\}. $$

From the Kauush-Kuhn-Tucker (KKT) conditions we obtain

$$\left\{\begin{array}{ll} {A_{i}^{T}}(\mathbf{y}-A\hat{\mathbf{x}})-m(1-w_{i})^{2}\hat{x}_{i}=mw_{i}sign(x^{\ast}_{i}), &\text{if}\,\,\hat{x}_{i}\neq 0 \\ \left|{A_{i}^{T}}\left( \mathbf{y}-A\hat{\mathbf{x}}\right)\right|\leq mw_{i}, & \text{otherwise} \end{array}\right. $$

which can be rewritten as,

$$ {A_{i}^{T}}A(\hat{\mathbf{x}}-\mathbf{x}^{\ast})-{A_{i}^{T}}\boldmath{\kappa} +m(1-w_{i})^{2}\hat{x}_{i}+md_{i}=0, $$

(14)

for some $\mathbf {d}\in \mathcal {D}$ with components d _i , $i=1,\dots ,n$. By substituting the equality y=A x ^∗+κ. From the above (14) the following two equations arise:

$$\begin{array}{@{}rcl@{}} A^{T}_{\mathcal{I}}A_{\mathcal{I}}(\hat{\mathbf{x}}_{\mathcal{I}}-\mathbf{x}^{\ast})-\frac{A^{T}_{\mathcal{I}}\boldmath{\kappa}} {m}+(1-W)^{2}\hat{\mathbf{x}}_{\mathcal{I}}&=&-\mathbf{b}, \end{array} $$

(15)

$$\begin{array}{@{}rcl@{}} A^{T}_{\mathcal{I}^{c}}A_{\mathcal{I}}(\hat{\mathbf{x}}_{\mathcal{I}}-\mathbf{x}^{\ast})-\frac{A^{T}_{\mathcal{I}^{c}}\boldmath(\kappa)} {m}&=&-\mathbf{d}_{\mathcal{I}^{c}}. \end{array} $$

(16)

Solving for $\mathbf {x}_{\mathcal {I}}$ in Eq. 15 and replacing in Eq. 16 to get b in terms of $\mathbf {x}_{\mathcal {I}}$ leave us with

$$ \ \hat{\mathbf{x}}_{\mathcal{I}}=\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}+(1-W)^{2}\right)^{-1}\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}^{\ast}_{\mathcal{I}}+\frac{A_{\mathcal{I}}\boldmath{\kappa}}{m}-\mathbf{b}\right), $$

(17)

$$\begin{array}{@{}rcl@{}} &&A^{T}_{\mathcal{I}^{c}}A_{\mathcal{I}}\left( \left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}+(1-W)^{2}\right)^{-1}\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}_{\mathcal{I}}^{\ast}+\frac{A_{\mathcal{I}}^{T}\boldmath{\kappa}} {m}-\mathbf{b}\right) -\mathbf{x}^{\ast}_{\mathcal{I}}\right)\\ &&\quad-\frac{A^{T}_{\mathcal{I}^{c}}\boldmath{\kappa}} m=-\mathbf{b}. \end{array} $$

(18)

From Eqs. 17 and 18, we finally get the next two equations:

$$ sign\left( \left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}+(1-W)^{2}\right)^{-1}\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}^{\ast}_{\mathcal{I}}+\frac{A_{\mathcal{I}}^{T}\boldmath{\kappa}}{m}-\mathbf{b}\right)\right)=sign(\mathbf{x}^{\ast}_{\mathcal{I}}) $$

(19)

and

$$\begin{array}{@{}rcl@{}} &&\left|{A_{i}^{T}}A_{\mathcal{I}}\left( \left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}+(1-W)^{2}\right)^{-1}\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}_{\mathcal{I}}^{\ast}+\frac{A_{\mathcal{I}}^{T}\boldmath(\kappa)} {m}-\mathbf{b}\right)\right.\right.\\ &&\left.\left.{\kern23pt}\phantom{\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}_{\mathcal{I}}^{\ast}+\frac{A_{\mathcal{I}}^{T}\boldmath(\kappa)} {m}-\mathbf{b}\right)}-\mathbf{x}^{\ast}_{\mathcal{I}}\right)-\frac{{A_{i}^{T}}\boldmath{\kappa}} {m}\right|\leq w_{i}, \end{array} $$

(20)

for $i\in \mathcal {I}^{c}$.

Now, let us assume that Eqs. 19 and 20 both hold. It will be proved there exist $\hat {\mathbf {x}}\in \mathbb {R}^{n}$ satisfying $sing(\hat {\mathbf {x}})=sign(\mathbf {x}^{\ast })$. Setting $\hat {\mathbf {x}}\in \mathbb {R}^{n}$ satisfying $\hat {\mathbf {x}}_{\mathcal {I}^{c}}=\mathbf {x}^{\ast }_{\mathcal {I}^{c}}=0$ and

$$\mathbf{x}_{\mathcal{I}}=\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}+(1-W)^{2}\right)^{-1} \left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}^{\ast}_{\mathcal{I}} +\frac{A_{\mathcal{I}}^{T}\boldmath{\kappa}} {m}-\mathbf{b}\right), $$

which guarantees the equality $sign(\hat {\mathbf {x}}_{\mathcal {I}})=sign(\mathbf {x}^{\ast }_{\mathcal {I}})$ due to Eq. 19. In the same manner, we define $\mathbf {d}\in \mathbb {R}^{n}$ satisfying $\mathbf {d}_{\mathcal {I}}=\mathbf {b}$ and

$$\begin{array}{@{}rcl@{}} &&\mathbf{d}_{\mathcal{I}^{c}}=-\left( A^{T}_{\mathcal{I}^{c}}A_{\mathcal{I}} \left( \left( A^{T}_{\mathcal{I}}A_{\mathcal{I}} +(1-W)^{2}\right)^{-1}\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}_{\mathcal{I}}^{\ast} +\frac{A_{\mathcal{I}}^{T}\boldmath{\kappa}}{m}-\mathbf{b}\right)\right.\right.\\ &&\left.\left.\phantom{\left( A^{T}_{\mathcal{I}}A_{\mathcal{I}}\mathbf{x}_{\mathcal{I}}^{\ast} +{} \frac{A_{\mathcal{I}}^{T}\boldmath{\kappa}}{m}-\mathbf{b}\right)}-\mathbf{x}^{\ast}_{\mathcal{I}}\right)- \frac{A_{\mathcal{I}^{c}}^{T}\boldmath{\kappa}}{m}\right), \end{array} $$

implying from Eq. 20 the inequality |d _i |≤w _i for $i\in \mathcal {I}^{c}$ and therefore $\mathbf {d}\in \mathcal {D}$. From previous, we have found a point a point $\hat {mathbf{x}}\in \mathbb {R}^{n}$ and $\mathbf {d}\in \mathcal {D}$ satisfying (15) and (16) respectively or equivalently (14). Moreover, we also have the equality $sign(\hat {\mathbf {x}})=sign(\mathbf {x}^{\ast })$. Under these assertions we can prove the sign recovery property of our model as illustrated next.

Appendix B: Sign Recovery Property

Under some regularity conditions on the proposed REN model, we intend to give an estimation for which the event $sign(\hat {\mathbf {x}})=sign(\mathbf {x}^{\ast })$ is true. Following similar notations and arguments to those used in [53, 55], we intend to prove that our model enjoys the following probabilistic property:

$$ Pr\left( \min_{i\in\mathcal{I}} \left|\hat{x}_{i}\right|>0\right)\rightarrow 1. $$

(21)

For theoretical analysis purposes, the problem (7) is written as

$$ \min_{\mathbf{x}}\left\{\|W\mathbf{x}\|_{1}+\|(1-W)\mathbf{x}\|_{2}^{2}+\|\mathbf{y}-A\mathbf{x}\|_{2}^{2}\right\}. $$

The following regularity conditions are also assumed:

• 1. Denoting with $\Lambda _{\min }(S)$ and $\Lambda _{\max }(S)$ the minimum and maximum eigenvalues of a symmetric matrix S, we assume the following inequalities hold:

  $$ \theta_{1}\leq\Lambda_{\min}\left( \frac{1}{m}A^{T}A\right)\leq\Lambda_{\max}\left( \frac{1}{m}A^{T}A\right)\leq\theta_{2}, $$

  where 𝜃 ₁ and 𝜃 ₂ are two positive constants.

• 2. $\lim _{m\rightarrow \infty }\frac {\log (n)}{\log (m)}=\nu$ for some 0≤ν<1

• 3. $\lim \limits _{m\rightarrow \infty }\sqrt {\frac {m}{n}}\frac {1}{\max _{i\in \mathcal {I}}w_{i}}=\infty$.

Let

$$ \tilde{\mathbf{x}}=arg\min_{\mathbf{x}}\left\{\left\|\mathbf{y}-A\mathbf{x}\right\|_{2}^{2}+\left\|(1-W)\mathbf{x}\right\|_{2}^{2}\right\}. $$

(22)

By using the definitions of $\hat {\mathbf {x}}$ and $\tilde {\mathbf {x}}$, the next two inequalities arise

$$ \left\|\mathbf{y}-A\hat{\mathbf{x}}\right\|_{2}^{2}+\left\|\left( 1-W\right)\hat{\mathbf{x}}\right\|_{2}^{2}\geq \left\|\mathbf{y}-A\tilde{\mathbf{x}}\right\|_{2}^{2}+\left\|\left( 1-W\right)\tilde{\mathbf{x}}\right\|_{2}^{2} $$

(23)

and

$$\begin{array}{@{}rcl@{}} &&\left\|\mathbf{y}-A\tilde{\mathbf{x}}\right\|_{2}^{2}+\left\|\left( 1-W\right)\tilde{\mathbf{x}}\right\|_{2}^{2} +\sum_{i=1}^{n}w_{i}|\tilde{x}_{i}|\\ &&{\kern20pt} \geq \left\|\mathbf{y}-A\hat{\mathbf{x}}\right\|_{2}^{2}+\left\|\left( 1-W\right)\hat{\mathbf{x}}\right\|_{2}^{2} +\sum_{i=1}^{n}w_{i}|\hat{x}_{i}|. \end{array} $$

(24)

The combination of Eqs. 23 and 24 give

$$\begin{array}{@{}rcl@{}} \sum\limits_{i=1}^{n}w_{i}(|\tilde{x}_{i}|\! -\! |\hat{x}_{i}|)\! &\geq&\! \left\|\mathbf{y}\! -\! A\hat{\mathbf{x}}\right\|_{2}^{2} +\! \left\|(1\!\ -\! W)\hat{\mathbf{x}} \right\|_{2}^{2}-\! \left\|\mathbf{y}\! -\! A\tilde{\mathbf{x}}\right\|_{2}^{2}-\! \left\|(1\! -\!W)\tilde{\mathbf{x}}\right\|_{2}^{2}\\[-6pt] &=&\left( \hat{\mathbf{x}}-\tilde{\mathbf{x}}\right)^{T} \left( A^{T}A+(1-W)^{2}\right) \left( \hat{\mathbf{x}}-\tilde{\mathbf{x}}\right)\\ \end{array} $$

(25)

On the other hand

$$ \sum\limits_{i=1}^{n}w_{i}\left( \left|\tilde{x}_{i}\right|-\left|\hat{x}_{i}\right|\right)\leq \sum\limits_{i=1}^{n}w_{i}\left|\tilde{x}_{i}-\hat{x}_{i}\right|\leq\sqrt{\sum\limits_{i=1}^{n}{w_{i}^{2}}}\left\|\tilde{\mathbf{x}}-\hat{\mathbf{x}}\right\|_{2}\\ $$

(26)

By combining Eqs. 25 and 26 we get

$$\begin{array}{@{}rcl@{}} \Lambda_{min}\left( \left( A^{T}A\right)+\left( 1-W\right)^{2}\right)\left\|\hat{\mathbf{x}}-\tilde{\mathbf{x}}\right\|_{2}^{2} &\leq&\left( \hat{\mathbf{x}}-\tilde{\mathbf{x}}\right)^{T} \left( A^{T}A+(1-W)^{2}\right) \left( \hat{\mathbf{x}}-\tilde{\mathbf{x}}\right)\\ &\leq& \sqrt{\sum_{i=1}^{n}{w_{i}^{2}}}\left\|\tilde{\mathbf{x}}-\hat{\mathbf{x}}\right\|_{2} \end{array} $$

which together with the identity

$$0\leq\theta_{1}\leq\Lambda_{min}\left( A^{T}A\right)\leq\Lambda_{min}\left( \left( A^{T}A\right)+\left( 1-W\right)^{2}\right) $$

allow us to prove

$$ \left\|\hat{\mathbf{x}}-\tilde{\mathbf{x}}\right\|_{2}\leq\frac{\sqrt{\sum_{i=1}^{n}{w_{i}^{2}}}}{\Lambda_{min}\left( A^{T}A\right)}, $$

(27)

Let us notice that

$$\begin{array}{@{}rcl@{}} E\left( \left\|\tilde{\mathbf{x}}-\mathbf{x}^{\ast}\right\|_{2}^{2}\right) &=&E\left( -\left( A^{T}A+\left( 1-W\right)^{2}\right)^{-1}\left( 1-W\right)^{2}\mathbf{x}^{\ast}\right.\\ &+&\left.\left( A^{T}A+\left( 1-W\right)^{2}\right)^{-1}A^{T}\boldmath{\kappa}\right)\\ &\leq&2\frac{\left\|(1-W)\mathbf{x}^{\ast}\right\|_{2}^{2}+n\Lambda_{\max}\left( A^{T}A\right)\sigma^{2}}{\Lambda_{\min}\left( A^{T}A\right)} \end{array} $$

(28)

From Eqs. 27 and 28 we conclude that

$$\begin{array}{@{}rcl@{}} E\left( \left\|\hat{\mathbf{x}}-\mathbf{x}^{\ast}\right\|_{2}^{2}\right) &\leq&2\left( E\left( \left\|\tilde{\mathbf{x}}-\mathbf{x}^{\ast}\right\|_{2}^{2}\right) -E\left( \left\|\hat{\mathbf{x}}-\mathbf{x}^{\ast}\right\|_{2}^{2}\right)\right)\\ &\leq&4\frac {\left\|\left( 1-W\right)\mathbf{x}^{\ast}\right\|_{2}^{2}+n\Lambda_{\max}(A^{T}A)\sigma^{2}+E\left( \sum_{i=1}^{n}{w_{i}^{2}}\right)} {\Lambda_{\min}\left( A^{T}A\right)}. \end{array} $$

(29)

Let $\eta =\min _{i\in \mathcal {I}}|x_{i}^{\ast }|$ and $\hat {\eta }=\max _{i\in \mathcal {I}}w_{i}$. Because of Eq. 27,

$$ \left\|\hat{\mathbf{x}}_{\mathcal{I}}-\tilde{\mathbf{x}}_{\mathcal{I}}\right\|_{2}^{2}\leq\frac{\sqrt{n}\hat{\eta}}{\theta_{1}m}. $$

Then

$$ \min_{i\in\mathcal{I}}|x^{\ast}_{i}|>\min_{i\in\mathcal{I}}|\tilde{x}_{i}|-\frac{\sqrt{n}\hat{\eta}}{\theta_{1}m}>\min_{i\in\mathcal{I}}|\hat{x}_{i}|- \left\|\tilde{\mathbf{x}}_{\mathcal{I}}-\mathbf{x}^{\ast}_{\mathcal{I}}\right\|_{2} -\frac{\sqrt{n}\hat{\eta}}{\theta_{1}m}. $$

(30)

Now, we notice that

$$ \displaystyle\frac{\sqrt{n}\hat{\eta}}{\theta_{1}m}=O\left( \frac{1}{\sqrt{n}}\right) \left( \sqrt{\frac{n}{m}}\eta^{-1}\right)\left( \hat{\eta}\eta\right). $$

Since

$$\begin{array}{@{}rcl@{}} E\left( \left( \hat{\eta}\eta\right)^{2}\right) &\leq& 2\eta^{2}+2\eta^{2}E\left( \left( \hat{\eta}-\eta\right)^{2}\right) \leq 2\eta^{2}+2\eta^{2}E\left( \left\|\hat{\mathbf{x}}-\mathbf{x}^{\ast}\right\|^{2}\right)\\ &\leq& 2 \eta^{2}+8\eta^{2}\frac{\left\|\left( 1-W\right)^{2}\mathbf{x}^{\ast}\right\|_{2}^{2}+\theta_{2}nm\sigma^{2}+E\left( \sum_{i=1}^{n}{w_{i}^{2}}\right)}{\theta_{1}m} \end{array} $$

and $\eta ^{2}m/n\rightarrow \infty$ as long as $m\rightarrow \infty$, it follows that

$$ \displaystyle\frac{\sqrt{n}\hat{\eta}^{-1}}{\theta_{1}m}=o\left( \frac{1}{\sqrt{n}}\right)O_{Pr}(1). $$

(31)

By using Eq. 29, we derive

$$ E\left( \left\|\hat{\mathbf{x}}_{\mathcal{I}}-\mathbf{x}^{\ast}_{\mathcal{I}}\right\|_{2}^{2}\right) \leq4\frac{\|\left( 1-W\right)^{2}\mathbf{x}^{\ast}\|_{2}+\theta_{2}nm\sigma^{2}}{(\theta_{1}m)^{2}}=\sqrt{\frac{n}{m}}O_{Pr}(1). $$

(32)

Substituting Eq. 31 and 32 in Eq. 30 allow us to conclude that

$$\min_{i\in\mathcal{I}}|x^{\ast}_{i}|>\eta-\sqrt{\frac{n}{m}}O_{Pr}(1)-o\left( \frac{1}{\sqrt{n}}\right)O_{Pr}(1). $$

Then Eq. 21 holds.

Remark 2

There is special interest in applying the REN model in the case the data satisfies the condition n≫m. For the LASSO model it was suggested in [6] to make use of the Dantzig selector which can achieve the ideal estimation up to a l o g(n) factor. In [13] a performing of the Dantzig selector called the Sure Independence Screening (SIS) was introduced in order to reduce the ultra-high dimensionality. We remark that the SIS technique can be combined with the REN model (7) for dealing the case n≫m. Then previous computations can be still applied to reach the sign recovery property.