Video-Based Human Respiratory Wavelet Extraction and Identity Recognition

Abstract

In this paper, we study the problem of human identity recognition using off-angle human faces. Our proposed system is composed of (i) a physiology-based human clustering module and (ii) an identification module based on facial features (nose, mouth, etc.) fetched from face videos. In our proposed methodology we, first, passively extract an important vital sign (breath). Next we cluster human subjects into nostril motion versus nostril non-motion groups, and, then, localize a set of facial features, before we apply feature extraction and matching. Our proposed human identity recognition system is very efficient. It is working well when dealing with breath signals and a combination of different facial components acquired under challenging luminous conditions. This is achieved by using our proposed Motion Classification approach and Feature Clustering technique based on the breathing waveforms we produce. The contributions of this work are three-fold. First, we generated a set of different datasets where we tested our proposed approach. Specifically, we considered six different types of facial components and their combination, to generate face-based video datasets, which present two diverse data collection conditions, i.e., videos acquired in head full frontal pose (baseline) and head looking up pose. Second, we propose an alternative way of passively measuring human breath from face videos. We demonstrate a comparatively identical breath waveform estimation when compared against the breath waveforms produced by an ADInstruments device (baseline) (Adinstruments, http://www.adinstruments.com/ [1]). Third, we demonstrate good human recognition performance based on partial facial features when using the proposed pre-processing Motion Classification and Feature Clustering techniques. Our approach achieves increased identification rates across all datasets used, and it yields a significantly high identification rate, ranging from 96 to 100% when using a single or a combination of facial features. The approach yields an average of 7% rank-1 rate increase, when compared to the baseline scenario. To the best of our knowledge, this is the first time that a biometric recognition system positively exploits human breath waveforms, which when fused with partial facial features, it increases a benchmark face-based recognition performance established using academic face matching algorithms.

Face Normalization Techniques

• Wavelet Based Normalization Technique (WA): The WA approach was proposed by Du and Ward [26] that applies the discrete wavelet transform to an image and processes the obtained sub-bands. This technique focuses on the detailed coefficient matrices and employs histogram equalization to the approximate transform coefficients. Finally, reconstructs the normalized image using the inverse wavelet transform after the manipulation of each individual sub-band.

• Anisotropic Diffusion Based Normalization Technique (AS): The AS approach adopts anisotropic smoothing of the input image to evaluate the luminance function, which was introduce by Gross et al. [41] to the face recognition area.

• Discrete Cosine Transform Based Normalization Technique (DCT): The DCT technique was introduced by Chen et al. [42] that truncates an appropriate number of DCT coefficients to minimize illumination variations under different lighting conditions.

• Gradientfaces Based Normalization Technique (GRA): The GRA approach proposed by Zhang et al. [43] is to transform image into the gradient domain and use the generated face representation as the illumination invariant version of the target image.

• Homomorphic Filtering Based Normalization Technique (HOMO): The HOMO is to transform the input image into the logarithm followed by the frequency domain in order to enhance the high-frequency components and weaken the low-frequency parts. Finally, apply the inverse Fourier transform to obtain the output image in the spatial domain [44].

• Isotropic Diffusion Based Normalization Technique (IS): The IS approach [44] is to estimate the luminance function of the input image using isotropic smoothing algorithm which is a simpler variance of the anisotropic diffusion based normalization technique [41].

• Large-Scale and Small-Scale Features Normalization Technique (LSSF): The LSSF proposed by Xie et al. [45] firstly computes the reflectance and luminance function of the input image and then further analyzes both generated functions using a second time of normalization. Within the INface toolbox used in our experiment, the SSR technique is implemented as the normalization approach in both steps of LSSF technique.

• Modified Anisotropic Diffusion Normalization Technique (MAS): The MAS approach included two main modification into the original anisotropic diffusion normalization technique [41]: (1) Introducing an additional atan function to estimate the local contrast; (2) Apply a robust post-processing procedure proposed by Tan et al. [28] in the final stage of this technique.

• Multi-Scale Retinex (MSR) Algorithm: The MSR method is to extend the previously designed single-scale center/surround retinex technique to a multi-scale version proposed by Jobson et al. [46].

• Multi-Scale Weberfaces Normalization Technique (MSW): The MSW as an extend the single-scale Weberfaces approach proposed by Wange et al. [47] is to compute the relative gradient using a modified Weber contrast method for diverse neighborhood sizes and to apply a linear combination of the produced face representation as an illumination invariant version of the target output.

• Steerable Filter Based Normalization Technique (SF): The SF approach produces the target normalized image by removing illumination induced appearance variation from the input facial image using steerable filters.

• Single Scale Retinex (SSR) Algorithm: The SSR approach was proposed by Jobson et al. [48] on the basis of the retinex theory [49] as the majority of photometric normalization techniques.

• Tan and Triggs Normalization Technique (TT): The TAT is to employ a processing chain on the input image by firstly using gamma correction, then applying DoG filtering and finally adopting a robust post-processor to generate the output normalized image [28].

• Single Scale Weberfaces Normalization Technique (WEB): The WEB method is to compute the relative gradient using a modified Weber contrast algorithm and treat the generated face representation as an illumination invariant version of the target image [47].