In this paper, we study the problem of human identity recognition using off-angle human faces. Our proposed system is composed of (i) a physiology-based human clustering module and (ii) an identification module based on facial features (nose, mouth, etc.) fetched from face videos. In our proposed methodology we, first, passively extract an important vital sign (breath). Next we cluster human subjects into nostril motion versus nostril non-motion groups, and, then, localize a set of facial features, before we apply feature extraction and matching. Our proposed human identity recognition system is very efficient. It is working well when dealing with breath signals and a combination of different facial components acquired under challenging luminous conditions. This is achieved by using our proposed Motion Classification approach and Feature Clustering technique based on the breathing waveforms we produce. The contributions of this work are three-fold. First, we generated a set of different datasets where we tested our proposed approach. Specifically, we considered six different types of facial components and their combination, to generate face-based video datasets, which present two diverse data collection conditions, i.e., videos acquired in head full frontal pose (baseline) and head looking up pose. Second, we propose an alternative way of passively measuring human breath from face videos. We demonstrate a comparatively identical breath waveform estimation when compared against the breath waveforms produced by an ADInstruments device (baseline) (Adinstruments, http://www.adinstruments.com/ ). Third, we demonstrate good human recognition performance based on partial facial features when using the proposed pre-processing Motion Classification and Feature Clustering techniques. Our approach achieves increased identification rates across all datasets used, and it yields a significantly high identification rate, ranging from 96 to 100% when using a single or a combination of facial features. The approach yields an average of 7% rank-1 rate increase, when compared to the baseline scenario. To the best of our knowledge, this is the first time that a biometric recognition system positively exploits human breath waveforms, which when fused with partial facial features, it increases a benchmark face-based recognition performance established using academic face matching algorithms.
This is a preview of subscription content, access via your institution.
Jain AK, Ross A, Pankanti S (2006) Biometrics: a tool for information security. IEEE Trans Inf Forensics Secur 1(2):125–143
Jain AK, Bolle R, Pankanti S (1999) Biometrics: personal identification in networked society. Springer
Maltoni D, Maio D, Jain AK, Prabhakar S (2009) Handbook of fingerprint recognition. Springer
Jain AK, Li SZ (2005) Handbook of face recognition. Springer
Sanchez-Reillo R, Sanchez-Avila C, Gonzalez-Marcos A (2000) Biometric identification through hand geometry measurements. IEEE Trans Pattern Anal Mach Intell 22(10):1168–1171
Daugman J (2004) How iris recognition works. IEEE Trans Circuits Syst Video Technol 14(1):21–30
Srinivasan D, Ng W, Liew A (2006) Neural-network-based signature recognition for harmonic source identification. IEEE Trans Power Deliv 21(1):398–405
Liu S, Silverman M (2001) A practical guide to biometric security technology. IT Prof 3(1):27–32
Kyoso M, Uchiyama A (2001) Development of an ECG identification system. In: Proceedings of the 23rd annual international conference of the IEEE engineering in medicine and biology society, 2001, vol 4. IEEE, pp 3721–3723
Irvine J, Wiederhold B, Gavshon L, Israel S, McGehee S, Meyer R, Wiederhold M (2001) Heart rate variability: a new biometric for human identification. In: Proceedings of the international conference on artificial intelligence (IC-AI01), pp 1106–1111
Israel SA, Scruggs WT, Worek WJ, Irvine JM (2003) Fusing face and ECG for personal identification. In: 2003 Proceedings of the 32nd applied imagery pattern recognition workshop. IEEE, pp 226–231
Israel SA, Irvine JM, Cheng A, Wiederhold MD, Wiederhold BK (2005) ECG to identify individuals. Pattern Recogn 38(1):133–142
Travaglini A, Lamberti C, DeBie J, Ferri M (1998) Respiratory signal derived from eight-lead ECG. In: Computers in cardiology 1998. IEEE, pp 65–68
Zhang T, Keller H, OBrien MJ, Mackie TR, Paliwal B (2003) Application of the spirometer in respiratory gated radiotherapy. Med Phys 30(12):3165–3171
Marks MK, South M, Carter BG (1995) Measurement of respiratory rate and timing using a nasal thermocouple. J Clin Monit 11(3):159–164
Allison RD, Holmes E, Nyboer J (1964) Volumetric dynamics of respiration as measured by electrical impedance plethysmography. J Appl Physiol 19(1):166–173
Heisele B, Ho P, Poggio T (2001) Face recognition with support vector machines: global versus component-based approach. In: 2001 Proceedings of the eighth IEEE international conference on computer vision, ICCV 2001, vol 2. IEEE, pp 688–694
Heisele B, Koshizen T (2004) Components for face recognition. In: 2004 Proceedings of the sixth IEEE international conference on automatic face and gesture recognition. IEEE, pp 153–158
Huang J, Heisele B, Blanz V (2003) Component-based face recognition with 3D morphable models. In: Audio-and video-based biometric person authentication. Springer, pp 27–34
Heisele B, Serre T, Pontil M, Vetter T, Poggio T (2001) Categorization by learning and combining object parts. In: NIPS, pp 1239–1245
Osia N, Bourlai T (2014) A spectral independent approach for physiological and geometric based face recognition in the visible, middle-wave and long-wave infrared bands. Image Vis Comput (in press)
Osia N, Bourlai T (2012) Holistic and partial face recognition in the MWIR band using manual and automatic detection of face-based features. In: 2012 IEEE conference on technologies for homeland security (HST). IEEE, pp 273–279
Kenney JF, Keeping ES (1962) Root mean square. In: Mathematics of statistics, 3rd edn. Princeton, NJ, Van Nostrand, pp 59–60
Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185
Du S, Ward R (2005) Wavelet-based illumination normalization for face recognition. In: 2005 IEEE international conference on image processing. ICIP 2005, vol 2. IEEE, pp II–954
Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. In: Computer vision-ECCV 2004. Springer, pp 469–481
Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650
Bourlai T, Whitelam C, Kakadiaris I (2011) Pupil detection under lighting and pose variations in the visible and active infrared bands. In: 2011 IEEE international workshop on information forensics and security (WIFS). IEEE, pp 1–6
Kaneshiro NK, Zieve D (2012) Nasal flaring. http://www.nlm.nih.gov/medlineplus/ency/article/003055.htm (MedlinePlus)
Wilkins LW (2007) Sensory system. In: Lippincott manual of nursing practice series: assessment. Lippincott Williams & Wilkins, pp 265–266
Kaneshiro NK, Zieve D. Nasal flaring, MedlinePlus. http://www.nlm.nih.gov/medlineplus/ency/imagepages/17279.htm
Tanaka M (2012) Face parts detection algorithm. http://www.mathworks.com/matlabcentral/fileexchange/36855-face-parts-detection (updated in 2014)
Biel L, Pettersson O, Philipson L, Wide P (2001) ECG analysis: a new approach in human identification. IEEE Trans Instrum Meas 50(3):808–812
Wang Y, Agrafioti F, Hatzinakos D, Plataniotis KN (2008) Analysis of human electrocardiogram for biometric recognition. EURASIP J Adv Signal Process 2008:19
Agrafioti F, Gao J, Hatzinakos D (2011) Heart biometrics: theory, methods and applications. In: Biometrics: book, vol 3, pp 199–216
Štruc V, Pavešić N (2009) Gabor-based kernel partial-least-squares discrimination features for face recognition. Informatica 20(1):115–138
Štruc V, Pavešic N (2011) Photometric normalization techniques for illumination invariance. Adv Face Image Anal Tech Technol IGI Global 279–300
Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041
Kalka ND, Bourlai T, Cukic B, Hornak L (2011) Cross-spectral face recognition in heterogeneous environments: a case study on matching visible to short-wave infrared imagery. In: 2011 International joint conference on biometrics (IJCB). IEEE, pp 1–8
Gross R, Brajovic V (2003) An image preprocessing algorithm for illumination invariant face recognition. In: Audio-and video-based biometric person authentication. Springer, pp 10–18
Chen W, Er MJ, Wu S (2006) Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. IEEE Trans Syst Man Cybern Part B Cybern 36(2):458–466
Zhang T, Tang YY, Fang B, Shang Z, Liu X (2009) Face recognition under varying illumination using gradientfaces. IEEE Trans Image Process 18(11):2599–2606
Heusch G, Cardinaux F, Marcel S (2005) Lighting normalization algorithms for face verification. In: IDIAP-com 05-3
Xie X, Zheng W-S, Lai J, Yuen PC, Suen CY (2011) Normalization of face illumination based on large-and small-scale features. IEEE Trans Image Process 20(7):1807–1821
Jobson DJ, Rahman Z-U, Woodell GA (1997) A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans Image Process 6(7):965–976
Wang B, Li W, Yang W, Liao Q (2011) Illumination normalization based on Weber’s law with application to face recognition. IEEE Signal Process Lett 18(8):462–465
Jobson DJ, Rahman Z-U, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462
Land EH, McCann J (1971) Lightness and retinex theory. JOSA 61(1):1–11
Editors and Affiliations
Face Normalization Techniques
Face Normalization Techniques
Wavelet Based Normalization Technique (WA): The WA approach was proposed by Du and Ward  that applies the discrete wavelet transform to an image and processes the obtained sub-bands. This technique focuses on the detailed coefficient matrices and employs histogram equalization to the approximate transform coefficients. Finally, reconstructs the normalized image using the inverse wavelet transform after the manipulation of each individual sub-band.
Anisotropic Diffusion Based Normalization Technique (AS): The AS approach adopts anisotropic smoothing of the input image to evaluate the luminance function, which was introduce by Gross et al.  to the face recognition area.
Discrete Cosine Transform Based Normalization Technique (DCT): The DCT technique was introduced by Chen et al.  that truncates an appropriate number of DCT coefficients to minimize illumination variations under different lighting conditions.
Gradientfaces Based Normalization Technique (GRA): The GRA approach proposed by Zhang et al.  is to transform image into the gradient domain and use the generated face representation as the illumination invariant version of the target image.
Homomorphic Filtering Based Normalization Technique (HOMO): The HOMO is to transform the input image into the logarithm followed by the frequency domain in order to enhance the high-frequency components and weaken the low-frequency parts. Finally, apply the inverse Fourier transform to obtain the output image in the spatial domain .
Isotropic Diffusion Based Normalization Technique (IS): The IS approach  is to estimate the luminance function of the input image using isotropic smoothing algorithm which is a simpler variance of the anisotropic diffusion based normalization technique .
Large-Scale and Small-Scale Features Normalization Technique (LSSF): The LSSF proposed by Xie et al.  firstly computes the reflectance and luminance function of the input image and then further analyzes both generated functions using a second time of normalization. Within the INface toolbox used in our experiment, the SSR technique is implemented as the normalization approach in both steps of LSSF technique.
Modified Anisotropic Diffusion Normalization Technique (MAS): The MAS approach included two main modification into the original anisotropic diffusion normalization technique : (1) Introducing an additional atan function to estimate the local contrast; (2) Apply a robust post-processing procedure proposed by Tan et al.  in the final stage of this technique.
Multi-Scale Retinex (MSR) Algorithm: The MSR method is to extend the previously designed single-scale center/surround retinex technique to a multi-scale version proposed by Jobson et al. .
Multi-Scale Weberfaces Normalization Technique (MSW): The MSW as an extend the single-scale Weberfaces approach proposed by Wange et al.  is to compute the relative gradient using a modified Weber contrast method for diverse neighborhood sizes and to apply a linear combination of the produced face representation as an illumination invariant version of the target output.
Steerable Filter Based Normalization Technique (SF): The SF approach produces the target normalized image by removing illumination induced appearance variation from the input facial image using steerable filters.
Single Scale Retinex (SSR) Algorithm: The SSR approach was proposed by Jobson et al.  on the basis of the retinex theory  as the majority of photometric normalization techniques.
Tan and Triggs Normalization Technique (TT): The TAT is to employ a processing chain on the input image by firstly using gamma correction, then applying DoG filtering and finally adopting a robust post-processor to generate the output normalized image .
Single Scale Weberfaces Normalization Technique (WEB): The WEB method is to compute the relative gradient using a modified Weber contrast algorithm and treat the generated face representation as an illumination invariant version of the target image .
Rights and permissions
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Yang, X., Bourlai, T. (2018). Video-Based Human Respiratory Wavelet Extraction and Identity Recognition. In: Karampelas, P., Bourlai, T. (eds) Surveillance in Action. Advanced Sciences and Technologies for Security Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-68533-5_3
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68532-8
Online ISBN: 978-3-319-68533-5
eBook Packages: Political Science and International StudiesPolitical Science and International Studies (R0)