Skip to main content
SpringerLink
Log in

Recognition in the near infrared spectrum for face, gender and facial expressions

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Visible face recognition systems are subjected to failure when recognizing the faces in unconstrained scenarios. So, recognizing faces under variable and low illumination conditions are more important since most of the security breaches happen during night time. Near Infrared (NIR) spectrum enables to acquire high quality images, even without any external source of light and hence it is a good method for solving the problem of illumination. Further, the soft biometric trait, gender classification and non verbal communication, facial expression recognition has also been addressed in the NIR spectrum. In this paper, a method has been proposed to recognize the face along with gender classification and facial expression recognition in NIR spectrum. The proposed method is based on transfer learning and it consists of three core components, i) training with small scale NIR images ii) matching NIR-NIR images (homogeneous) and iii) classification. Training on NIR images produce features using transfer learning which has been pre-trained on large scale VIS face images. Next, matching is performed between NIR-NIR spectrum of both training and testing faces. Then it is classified using three, separate SVM classifiers, one for face recognition, the second one for gender classification and the third one for facial expression recognition. It has been observed that the method gives state-of-the-art accuracy on the publicly available, challenging, benchmark datasets CASIA NIR-VIS 2.0, Oulu-CASIA NIR-VIS, PolyU, CBSR, IIT Kh and HITSZ for face recognition. Further, for gender classification the Oulu-CASIA NIR-VIS, PolyU,and IIT Kh has been analyzed and for facial expression the Oulu-CASIA NIR-VIS dataset has been analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Zhao G, Huang X, Taini M, Li SZ, PietikäInen M (2011) Facial expression recognition from near-infrared videos. Image and Vision Computing 29(9):607–619

    Article  Google Scholar 

  2. Cerna L, Cámara-Chávez G, Menotti D (2013) Face detection: histogram of oriented gradients and bag of feature method. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition (IPCV). The Steering Committee of The World Congress in Computer Science, Computer, pp 1

  3. Chen C, Ross A (2011) Evaluation of gender classification methods on thermal and near-infrared face images. In: 2011 International joint conference on biometrics (IJCB). IEEE, pp 1–8

  4. Dhamecha TI, Sharma P, Singh R, Vatsa M (2014) On effectiveness of histogram of oriented gradient features for visible to near infrared face matching. IEEE, pp 1788–1793

  5. Duan M, Li K, Yang C, Li K (2018) A hybrid deep learning cnnelm for age and gender classification. Neurocomputing 275(C):448–461

    Article  Google Scholar 

  6. Fang J, Yuan Y, Lu X, Feng Y (2019) Muti-stage learning for gender and age prediction. Neurocomputing 334:114–124

    Article  Google Scholar 

  7. Farokhi S, Flusser J, Sheikh UU (2016) Near infrared face recognition: a literature survey. Computer Science Review 21:1–17

    Article  MathSciNet  Google Scholar 

  8. Fu C, Wu X, Hu Y, Huang H, He R (2021) Dvg-face: dual variational generation for heterogeneous face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence

  9. Happy S, Dasgupta A, George A, Routray A (2012) A video database of human faces under near infra-red illumination for human computer interaction applications. In: 2012 4th international conference on intelligent human computer interaction (IHCI). IEEE, pp 1–4

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  11. He R, Wu X, Sun Z, Tan T (2019) Wasserstein cnn: learning invariant features for nir-vis face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(7):1761–1773

    Article  Google Scholar 

  12. Kim J, Jo H, Ra M, Kim WY (2019) Fine-tuning approach to nir face recognition. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2337–2341

  13. Li S, Yi D, Lei Z, Liao S (2013) The casia nir-vis 2.0 face database. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 348–353

  14. Li SZ, Chu R, Liao S, Zhang L (2007) Illumination invariant face recognition using near-infrared images. IEEE Transactions on Pattern Analysis and Machine Intelligence 29(4):627–639

    Article  Google Scholar 

  15. Li SZ, Zhang L, Liao S, Zhu X, Chu R, Ao M, He R (2006) A near-infrared image based face recognition system. In: 7th international conference on automatic face and gesture recognition (FGR06). IEEE, pp 455–460

  16. Liang D, Liang H, Yu Z, Zhang Y (2020) Deep convolutional bilstm fusion network for facial expression recognition. The Visual Computer 36(3):499–508

    Article  Google Scholar 

  17. Maji S, Berg AC, Malik J (2013) Efficient classification for additive kernel svms. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(1):66–77. https://doi.org/10.1109/TPAMI.2012.62

    Article  Google Scholar 

  18. Makinen E, Raisamo R (2008) Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(3):541–547

    Article  Google Scholar 

  19. Narang N, Bourlai T (2016) Gender and ethnicity classification using deep learning in heterogeneous face recognition. In: 2016 international conference on biometrics (ICB). IEEE, pp 1–8

  20. Nordstrøm MM, Larsen M, Sierakowski J, Stegmann MB (2004) The imm face database-an annotated dataset of 240 face images

  21. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. Journal of Machine Learning Research 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  22. Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image and Vision Computing 16(5):295–306

    Article  Google Scholar 

  23. Singh M, Nagpal S, Singh R, Vatsa M (2017) Class representative autoencoder for low resolution multi-spectral gender classification. In: 2017 international joint conference on neural networks (IJCNN). IEEE, pp 1026–1033

  24. Wu X, Huang H, Patel VM, He R, Sun Z (2019) Disentangled variational representation for heterogeneous face recognition. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 9005–9012

  25. Xu Y, Zhong A, Yang J, Zhang D (2011) Bimodal biometrics based on a representation and recognition approach. Optical Engineering 50(3):037,202

  26. Yang H, Ciftci U, Yin L (2018) Facial expression recognition by de-expression residue learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2168–2177

  27. Zhang B, Zhang L, Zhang D, Shen L (2010) Directional binary code with application to polyu near-infrared face database. Pattern Recognition Letters 31(14):2337–2344

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of CSE, IIITDM Kancheepuram, Chennai, India

    Nilu R. Salim & Umarani Jayaraman

  2. Department of IT, SSN College of Engineering, Chennai, India

    Srinath V.

  3. GLA University, Mathura, India

    Phalguni Gupta

Authors
  1. Nilu R. Salim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Srinath V.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Umarani Jayaraman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Phalguni Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nilu R. Salim.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Salim, N.R., V., S., Jayaraman, U. et al. Recognition in the near infrared spectrum for face, gender and facial expressions. Multimed Tools Appl 81, 4143–4162 (2022). https://doi.org/10.1007/s11042-021-11728-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11728-9

Keywords

Search

Navigation