A Model for Real-Time Biometric Authentication Using Facial and Hand Gesture Recognition

  • Astitva Narayan Pandey
  • Ajay Vikram Singh
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 103)


Facial recognition is the process of identifying and/or verifying distinct facial characteristics of an individual in an image or a video stream, pre-recorded or in real time. Hand gesture recognition refers to the process of identifying the configuration of an individual’s hand posture in an image and can be used to verify concurrency of it with respect to a pre-set randomized prompt example to use it in a way similar to CAPTCHA. This paper aims to unify these two similar, yet varying image processing technologies in order to be able to provide a multi-factor authentication system for web applications which cannot be tricked using pre-recorded footage. The face would be identified in the image using convolutional neural networks and then be masked from the image for the hand to be isolated by identifying them using the skin tone. Once the hand has been isolated, SIFT points identified would determine the gesture being performed by the subject, followed by matching the face with the requested credentials if approved. The facial recognition would act like a password and the hand gesture recognition as a means to verify the authenticity of the facial credentials provided.


Facial recognition Hand gesture recognition Authentication CAPTCHA Image processing 


  1. 1.
    Chen G, Shao Y, Tang C, Jin Z, Zhang J (2018) Deep transformation learning for face recognition in the unconstrained scene. Mach Vis Appl 1–11Google Scholar
  2. 2.
    Abdalmageed W, Wu Y, Rawls S, Harel S, Hassner T, Masi I, Choi J, Lekust J, Kim J, Natarajan P (2016) Face recognition using deep multi-pose representations. In: WACV, pp 1–9Google Scholar
  3. 3.
    Bansal A, Castillo C, Ranjan R, Chellappa R (2017) The dos and donts for cnn-based face verification. arXiv preprint, arXiv:1705.07426, p 5
  4. 4.
    Bansal A, Nanduri A, Castillo C, Ranjan R, Chellappa R (2016) Umdfaces: an annotated face dataset for training deep networks. arXiv preprint, arXiv:1611.01484
  5. 5.
    Bodla N, Zheng J, Xu H, Chen J-C, Castillo C, Chellappa R (2017) Deep heterogeneous feature fusion for template-based face recognition. In: WACV. IEEE, pp 586–595Google Scholar
  6. 6.
    Cao Z, Yin Q, Tang X, Sun J (2010) Face recognition with learning based descriptor. In: CVPR. IEEE, pp 2707–2714Google Scholar
  7. 7.
    Chen B, Deng W, Du J (2017) Noisy softmax: improving the generalization ability of dcnn via postponing the early softmax saturation. arXiv preprint, arXiv:1708.03769
  8. 8.
    Agarwal A, Singh R, Vatsa M, Ratha N (2018) Are imageagnostic universal adversarial perturbations for face recognition difficult to detect. IEEE BTASGoogle Scholar
  9. 9.
    Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition. Comput Vis Image Underst 101(1):1–15CrossRefGoogle Scholar
  10. 10.
    Dorner B (1993) Hand shape identification and tracking for sign language interpretation. In: IJCAI workshop on looking at peopleGoogle Scholar
  11. 11.
    Cui Y, Weng JJ (1996) Hand segmentation using learning based prediction and verification for hand sign recognition. In: Proceedings of international conference on automatic face and gesture recognition, Killington, Vt., pp 88–93, Oct 1996Google Scholar
  12. 12.
    Olszewska, JI (2018) Chapter 4: Automated face recognition: challenges and solutions. Pattern recognition-analysis and applications. IntechOpen, pp 59–79. Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Astitva Narayan Pandey
    • 1
  • Ajay Vikram Singh
    • 1
  1. 1.Amity Institute of Information TechnologyAmity UniversityNoidaIndia

Personalised recommendations