Multimedia Tools and Applications

, Volume 78, Issue 6, pp 6913–6937 | Cite as

In-air hand gesture signature recognition system based on 3-dimensional imagery

  • Wee How KhohEmail author
  • Ying Han Pang
  • Andrew Beng Jin Teoh


A traditional online handwritten signature recognition system requires direct contact to acquisition device and usually will leave a traceable print on the surface. This made a signature possible and vulnerable to certain attempts of tracking and imitated. Looking into this shortfall, this paper proposes a novel approach to recognise an individual based on his/ her in-air hand motion while signing his/her signature. In this study, a low-cost acquisition device – Microsoft Kinect sensor is adopted to capture an image sequence of hand gesture signature. Palm region is first located and segmented through a predictive palm segmentation algorithm, which are then combined to generate a volume data. The volume data is condensed and reduced into a motion representation image by means of Motion History Image (MHI), which produces rich motion and temporal information. Several features are extracted from the MHI for empirical evaluation. Two classical recognition modes – identification and verification, are testified with an in-house database (HGS database). The proposed system achieves 90.4% identification accuracy and 3.22% equal error rate in verification mode. The experimental results substantiated the potential of the proposed system.


Dynamic signature Hand gesture signature Biometrics Motion analysis Gesture recognition 



  1. 1.
    Babu RV, Ramakrishnan KR (2004) Recognition of human actions using motion history information extracted from the compressed video. Image Vis Comput 22:597–607. CrossRefGoogle Scholar
  2. 2.
    Chen CP, Chen YT, Lee PH, Tsai YP, Lei S (2011) Real-time hand tracking on depth images. In Proceedings of 2011 IEEE Visual Communications and Image Processing, VCIP 2011.
  3. 3.
    Cheng H, Yang L, Liu Z (2015) A survey on 3D hand gesture recognition. IEEE Trans Circuits Syst Video Technol PP (99), 1. CrossRefGoogle Scholar
  4. 4.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In Proceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005 (Vol I pp 886–893).
  5. 5.
    Davis JW, Bobick AF (1997) The representation and recognition of human movement using temporal templates. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 23(402):928–934. CrossRefGoogle Scholar
  6. 6.
    Dominio F, Donadeo M, Marin G, Zanuttigh P, Cortelazzo GM (2013) Hand gesture recognition with depth data. In Proceedings of the 4th ACM/IEEE International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Stream (pp 9–16). Barcelona, SpainGoogle Scholar
  7. 7.
    Fierrez J, Ortega-Garcia J (2008) On-Line Signature Verification. In Handbook of Biometrics (pp 189–209).
  8. 8.
    Fierrez J, Ortega-Garcia J, Ramos D, Gonzalez-Rodriguez J (2007) HMM-based on-line signature verification: feature extraction and signature modeling. Pattern Recogn Lett 28(16):2325–2334. CrossRefGoogle Scholar
  9. 9.
    Fischer A, Diaz M, Plamondon R, Ferrer MA (2015) Robust score normalization for DTW-based on-line signature verification. In Proceedings of the International Conference on Document Analysis and Recognition, ICDAR (Vol 2015–Novem, pp 241–245).
  10. 10.
    Gruber C, Gruber T, Krinninger S, Sick B (2010) Online signature verification with support vector machines based on LCSS kernel functions. IEEE Trans Syst Man Cybern B, Cybern Publ IEEE Syst Man Cybern Soc 40(4):1088–1100. CrossRefGoogle Scholar
  11. 11.
    Han J, Shao L, Member S, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. 43(5):1318–1334Google Scholar
  12. 12.
    Iranmanesh V, Ahmad SMS, Adnan WAW, Yussof S, Arigbabu OA, Malallah FL et al (2014) Online handwritten signature verification using neural network classifier based on principal component analysis. TheScientificWorldJournal 2014:381469. CrossRefGoogle Scholar
  13. 13.
    Jaemin L, Takimoto H, Yamauchi H, Kanazawa A, Mitsukura Y (2013) A robust gesture recognition based on depth data. FCV 2013 - Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 127–131.
  14. 14.
    Jain AK, Ross A, Prabhakar S (2004) An introduction to biometric recognition. IEEE Trans Circuits Syst Video Technol 14(1):4–20. CrossRefGoogle Scholar
  15. 15.
    Jeon JH, Oh BS, Toh KA (2012) A system for hand gesture based signature recognition. In Proceedings of 2012 12th International Conference on Control, Automation, Robotics and Vision, ICARCV 2012 (pp 171–175).
  16. 16.
    Ju Z, Gao D, Cao J, Liu H (2015) A novel approach to extract hand gesture feature in depth images. Multimed Tools Appl. CrossRefGoogle Scholar
  17. 17.
    Kashi R, Hu J, Nelson WL, Turin W (1997) On-line handwritten signature verification using hidden Markov model features. Proc Fourth Int Conf Doc Anal Recogn 1:253–257. CrossRefGoogle Scholar
  18. 18.
    Khalil MI, Moustafa M, Abbas HM (2009) Enhanced DTW based on-line signature verification. Image Processing (ICIP), 2009 16th IEEE International Conference on, 2713–2716.
  19. 19.
    Khoh WH, Ong TS, Pang YH, Teoh ABJ (2014a) Score level fusion approach in dynamic signature verification based on hybrid wavelet-fourier transform. Secur Commun Netw (July 2013), 1067–1078.
  20. 20.
    Khoh WH, Yap HY, Pang YH (2014b) Dynamic signature verification based on hybrid wavelet- Fourier transform. Sunway Acad J 11:11–21Google Scholar
  21. 21.
    Kholmatov A, Yanikoglu B (2005) Identity authentication using improved online signature verification method. Pattern Recogn Lett 26(15):2400–2408CrossRefGoogle Scholar
  22. 22.
    Kour J, Hanmandlu M, Ansari AQ (2011) Online signature verification using GA-SVM. 2011 International Conference on Image Information Processing, (Iciip), 1–4.
  23. 23.
    Le VB, Nguyen AT, Zhu Y (2014) Hand detecting and positioning based on depth image of Kinect sensor. Int J Inf Electron Eng 4(3):176–179. CrossRefGoogle Scholar
  24. 24.
    Ling X, Wang Y, Zhang Z, Wang Y (2010) On-line signature verification based on Gabor features. The 19th Annual Wireless and Optical Communications Conference (WOCC 2010), 1–4.
  25. 25.
    Manabe H, Yamakawa Y, Sasamoto T, Sasaki R (2009) Security evaluation of biometrics authentications for cellular phones. IIH-MSP 2009 - 2009 5th International Conference on Intelligent Information Hiding and Multimedia Signal Processing (1) 34–39.
  26. 26.
    Mendels O, Stern H, Berman S (2014) User identification for home entertainment based on free-air hand motion signatures. IEEE Trans Syst Man Cybern Syst 44(11). CrossRefGoogle Scholar
  27. 27.
    Meshoul S, Batouche M (2010) A novel approach for online signature verification using fisher based probabilistic neural network. IEEE Symp Comput Commun 314–319.
  28. 28.
    Park MS, Hasan MM, Kim JM, Chae OS (2012) Hand detection and tracking using depth and color information. Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV’12), 2, 779–785Google Scholar
  29. 29.
    Pisharady PK, Saerbeck M (2015) Recent methods and databases in vision-based hand gesture recognition: a review. Comput Vis Image Underst 141:152–165. CrossRefGoogle Scholar
  30. 30.
    Raheja JL, Chaudhary A, Singal K (2011) Tracking of fingertips and centers of palm using KINECT. In Proceedings of CIMSim 2011: 3rd International Conference on Computational Intelligence, Modelling and Simulation.
  31. 31.
    Suarez J, Murphy R (2012) Hand gesture recognition with depth images: a review. In IEEE International Symposium on Robot and Human Interactive COmmunication (pp. 411–417)Google Scholar
  32. 32.
    Sun Z, Wang Y, Qu G, Zhou Z (2016) A 3-D hand gesture signature based biometric authentication system for smartphones. Secur Commun Netw 9(11):1359–1373. CrossRefGoogle Scholar
  33. 33.
    Tian J, Qu C, Xu W, Wang S (2013) KinWrite: handwriting-based authentication using kinect. In Proceedings of NDSS ‘13Google Scholar
  34. 34.
    Yang L, Widjaja BK, Prasad R (1995) Application of hidden Markov models for signature verification. Pattern Recogn 28(2):161–170. CrossRefGoogle Scholar
  35. 35.
    Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimedia 19(2):4–10. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Multimedia UniversityMelakaMalaysia
  2. 2.Yonsei UniversitySeoulSouth Korea

Personalised recommendations