Skip to main content
SpringerLink
Log in

In-air hand gesture signature recognition system based on 3-dimensional imagery

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A traditional online handwritten signature recognition system requires direct contact to acquisition device and usually will leave a traceable print on the surface. This made a signature possible and vulnerable to certain attempts of tracking and imitated. Looking into this shortfall, this paper proposes a novel approach to recognise an individual based on his/ her in-air hand motion while signing his/her signature. In this study, a low-cost acquisition device – Microsoft Kinect sensor is adopted to capture an image sequence of hand gesture signature. Palm region is first located and segmented through a predictive palm segmentation algorithm, which are then combined to generate a volume data. The volume data is condensed and reduced into a motion representation image by means of Motion History Image (MHI), which produces rich motion and temporal information. Several features are extracted from the MHI for empirical evaluation. Two classical recognition modes – identification and verification, are testified with an in-house database (HGS database). The proposed system achieves 90.4% identification accuracy and 3.22% equal error rate in verification mode. The experimental results substantiated the potential of the proposed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Babu RV, Ramakrishnan KR (2004) Recognition of human actions using motion history information extracted from the compressed video. Image Vis Comput 22:597–607. https://doi.org/10.1016/j.imavis.2003.11.004

    Article  Google Scholar 

  2. Chen CP, Chen YT, Lee PH, Tsai YP, Lei S (2011) Real-time hand tracking on depth images. In Proceedings of 2011 IEEE Visual Communications and Image Processing, VCIP 2011. https://doi.org/10.1109/VCIP.2011.6115983

  3. Cheng H, Yang L, Liu Z (2015) A survey on 3D hand gesture recognition. IEEE Trans Circuits Syst Video Technol PP (99), 1. https://doi.org/10.1109/TCSVT.2015.2469551

    Article  Google Scholar 

  4. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In Proceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005 (Vol I pp 886–893). https://doi.org/10.1109/CVPR.2005.177

  5. Davis JW, Bobick AF (1997) The representation and recognition of human movement using temporal templates. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 23(402):928–934. https://doi.org/10.1109/CVPR.1997.609439

    Article  Google Scholar 

  6. Dominio F, Donadeo M, Marin G, Zanuttigh P, Cortelazzo GM (2013) Hand gesture recognition with depth data. In Proceedings of the 4th ACM/IEEE International Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Stream (pp 9–16). Barcelona, Spain

  7. Fierrez J, Ortega-Garcia J (2008) On-Line Signature Verification. In Handbook of Biometrics (pp 189–209). https://doi.org/10.1007/978-0-387-71041-9_10

  8. Fierrez J, Ortega-Garcia J, Ramos D, Gonzalez-Rodriguez J (2007) HMM-based on-line signature verification: feature extraction and signature modeling. Pattern Recogn Lett 28(16):2325–2334. https://doi.org/10.1016/j.patrec.2007.07.012

    Article  Google Scholar 

  9. Fischer A, Diaz M, Plamondon R, Ferrer MA (2015) Robust score normalization for DTW-based on-line signature verification. In Proceedings of the International Conference on Document Analysis and Recognition, ICDAR (Vol 2015–Novem, pp 241–245). https://doi.org/10.1109/ICDAR.2015.7333760

  10. Gruber C, Gruber T, Krinninger S, Sick B (2010) Online signature verification with support vector machines based on LCSS kernel functions. IEEE Trans Syst Man Cybern B, Cybern Publ IEEE Syst Man Cybern Soc 40(4):1088–1100. https://doi.org/10.1109/TSMCB.2009.2034382

    Article  Google Scholar 

  11. Han J, Shao L, Member S, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. 43(5):1318–1334

  12. Iranmanesh V, Ahmad SMS, Adnan WAW, Yussof S, Arigbabu OA, Malallah FL et al (2014) Online handwritten signature verification using neural network classifier based on principal component analysis. TheScientificWorldJournal 2014:381469. https://doi.org/10.1155/2014/381469

    Article  Google Scholar 

  13. Jaemin L, Takimoto H, Yamauchi H, Kanazawa A, Mitsukura Y (2013) A robust gesture recognition based on depth data. FCV 2013 - Proceedings of the 19th Korea-Japan Joint Workshop on Frontiers of Computer Vision, 127–131. https://doi.org/10.1109/FCV.2013.6485474

  14. Jain AK, Ross A, Prabhakar S (2004) An introduction to biometric recognition. IEEE Trans Circuits Syst Video Technol 14(1):4–20. https://doi.org/10.1109/TCSVT.2003.818349

    Article  Google Scholar 

  15. Jeon JH, Oh BS, Toh KA (2012) A system for hand gesture based signature recognition. In Proceedings of 2012 12th International Conference on Control, Automation, Robotics and Vision, ICARCV 2012 (pp 171–175). https://doi.org/10.1109/ICARCV.2012.6485153

  16. Ju Z, Gao D, Cao J, Liu H (2015) A novel approach to extract hand gesture feature in depth images. Multimed Tools Appl. https://doi.org/10.1007/s11042-015-2609-2

    Article  Google Scholar 

  17. Kashi R, Hu J, Nelson WL, Turin W (1997) On-line handwritten signature verification using hidden Markov model features. Proc Fourth Int Conf Doc Anal Recogn 1:253–257. https://doi.org/10.1109/ICDAR.1997.619851

    Article  Google Scholar 

  18. Khalil MI, Moustafa M, Abbas HM (2009) Enhanced DTW based on-line signature verification. Image Processing (ICIP), 2009 16th IEEE International Conference on, 2713–2716. https://doi.org/10.1109/ICIP.2009.5414166

  19. Khoh WH, Ong TS, Pang YH, Teoh ABJ (2014a) Score level fusion approach in dynamic signature verification based on hybrid wavelet-fourier transform. Secur Commun Netw (July 2013), 1067–1078. https://doi.org/10.1002/sec

  20. Khoh WH, Yap HY, Pang YH (2014b) Dynamic signature verification based on hybrid wavelet- Fourier transform. Sunway Acad J 11:11–21

    Google Scholar 

  21. Kholmatov A, Yanikoglu B (2005) Identity authentication using improved online signature verification method. Pattern Recogn Lett 26(15):2400–2408

    Article  Google Scholar 

  22. Kour J, Hanmandlu M, Ansari AQ (2011) Online signature verification using GA-SVM. 2011 International Conference on Image Information Processing, (Iciip), 1–4. https://doi.org/10.1109/ICIIP.2011.6108923

  23. Le VB, Nguyen AT, Zhu Y (2014) Hand detecting and positioning based on depth image of Kinect sensor. Int J Inf Electron Eng 4(3):176–179. https://doi.org/10.7763/IJIEE.2014.V4.430

    Article  Google Scholar 

  24. Ling X, Wang Y, Zhang Z, Wang Y (2010) On-line signature verification based on Gabor features. The 19th Annual Wireless and Optical Communications Conference (WOCC 2010), 1–4. https://doi.org/10.1109/WOCC.2010.5510683

  25. Manabe H, Yamakawa Y, Sasamoto T, Sasaki R (2009) Security evaluation of biometrics authentications for cellular phones. IIH-MSP 2009 - 2009 5th International Conference on Intelligent Information Hiding and Multimedia Signal Processing (1) 34–39. https://doi.org/10.1109/IIH-MSP.2009.51

  26. Mendels O, Stern H, Berman S (2014) User identification for home entertainment based on free-air hand motion signatures. IEEE Trans Syst Man Cybern Syst 44(11). https://doi.org/10.1109/TSMC.2014.2329652

    Article  Google Scholar 

  27. Meshoul S, Batouche M (2010) A novel approach for online signature verification using fisher based probabilistic neural network. IEEE Symp Comput Commun 314–319. https://doi.org/10.1109/ISCC.2010.5546760

  28. Park MS, Hasan MM, Kim JM, Chae OS (2012) Hand detection and tracking using depth and color information. Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV’12), 2, 779–785

  29. Pisharady PK, Saerbeck M (2015) Recent methods and databases in vision-based hand gesture recognition: a review. Comput Vis Image Underst 141:152–165. https://doi.org/10.1016/j.cviu.2015.08.004

    Article  Google Scholar 

  30. Raheja JL, Chaudhary A, Singal K (2011) Tracking of fingertips and centers of palm using KINECT. In Proceedings of CIMSim 2011: 3rd International Conference on Computational Intelligence, Modelling and Simulation. https://doi.org/10.1109/CIMSim.2011.51

  31. Suarez J, Murphy R (2012) Hand gesture recognition with depth images: a review. In IEEE International Symposium on Robot and Human Interactive COmmunication (pp. 411–417)

  32. Sun Z, Wang Y, Qu G, Zhou Z (2016) A 3-D hand gesture signature based biometric authentication system for smartphones. Secur Commun Netw 9(11):1359–1373. https://doi.org/10.1002/sec

    Article  Google Scholar 

  33. Tian J, Qu C, Xu W, Wang S (2013) KinWrite: handwriting-based authentication using kinect. In Proceedings of NDSS ‘13

  34. Yang L, Widjaja BK, Prasad R (1995) Application of hidden Markov models for signature verification. Pattern Recogn 28(2):161–170. https://doi.org/10.1016/0031-3203(94)00092-Z

    Article  Google Scholar 

  35. Zhang Z (2012) Microsoft kinect sensor and its effect. IEEE Multimedia 19(2):4–10. https://doi.org/10.1109/MMUL.2012.24

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Multimedia University, Jalan Ayer Keroh Lama, Bukit Beruang, 75450, Melaka, Malaysia

    Wee How Khoh & Ying Han Pang

  2. Yonsei University, Seoul, South Korea

    Andrew Beng Jin Teoh

Authors
  1. Wee How Khoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Han Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andrew Beng Jin Teoh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wee How Khoh.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khoh, W.H., Pang, Y.H. & Teoh, A.B.J. In-air hand gesture signature recognition system based on 3-dimensional imagery. Multimed Tools Appl 78, 6913–6937 (2019). https://doi.org/10.1007/s11042-018-6458-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6458-7

Keywords

Search

Navigation