Multimedia Tools and Applications

, Volume 76, Issue 3, pp 4571–4598 | Cite as

Real-time recognition of medial structures within hand postures through Eigen-space and geometric skeletal shape features

Article
  • 113 Downloads

Abstract

Skeletons provide landmark points that preserve implicit strokes or medial structure within a shape for compact representation. Though, hand posture shapes are usually recognized by region or contour representations, some applications may only be interested in the recognition of medial structures within the postures rather than their exact outlines and regions. Proposed work identifies several unique medial structures formed by a set of both one and two-handed postures and demonstrates their pure skeletal recognition in real-time. Existing skeleton-based recognition schemes apply the complex segmental processing on underlying skeleton and rely on contour information which is not suitable for fast recognition of medial structures. Presented work applies intuitive Eigen-space based Principal Components of Symbolic Structure (PCSS) and geometric Equi-Polar Signature (EPS) features to accomplish the recognition task. Both PCV and EPS process the skeleton globally without sections without associating contour information. Recognition accuracy up to 94% is obtained on a 22 posture dataset comprising of 10,560 depth frames with 480 samples for each posture. Depth sensor based acquisition is employed to meet the real-time requirements.

Keywords

Skeletonization Hand posture recognition Principal component vector Polar normalization 

References

  1. 1.
    Bai X, Latecki LJ (2008) Path similarity skeleton graph matching. IEEE Trans Pattern Anal Mach Intell 30(7):1282–1292CrossRefGoogle Scholar
  2. 2.
    Barkoky A, Charkari NM (2011) Static hand gesture recognition of Persian sign numbers using thinning method. In: IEEE International Conference on Multimedia Technology (ICMT), pp. 6548–6551Google Scholar
  3. 3.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522CrossRefGoogle Scholar
  4. 4.
    Bourennane S, Fossati C (2012) Comparison of shape descriptors for hand posture recognition in video. SIViP 6(1):147–157CrossRefGoogle Scholar
  5. 5.
    Chang CC, Wu TC (1995) An exact match retrieval scheme based upon principal component analysis. Pattern Recogn Lett 16(5):465–470CrossRefGoogle Scholar
  6. 6.
    Chen C, Liu M, Zhang B, Han J, Jiang J, Liu H (2016) 3D Action Recognition Using Multi-temporal Depth Motion Maps and Fisher Vector. IJCAI:3331–3337Google Scholar
  7. 7.
    Cobos S, Ferre M, Uran S, Ortego J, Pena C (2008) Efficient human hand kinematics for manipulation tasks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, pp 2246–2251Google Scholar
  8. 8.
    Deng W, Iyengar SS, Brener NE (2000) A fast parallel thinning algorithm for the binary image skeletonization. Int J High Perform Comput Appl 14(1):65–81CrossRefGoogle Scholar
  9. 9.
    Dominio F, Donadeo M, Zanuttigh P (2014) Combining multiple depth-based descriptors for hand gesture recognition. Pattern Recogn Lett 50:101–111CrossRefGoogle Scholar
  10. 10.
    El-Khoury S, Li M, Billard A (2013) On the generation of a variety of grasps. Robot Auton Syst 61(12):1335–1349CrossRefGoogle Scholar
  11. 11.
    Fujimura K, Liu X (2006) Sign recognition using depth image streams. In: IEEE 7th International Conference on Automatic Face and Gesture Recognition (FGR), pp. 381–386Google Scholar
  12. 12.
    Goh WB (2008) Strategies for shape matching using skeletons. Comput Vis Image Underst 110(3):326–345CrossRefGoogle Scholar
  13. 13.
    Gonzalez RC, Woods RE, Eddins SL (2012) Digital image processing using MATLAB, 2nd edn. Tata McGraw-Hill, New DelhiGoogle Scholar
  14. 14.
    Guru DS, Punitha P (2004) An invariant scheme for exact match retrieval of symbolic images based upon principal component analysis. Pattern Recogn Lett 25(1):73–86CrossRefGoogle Scholar
  15. 15.
    Guru DS, Punitha P, Nagabhushan P (2003) Archival and retrieval of symbolic images: an invariant scheme based on triangular spatial relationship. Pattern Recogn Lett 24(14):2397–2408CrossRefMATHGoogle Scholar
  16. 16.
    Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334CrossRefGoogle Scholar
  17. 17.
    Jiang M, Kong J, Bebis G, Huo H (2015) Informative joints based human action recognition using skeleton contexts. Signal Process Image Commun 33:29–40CrossRefGoogle Scholar
  18. 18.
    Kimia BB, Tannenbaum AR, Zucker SW (1995) Shapes, shocks, and deformations I: the components of two-dimensional shape and the reaction-diffusion space. Int J Comput Vis 15(3):189–224CrossRefGoogle Scholar
  19. 19.
    Kirac F, Kara YE, Akarun L (2014) Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data. Pattern Recogn Lett 50:91–100CrossRefGoogle Scholar
  20. 20.
    Krinidis S, Krinidis M (2014) Skeletonization based on angle maps. Pattern Anal Applic 17(3):517–528MathSciNetCrossRefGoogle Scholar
  21. 21.
    Lam L, Lee SW, Suen CY (1992) Thinning methodologies-a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 14(9):869–885CrossRefGoogle Scholar
  22. 22.
    Lee AJ, Chiu HP (2003) 2D Z-string: a new spatial knowledge representation for image databases. Pattern Recogn Lett 24(16):3015–3026CrossRefGoogle Scholar
  23. 23.
    Li X., An JH, Min JH, Hong KS (2011) Hand gesture recognition by stereo camera using the thinning method. In: IEEE International Conference on Multimedia Technology (ICMT), pp. 3077–3080Google Scholar
  24. 24.
    Liu L, Xing J, Ai H, & Ruan X (2012). Hand posture recognition using finger geometric feature. In IEEE 21st International Conference on Pattern Recognition (ICPR), pp. 565–568Google Scholar
  25. 25.
    Ma J, Choi S (2014) Kinematic skeleton extraction from 3D articulated models. Comput Aided Des 46:221–226CrossRefGoogle Scholar
  26. 26.
    Narayana M, Sandeep VM, Kulkarni S (2011) Skeleton based signatures for content based image retrieval. Int J Comput Appl 23(7):29–34Google Scholar
  27. 27.
    Pedersoli F, Benini S, Adami N, Leonardi R (2014) XKin: an open source framework for hand pose and gesture recognition using kinect. Vis Comput 30(10):1107–1122CrossRefGoogle Scholar
  28. 28.
    Plouffe G, Cretu AM (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–316CrossRefGoogle Scholar
  29. 29.
    Pugeault N, and Bowden R (2011) Spelling it out: Real-time asl fingerspelling recognition. In IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1114–1119Google Scholar
  30. 30.
    Qian C, Sun X, Wei Y, Tang X, Sun J (2014) Realtime and robust hand tracking from depth. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 1106–1113Google Scholar
  31. 31.
    Rempel D, Camilleri MJ, Lee DL (2014) The design of hand gestures for human–computer interaction: lessons from sign language interpreters. Int J Hum Comput Stud 72(10):728–735CrossRefGoogle Scholar
  32. 32.
    Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimed 15(5):1110–1120CrossRefGoogle Scholar
  33. 33.
    Roberts, L., Singhal, G., Kaliki, R. (2011) : Slip detection and grip adjustment using optical tracking in prosthetic hands. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, pp. 2929–2932Google Scholar
  34. 34.
    Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distanceas a metric for image retrieval. Int J Comput Vis 40:99–121CrossRefMATHGoogle Scholar
  35. 35.
    Sebastian TB, Kimia BB (2005) Curves vs. skeletons in object recognition. Signal Process 85(2):247–263CrossRefMATHGoogle Scholar
  36. 36.
    Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124CrossRefGoogle Scholar
  37. 37.
    Soffer A, Samet H (1999) Two data organizations for storing symbolic images in a relational database system. In: Database semantics, Springer, US, pp 435–456Google Scholar
  38. 38.
    Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158CrossRefGoogle Scholar
  39. 39.
    Su Z, Cao Z, Wang Y, Zhen X (2011) Identification of unreliable segments to improve skeletonization of handwriting images. Pattern Anal Applic 14(1):77–86MathSciNetCrossRefGoogle Scholar
  40. 40.
    Suau X, Alcoverro M, López-Méndez A, Ruiz-Hidalgo J, Casas JR (2014) Real-time fingertip localization conditioned on hand gesture classification. Image Vis Comput 32(8):522–532CrossRefGoogle Scholar
  41. 41.
    Tang D, Yu TH, Kim TK (2013) Real-time articulated hand pose estimation using semi-supervised transductive regression forests. In: IEEE International Conference on Computer Vision (ICCV), pp. 3224–3231Google Scholar
  42. 42.
    Torsello A, Hancock ER (2003) Curvature dependent skeletonization. In: Image analysis. Springer, Berlin, pp. 200–207CrossRefGoogle Scholar
  43. 43.
    Wang J, Liu Z, Chorowski J, Chen Z, Wu Y (2012) Robust 3d action recognition with random occupancy patterns. In: Computer vision–ECCV 2012. Springer, Berlin, pp. 872–885CrossRefGoogle Scholar
  44. 44.
    Wang X, Yang W, Peng H, Wang G (2013) Shape-aware skeletal deformation for 2D characters. Vis Comput 29:545–553CrossRefGoogle Scholar
  45. 45.
    Wang X, Wang R, Zhou F (2014) Fingertips detection and hand tracking based on curve fitting. In IEEE 7th International Congress on Image and Signal Processing (CISP), pp. 99–103Google Scholar
  46. 46.
    Wu D and Shao L (2014) Leveraging hierarchical parametric networks for skeletal joints based action segmentation and recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 724–731Google Scholar
  47. 47.
    Wu D, Pigou L, Kindermans PJ, Nam LE, Shao L, Dambre J, Odobez JM (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell:1–16Google Scholar
  48. 48.
    Xie J, Heng PA, Shah M (2008) Shape matching and modeling using skeletal context. Pattern Recogn 41(5):1756–1767CrossRefMATHGoogle Scholar
  49. 49.
    Xie S., Liu J, Smith CD (2012) Curve skeleton-based shape representation and classification. In: IEEE 19th International Conference on Image Processing (ICIP), pp 529–532Google Scholar
  50. 50.
    Yu D, Jin JS, Luo S, Lai W, Park M, Pham TD (2010) Shape analysis and recognition based on skeleton and morphological structure. In: Seventh International Conference on Computer Graphics, Imaging and Visualization (CGIV), pp. 118–123Google Scholar
  51. 51.
    Yu M, Liu L, Shao L (2016) Structure-preserving binary representations for RGB-D action recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1651–1664CrossRefGoogle Scholar
  52. 52.
    Zhang TY, Suen CY (1984) A fast parallel algorithm for thinning digital patterns. Commun ACM 27(3):236–239CrossRefGoogle Scholar
  53. 53.
    Zhang B, Perina A, Murino V and Del Bue A (2015) Sparse representation classification with manifold constraints transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4557–4565Google Scholar
  54. 54.
    Zhang B, Perina A, Li Z, Murino V, Liu J and Ji R (2016) Bounding multiple gaussians uncertainty with application to object tracking. International Journal of Computer Vision, pp1–16Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Discipline of Computer Science and EngineeringPDPM Indian Institute of Information Technology, Design and ManufacturingJabalpurIndia

Personalised recommendations