Attention Based Detection and Recognition of Hand Postures Against Complex Backgrounds
- 2.4k Downloads
A system for the detection, segmentation and recognition of multi-class hand postures against complex natural backgrounds is presented. Visual attention, which is the cognitive process of selectively concentrating on a region of interest in the visual field, helps human to recognize objects in cluttered natural scenes. The proposed system utilizes a Bayesian model of visual attention to generate a saliency map, and to detect and identify the hand region. Feature based visual attention is implemented using a combination of high level (shape, texture) and low level (color) image features. The shape and texture features are extracted from a skin similarity map, using a computational model of the ventral stream of visual cortex. The skin similarity map, which represents the similarity of each pixel to the human skin color in HSI color space, enhanced the edges and shapes within the skin colored regions. The color features used are the discretized chrominance components in HSI, YCbCr color spaces, and the similarity to skin map. The hand postures are classified using the shape and texture features, with a support vector machines classifier. A new 10 class complex background hand posture dataset namely NUS hand posture dataset-II is developed for testing the proposed algorithm (40 subjects, different ethnicities, various hand sizes, 2750 hand postures and 2000 background images). The algorithm is tested for hand detection and hand posture recognition using 10 fold cross-validation. The experimental results show that the algorithm has a person independent performance, and is reliable against variations in hand sizes and complex backgrounds. The algorithm provided a recognition rate of 94.36 %. A comparison of the proposed algorithm with other existing methods evidences its better performance.
KeywordsComputer vision Pattern recognition Hand gesture recognition Complex backgrounds Visual attention Biologically inspired features
The authors would like to thank Ms. Ma Zin Thu Shein for taking part in the shooting of NUS hand posture dataset-II. Also the authors express their appreciation to all the 40 subjects volunteered for the development of the dataset.
- Athitsos, V., & Sclaroff, S. (2003). Estimating 3d hand pose from a cluttered image. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 432–439). Google Scholar
- Bishop, C. (1995). Neural networks for pattern recognition. London: Oxford University Press. Google Scholar
- Eng-Jon, O., & Bowden, R. (2004). A boosted classifier tree for hand shape detection. In IEEE conference on automatic face and gesture recognition (pp. 889–894). Google Scholar
- Jones, J. P., & Palmer, L. A. (1987). An evaluation of the twodimensional gabor filter model of simple receptive fields in cat striate cortex. Journal of Neurophysiology, 58(6), 1233–1258. Google Scholar
- Jones, M., & Rehg, J. (1999). Statistical color models with application to skin detection. In IEEE conference on computer vision and pattern recognition (Vol. 1). Google Scholar
- Kolsch, M., & Turk, M. (2004). Robust hand detection. In IEEE conference on automatic face and gesture recognition (pp. 614–619). Google Scholar
- Murphy, K. (2003). Bayes net toolbox for Matlab. Google Scholar
- Niebur, E., & Koch, C. (1998). Computational architectures for attention. In R. Parasuraman (Ed.), The attentive brain (pp. 163–186). Cambridge: MIT Press. Google Scholar
- Pearl, J. (1988). Probabilistic reasoning in intelligent systems: networks of plausible inference. San Mateo: Morgan Kaufmann. Google Scholar
- Pramod Kumar, P., Vadakkepat, P., & Loh, A. P. (2010b). Graph matching based hand posture recognition using neuro-biologically inspired features. In International conference on control, automation, robotics and vision (ICARCV) 2010, Singapore. Google Scholar
- Pramod Kumar, P., Stephanie, Q. S. H., Vadakkepat, P., & Loh, A. P. (2010c). Hand posture recognition using neuro-biologically inspired features. In International conference on computational intelligence, robotics and autonomous systems (CIRAS) 2010, Bangalore. Google Scholar
- Serre, T., Wolf, L., & Poggio, T. (2005). Object recognition with features inspired by visual cortex. In C. Schmid, S. Soatto, & C. Tomasi (Eds.), Conference on computer vision and pattern recognition, San Diego, CA (pp. 994–1000). Google Scholar
- Su, M. C. (2000). A fuzzy rule-based approach to spatio-temporal hand gesture recognition. IEEE Transactions on Systems, Man and Cybernetics. Part C, Applications and Reviews, 30(2), 276–281. Google Scholar
- Triesch, J., & Malsburg, C. (1996b). Sebastien Marcel hand posture and gesture datasets: Jochen Triesch static hand posture database [online]: http://www.idiap.ch/resources/gestures/.
- Wang, W. H. A., & Tung, C. L. (2008). Dynamic hand gesture recognition using hierarchical dynamic Bayesian networks through low-level image processing. In 7th international conference on machine learning and cybernetics, Kunming, P.R. China (pp. 3247–3253). Google Scholar
- Wiesel, T. N., & Hubel, D. H. (1962). Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. Journal of Physiology, 160, 106–154. Google Scholar
- Wu, Y., & Huang, T. S. (1999). Vision-based gesture recognition: a review. In A. Braffort, R. Gherbi, S. Gibet, J. Richardson, & D. Teil (Eds.), International gesture workshop on gesture-based communication in human computer interaction, Gif Sur Yvette, France (pp. 103–115). Berlin: Springer CrossRefGoogle Scholar
- Wu, Y., & Huang, T. S. (2000). View-independent recognition of hand postures. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 88–94). Google Scholar
- Yang, M. H., & Ahuja, N. (1998). Extraction and classification of visual motion patterns for hand gesture recognition. In Proceedings, IEEE computer society conference on computer vision and pattern recognition, Santa Barbara, CA, USA (pp. 892–897). Google Scholar