Abstract
A gesture recognition based interface was developed to facilitate people with upper extremity physical impairments as an alternative way to perform laboratory experiments that require ‘physical’ manipulation of components. A color, depth and spatial information based particle filter framework was constructed with unique descriptive features for face and hands representation. The same feature encoding policy was subsequently used to detect, track and recognize users’ hands. Motion models were created employing dynamic time warping (DTW) method for better observation encoding. Finally, the hand trajectories were classified into different classes (commands) by applying the CONDENSATION method and, in turn, an interface was designed for robot control, with a recognition accuracy of 97.5%. To assess the gesture recognition and control policies, a validation experiment consisting in controlling a mobile service robot and a robotic arm in a laboratory environment was conducted.
Keywords
- Gesture recognition
- particle filter
- dynamic time warping (DTW)
- CONDENSATION
Download conference paper PDF
References
Jacko, J.A.: Human-Computer Interaction Design and Development Approaches. In: 14th HCI International Conference, pp. 169–180 (2011)
Moon, I., Lee, M., Ryu, J., Mun, M.: Intelligent Robotic Wheelchair with EMG-, Gesture-, and Voice-based Interfaces. In: International Conference on Intelligent Robots and Systems, pp. 3453–3458. IEEE Press (2003)
Reale, M., Liu, P.: Yin. L.J.: Using eye gaze, head pose and facial expression for personalized non-player character interaction. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 13–18. IEEE Press (2011)
Soriano, M., Martinkauppi, B., Huovinen, S., Laaksonen, M.: Skin detection in video under changing illumination conditions. In: 15th International Conference on Pattern Recognition, vol. 1, pp. 839–842 (2000)
Bradski, G.R.: Computer vision face tracking as a component of a perceptual user interface. In: Workshop on Applications of Computer Vision, Princeton, NJ, pp. 214–219 (1998)
Isard, M., Black, A.: CONDENSATION: Conditional density propagation for visual tracking. International Journal of Computer Vision 29, 5–28 (1998)
Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-Based Probabilistic Tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 661–675. Springer, Heidelberg (2002)
Bilal, S., Akmeliawati, R., Shafie, A.A., Salami, M.J.E.: Hidden Markov Model for human to computer interaction: a study on human hand gesture recognition. In: Artificial Intelligence (2011)
Black, M.J., Jepson, A.D.: A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998, Part I. LNCS, vol. 1406, pp. 909–924. Springer, Heidelberg (1998)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: International Conference on Computer Vision and Pattern Recognition, pp. 511–518 (2001)
Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 46, pp. 81–96 (2002)
Hess, R., Fern, A.: Discriminatively Trained Particle Filters for Complex Multi-Object Tracking. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 240–247 (2009)
Aach, J., Church, G.M.: Alignment gene expression time series with time warping algorithms. J. Bioinformatics 17(6), 495–508 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jiang, H., Wachs, J.P., Duerstock, B.S. (2012). Facilitated Gesture Recognition Based Interfaces for People with Upper Extremity Physical Impairments. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2012. Lecture Notes in Computer Science, vol 7441. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33275-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-642-33275-3_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33274-6
Online ISBN: 978-3-642-33275-3
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://www.iapr.org/
