Real Time Hand Based Robot Control Using 2D/3D Images
In the interaction between man and machine, an efficient, natural and intuitive commanding system plays a key role. Vision based techniques are usually used to provide such a system. This paper presents a new solution using 2D/3D images for real time hand detection, tracking and classification which is used as an interface for sending the commands to an industrial robot. 2D/3D images, including low resolution range data and high resolution color information, are provided by a novel monocular hybrid vision system, called MultiCam, at video frame rates. After region extraction and applying some preprocessing techniques, the range data are segmented using an unsupervised clustering approach. The segmented range image is then mapped to the corresponding 2D color image. Due to the monocular setup of the vision system, mapping 3D range data to the 2D color information is trivial and does not need any complicated calibration and registration techniques. Consequently, the segmentation of 2D color image becomes simple and fast. Haar-like features are then extracted from the segmented color image and used as the input features for an AdaBoost classifier to find the region of the hand in the image and track it in each frame. The hand region found by AdaBoost is improved through postprocessing techniques and finally the hand posture (palm and fist) is classified based on a very fast heuristic method. The proposed approach has shown promising results in real time application, even under challenging variant lighting conditions which was demonstrated at the Hannover fair in 2008.
KeywordsHand Gesture Recognition Human Robot Interaction Video Frame Rate Hand Detection Hand Posture Recognition
Unable to display preview. Download preview PDF.
- 1.Wang, C., Wang, K.: Hand posture recognition using adaboost with sift for human robot interaction. In: International Conference on Advanced Robotics (2007)Google Scholar
- 2.Rogalla, O., Ehrenmann, M., Zoellner, R., Becher, R., Dillmann, R.: Using gesture and speech control for commanding a robot assistant. In: 11th IEEE International Workshop on Robot and Human Interactive Communication (2002)Google Scholar
- 3.Malima, A., Ozgur, E., Cetin, M.: A fast algorithm for vision-based hand gesture recognition for robot control. In: IEEE Conference on Signal Processing and Communications Applications (2006)Google Scholar
- 4.Cerlinca, T., Pentiuc, S., Cerlinca, M.: Hand posture recognition for human-robot interaction. In: Proceedings of the 2007 workshop on Multimodal interfaces in semantic interaction (2007)Google Scholar
- 5.Fang, Y., Wang, K., Cheng, J., Lu, H.: A real-time hand gesture recognition method. In: 2007 IEEE International Conference on Multimedia and Expo (2007)Google Scholar
- 6.Ghobadi, S., Hartmann, K., Weihs, W., Netramai, C., Loffeld, O., Roth, H.: Detection and classification of moving objects-stereo or time-of-flight images. In: Computational Intelligence and Security, pp. 11–16. IEEE, Los Alamitos (2006)Google Scholar
- 7.PMD: Photoics pmd 3k-s 3d video sensor array with active sbi (2007), www.pmdtec.com
- 8.Lottner, O., Hartmann, K., Weihs, W., Loffeld, O.: Image registration and calibration aspects for a new 2d / 3d camera. In: EOS Conference on Frontiers in Electronic Imaging (2007)Google Scholar
- 10.Ghobadi, S., Loepprich, O., Hartmann, K., Loffeld, O.: Hand segmentation using 2d/3d images. In: IVCNZ 2007 Conference, Hamilton, New Zealand (2007)Google Scholar
- 11.Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer vision and Pattern Recognition (2001)Google Scholar
- 12.OpenCV: (The open computer vision library, intel)Google Scholar