Advertisement

Real Time Hand Based Robot Control Using 2D/3D Images

  • Seyed Eghbal Ghobadi
  • Omar Edmond Loepprich
  • Farid Ahmadov
  • Jens Bernshausen
  • Klaus Hartmann
  • Otmar Loffeld
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5359)

Abstract

In the interaction between man and machine, an efficient, natural and intuitive commanding system plays a key role. Vision based techniques are usually used to provide such a system. This paper presents a new solution using 2D/3D images for real time hand detection, tracking and classification which is used as an interface for sending the commands to an industrial robot. 2D/3D images, including low resolution range data and high resolution color information, are provided by a novel monocular hybrid vision system, called MultiCam, at video frame rates. After region extraction and applying some preprocessing techniques, the range data are segmented using an unsupervised clustering approach. The segmented range image is then mapped to the corresponding 2D color image. Due to the monocular setup of the vision system, mapping 3D range data to the 2D color information is trivial and does not need any complicated calibration and registration techniques. Consequently, the segmentation of 2D color image becomes simple and fast. Haar-like features are then extracted from the segmented color image and used as the input features for an AdaBoost classifier to find the region of the hand in the image and track it in each frame. The hand region found by AdaBoost is improved through postprocessing techniques and finally the hand posture (palm and fist) is classified based on a very fast heuristic method. The proposed approach has shown promising results in real time application, even under challenging variant lighting conditions which was demonstrated at the Hannover fair in 2008.

Keywords

Hand Gesture Recognition Human Robot Interaction Video Frame Rate Hand Detection Hand Posture Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Wang, C., Wang, K.: Hand posture recognition using adaboost with sift for human robot interaction. In: International Conference on Advanced Robotics (2007)Google Scholar
  2. 2.
    Rogalla, O., Ehrenmann, M., Zoellner, R., Becher, R., Dillmann, R.: Using gesture and speech control for commanding a robot assistant. In: 11th IEEE International Workshop on Robot and Human Interactive Communication (2002)Google Scholar
  3. 3.
    Malima, A., Ozgur, E., Cetin, M.: A fast algorithm for vision-based hand gesture recognition for robot control. In: IEEE Conference on Signal Processing and Communications Applications (2006)Google Scholar
  4. 4.
    Cerlinca, T., Pentiuc, S., Cerlinca, M.: Hand posture recognition for human-robot interaction. In: Proceedings of the 2007 workshop on Multimodal interfaces in semantic interaction (2007)Google Scholar
  5. 5.
    Fang, Y., Wang, K., Cheng, J., Lu, H.: A real-time hand gesture recognition method. In: 2007 IEEE International Conference on Multimedia and Expo (2007)Google Scholar
  6. 6.
    Ghobadi, S., Hartmann, K., Weihs, W., Netramai, C., Loffeld, O., Roth, H.: Detection and classification of moving objects-stereo or time-of-flight images. In: Computational Intelligence and Security, pp. 11–16. IEEE, Los Alamitos (2006)Google Scholar
  7. 7.
    PMD: Photoics pmd 3k-s 3d video sensor array with active sbi (2007), www.pmdtec.com
  8. 8.
    Lottner, O., Hartmann, K., Weihs, W., Loffeld, O.: Image registration and calibration aspects for a new 2d / 3d camera. In: EOS Conference on Frontiers in Electronic Imaging (2007)Google Scholar
  9. 9.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  10. 10.
    Ghobadi, S., Loepprich, O., Hartmann, K., Loffeld, O.: Hand segmentation using 2d/3d images. In: IVCNZ 2007 Conference, Hamilton, New Zealand (2007)Google Scholar
  11. 11.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Conference on Computer vision and Pattern Recognition (2001)Google Scholar
  12. 12.
    OpenCV: (The open computer vision library, intel)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Seyed Eghbal Ghobadi
    • 1
  • Omar Edmond Loepprich
    • 1
  • Farid Ahmadov
    • 1
  • Jens Bernshausen
    • 1
  • Klaus Hartmann
    • 1
  • Otmar Loffeld
    • 1
  1. 1.Center for Sensor Systems (ZESS)University of SiegenSiegenGermany

Personalised recommendations