Hand Motion Detection for Observation-Based Assistance in a Cooperation by Multiple Robots

  • Toyomi FujitaEmail author
  • Tetsuya Endo
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 325)


In a cooperation task by multiple robots, it may happen that a working robot can not detect a target object for handling due to a sensor occlusion. In this situation, if another cooperative robot observes the working robot with the target object and detects their positions and orientations, it will be possible for the working robot to complete the handling task. Such behavior is a kind of indirect cooperation. This study proposes a method for such an indirect cooperation based on an observation by the partner robot. The observing robot obtains corresponding points of Scale-Invariant Feature Transformation (SIFT) on the working robot with hand and the target object from multiple captured images. The 3-D position of the target object and hand motion of the working robot can be detected by applying stereo vision theory to the points. The working robot is then able to get the relation between its hand and the target object indirectly from the observing robot. We describe each process to establish the indirect cooperation. Fundamental experiments confirmed the validity of presented method.


Robot vision Cooperation by observation Scale-Invariant Feature Transformation (SIFT) Stereo vision 


  1. 1.
    Kuniyoshi, Y., Rickki, J., Ishii, M., Rougeaux, S., Kita, N., Sakane, S., Kakikura, M.: Vision based behaviors for multi-robot cooperation. In: Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems ’94, vol. 2, pp. 925–932 (1994)Google Scholar
  2. 2.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceeding of IEEE International Conference on Computer Vision (ICCV), pp. 1150–1157 (1999)Google Scholar
  3. 3.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)Google Scholar
  4. 4.
    Luong, Q.-T., Faugeras, O.D.: Self-calibration of a moving camera from point correspondences and fundamental matrices. Int. J. Comput. Vis. 23(3), 261–289 (1997)CrossRefGoogle Scholar
  5. 5.
    Luong, Q.T., Faugeras O.D.: The fundamental matrix: theory, algorithms, and stability analysis. Int. J. Comput. Vis. 17(1), 43–75(33) (1996)Google Scholar
  6. 6.
    Weng, J., Ahuja, N., Huang, T.S.: Motion and structure from two perspective views: algorithms, error analysis, and error estimation. IEEE Trans. Pattern Anal. Machine Intell. 11(5), 451–476 (1989)CrossRefGoogle Scholar
  7. 7.
    Hartley, R.I.: In defense of the eight-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 19(6), 580–593 (1997)CrossRefGoogle Scholar
  8. 8.

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Electronics and Intelligent SystemsTohoku Institute of TechnologySendaiJapan

Personalised recommendations