Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse

  • Antonis A. Argyros
  • Manolis I. A. Lourakis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3979)


This paper presents a vision-based interface for controlling a computer mouse via 2D and 3D hand gestures. The proposed interface builds upon our previous work that permits the detection and tracking of multiple hands that can move freely in the field of view of a potentially moving camera system. Dependable hand tracking, combined with fingertip detection, facilitates the definition of simple and, therefore, robustly interpretable vocabularies of hand gestures that are subsequently used to enable a human operator convey control information to a computer system. Two such vocabularies are defined, implemented and validated. The first one depends only on 2D hand tracking results while the second also makes use of 3D information. As confirmed by several experiments, the proposed interface achieves accurate mouse positioning, smooth cursor movement and reliable recognition of gestures activating button events. Owing to these properties, our interface can be used as a virtual mouse for controlling any Windows application.


Hand Gesture Iterative Close Point Stereo Pair Mouse Button Iterative Close Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Argyros, A.A., Lourakis, M.I.A.: Real-Time Tracking of Multiple Skin-Colored Objects with a Possibly Moving Camera. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 368–379. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Argyros, A.A., Lourakis, M.I.A.: Binocular Hand Tracking and Reconstruction Based on 2D Shape Matching. In: ICPR 2006, under review (submitted, 2006)Google Scholar
  3. 3.
    Besl, P., McKay, N.: A Method for Registration of 3-d Shapes. IEEE Trans. on PAMI 14(2) (1992)Google Scholar
  4. 4.
    Chetverikov, D., Svirko, D., Stepanov, D., Krsek, P.: The Trimmed Iterative Closest Point Algorithm. In: Proceedings of ICPR 2002 (2002)Google Scholar
  5. 5.
    Corso, J.J., Burschka, D., Hager, G.D.: The 4D Touchpad: Unencumbered HCI With VICs. In: Proceedings of CVPR-HCI 2003, p. 55 (2003)Google Scholar
  6. 6.
    Gale, D., Shapley, L.S.: College Admissions and the Stability of Marriage. American Mathematical Monthly 69(9) (1962)Google Scholar
  7. 7.
    Goldman, R.: Intersection of two lines in three-space. In: Graphics Gems I, p. 304 (1990)Google Scholar
  8. 8.
    Malik, S., Laszlo, J.: Visual Touchpad: A Two-Handed Gestural Input Device. In: Proceedings of ICMI 2004, pp. 289–296 (2004)Google Scholar
  9. 9.
    Porta, M.: Vision-based User Interfaces: Methods and Applications. Int. J. Human-Computer Studies 57(1), 27–73 (2002)CrossRefGoogle Scholar
  10. 10.
    Robertson, P., Laddaga, R., van Kleek, M.: Virtual Mouse Vision Based Interface. In: Proceedings of IUI 2004, pp. 177–183 (2004)Google Scholar
  11. 11.
    Rousseeuw, P.J.: Least Median of Squares Regression. Journal of American Statistics Association 79, 871–880 (1984)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    van Dam, A.: Post-WIMP User Interfaces. Commun. ACM 40(2), 63–67 (1997)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Zhang, Z.: Determining the Epipolar Geometry and its Uncertainty: a Review. Int. Journal of Computer Vision 27, 161–195 (1998)CrossRefGoogle Scholar
  14. 14.
    Zhang, Z., Wu, Y., Shan, Y., Shafer, S.: Visual Panel: Virtual Mouse, Keyboard and 3D Controller With an Ordinary Piece of Paper. In: Proceedings of PUI 2001, pp. 1–8 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Antonis A. Argyros
    • 1
  • Manolis I. A. Lourakis
    • 1
  1. 1.Institute of Computer ScienceFoundation for Research and Technology – Hellas (FORTH), Vassilika VoutonHeraklion, CreteGreece

Personalised recommendations