Neural Processing Letters

, Volume 3, Issue 2, pp 61–71 | Cite as

Neural recognition of human pointing gestures in real images

  • Enno Litmann
  • Andrea Drees
  • Helge Ritter


We present a neural network based system for the visual recognition of human hand pointing gestures from stereo pairs of video camera images. The accuracy of the current system allows to estimate the pointing target to an accuracy of 2 cm in a workspace area of 50×50 cm. The system consists of several neural networks that perform the tasks of image segmentation, estimation of hand location, estimation of 3D-pointing direction and necessary coordinate transforms. Drawing heavily on the use of learning algorithms, the functions of all network modules were created from data examples only.

Key words

gesture recognition man-machine interface modular neural system pointing gestures 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S.J.Nowlan and J.C.Platt. “A convolutional neural network hand tracker”, in G.Tesauro, D.S.Touretzky and T.K.Leen (eds) Neural Information Processing Systems 7, MIT Press: Cambridge, MA, 1995.Google Scholar
  2. 2.
    T.J.Darell and A.P.Pentland, “Classifying hand gestures with a view-based distributed representation”, in J.D.Cowan, G.Tesauro, and J.Alspector (eds) Neural Information Processing Systems 6, pp. 945–953, Morgan Kaufman: San Mateo, CA, 1994.Google Scholar
  3. 3.
    J.Davis and M.Shah, “Recognizing hand gestures’, in J.-O.Eklundh (ed) Computer Vision-ECCV '94, Vol. 800 of Lecture Notes in Computer Science, pp. 331–340, Springer-Verlag: Berlin, Heidelberg, New York, 1994.Google Scholar
  4. 4.
    C. Maggioni, “A novel device for using the hand as a human-computer interface”, in Proc. HCI'93-Human Control Interface, Loughborough, UK, 1993.Google Scholar
  5. 5.
    A.Meyering and H.Ritter, “Learning 3D hand postures from perspective pixel images”, in I.Aleksander and J.Taylor (eds) Artificial Neural Networks 2, pp. 821–824, Elsevier Science Publishers: Amsterdam, 1992.Google Scholar
  6. 6.
    F.Kummert, E.Littmann, A.Meyering, S.Posch, H.Ritter and G.Sagerer, “Recognition of 3d-hand orientation from monocular color images by neural semantic networks”, Pattern Recognition and Image Analysis, Vol. 3, No. 3, pp. 311–316, 1993.Google Scholar
  7. 7.
    E.Littmann and H.Ritter, “Neural and statistical methods for adaptive color segmentation-a comparison”, in G.Sagerer, S.Posch, and F.Kummert (eds) Mustererkennung 1995, pp. 84–93, Springer-Verlag: Heidelberg, 1995.Google Scholar
  8. 8.
    A.Drees, F.Kummert, E.Littmann, S.Posch, H.Ritter and G.Sagerer, “A hybrid system to detect hand orientation in stereo images”, in E.S.Gelsema and L.N.Kanal (eds) Pattern Recognition in Practice IV: Multiple Paradigms, Comparative Studies and Hybrid Systems, pp. 551–562, Elsevier Science Publishers: Amsterdam, 1994.Google Scholar
  9. 9.
    H.Ritter, “Learning with the self-organizing map”, in T.Kohonen, K.Mäkisara, O.Simula and J.Kangas (eds) Artificial neural Networks 1, pp. 357–364. Elsevier Science Publishers: Amsterdam, 1991.Google Scholar
  10. 10.
    H.Ritter, T.Martinetz, and K.Schulten, Neural Computation and Self-Organizing Maps: An Introduction, Addison-Wesley: New York, 1992 (English and German).Google Scholar
  11. 11.
    T.Kohonen, Self Organization and Associative Memory, Springer Series in Information Sciences 8, Springer-Verlag: Berlin, Heidelberg, New York, 1982 (3rd edition 1989).Google Scholar
  12. 12.
    T.Kohonen, “The self-organizing map”, Proc. IEEE., Vol. 78, pp. 1464–1480, 1990.Google Scholar
  13. 13.
    T.Poggio and F.Girosi, “Regularization algorithms for learning that are equivalent to multilayer networks”, Science, Vol. 247, pp. 978–982, 1990.Google Scholar
  14. 14.
    J.D.Farmer and J.J.Sidorowich, “Predicting chaotic time series”, Physical Review Letters, Vol. 59, pp. 845–848, 1987.Google Scholar
  15. 15.
    K.Stokbro, D.K.Umberger and J.A.Hertz, “Exploiting neurons with localized receptive fields to learn chaos”, Complex Systems, Vol. 4, pp. 603–622, 1990.Google Scholar
  16. 16.
    C.G.Atkeson, “Memory-based approaches to approximating continuous functions”, in M.Casdagli and S.Eubank (eds) Nonlinear Modeling and Forecasting Vol. XII of SFI Studies in the Sciences of Complexity, pp. 503–521, Addison-Wesley: New York, 1992.Google Scholar
  17. 17.
    H.Ritter, “Parametrized self-organizing maps for vision learning tasks”, in P.Morasso (ed) ICANN '94, Springer-Verlag: Berlin, Heidelberg, New York, 1994.Google Scholar
  18. 18.
    J. Walter and H. Ritter, “Rapid learning with parametrized self-organizing maps”, Neural Computing, 1996 (in press).Google Scholar
  19. 19.
    E.Littmann, Strukturierung Neuronaler Netze zwischen Biologie und Anwendung, DISKI, Infix Verlag: St. Augustin, Germany, 1995.Google Scholar
  20. 20.
    A. Meyering and H. Ritter, “Learning 3D shape perception with local linear maps” in Proceedings of the International Joint Conference on Neural Networks, Vol. IV, pp. 432–436, Baltimore MD, June 7–11, 1992.Google Scholar
  21. 21.
    F.Kummert, E.Littmann, A.Meyering, S.Posch, H.Ritter and G.Sagerer, “A hybrid approach to signal interpretation using neural and semantic networks”, in S.J.Pöppl and H.Handels (eds) Mustererkennung 1993, pp. 245–252, Springer-Verlag: Berlin, Heidelberg, New York, 1993.Google Scholar

Copyright information

© Kluwer Academic Publishers 1996

Authors and Affiliations

  • Enno Litmann
    • 1
  • Andrea Drees
    • 2
  • Helge Ritter
    • 2
  1. 1.Ableilung NeuroinformatikUniversität UlmUlmGermany
  2. 2.AG Neuroinformatik, Technische FakultätUniversität BielefeldBielefeldGermany

Personalised recommendations