Visual gesture recognition by a modular neural system
The visual recognition of human hand pointing gestures from stereo pairs of video camera images provides a very intuitive kind of man-machine interface. We show that a modular, neural network based system can solve this task in a realistic laboratory environment. Several neural networks account for image segmentation, estimation of hand location, estimation of 3D-pointing direction, and necessary transforms from image to world coordinates and vice versa. The functions of all network modules can be learned from data examples only, by exploiting various learning algorithms. We investigate the performance of such a system and dicuss the problem of operator-independent recognition.
Unable to display preview. Download preview PDF.
- 1.T. J. Darell and A. P. Pentland. Classifying hand gestures with a view-based distributed representation. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Neural Information Processing Systems 6, pages 945–952. Morgan Kaufman, 1994.Google Scholar
- 2.J. Davis and M. Shah. Recognizing hand gestures. In J.-O. Eklundh, editor, Computer Vision — ECCV '94, volume 800 of Lecture Notes in Computer Science, pages 331–340. Springer-Verlag, Berlin Heidelberg New York, 1994.Google Scholar
- 3.F. Kummert, E. Littmann, A. Meyering, S. Posch, H. Ritter, and G. Sagerer. Recognition of 3d-hand orientation from monocular color images by neural semantic networks. Pattern Recognition and Image Analysis, 3(3):311–316, 1993.Google Scholar
- 4.E. Littmann, A. Drees, and H. Ritter. Visual gesture-based robot guidance with a modular neural system. In D. Touretzky, M. Mozer, and M. Hasselmo, editors, NIPS 8. Morgan Kaufman Publishers, San Mateo, CA, 1996. To appear.Google Scholar
- 5.E. Littmann and H. Ritter. Neural and statistical methods for adaptive color segmentation — a comparison. In G. Sagerer, S. Posch, and F. Kummert, editors, Mustererkennung 1995, pages 84–93. Springer-Verlag, Heidelberg, 1995.Google Scholar
- 6.C. Maggioni. A novel device for using the hand as a human-computer interface. In Proc. HCI'93 — Human Control Interface, Loughborough, Great Britain, 1993.Google Scholar
- 7.A. Meyering and H. Ritter. Learning 3D hand postures from perspective pixel images. In I. Aleksander and J. Taylor, editors, Artificial Neural Networks 2, pages 821–824. Elsevier Science Publishers B.V., North Holland, 1992.Google Scholar
- 8.A. Meyering and H. Ritter. Learning 3D shape perception with local linear maps. In Proc. of the IJCNN, volume IV, pages 432–436, Baltimore, MD, 1992.Google Scholar
- 9.Steven J. Nowlan and John C. Platt. A convolutional neural network hand tracker. In Neural Information Processing Systems 7. Morgan Kaufman Publishers, 1995.Google Scholar
- 10.K. Väänänen and K. Böhm. Gesture driven interaction as a human factor in virtual environments — an approach with neural networks. In R. Earnshaw, M. Gigante, and H. Jones, editors, Virtual reality systems, pages 93–106. Academic Press, 1993.Google Scholar
- 11.T. G. Zimmermann, J. Lanier, C. Blanchard, S. Bryson, and Y. Harvill. A hand gesture interface device. In Proc. CHI+GI, pages 189–192, 1987.Google Scholar