Abstract
We present a cognitively motivated vision architecture for the evaluation of pointing gestures. The system views a scene of several structured objects and a pointing human hand. A neural classifier gives an estimation of the pointing direction, then the object correspondence is established using a sub-symbolic representation of both the scene and the pointing direction. The system achieves high robustness because the result (the indicated location) does not primarily depend on the accuracy of the pointing direction classification. Instead, the scene is analysed for low level saliency features to restrict the set of all possible pointing locations to a subset of highly likely locations. This transformation of the “continuous” to a “discrete” pointing problem simultaneously facilitates an auditory feedback whenever the object reference changes, which leads to a significantly improved human-machine interaction.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
G. Backer, B. Mertsching, and M. Bollmann. Data-and Model-Driven Gaze Control for an Active-Vision System. IEEE Trans. PAMI, 23(12):1415–1429, 2001.
M. Fislage, R. Rae, and H. Ritter. Using visual attention to recognize human pointing gestures in assembly tasks. In 7th IEEE Int’l Conf. Comp. Vision, 1999.
K. Fukushima. Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Pattern Recognition unaffected by Shift in Position. Biol. Cybern., 36:193–202, 1980.
C. Harris and M. Stephens. A Combined Corner and Edge Detector. In Proc. 4th Alvey Vision Conf., pages 147–151, 1988.
G. Heidemann. Ein flexibel einsetzbares Objekterkennungssystem auf der Basis neuronaler Netze. PhD thesis, Univ. Bielefeld, 1998. Infix, DISKI 190.
G. Heidemann, D. Lücke, and H. Ritter. A System for Various Visual Classification Tasks Based on Neural Networks. In A. Sanfeliu et al., editor, Proc. 15th Int’l Conf. on Pattern Recognition ICPR 2000, Barcelona, volume I, pages 9–12, 2000.
G. Heidemann, R. Rae, H. Bekel, I. Bax, and H. Ritter. Integrating Context-Free and Context-Dependent Attentional Mechanisms for Gestural Object Reference. In Proc. Int’l Conf. Cognitive Vision Systems, Graz, Austria, 2003.
G. Heidemann and H. Ritter. Efficient Vector Quantization Using the WTA-rule with Activity Equalization. Neural Processing Letters, 13(1):17–30, 2001.
L. Itti, C. Koch, and E. Niebur. A Model of Saliency-Based Visual Attention for Rapid Scene Analysis. IEEE Trans. PAMI, 20(11):1254–1259, 1998.
T. Kalinke and W. v. Seelen. Entropie als Maß des lokalen Informationsgehalts in Bildern zur Realisierung einer Aufmerksamkeitssteuerung. In B. Jähne et al., editor, Mustererkennung 1996. Springer, Heidelberg, 1996.
T. Kohonen. Self-organization and associative memory. In Springer Series in Information Sciences 8. Springer-Verlag Heidelberg, 1984.
P. J. Locher and C. F. Nodine. Symmetry Catches the Eye. In A. Levy-Schoen and J. K. O’Reagan, editors, Eye Movements: From Physiology to Cognition, pages 353–361. Elsevier Science Publishers B. V. (North Holland), 1987.
D. Reisfeld, H. Wolfson, and Y. Yeshurun. Context-Free Attentional Operators: The Generalized Symmetry Transform. Int’l J. Comp. Vision, 14, 1995.
T. D. Sanger. Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network. Neural Networks, 2:459–473, 1989.
C. Schmid, R. Mohr, and C. Bauckhage. Evaluation of Interest Point Detectors. Int’l J. of Computer Vision, 37(2):151–172, 2000.
M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analyzers. Neural Computation, 11(2):443–482, 1999.
D. Walther, L. Itti, M. Riesenhuber, T. Poggio, and C. Koch. Attentional Selection for Object Recognition — a Gentle Way. In Proc. 2nd Workshop on Biologically Motivated Computer Vision (BMCV’02), Tübingen, Germany, 2002.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bax, I., Bekel, H., Heidemann, G. (2003). Recognition of Gestural Object Reference with Auditory Feedback. In: Kaynak, O., Alpaydin, E., Oja, E., Xu, L. (eds) Artificial Neural Networks and Neural Information Processing — ICANN/ICONIP 2003. ICANN ICONIP 2003 2003. Lecture Notes in Computer Science, vol 2714. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44989-2_51
Download citation
DOI: https://doi.org/10.1007/3-540-44989-2_51
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40408-8
Online ISBN: 978-3-540-44989-8
eBook Packages: Springer Book Archive