Abstract
This paper is concerned with the video-based recognition of signs. Concentrating on the manual parameters of sign language, the system aims for the signer dependent recognition of 262 different signs taken from Sign Language of the Netherlands. For Hidden Markov Modelling a sign is considered a doubly stochastic process, represented by an unobservable state sequence. The observations emitted by the states are regarded as feature vectors, that are extracted from video frames. This work deals with three topics: Firstly the recognition of isolated signs, secondly the influence of variations of the feature vector on the recognition rate and thirdly an approach for the recognition of connected signs. The system achieves recognition rates up to 94% for isolated signs and 73% for a reduced vocabulary of connected signs.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
M. Assan. Videobasierte Gebärdenspracherkennung mit Hidden-Markov-Modellen. Diplomarbeit, Lehrstuhl für Technische Informatik, RWTH Aachen, 1997.
P. Boyes-Braem, Einführung in die Gebärdensprache und ihre Erforschung. Signum Verlag, Hamburg, 1995.
D. Brentari. Sign Language Phonology: ASL. In Handbook of Phonological Theory, Basil Blackwell, New York, 1994, pp. 615–637.
L.W. Campbell, D.A. Becker, A. Azarbayejani, A.F. Bobick and A. Pentland. Invariant Features for 3-D Gesture Recognition. In Proc. of the 2. Int. Conf. on Automatic Face & Gesture Recognition, Killington, 1996, pp. 617–621.
Y. Cui and J.J. Weng. View-based Hand Segmentation and Hand-Sequence Recognition with Complex Backgrounds, Proc. of the Int. Conf. on Pattern Recognition, Vienna, 1996, pp. 617–621.
K. Grobel and H. Hienz. Video-based Recognition of Fingerspelling in Realtime. In Proc. des Workshops Bildverarbeitung für die Medizin, Aachen, 1996, pp. 197–202.
K. Grobel und H. Hienz. Videobasierte Erkennung von Körperregionen zur Bestimmung der Ausführungsstelle einer Gebärde. In Proc. des 9. Aachener Kolloquiums Signaltheorie, März 18–20, 1997.
K. Grobel and M. Assan. Isolated Sign Language Recognition using Hidden Markov Models. In Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics, Orlando/Florida, Oct. 12–15, 1997.
H. Liang and M. Ouhyoung. A Sign Language Recognition System using Hidden Markov Model and Context Sensitive Search. In ACM Symposium on Virtual Reality Software and Technology, Hong Kong, July 1996.
R. Rabiner. A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. In Proc. of the IEEE, vol. 77, no. 2, Feb. 1989, pp. 257–285.
J. Schlenzig, E. Hunter and R. Jain. Video Based Hand Gesture Interpretation Using Recursive Estimation. In Proc. of Asilomar Conference on Signals, Systems and Computers, Pacific Grove, Nov. 1995, pp.1267–1271.
E.G. Schukakat-Talamanzini. Automatische Spracherkennung, Vieweg Verlag, Braunschweig, 1995, pp. 121–164.
T.E. Starner and A. Pentland. Real-Time American Sign Language Recognition from Video Using Hidden Markov Models. In Technical Report no. 375, MIT Cambridge, Media Laboratory, 1995.
J. Yamato, J. Ohya and K. Ishii. Recognizing Human Action in Time-Sequential Images Using Hidden Markov Model. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, 1992, pp. 379–385.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag
About this paper
Cite this paper
Assan, M., Grobel, K. (1998). Video-based sign language recognition using Hidden Markov Models. In: Wachsmuth, I., Fröhlich, M. (eds) Gesture and Sign Language in Human-Computer Interaction. GW 1997. Lecture Notes in Computer Science, vol 1371. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0052992
Download citation
DOI: https://doi.org/10.1007/BFb0052992
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64424-8
Online ISBN: 978-3-540-69782-4
eBook Packages: Springer Book Archive