MMSDS: Ubiquitous Computing and WWW-Based Multi-modal Sentential Dialog System
In this study, we suggest and implement Multi-Modal Sentential Dialog System (MMSDS) integrating 2 sensory channels with speech and haptic information based on ubiquitous computing and WWW for clear communication. The importance and necessity of MMSDS for HCI as following: 1) it can allow more interactive and natural communication functions between the hearing-impaired and hearing person without special learning and education, 2) according as it recognizes a sentential Korean Standard Sign Language (KSSL) which is represented with speech and haptics and then translates recognition results into a synthetic speech and visual illustration in real-time, it may provide a wider range of personalized and differentiated information more effectively to them, and 3) above all things, a user need not be constrained by the limitations of a particular interaction mode at any given moment because it can guarantee mobility of WPS (Wearable Personal Station for the post PC) with a built-in sentential sign language recognizer. In experiment results, while an average recognition rate of uni-modal recognizer using KSSL only is 93.1% and speech only is 95.5%, advanced MMSDS deduced an average recognition rate of 96.1% for 32 sentential KSSL recognition models.
KeywordsUbiquitous Computing Hand Gesture Fuzzy Relation Average Recognition Rate Data Glove
Unable to display preview. Download preview PDF.
- 3.Jiangqin, W., et al.: A Simple Sign Language Recognition System Based on Data Glove. In: ICSP 1998, IEEE International Conference Proceedings, pp. 1257–1260 (1998)Google Scholar
- 6.Barnard, M., et al.: Multi-Modal Audio-Visual Event Recognition for Football Analysis. In: IEEE XI11 Workshop on Neural Networks for Signal Processing, IEEE Workshop Proceedings, pp. 469–478 (2003)Google Scholar
- 7.Use of Signs in Hearing Communities: http://en.wikipedia.org/wiki/Sign_language
- 8.Kim, S.-G.: Korean Standard Sign Language Tutor, 1st edn. Osung Publishing Company, Seoul (2000)Google Scholar
- 10.5DT Data Glove 5 Manual and FASTRAK® Data Sheet: http://www.5dt.com
- 12.Paulus, D., Hornegger, J.: Applied Pattern Recognition, 2nd edn. Vieweg (1998)Google Scholar
- 13.Schuermann, J.: Pattern Classification: A Unified View of Statistical and Neural Approaches. Wiley & Sons, Chichester (1996)Google Scholar
- 14.Relational DataBase Management System: http://www.auditmypc.com/acronym/RDBMS.asp
- 15.Oracle 10g DW Guide: http://www.oracle.com
- 16.Vasantha kandasamy, W.B.: Smaranda Fuzzy Algebra. American Research Press, Seattle (2003)Google Scholar
- 17.McGlashan, S., et al.: Voice Extensible Markup Language (VoiceXML) Version 2.0. W3C Recommendation (1992), http://www.w3.org