Lipreading Using the Fuzzy Degree of Similarity
Lipreading through visual processing techniques help provide some useful systems for the hearing impaired to learn communication assistance. This paper proposes a method to understand spoken words by using visual images taken by a camera with a video-digitizer. The image is processed to obtain contours of lips, which are converted into approximated hexagons. The pattern lists, consisting of lengths and angles of hexagons, are compared to get the fuzzy similarity between two hexagons. By similarity matching, the mouth shape is recognized as the one which has the pronounced voice. Some experiments, exemplified by recognition of the Japanese vowels, are given to show feasibility of this method.
KeywordsFuzzy Number Speech Recognition Representative Point Standard Pattern Similarity Match
Unable to display preview. Download preview PDF.
- 1.Fukuda, Y., Hiki, S.: Characteristics of the mouth shape in the production of Japanese stroboscopic observation, J.Acoust.Soc.Japan (E) 3-2 (1982), 75–91.Google Scholar
- 2.Mase, K., Pentland, A.: Automatic lipreading by optical flow analysis, Trans. Inst. Electron. Inf. Commun. Eng. D-II-6 (1990), 796–803.Google Scholar
- 3.Okada, K., Ohira,C., Nakamura, H: A method for lipreading, Trans. Inst. Electron. Inf. Commun. Eng. D-11-9 (1989), 1532–1583.Google Scholar
- 4.Gulliiams, I., Segui, A.: Interactive videodisc for teaching and evaluating lipreading, Proceedings of An. Intern. Conf. of IEEE Eng. in Medicine and Biology Soc., Cat.88Ch2566-8 (1989), 1551–1552.Google Scholar
- 5.Pentajan, E.D.: Automatic lipreading to enhance speech recognition, Proceedings of CVPR’85 Cat.85Ch2145-I (1985), 40–47.Google Scholar
- 6.Watanabe, T.: Machine lip reading of two concatenated vowels, Trans. JPN. Soc. Mech. Eng. (C) 55-509 (1989), 134–138.Google Scholar
- 7.Matsuoka, K., Furuya,T., Kurosu, K.: Speech Recognition by image processing of lip movements, Trans, of SICE 22-2 (1986), 67–74.Google Scholar
- 8.Kurosu, K., Furuya, T., Takeuchi, S.: Lip reading by fuzzy logic, Proc. of 2nd Congress of IFSA (1987), 23-4C.Google Scholar
- 9.Furuya, T., Soeda, M., Kurosu, K., Tamura, S.: Speech recognition with lip movement data using an X-Y tracker, Trans, of SICE 27-8 (1991), 958–965.Google Scholar
- 10.Kurosu, K., Furuya, T., etal: Lipreading by a neural net, Proc. of 8th European Annual Conf on Human Decision Making and Manual Control (1989), 329–339.Google Scholar
- 11.Weijing, Z., Furuya, T., Kurosu, K.: A list processing method on Composition and decomposition of figure patterns, Tran. of SICE 26-1 (1990), 81–86.Google Scholar