Lipreading Using the Fuzzy Degree of Similarity

  • K. Kurosu
  • T. Furuya
  • M. Soeda
  • S. Takeuchi
Part of the Theory and Decision Library book series (TDLD, volume 16)


Lipreading through visual processing techniques help provide some useful systems for the hearing impaired to learn communication assistance. This paper proposes a method to understand spoken words by using visual images taken by a camera with a video-digitizer. The image is processed to obtain contours of lips, which are converted into approximated hexagons. The pattern lists, consisting of lengths and angles of hexagons, are compared to get the fuzzy similarity between two hexagons. By similarity matching, the mouth shape is recognized as the one which has the pronounced voice. Some experiments, exemplified by recognition of the Japanese vowels, are given to show feasibility of this method.


Fuzzy Number Speech Recognition Representative Point Standard Pattern Similarity Match 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fukuda, Y., Hiki, S.: Characteristics of the mouth shape in the production of Japanese stroboscopic observation, J.Acoust.Soc.Japan (E) 3-2 (1982), 75–91.Google Scholar
  2. 2.
    Mase, K., Pentland, A.: Automatic lipreading by optical flow analysis, Trans. Inst. Electron. Inf. Commun. Eng. D-II-6 (1990), 796–803.Google Scholar
  3. 3.
    Okada, K., Ohira,C., Nakamura, H: A method for lipreading, Trans. Inst. Electron. Inf. Commun. Eng. D-11-9 (1989), 1532–1583.Google Scholar
  4. 4.
    Gulliiams, I., Segui, A.: Interactive videodisc for teaching and evaluating lipreading, Proceedings of An. Intern. Conf. of IEEE Eng. in Medicine and Biology Soc., Cat.88Ch2566-8 (1989), 1551–1552.Google Scholar
  5. 5.
    Pentajan, E.D.: Automatic lipreading to enhance speech recognition, Proceedings of CVPR’85 Cat.85Ch2145-I (1985), 40–47.Google Scholar
  6. 6.
    Watanabe, T.: Machine lip reading of two concatenated vowels, Trans. JPN. Soc. Mech. Eng. (C) 55-509 (1989), 134–138.Google Scholar
  7. 7.
    Matsuoka, K., Furuya,T., Kurosu, K.: Speech Recognition by image processing of lip movements, Trans, of SICE 22-2 (1986), 67–74.Google Scholar
  8. 8.
    Kurosu, K., Furuya, T., Takeuchi, S.: Lip reading by fuzzy logic, Proc. of 2nd Congress of IFSA (1987), 23-4C.Google Scholar
  9. 9.
    Furuya, T., Soeda, M., Kurosu, K., Tamura, S.: Speech recognition with lip movement data using an X-Y tracker, Trans, of SICE 27-8 (1991), 958–965.Google Scholar
  10. 10.
    Kurosu, K., Furuya, T., etal: Lipreading by a neural net, Proc. of 8th European Annual Conf on Human Decision Making and Manual Control (1989), 329–339.Google Scholar
  11. 11.
    Weijing, Z., Furuya, T., Kurosu, K.: A list processing method on Composition and decomposition of figure patterns, Tran. of SICE 26-1 (1990), 81–86.Google Scholar

Copyright information

© Kluwer Academic Publishers 1995

Authors and Affiliations

  • K. Kurosu
    • 1
  • T. Furuya
    • 2
  • M. Soeda
    • 2
  • S. Takeuchi
    • 3
  1. 1.Kinki UniversityIizuka-shi, Fukuoka-ken, 820Japan
  2. 2.Kitakyushu College of TechnologyKokura-minami-ku, Kitakyushu, 803Japan
  3. 3.NTTKumamoto-shi, 860Japan

Personalised recommendations