Abstract
Lip image analysis is a critical problem in many vision-based applications such as lip reading, speaker authentication, video conferencing and language learning. We design three dynamic probability maps based on the color, shape, and edge features, respectively. These probability maps are then utilized to extract both inner and outer lip contours using a grid-based gradient-ascent approach. The experiments show that the proposed algorithm can extract inner/outer lip contours efficiently and reliably. The proposed lip contour extraction algorithm is applied successfully in a language-learning application called VEC3D.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aleksic, P., Williams, J., Wu, Z., Katsaggelos, A.: Audio-visual speech recognition using MPEG-4 compliant visual features. EURASIP J. Appl. Signal Process. 1213–1227 (2002)
Balci, K.: Xface: MPEG-4 based open source toolkit for 3d facial animation. In: Working Conference on Advanced Visual Interfaces, Italy, May 2004
Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)
Dansereau, R., Li, C., Goubran, R.: Lip feature extraction using motion, color, and edge information. In: IEEE Internatioal Workshop on Haptic, Audio and Visual Environments and Their Applications, pp. 1–6, Sept. 2003
Delmas, P., Coulon, P., Fristot, V.: Automatic snakes for robust lip boundaries extraction. ICASSP, pp. 3069–3072 (1999)
Eveno, N., Caplier, A., Coulon, P.: A new color transformation for lips segmentation. Multimed. Signal Process. France, Oct. 2001
Eveno, N., Caplier, A., Coulon, P.: Key point based segmentation of lips. IEEE Int. Conf. Multimed. Expo 2, 125–128 (2002)
Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1988)
Lavagetto, F., Pockaj, R.: The facial animation engine: toward a high-level interface for the design of MPEG-4 compliant animated faces. IEEE Trans. Circuits Syst. Video Technol. (1999)
Leung, S., Wang, S., Lau, W.: Lip Image segmentation using fuzzy clustering incorporating an elliptic shape function. IEEE Trans. Image Process. 13(1), 51–62 (2004)
Luettin, J., Tracker, N., Beet, S.: Active shape models for visual speech feature extraction. Electronic System Group Rep. Univ. of Sheffield, UK, 95/44 (1995)
Patterson, E., Gurbuz, S., Tufekci, Z., Gowdy, J.: Moving-talker, speaker-independent feature study and baseline results using the CUAVE multimodal speech corpus. EURASIP J. Appl. Signal Process. (2002)
Shih, Y., Yang, M.T.: A collaborative virtual environment for situated language learning using VEC3D. J. Educ. Technol. Soc. 11(1), 56–68 (2008)
Terzopoulos, D., Waters, K.: Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Trans. Pattern Anal. Mach. Intell. 25, 569–579 (1993)
Wakasugi, T., Nishiura, M., Fukui, K.: Robust lip contour extraction using separability of multi-dimensional distributions. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 415–420, May 2004
Wang, G., Yang, M.T., Chiang, C., Tai, W.: A talking face driven by voice using hidden Markov model. J. Inf. Sci. Eng. 22, 1059–1075 (2006)
Zhang, X., Mersereau, R., Clements, M., Broun, C.: Visual speech feature extraction for improved speech recognition. ICASSP 1993–1996 (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yang, MT., You, ZW. & Shih, YC. Lip contour extraction for language learning in VEC3D. Machine Vision and Applications 21, 33 (2009). https://doi.org/10.1007/s00138-008-0139-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00138-008-0139-x