Skip to main content
SpringerLink
Log in

Lip segmentation using localized active contour model with automatic initial contour

  • Neural Computing in Next Generation Virtual Reality Technology
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Lip-reading is one of important approaches for human–computer interaction (HCI). Its development would have a large range of applications, especially in augmented reality. Lip segmentation is the first and foremost step in the lip-reading system. Conventional method of region-based active contour model adopts the global information of image and is unable to perform well. In this paper, from a localized perspective, we introduce the methodology of localized active contour model (LACM) and, meanwhile, propose the method that using LACM to perform the lip segmentation with the initial contour automatically generated. The scope for active contour model is reduced to the local region that reduces the disturbances of unrelated factors. The experimental results demonstrate the method adopts this model would dramatically improve the robustness for lip segmentation. On this basis, we analyze the influence of initial contours and local radiuses, study the efficiencies under different initial contours and compare it with the conventional active contour model which adopts the global information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Dupont S, Luettin J (2000) Audio-visual speech modeling for continuous speech recognition. IEEE Trans Multimed 2(3):141–151

    Article  Google Scholar 

  2. Rong C, Yue Z, Jia Y, Wang Y, Yang Y (2012) Research advances in key technology of lip-reading. J Data Acquis Process 27(s2):277–283

    Google Scholar 

  3. Lewis T, Powers D (2002) Lip feature extraction using red exclusion. In: Pan-sydney workshop visualization, pp 61–67

  4. Wang S, Lau W, Leung S, Liew A (2004) Lip segmentation with the presence of beards. Int Conf Acoust Speech Signal Process ICASSP 3:529–532

    Google Scholar 

  5. Leung S, Wang S, Lau W (2004) Lip image segmentation using fuzzy clustering incorporating an elliptic shape function. IEEE Trans Image Process 3(1):51–62

    Article  Google Scholar 

  6. Matthews I, Cootes T, Bangham J, Cox S, Harvey R (2002) Extraction of visual features for lipreading. IEEE Trans Pattern Anal Mach Intell 24(2):198–213

    Article  Google Scholar 

  7. Pardàs M, Sayrol E (2001) Motion estimation based tracking of active contours. Pattern Recognit Lett 22(13):1447–1456

    Article  MATH  Google Scholar 

  8. Eveno N, Caplier A, Coulon P-Y (2004) Accurate and quasi-automatic lip tracking. IEEE Trans Circuits Syst Video Technol 14(5):706–715

    Article  Google Scholar 

  9. Liévin M, Delmas P, Coulon PY, Luthon F, Fristol V (1999) Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme. In: Proceedings of IEEE international conference multimedia computer systems. Barcelona, Spain, pp 691–696

  10. Liévin M, Luthon F (1999) Unsupervised lip segmentation under natural conditions. In: Proceedings of IEEE international conference acoustics speech signal processing, Phoenix, AZ, USA, pp 3065–3068

  11. Shawn L, Allen T (2008) Localizing region-based active contours. IEEE Trans Image Process 17(11):2029–2039

    Article  MathSciNet  MATH  Google Scholar 

  12. Liévin M, Luthon F (2004) Nonlinear color space and spatiotemporal MRF for hierarchical segmentation of face features in video. IEEE Trans Image Process 13(1):63–71

    Article  Google Scholar 

  13. Patterson EK, Gurbuz S, Tufekci Z, Gowdy J (2002) CUAVE: a new audio–visual database for multimodal human–computer interface research. In: IEEE international conference on acoustics, speech and signal processing

  14. http://fei.edu.br/~cet/facedatabase.html

  15. Srikham M (2012) Active contours segmentation with edge based and local region based. In: 21st international conference on pattern recognition, pp 1989–1992

  16. Mehrotra H, Agrawal G, Srivastava MC (2009) Automatic lip contour tracking and visual character recognition for computerized lip reading. Int J Comput Sci 1:808–817

    Google Scholar 

  17. Leung SH, Wang SL, Lau WH (2004) Lip image segmentation using fuzzy clustering incorporating an elliptic shape function. IEEE Trans Image Process Publ IEEE Signal Process Soc 13(1):51–62

    Article  Google Scholar 

  18. Eveno N, Caplier A, Coulon PY (2001) New color transformation for lips segmentation. In: IEEE fourth workshop on multimedia signal processing, pp 3–8

Download references

Acknowledgements

The research was supported by the National Natural Science Foundation of China (61571013) and by the Beijing Natural Science Foundation of China (4143061). The authors thank all the partners and the participants in the experiment for their help.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, North China University of Technology, No. 5, Jinyuanzhuang Road, Shijingshan District, Beijing, 100144, China

    Yuanyao Lu & Tenghe Zhou

Authors
  1. Yuanyao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tenghe Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanyao Lu.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, Y., Zhou, T. Lip segmentation using localized active contour model with automatic initial contour. Neural Comput & Applic 29, 1417–1424 (2018). https://doi.org/10.1007/s00521-017-3046-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3046-0

Keywords

Search

Navigation