Lip Contour Segmentation Using Kernel Methods and Level Sets

  • A. Khan
  • W. Christmas
  • J. Kittler
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4842)


This paper proposes a novel method for segmenting lips from face images or video sequences. A non-linear learning method in the form of an SVM classifier is trained to recognise lip colour over a variety of faces. The pixel-level information that the trained classifier outputs is integrated effectively by minimising an energy functional using level set methods, which yields the lip contour(s). The method works over a wide variety of face types, and can elegantly deal with both the case where the subjects’ mouths are open and the mouth contour is prominent, and with the closed mouth case where the mouth contour is not visible.


Face Image Active Contour Decision Function Kernel Method Visual Speech 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Mcgurck, H., Macdonald, J.W.: Hearing lips and seeing voices. Nature 264 (1976)Google Scholar
  2. 2.
    Hennecke, M., Prasad, K., Stork, D.: Using deformable templates to infer visual speech dynamics. In: 9th Asilomar Conference on Signals Systems and Computers, Pacific Grove, CA, pp. 578–582 (1994)Google Scholar
  3. 3.
    Kaucic, R., Blake, A.: Accurate, real-time, unadorned lip tracking. In: ICCV 1998: Proceedings of the Sixth International Conference on Computer Vision, p. 370. IEEE Computer Society, Washington, DC, USA (1998)Google Scholar
  4. 4.
    Luettin, J., Thacker, N.A., Beet, S.W.: Visual speech recognition using active shape models and hidden markov models. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 1996), vol. 2, pp. 817–820 (1996)Google Scholar
  5. 5.
    Lievin, M., Delmas, P., Coulon, P., Luthon, F., Fristot, V.: Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme. In: ICMCS, vol. 1, pp. 691–696 (1999)Google Scholar
  6. 6.
    Bregler, C., Omohundro, S.M.: Nonlinear manifold learning for visual speech recognition. In: ICCV 1995: Proceedings of the Fifth International Conference on Computer Vision, p. 494. IEEE Computer Society, Washington, DC, USA (1995)CrossRefGoogle Scholar
  7. 7.
    Tian, Y.-L., Kanade, T., Cohn, J.: Robust lip tracking by combining shape, color and motion. In: Proceedings of the 4th Asian Conference on Computer Vision (ACCV 2000) (2000)Google Scholar
  8. 8.
    Liévin, M., Luthon, F.: Nonlinear color space and spatiotemporal mrf for hierarchical segmentation of face features in video. IEEE Transactions on Image Processing 13, 63–71 (2004)CrossRefGoogle Scholar
  9. 9.
    Osher, S., Fedkiw, R.: Level set methods and dynamic implicit surfaces. Springer, Heidelberg (2003)zbMATHGoogle Scholar
  10. 10.
    Morgan, F.: Geometric Measure Theory: A Beginner’s Guide, 3rd edn. Academic Press, London (2000)zbMATHGoogle Scholar
  11. 11.
    Bakir, G.H., Bottou, L., Weston, J.: Breaking svm complexity with cross-training. In: NIPS (2004)Google Scholar
  12. 12.
    Burges, C.J.C.: Simplified support vector decision rules. In: International Conference on Machine Learning, pp. 71–77 (1996)Google Scholar
  13. 13.
    Chan, T., Zhu, W.: Level set based shape prior segmentation. In: CVPR 2005: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), vol. 2, pp. 1164–1170. IEEE Computer Society, Washington, DC, USA (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • A. Khan
    • 1
  • W. Christmas
    • 1
  • J. Kittler
    • 1
  1. 1.Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, GU2 7XHUK

Personalised recommendations