Active Shape Model Based Segmentation and Tracking of Facial Regions in Color Images

  • Bogdan Kwolek
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4141)


An approach for segmenting and tracking a face in a sequence of color images is presented. It enables reliable segmentation of facial region despite variation of skin-color perceived by a camera. A second order Markov model is utilized to forecast the skin distribution of facial regions in the next frame. The histograms that are constructed from the predicted distribution are backprojected to generate candidates of facial regions. Afterwards, a connected component labeling takes place. Spatial morphological operations, such as size and hole filtering are employed next. The Active Shape Model seeks to match a set of model points to the image. This statistical model of shape supports the segmentation of facial region undergoing tracking. Histograms are accommodated over time using feedback from shape, newly classified skin pixels and predictions of the skin-color evolution. This evolution is described by translation, rotation and scaling. In this context, the novelty of our approach lies in the introduction of Active Shape Model dealing with translation, rotation and scaling of the target to support face verification as well as to guide the evolution of skin distribution. The kernel histograms characterize the face during tracking in subsequent frames. The proposed algorithm achieves reliable detection and tracking results. The resulting system runs in real-time on standard PC computer.


Skin Color Gaussian Mixture Model Face Detection Facial Region Statistical Shape Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Birchfield, S.: Elliptical Head Tracking Using Intensity Gradients and Color Histograms. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Rec., pp. 232–237 (1998)Google Scholar
  2. 2.
    Blake, A., Isard, M., Reynard, D.: Learning to Track the Visual Motion of Contours. Artificial Intelligence 78, 101–133 (1995)CrossRefGoogle Scholar
  3. 3.
    Blake, A., Isard, M.: Active Contours. Springer, Heidelberg (1998)Google Scholar
  4. 4.
    Bradski, G.R.: Computer Vision Face Tracking as a Component of a Perceptual User Interface. In: Proc. IEEE Workshop on Appl. of Comp. Vision, pp. 214–219 (1998)Google Scholar
  5. 5.
    Chen, Y., Rui, Y., Huang, T.: Mode-based Multi-Hypothesis Head Tracking Using Parametric Contours. In: Proc. IEEE Int. Conf. on Aut. Face and Gesture Rec., pp. 112–117 (2002)Google Scholar
  6. 6.
    Cho, K.M., Jang, J.H., Hong, K.S.: Adaptive Skin Color Filter. Pattern Recognition 34(5), 1067–1073 (2001)CrossRefMATHGoogle Scholar
  7. 7.
    Comaniciu, D., Ramesh, V., Meer, P.: Real-Time Tracking of Non-Rigid Objects Using Mean Shift. In: Proc. IEEE Conf. on Comp. Vis. Patt. Rec., pp. 142–149 (2000)Google Scholar
  8. 8.
    Cootes, T.: An Introduction to Active Shape Models, Model-Based Methods in Analysis of Biomedical Images. In: Baldock, R., Graham, J. (eds.) Image Processing and Analysis. Oxford University Press, Oxford (2000)Google Scholar
  9. 9.
    Elgammal, A., Duraiswami, R., Davis, L.S.: Probabilistic Tracing in Joint Feature-Spatial Spaces. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Rec., pp. 16–22 (2003)Google Scholar
  10. 10.
    Fieguth, P., Terzopoulos, D.: Color-Based Tracking of Heads and Other Mobile Objects at Video Frame Rates. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Rec., Hilton Head Island, pp. 21–27 (1997)Google Scholar
  11. 11.
    Han, B., Davis, L.: Robust Observations for Object Tracking. In: Proc. Int. Conf. on Image Processing, pp. 442–445 (2005)Google Scholar
  12. 12.
    Isard, M., Blake, A.: Contour Tracking by Stochastic Propagation of Conditional Density. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1064, pp. 343–356. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  13. 13.
    Koschan, A., Kang, A., Paik, J., Abidi, B., Abidi, M.: Color Active Shape Models for Tracking Non-Rigid Objects. Pattern Recognition Letters 24, 1751–1765 (2003)CrossRefGoogle Scholar
  14. 14.
    Perez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-Based Probabilistic Tracking. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 661–675. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  15. 15.
    Raja, Y., McKenna, S.J., Gong, S.: Color Model Selection and Adaptation in Dynamic Scenes. In: Burkhardt, H.-J., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1406, pp. 460–474. Springer, Heidelberg (1998)Google Scholar
  16. 16.
    Soriano, M., Martinkauppi, B., Huovinen, S., Laaksonen, M.: Adaptive Skin Color Modelling Using the Skin Locus for Selecting Training Pixels. Pattern Recognition 36, 681–690 (2003)CrossRefGoogle Scholar
  17. 17.
    Soriano, M., Martinkauppi, B., Pietikainen, M.: Detection of Skin under Changing Illumination: A Comparative Study. In: Int. Conf. on Image Analysis and Proc., pp. 652–657 (2003)Google Scholar
  18. 18.
    Sigal, L., Sclaroff, S., Athitsos, V.: Estimation and Prediction of Evolving Color Distributions for Skin Segmentation under Varying Illumination. In: Proc. IEEE Conf. on Comp. Vis. and Patt. Rec., pp. 2152–2159 (2000)Google Scholar
  19. 19.
    Sobottka, K., Pitas, I.: Segmentation and Tracking of Faces in Color Images. In: Proc. of the Sec. Int. Conf. on Aut. Face and Gesture Rec., pp. 236–241 (1996)Google Scholar
  20. 20.
    Srisuk, S., Kurutach, W., Limpitikeat, K.: A Novel Approach for Robust Fast and Accurate Face Detection. Int. J. of Uncertainty, Fuzziness and Knowledge-Based Systems 9(6), 769–779 (2001)MATHGoogle Scholar
  21. 21.
    Swain, M.J., Ballard, D.H.: Color Indexing. Int. J. of Comp. Vision 7(1), 11–32 (1991)CrossRefGoogle Scholar
  22. 22.
    Yang, M.-H., Krigman, D., Ahuja, N.: Detecting Faces in Images: A survey. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(1), 34–58 (2002)CrossRefGoogle Scholar
  23. 23.
    Yang, J., Weier, L., Waibel, A.: Skin-Color Modelling in Color Images. In: Proc. Asian Conf. on Computer Vision, vol. II, pp. 687–694 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Bogdan Kwolek
    • 1
  1. 1.Rzeszów University of TechnologyRzeszówPoland

Personalised recommendations