Robust Lip Contours Localization and Tracking Using Multi Features – Statistical Shape Models

Abstract

We propose and evaluate methods for enhancing performances of lip contours localization and tracking, which are based on the concepts of Statistical Shape Models (e.g. Active Shape Models, Hybrid Active Shape Models) and optimization of multi features. A single feature-based ASM gets good performance only in particular conditions but gets stuck in local minimum or gives bad performance in noisy conditions. In this paper, we propose to use 3 features: Normal Profile, Grey Level Patches and Gabor Wavelets and combine them by using a voting approach to derive a robust method (MF-ASM) on lip contours detection. Since the original ASM does not take into account the temporal information from previous frames, the lip contours are tracked by replacing the standard ASM with our hybrid ASM which is capable to take advantage of temporal information. Initial experimental results using popular audio-visual database show that our methods are more robust to the local minimum problem and give higher accuracy than traditional single feature-based ASM in lip contours detection and tracking.