Human Face Motion Analysis

Abstract

Motivated by model-based video compression, we are studying the modeling, analysis, and synthesis of human face motion. The geometrical shape of the face is modeled by a triangular mesh. Motion can be described at three levels. At the lowest level, motion is represented by the 3D displacements of the vertices of the triangles from one time instant to the next. At the intermediate level, one has the so-called “action units” (e.g., raising eye brows), each of which consists of the movements of several specific vertices. Finally, at the top level, one is concerned with the meaning of the motion. Restricting to the emotional aspect of face motion (not the speech aspect), one considers meanings such as: Happy, angry, anxious, etc.

In this paper, we describe some of our preliminary results on human face motion analysis. Specifically, algorithms are presented for: (i) extracting and tracking key feature points on the face, and (ii) estimating the global and local motion of the head/face. These two sets of algorithms interact with each other. Based on feature correspondences over successive time instants, motion is estimated. Then the estimated motion is used to help track the feature points at future time instants. We attempt to determine the motion of 26 key feature points on the face, from which the action units and meaning of the motion can be deduced.