Image Representations for Facial Expression Analysis: Comparative Study I

Abstract

Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System, (Ekman and Friesen, 1978), is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. In our first study we compared three approaches: Holistic spatial analysis (eigenfaces), explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system which classified six upper facial actions with 91% accuracy, including low, medium, and high magnitude facial actions. The hybrid system outperformed human non-experts on this task, and performed as well as highly trained experts. These comparisons supported the theory that unsupervised feature extraction based on dependencies in the image ensemble is more effective for face image analysis than explicit measurement of facial features.