Detecting Anomaly Features in Facial Expression Recognition

Abstract

In facial expression recognition task, humans and machines may give very different result on the same facial expression image. To solve this problem, this paper analyzes the reason that causes this phenomenon. Our research find that expression features of anomaly samples often deviate from standard expression features, which makes the recognition results different from that of human’s. In order to detect anomaly of facial expression, we propose Emo-Encoder Network (EEN), which consists of three modules: feature extractor, classifier and anomaly detector. Feature extractor is used to extract facial expression features. Based on extracted features, we can get classification results by classifier and anomaly degree by anomaly detector, where classification results denotes what kind of expression is, while anomaly degree displays how likely is this expression anomaly. To demonstrate our idea, we build a mixed dataset consisting of images collected from the Internet as well as the RAFDB dataset, and we relabel them with anomaly labels. Through our experiments, our model achieve anomaly detection accuracy of 87.09% on the mixed dataset. We find that expressions with high classification confidence cannot be directly adopted as the final results, unless the results is at high confidence and low anomaly degree.

This research is partially supported by National Key Research and Development Program of China with ID 2018AAA0103203.