Emotional Scene Retrieval from Lifelog Videos Using Evolutionary Feature Creation

Abstract

For the purpose of promoting the utilization of a large amount of lifelog videos, an emotional scene retrieval framework is proposed. It detects emotional scenes on the basis of facial expression recognition assuming that a kind of emotion will be aroused with a certain facial expression in an important scene which is likely to be a target of the retrieval. The emotional scene retrieval has a critical issue that it is quite hard to accurately and efficiently detect the emotional scenes because of the difficulty in discriminating spontaneous facial expressions. One of the most effective way to enhance the performance of the retrieval is to select discriminative facial features used for the facial expression recognition. It is, however, not easy to manually select good facial features because very subtle and complex movements of several facial parts will be observed in the appearance of a facial expression. We thus propose a method to automatically generate discriminative facial features on the basis of genetic programming. It produces discriminative facial features by combining a number of points on some salient facial parts using various arithmetic operators. The proposed method is evaluated through an emotional scene detection experiment using a lifelog video dataset containing spontaneous facial expressions.