Emotional Scene Retrieval from Lifelog Videos Using Evolutionary Feature Creation
For the purpose of promoting the utilization of a large amount of lifelog videos, an emotional scene retrieval framework is proposed. It detects emotional scenes on the basis of facial expression recognition assuming that a kind of emotion will be aroused with a certain facial expression in an important scene which is likely to be a target of the retrieval. The emotional scene retrieval has a critical issue that it is quite hard to accurately and efficiently detect the emotional scenes because of the difficulty in discriminating spontaneous facial expressions. One of the most effective way to enhance the performance of the retrieval is to select discriminative facial features used for the facial expression recognition. It is, however, not easy to manually select good facial features because very subtle and complex movements of several facial parts will be observed in the appearance of a facial expression. We thus propose a method to automatically generate discriminative facial features on the basis of genetic programming. It produces discriminative facial features by combining a number of points on some salient facial parts using various arithmetic operators. The proposed method is evaluated through an emotional scene detection experiment using a lifelog video dataset containing spontaneous facial expressions.
KeywordsLifelog Video retrieval Facial expression recognition Genetic programming
This research is supported by Japan Society for the Promotion of Science, Grant-in-Aid for Young Scientists (B), 15K15993.
- 1.Aizawa, K., Hori, T., Kawasaki, S., Ishikawa, T.: Capture and efficient retrieval of life log. In: Proceedings of Pervasive 2004 Workshop on Memory and Sharing Experiences, pp. 15-20 (2004)Google Scholar
- 2.Gemmell, J., Bell, G., Luederand, R., Drucker, S., Wong, C.: MyLifeBits: fulfilling the memex vision. In: Proceedings of the 10th ACM International Conference on Multimedia, pp. 235-238 (2002)Google Scholar
- 3.Datchakorn, T., Yamasaki, T., Aizawa, K.: Practical experience recording and indexing of life log video. Proceedings of the 2nd ACM Workshop on Continuous Archival and Retrieval of Personal Experiences, pp. 61-66 (2005)Google Scholar
- 4.Nomiya, H., Morikuni, A., Hochin, T.: An unsupervised ensemble approach for emotional scene detection from lifelog videos. Softw. Eng. Artif. Intell. Netw. Parallel/Distrib. Comput. Stud. Comput. Intell. 569, 145-159 (2015)Google Scholar
- 5.Datcu, D., Rothkrantz, L.: Facial expression recognition in still pictures and videos using active appearance models: a comparison approach. In: Proceedings of the 2007 International Conference on Computer Systems and Technologies, pp. 1-6 (2007)Google Scholar
- 6.Fanelli, G., Yao, A., Noel, P.-L., Gall, J., Gool, L.V.: Hough forest-based facial expression recognition from video sequences. In: Proceedings of the 11th European Conference on Trends and Topics in Computer Vision, pp. 195-206 (2010)Google Scholar
- 8.Tian, Y., Kanade, T., Cohn, J.F.: In: Li, S.Z., Jain, A.K. (eds.) Handbook of face recognition. Facial Expression Recognition. Springer, London (2011)Google Scholar
- 9.Soyel, H., Demirel, H.: Facial expression recognition using 3d facial feature distances. In: Proceedings of the 4th International Conference on Image Analysis and Recognition, pp. 831-838 (2007)Google Scholar
- 10.Hupont, I., Cerezo, E., Baldassarri, S.: Sensing facial emotion in a continuous 2D affective space. In: Proceedings of International Conference on Systems, Man, and Cybernetics, pp. 2045-2051 (2010)Google Scholar
- 11.Luxand Inc.: Luxand FaceSDK 4.0 (2015). http://www.luxand.com/facesdk (The latest version is 5.0.1)
- 13.Fraser, A., Weinbrenner, T.: GPC++ - Genetic Programming C++ Class Library, Version 0.5.2 (2015). http://www0.cs.ucl.ac.uk/staff/ucacbbl/ftp/weinbenner/gp.html
- 14.Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3, 27):1-27 (2011)Google Scholar