Facial Expression Recognition Using Local Region Specific Dense Optical Flow and LBP Features
Recognition of facial expression has many applications including human-computer interaction, human emotion analysis, personality development, cognitive science, health-care, virtual reality, image retrieval, etc. In this paper we propose a new method for recognition of facial expression using local region specific mean optical flow and local binary pattern feature descriptor with support vector machine classification. In general, facial expression recognition techniques divide the face into regular grid (holistic representation) and the facial features are extracted. However, in this paper we divide the face into domain specific local regions. At first a robust optical flow is utilized to get mean optical flow in different directions for each local region which considers both local statistic motion information and its spatial location. The features are used only from the key frames; which are detected based on maximal mean optical flow magnitude within a sequence w.r.t. neutral frame. Now, the region specific local binary pattern is extracted from key frame and concatenated with mean optical flow features. The performance of the proposed facial expression recognition system has been validated on CK+ facial expression dataset.
KeywordsFacial expression Local representation Optical flow Local binary pattern Support vector machines
This work was supported by the Technology Innovation Program (10052289, Development of HD high-reliability stereo ADAS vision system) funded By the Ministry of Trade, Industry & Energy (MI, Korea).
- 1.Mehrabian, A.: Communication without words. Psychol. Today 2, 53–56 (1968)Google Scholar
- 13.Rudovic, O., Pantic, M., Patras, I.: Coupled Gaussian processes for pose-invariant facial expression recognition. IEEE Trans. Pattern Anal. Mach. Intell. 25, 1357–1369 (2012)Google Scholar
- 18.Jiang, B., Martinez, B., Valster, M.F., Pantic, M.: Decision level fusion of domain specific regions for facial action recognition. In: 22nd International Conference on Pattern Recognition, Stockholm, Sweden, pp. 1776–1781, 24–28 August 2014Google Scholar
- 19.Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychological Press, Palo Alto (1978)Google Scholar
- 20.Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression tree. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, pp. 1867–1874, 23-28 June 2014Google Scholar
- 21.King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)Google Scholar
- 22.Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools (2000)Google Scholar
- 25.Chaudhry, R., Ravichandran, A., Hager, G., Vidal, R.: Histogram of oriented optical flow and Binet-Cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: Proceeding of IEEE Conference Computer Vision and Pattern Recognition (CVPR), Miami Beach, Floridia, USA, pp. 1932–1939. 20–25 June 2009Google Scholar
- 27.Pantic, M., Valster, M., Rademaker, R., Maat, L.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specific expressions. In: Proceeding of 3rd IEEE Workshop on CVPR for Human Communication Behavior Analysis, pp. 94–101, June 2010Google Scholar
- 28.Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm. Accessed on 15 Apr 2017
- 29.Hsu, C.W., Chang, C.-C., Lin, C.-J.: A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University, Taiwan (2010)Google Scholar
- 30.Poursaberi, A., Noubari, H.A., Gavrolova, M., Yanushkevich, S.N.: Gauss-Laguerre wavelet textural feature fusion with geometrical information for facial expression identification. EURASIP J. Image Video Process. 17, 1–13 (2012)Google Scholar