Multimedia Tools and Applications

, Volume 76, Issue 2, pp 2995–3010 | Cite as

Facial expression recognition using ELBP based on covariance matrix transform in KLT

  • Min GuoEmail author
  • Xiaohong Hou
  • Yuting Ma
  • Xiaojun Wu


According to the deficiencies of Local Binary Pattern (LBP), the dimension of extraction is large, and it is not conducive to describe all characteristics of image texture, this paper proposes a novel facial expression recognition algorithm “K-ELBP” which uses uniform patterns of Extended Local Binary Pattern (ELBP), and combines with the covariance matrix transform in K-L transform (KLT). In this paper, ELBP is used for the first step to extract the feature matrix of expression images, then covariance matrix transform is applied to the ELBP matrix for reducing the dimension, which aims at extracting the main feature vectors. And the best recognition performance is obtained by using SVM for classification. A series of experiments by using different divided methods are designed to evaluate the effects of characteristics which are extracted by K-ELBP algorithm. According to the results of the experiments, the proposed K-ELBP algorithm can extract facial expression features effectively, and the rates of recognition are satisfying.


Facial expression recognition Extended local binary pattern Covariance matrix transform KLT Machine learning 



This work was supported by the National Natural Science Foundation of China (No.11172342, 11372167), the Fundamental Research Funds for the Central Universities (No.GK201405007), Interdisciplinary Incubation Project of Learning Science of Shaanxi Normal University, the Program of Key Science and Technology Innovation Team in Shaanxi Province (No. 2014KTC-18).


  1. 1.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefzbMATHGoogle Scholar
  2. 2.
    Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27Google Scholar
  3. 3.
    Cheon Y, Kim D (2009) Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recogn 42(7):1340–1350CrossRefzbMATHGoogle Scholar
  4. 4.
    Gao T, Feng X-L, Lu ZJ-H (2013) A novel face feature descriptor using adaptively weighted extended LBP pyramid. Optik Int J Light and Electron Opt 124(23):6286–6291CrossRefGoogle Scholar
  5. 5.
    Ge K-B, Wen J, Fang B (2011) Adaboost algorithm based on MB-LBP features with skin color segmentation for face detection. Int Conf Wavelet Anal Pattern Recogn (ICWAPR) 2011:40–43Google Scholar
  6. 6.
    Guo G-D, Dyer C-R (2003) Simultaneous feature selection and classifier training via linear programming: a case study for face expression recognition. Comput Soc Conf IEEE 1:I-346–I-352Google Scholar
  7. 7.
    Hadid A, Pietikäinen M, Ahonen T (2004) A discriminative feature space for detecting and recognizing faces. Comput Soc Conf IEEE 2:II-797–II-804Google Scholar
  8. 8.
    Han Y-H, Yang Y, Wu F, Hong R-C (2015) Compact and discriminative descriptor inference using multi-cues. IEEE Trans Image Process 24(12):5114–5126MathSciNetCrossRefGoogle Scholar
  9. 9.
    Han Y-H, Yang Y, Wu F, Hong R-C (2015) Semisupervised feature selection via spline regression for video semantic recognition. IEEE Trans Neural Netw Learn Syst 26(2):252–264MathSciNetCrossRefGoogle Scholar
  10. 10.
    Kumbhar M, Jadhav A, Patil M (2012) Facial expression recognition based on image feature. Int J Comput Commun Eng 1(2):117–119CrossRefGoogle Scholar
  11. 11.
    Londhe R, Pawar V (2012) Facial expression recognition based on affine moment invariants. Int J Comput Sci Issues 9(6):388–392Google Scholar
  12. 12.
    Mäenpää T, Pietikäinen M (2005) Texture analysis with local binary patterns. Handb Pattern Recogn Comput Vis 3:197–216CrossRefzbMATHGoogle Scholar
  13. 13.
    Mohammadi MR, Fatemizadeh E, Mahoor MH (2014) PCA-based dictionary building for accurate facial expression recognition via sparse representation. J Vis Commun Image Represent 25(5):1082–1092CrossRefGoogle Scholar
  14. 14.
    Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29(1):51–59CrossRefGoogle Scholar
  15. 15.
    Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefzbMATHGoogle Scholar
  16. 16.
    Pietikäinen M, Ojala T, Xu Z (2000) Rotation-invariant texture classification using feature distributions. Pattern Recogn 33(1):43–52CrossRefGoogle Scholar
  17. 17.
    Shan CF, Gong SG, W. P (2006) A comprehensive empirical study on linear subspace methods for facial expression analysis. Comput Vis Pattern Recognit Workshop 153–153Google Scholar
  18. 18.
    Topi M, Matti P, Timo O (2000) Texture classification by multi-predicate local binary pattern operators. ICPR 3951Google Scholar
  19. 19.
    Turtinen M, Pietikainen M, SilvÉn O (2006) Visual characterization of paper using isomap and local binary patterns. IEICE Trans Inf Syst 89(7):2076–2083CrossRefGoogle Scholar
  20. 20.
    Yang B, Chen S-C (2013) A comparative study on local binary pattern (LBP) based face recognition: LBP histogram versus LBP image. Neurocomputing 120:365–379CrossRefGoogle Scholar
  21. 21.
    Yang J, Yang J-Y (2002) Generalized K–L transform based combined feature extraction. Pattern Recogn 35(1):295–297CrossRefzbMATHGoogle Scholar
  22. 22.
    Zhang SQ, Zhao XM, Lei BC (2011) Facial expression recognition using local fisher discriminant analysis. Adv Comput Sci Environ Ecoinformatics Educ 443–448Google Scholar
  23. 23.
    Zhong L, Liu Q-S, Yang P, Liu B (2012) Learning active facial patches for expression analysis. Comput Vis Pattern Recognit (CVPR) 2562–2569Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Min Guo
    • 1
    • 2
    Email author
  • Xiaohong Hou
    • 1
    • 2
  • Yuting Ma
    • 1
    • 2
  • Xiaojun Wu
    • 1
    • 2
  1. 1.Key Laboratory of Modern Teaching TechnologyMinistry of EducationXi’anChina
  2. 2.School of Computer ScienceShaanxi Normal UniversityXi’anChina

Personalised recommendations