Image-Based Facial Expression Recognition Using Local Neighborhood Difference Binary Pattern

  • Sumeet SauravEmail author
  • Sanjay Singh
  • Madhulika Yadav
  • Ravi Saini
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1022)


Automatic facial expression recognition (FER) has gained enormous interest among the computer vision researchers in recent years because of its potential deployment in many industrial, consumer, automobile, and societal applications. There are a number of techniques available in the literature for FER; among them, many appearance-based methods such as local binary pattern (LBP), local directional pattern (LDP), local ternary pattern (LTP), gradient local ternary pattern (GLTP), and improved local ternary pattern (IGLTP) have been shown to be very efficient and accurate. In this paper, we propose a new descriptor called local neighborhood difference binary pattern (LNDBP). This new descriptor is motivated by the recent success of local neighborhood difference pattern (LNDP) which has been proven to be very effective in image retrieval. The basic characteristic of LNDP as compared with the traditional LBP is that it generates binary patterns based on a mutual relationship of all neighboring pixels. Therefore, in order to use the benefit of both LNDP and LBP, we have proposed LNDBP descriptor. Moreover, since the extracted LNDBP features are of higher dimension, therefore a dimensionality reduction technique has been used to reduce the dimension of the LNDBP features. The reduced features are then classified using the kernel extreme learning machine (K-ELM) classifier. In order to, validate the performance of the proposed method, experiments have been conducted on two different FER datasets. The performance has been observed using well-known evaluation measures, such as accuracy, precision, recall, and F1-score. The proposed method has been compared with some of the state-of-the-art works available in the literature and found to be very effective and accurate.


Facial expression recognition (FER) Local neighborhood difference pattern (LNDP) Principal component analysis (PCA) Kernel extreme learning machine (K-ELM) 


  1. 1.
    Rivera, A.R., Castillo, J.R., Chae, O.O.: Local directional number pattern for face analysis: face and expression recognition. IEEE Trans. Image Process. 22(5), 1740–1752 (2013)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Ryu, B., Rivera, A.R., Kim, J., Chae, O.: Local directional ternary pattern for facial expression recognition. IEEE Trans. Image Process. 26(12), 6006–6018 (2017)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)CrossRefGoogle Scholar
  4. 4.
    Zhou, H., Wang, R., Wang, C.: A novel extended local-binary-pattern operator for texture analysis. Inf. Sci. 178(22), 4314–4325 (2008)CrossRefGoogle Scholar
  5. 5.
    Zhao, S., Gao, Y., Zhang, B.: Sobel-lbp. In: 15th IEEE International Conference on Image Processing, pp. 2144–2147 (2008)Google Scholar
  6. 6.
    Jabid, T., Kabir, M.H., Chae, O.: Facial expression recognition using local directional pattern (LDP). In: 17th IEEE International Conference on Image Processing, pp. 1605–1608 (2010)Google Scholar
  7. 7.
    Ahmed, F., Hossain, E.: Automated facial expression recognition using gradient-based ternary texture patterns. Chin. J. Eng. (2013)Google Scholar
  8. 8.
    Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: a robust local image descriptor. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1705–1720 (2010)CrossRefGoogle Scholar
  9. 9.
    Alhussein, M.: Automatic facial emotion recognition using weber local descriptor for e-Healthcare system. Clust. Comput. 19(1), 99–108 (2016)CrossRefGoogle Scholar
  10. 10.
    Holder, R.P., Tapamo, J.R.: Improved gradient local ternary patterns for facial expression recognition. EURASIP J. Image Video Process. (1), 42 (2017)Google Scholar
  11. 11.
    Al-Sumaidaee, S.A.M., Abdullah, M.A.M., Al-Nima, R.R.O., Dlay, S.S., Chambers, J.A.: Multi-gradient features and elongated quinary pattern encoding for image-based facial expression recognition. Pattern Recogn. 71, 249–263 (2017)CrossRefGoogle Scholar
  12. 12.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004)CrossRefGoogle Scholar
  13. 13.
    Martin, K.: Efficient metric learning for real-world face recognition.
  14. 14.
    Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 532–539 (2013)Google Scholar
  15. 15.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  16. 16.
    Verma, M., Raman, B.: Local neighborhood difference pattern: a new feature descriptor for natural and texture image retrieval. Multimed. Tools Appl., 1–24 (2017)Google Scholar
  17. 17.
    Jolliffe, I.: Principal component analysis. In: International encyclopedia of statistical science, pp. 1094–1096. Springer, Berlin, Heidelberg (2011)Google Scholar
  18. 18.
    Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst., Man, Cybern., Part B (Cybernetics), 42(2), 513–529 (2012)Google Scholar
  19. 19.
    Huang, Z., Yu, Y., Gu, J., Liu, H.: An efficient method for traffic sign recognition based on extreme learning machine. IEEE Trans. Cybern. 47(4), 920–933 (2017)CrossRefGoogle Scholar
  20. 20.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101 (2010)Google Scholar
  21. 21.
    Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H., Hawk, S.T., Van Knippenberg, A.D.: Presentation and validation of the Radboud Faces Database. Cogn. Emot. 24(8), 1377–1388 (2010)CrossRefGoogle Scholar
  22. 22.
    Carcagnì, P., Coco, M., Leo, M., Distante, C.: Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus 4(1), 645 (2015)CrossRefGoogle Scholar
  23. 23.
    Ahmed, F., Kabir, M. H.: Directional ternary pattern (dtp) for facial expression recognition. In: IEEE International Conference on Consumer Electronics (ICCE), pp. 265–266 (2012)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  • Sumeet Saurav
    • 1
    • 2
    Email author
  • Sanjay Singh
    • 1
    • 2
  • Madhulika Yadav
    • 3
  • Ravi Saini
    • 1
    • 2
  1. 1.Academy of Scientific & Innovative Research (AcSIR), ChennaiChennaiIndia
  2. 2.CSIR-Central Electronics Engineering Research Institute, PilaniPilaniIndia
  3. 3.Department of ElectronicsBanasthali VidyapithVanasthaliIndia

Personalised recommendations