Cluster Computing

, Volume 21, Issue 1, pp 323–331 | Cite as

Multi-stage binary patterns for facial expression recognition in real world

  • Sadia Arshid
  • Ayyaz HussainEmail author
  • Asim Munir
  • Anum Nawaz
  • Sanneya Aziz


Facial expression recognition in real world has always been a challenging task in computer vision. Expressions are spawned from highly bendable features and vary in cultures, genders and ages among individuals. Moreover, complex backgrounds, intricate settings and varied illumination conditions add to complexity of recognition. Local binary patterns (LBP) capture the local region properties from images so these are extensively used for facial expression recognition. But still LBP and its variations including local gradient coding horizontal diagonal direction (LGC-HD), local gradient coding horizontal vertical diagonal direction (LGC-HVD) and compound local binary pattern (CLBP) are not able to resolve the issues of local illuminations that might result from disproportionate light, certain obscurities, shadows or use of accessories on the face. In proposed technique, multi-stage binary code is generated for each comparison against neighbouring pixel. LBP records only sign difference which eventually discards some significant texture information. In MSBP sign difference along with gradient difference accumulates changes along edges such as area around eye brows, eyes, mouth, bulges and wrinkles. MSBP is then compared with LBP, LGC-HD, LGC-HVD and CLBP. LGC-HD, LGCHVD and CLBP centered on experimentation using two approaches. These techniques are tested on dataset containing full face image named holistic and then division based approach. Experiments are performed using static facial expression in wild dataset. Results show that proposed approach MSBP outperforms other techniques with 96% accuracy in holistic approach and 60% accuracy in division based approach. Further, it is also ascertained from the experiments that all these approaches work better in holistic approach than division based approach.


Facial expression recognition Local binary patterns Compound local binary patterns 


  1. 1.
    Lai, C.-C., Ko, C.-H.: Facial expression recognition based on two-stage features extraction. Opt. Int. J. Light Electron Opt. 125(22), 6678–6680 (2014)CrossRefGoogle Scholar
  2. 2.
    Chao, W.-L., Ding, J.-J., Liu, J.-Z.: Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection. Signal Process. 117, 1–10 (2015)CrossRefGoogle Scholar
  3. 3.
    Ali, G., Iqbal, M.A., Choi, T.-S.: Boosted NNE collections for multicultural facial expression recognition. Pattern Recognit. 55, 14–27 (2016)CrossRefGoogle Scholar
  4. 4.
    Jiang, T., Wang, L., Zhao, X.: Facial expression recognition based on adaptively weighted improved local binary pattern. In: Eighth International Conference on Digital Image Processing (ICDIP 2016), 2016, pp. 100330W–100330W–5 (2016)Google Scholar
  5. 5.
    Lyons, M.J., Akamatsu, S., Kamachi, M., Gyoba, J., Budynek, J.: The Japanese female facial expression (JAFFE) database (1998)Google Scholar
  6. 6.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops 2010, pp. 94–101 (2010)Google Scholar
  7. 7.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (1992)CrossRefGoogle Scholar
  8. 8.
    Dhall, A., Goecke, R., Lucey, S., Gedeon, T.: Acted facial expressions in the wild database. Australian National University Canberra, Australia Technical Report TR-CS-11, vol. 2 (2011)Google Scholar
  9. 9.
    Ghimire, D., Lee, J., Li, Z.-N., Jeong, S., Park, S.H., Choi, H.S.: Recognition of facial expressions based on tracking and selection of discriminative geometric features. Int. J. Multimed. Ubiquitous Eng. 10(3), 35–44 (2015)CrossRefGoogle Scholar
  10. 10.
    Ben, X., Zhang, P., Yan, R., Yang, M., Ge, G.: Gait recognition and micro-expression recognition based on maximum margin projection with tensor representation. Neural Comput. Appl. 26, 1–18 (2016)Google Scholar
  11. 11.
    Zhang, Z., Lyons, M., Schuster, M., Akamatsu, S.: Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition 1998, pp. 454–459 (1998)Google Scholar
  12. 12.
    Tan, X., Triggs, B.: Fusing Gabor and LBP feature sets for kernel-based face recognition. In: International Workshop on Analysis and Modeling of Faces and Gestures, 2007, pp. 235–249 (2007)Google Scholar
  13. 13.
    Anil, J., Suresh, L.P.: Literature survey on face and face expression recognition. In: International Conference on Circuit, Power and Computing Technologies (ICCPCT), 2016, pp. 1–6 (2016)Google Scholar
  14. 14.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29(1), 51–59 (1996)CrossRefGoogle Scholar
  15. 15.
    Topi, M., Timo, O., Matti, P., Maricor, S.: Robust texture classification by subsets of local binary patterns. In: Proceedings of the 15th International Conference on Pattern Recognition, 2000, vol. 3, pp. 935–938 (2000)Google Scholar
  16. 16.
    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)CrossRefzbMATHGoogle Scholar
  17. 17.
    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)CrossRefzbMATHGoogle Scholar
  18. 18.
    Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRefGoogle Scholar
  19. 19.
    Huang, X., Wang, S.-J., Zhao, G., Piteikainen, M.: Facial micro-expression recognition using spatiotemporal local binary pattern with integral projection. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)Google Scholar
  20. 20.
    Ahmed, F., Bari, H., Hossain, E.: Person-independent facial expression recognition based on compound local binary pattern (CLBP). Int. Arab. J. Inf. Technol. 11(2), 195–203 (2014)Google Scholar
  21. 21.
    Jun, B., Kim, D.: Robust face detection using local gradient patterns and evidence accumulation. Pattern Recognit. 45(9), 3304–3316 (2012)CrossRefGoogle Scholar
  22. 22.
    Subramanyam, P.S., Fegade, S.A.: Face and facial expression recognition: a comparative study. Int. J. Comput. Sci. Mob. Comput. 2, 28 (2013)Google Scholar
  23. 23.
    Faraji, M.R., Qi, X.: Face recognition under varying illuminations using logarithmic fractal dimension-based complete eight local directional patterns. Neurocomputing 199, 16–30 (2016)CrossRefGoogle Scholar
  24. 24.
    Hussain, A. Khan, M.S., Nazir, M., Iqbal, M.A.: Survey of various feature extraction and classification techniques for facial expression recognition. In: Proceedings of the 11th WSEAS International Conference on Electronics, Hardware, Wireless and Optical Communications, and Proceedings of the 11th WSEAS International Conference on Signal Processing, Robotics and Automation, and Proceedings of the 4th WSEAS International Conference on Nanotechnology; World Scientific and Engineering Academy and Society (WSEAS), 2012, pp. 138–142 (2012)Google Scholar
  25. 25.
    Yuan, L.U.O., Wu, C., Zhang, Y.: Facial expression feature extraction using hybrid PCA and LBP. J. China Univ. Posts Telecommun. 20(2), 120–124 (2013)CrossRefGoogle Scholar
  26. 26.
    Khan, S.A., Hussain, A., Usman, M.: Facial expression recognition on real world face images using intelligent techniques: A survey. Opt. Int. J. Light Electron Opt. 127(15), 6195–6203 (2016)CrossRefGoogle Scholar
  27. 27.
    Al-Sumaidaee, S.A.M., Dlay, S.S., Woo, W.L., Chambers, J.A.: Facial expression recognition using local Gabor gradient code-horizontal diagonal descriptor (2015)Google Scholar
  28. 28.
    Kristensen, R.L., Tan, Z.-H., Ma, Z., Gou, J.: Emotion recognition in blurred images with local features and machine learning (2014)Google Scholar
  29. 29.
    Carcagnì, P., Coco, M., Leo, M., Distante, C.: Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus 4(1), 1 (2015)CrossRefGoogle Scholar
  30. 30.
    Kumari, J., Rajesh, R., Pooja, K.M.: Facial expression recognition: a survey. Proc. Comput. Sci. 58, 486–491 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  • Sadia Arshid
    • 1
  • Ayyaz Hussain
    • 1
    • 2
    Email author
  • Asim Munir
    • 1
  • Anum Nawaz
    • 1
  • Sanneya Aziz
    • 1
  1. 1.Department of Computer ScienceInternational Islamic UniversityIslamabadPakistan
  2. 2.College of Computing & InformaticsSaudi Electronic UniversityRiyadhKingdom of Saudi Arabia

Personalised recommendations