Skip to main content
SpringerLink
Log in

FER based on the improved convex nonnegative matrix factorization feature

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Facial expression recognition is an important research issue in the pattern recognition field. In this paper, we intend to present an accurate facial expression recognition (FER) system, which employs an improved convex non-negative matrix factorization (ICNMF) method based on a novel objective function and smaller iterative step sizes for feature extraction. Since negative values appearing in the facial expression feature will weaken the features and reduce the recognition rate, the nonnegative matrix factorization (NMF) methods are adopted to guarantee the non-negativity of the extracted feature value to improve the recognition rate. To enhance the performance of NMF method for FER, the ICNMF approach based on a novel convergent objective function and smaller iterative step sizes is proposed, and the FER rate can be improved effectively. In the FER system, the face region is detected firstly, and is enhanced by histogram specification, secondly the ICNMF approach is adopted to extract features and then the feature coefficient matrix is achieved. Finally, the SVM classifier is applied to recognize the extracted features. To validate the effectiveness of FER system, four public available datasets of MultiPIE, CK+, FER2013 and SFEW are tested and then high recognition rates can be achieved based on ICNMF method. In addition, the proposed ICNMF approach is compared with the methods of multi-layer NMF, sparse non-negative matrix factorization (SNMF), the traditional convex non-negative matrix factorization (CNMF), deep belief networks (DBN) and stacked auto-encoder (SAE), and results of experiments show that the proposed ICNMF approach is significantly effective contrasting to the other five expression extraction methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ahsan T, Jabid T, Chong UP (2013) Facial expression recognition using local transitional pattern on Gabor filtered facial images. IETE Tech Rev 30(1):47–52

    Article  Google Scholar 

  2. Ali HB, Powers DMW, Jia X, Zhang YH (2015) Extended non-negative matrix factorization for face and facial expression recognition. International Journal of Machine Learning and Computing 5(2):142–147

    Article  Google Scholar 

  3. Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Statistics surveys 4:40–79

    Article  MathSciNet  Google Scholar 

  4. Bejaoui H, Ghazouani H, Barhoumi W (2019) Sparse coding-based representation of LBP difference for 3D/4D facial expression recognition. Multimed Tools Appl 78(16):22773–22796

    Article  Google Scholar 

  5. Bhowmik MK, Saha P, Singha A, Bhattacharjee D (2019) Enhancement of robustness of face recognition system through reduced gaussianity in log-ICA. Expert Syst Appl 116:96–107

    Article  Google Scholar 

  6. Ding CHQ, Li T, Jordan MI (2008) Convex and semi-nonnegative matrix factorizations. IEEE Trans Pattern Anal Mach Intell 32(1):45–55

    Article  Google Scholar 

  7. Fernandez PDM, Peña FAG, Ren TI, Alexandre C. 2019 FERAtt: Facial Expression Recognition with Attention Net. arXiv preprint arXiv:1902.03284

  8. Gong J, Liu JY, Wang LJ, Sun XW, Nie SD (2018) Automatic detection of pulmonary nodules in CT images by incorporating 3D tensor filtering with local image feature analysis. Physica Medica 46:124–133

    Article  Google Scholar 

  9. Happy SL, Routray A (2014) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6(1):1–12

    Article  Google Scholar 

  10. Issa SM, Peng Q, You X, Shah WA (2019) Emotion assessment using EEG brain signals and stacked sparse autoencoder. Journal of Information Assurance & Security 14(1):20–29

    Google Scholar 

  11. Khan RA, Meyer A, Konik H, Bouakaz S (2019) Saliency-based framework for facial expression recognition. Frontiers of Computer Science 13(1):183–198

    Article  Google Scholar 

  12. Kumar S, Gupta A (2015) Facial expression recognition: a review. Proceedings of the National Conference on Cloud Computing and Big Data, Shanghai, pp 4–6

    Google Scholar 

  13. Lai SZ (2014) Research on several non-negative matrix factorization methods and applications. A Master Thesis of University of Electronic Science and Technology:17–29

  14. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  15. Lee DD, Seung HS (2015) Algorithms for non-negative matrix factorization. Adv Neural Inf Proces Syst 13(6):556–562

    Google Scholar 

  16. Mollahosseini A, Chan D, Mahoor MH. Going Deeper in Facial Expression Recognition using Deep Neural Networks. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 2016, 3: 1–10.

  17. Nigam S, Singh R, Misra AK. Local Binary Patterns Based Facial Expression Recognition for Efficient Smart Applications. Security in Smart Cities: Models, Applications, and Challenges. Springer, Cham, 2019: 297–322.

  18. Patil MN, Iyer B, Arya R (2016) Performance evaluation of PCA and ICA algorithm for facial expression recognition application. In: Proceedings of Fifth International Conference on Soft Computing for Problem Solving. Springer, Singapore, pp 965–976

    Chapter  Google Scholar 

  19. Rezaei M, Derhami V (2019) Improving LNMF performance of facial expression recognition via significant parts extraction using Shapley value. Journal of AI and Data Mining 7(1):17–25

    Google Scholar 

  20. Saha A, Pradhan SN (2018) Facial expression recognition based on eigenspaces and principle component analysis. International journal of computational vision and robotics. Gurgaon, India: IEEE 8(2):190–200

    Google Scholar 

  21. Sajjad M, Nasir M, Ullah FUM, Muhammad K, Kumar A, Balk SW (2019) Raspberry pi assisted facial expression recognition framework for smart security in law-enforcement services. Inf Sci 479:416–431

    Article  Google Scholar 

  22. Siddiqi MH, Ali R, Khan AM, Park YT (2015) Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields. IEEE Trans Image Process 24(4):1386–1398

    Article  MathSciNet  Google Scholar 

  23. Song HA, Kim BK, Xuan TL, Lee SY (2015) Hierarchical feature extraction by multi-layer non-negative matrix factorization network for classification task. Neurocomputing 165:63–74

    Article  Google Scholar 

  24. Uddin MZ, Hassan MM, Almogren A, Alamri A (2017) Facial expression recognition utilizing local direction-based robust features and deep belief network. IEEE Access 5:4525–4536

    Article  Google Scholar 

  25. Wang Y (2015) Proximal gradient method for convex and semi-nonnegative matrix factorization. A Dissertation of Northeast Normal University for Master Degree:5–6

  26. Xue ML, Duan XD, Liu WQ, Wang YH (2018) An ICA-Based Other-Race Effect Elimination for Facial Expression Recognition. Chinese Conference on Biometric Recognition, Springer, pp 367–376

  27. Yang Y, Fang D, Zhu D (2016) Facial expression recognition using deep belief network. Rev Tec Ing Univ Zulia 39(2):384–392

    Google Scholar 

  28. Yang H, Ciftci U, Yin L (2018) Facial expression recognition by de-expression residue learning. Proc IEEE Conf Comput Vis Pattern Recognit:2168–2177

  29. Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie AM (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273(17):643–649

    Article  Google Scholar 

  30. Zhang ZP, Luo P, Loy CC, Tang XO (2018) From facial expression recognition to interpersonal relation prediction. Int J Comput Vis 126(5):550–569

    Article  MathSciNet  Google Scholar 

  31. Zhao X, Shi X, Zhang S (2015) Facial expression recognition via deep learning. IETE Tech Rev 32(5):347–355

    Article  Google Scholar 

  32. Zhou J, Zhang S, Mei HY, Wang DW (2016) A method of facial expression recognition based on Gabor and NMF. Pattern Recognition and Image Analysis 26(1):119–124

    Article  Google Scholar 

  33. Zhou H, Chen P, Shen WA (2018) multi-view face recognition system based on cascade face detector and improved Dlib. MIPPR 2017: Pattern Recognition and Computer Vision. International Society for Optics and Photonics 10609:1060908

    Google Scholar 

Download references

Acknowledgements

This paper is sponsored by key research funding of the 13th five year plan for education science of Wuhan in 2017(2017A073).

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Science, Jianghan University, Wuhan, 430056, Hubei, China

    Jing Zhou

  2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, China

    Tianjiang Wang

Authors
  1. Jing Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianjiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Zhou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Wang, T. FER based on the improved convex nonnegative matrix factorization feature. Multimed Tools Appl 79, 26305–26325 (2020). https://doi.org/10.1007/s11042-020-08919-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08919-1

Keywords

Search

Navigation