Skip to main content
SpringerLink
Log in

Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes an efficient approach for person-independent facial expression recognition based on the fusion of Histogram of Oriented Gradients (HOG) descriptor and Cuttlefish Algorithm (CFA). The proposed approach employs HOG descriptor due to its outstanding performance in pattern recognition, which results in features that are robust against small local pose and illumination variations. However, it produces some irrelevant and noisy features that slow down and degrade the classification performance. To address this problem, a wrapper-based feature selector, called CFA, is used. This is because CFA is a recent bio-inspired feature selection algorithm, which has been shown to effectively select an optimal subset of features while achieving a high accuracy rate. Here, support vector machine classifier is used to evaluate the quality of the features selected by the CFA. Experimental results validated the effectiveness of the proposed approach in attaining a high recognition accuracy rate on three widely adopted datasets: CK+ (97.86%), RaFD (95.15%), and JAFFE (90.95%). Moreover, the results also indicated that the proposed approach yields competitive or even superior results compared to state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Ahmed F, Bari H, Hossain E (2014) Person-independent facial expression recognition based on compound local binary pattern (CLBP). Int Arab J Inform Technol 11(2):195–203

    Google Scholar 

  2. An L, Yang S, Bhanu B (2014) Efficient smile detection by extreme learning machine. Neurocomputing 149:354–363. https://doi.org/10.1016/j.neucom.2014.04.072

    Article  Google Scholar 

  3. Bastanfard A, Bastanfard O, Takahashi H, Nakajima M (2004) Toward anthropometries simulation of face rejuvenation and skin cosmetic. Comp Anim Virtual Worlds 15(3–4):347–352. https://doi.org/10.1002/cav.38

    Article  Google Scholar 

  4. Bastanfard A, Takahashi H, Nakajima M (2004) Toward E-appearance of human face and hair by age, expression and rejuvenation. Proceed - 2004 Int Conf Cyberworlds, CW 2004:306–311. https://doi.org/10.1109/CW.2004.65

  5. Bin Iqbal MT, Ryu B, Rivera AR, Makhmudkhujaev F, Chae O, Bae S-H (2020) Facial expression recognition with active local shape pattern and learned-size block representations. IEEE Trans Affect Comput:1–15. https://doi.org/10.1109/TAFFC.2020.2995432

  6. Carcagnì P, Del Coco M, Leo M, Distante C (2015) Facial expression recognition and histograms of oriented gradients: a comprehensive study. SpringerPlus 4. https://doi.org/10.1186/s40064-015-1427-3

  7. Chen J, Gong Y, Zhang K, Chen D, Yu M, Wang L (2012) Facial expression recognition using geometric and appearance features. Proceed 4th Int Conf Internet Multimedia Comput Service:29–33. https://doi.org/10.1145/2382336.2382345

  8. Dagher I, Dahdah E, Al Shakik M (2019) Facial expression recognition using three- stage support vector machines. Visual Comput Industry, Biomed Art 2(24):1–9. https://doi.org/10.1186/s42492-019-0034-5

    Article  Google Scholar 

  9. Dalal N, Triggers B (2005) Histograms of oriented gradients for human detection. Proceed IEEE Comput Soc Conf Comput Vision Patt Recogn (CVPR’05). https://doi.org/10.1109/CVPR.2005.177

  10. Dapogny A, Bailly K, Dubuisson S (2017) Dynamic pose-robust facial expression recognition by multi-view pairwise conditional random forests. IEEE Trans Affect Comput 10(2):1–14. https://doi.org/10.1109/TAFFC.2017.2708106

    Article  Google Scholar 

  11. Eesa AS, Brifcani AMA, Orman Z (2013) Cuttlefish algorithm – a novel bio-inspired optimization algorithm. Int J Sci Eng Res 4(9):1978–1986

    Google Scholar 

  12. Eesa AS, Orman Z, Brifcani AMA (2015) A novel feature-selection approach based on the cuttlefish optimization algorithm for intrusion detection systems. Expert Syst Appl 42(5):2670–2679. https://doi.org/10.1016/j.eswa.2014.11.009

    Article  Google Scholar 

  13. Ekundayo O, Viriri S (2019) Facial expression recognition: A review of methods, performances and limitations. Proceed IEEE Conf Inform Comm Technol Soc (ICTAS):1–6. https://doi.org/10.1109/ICTAS.2019.8703619

  14. Gupta D, … de Albuquerque VHC (2018) Optimized cuttlefish algorithm for diagnosis of Parkinson’s disease. Cogn Syst Res 52:36–48. https://doi.org/10.1016/j.cogsys.2018.06.006

    Article  Google Scholar 

  15. Gupta N, Gupta D, Khanna A, Filho PPR, de Albuquerque VHC (2019) Evolutionary algorithms for automatic lung disease detection. Measurement 140:590–608. https://doi.org/10.1016/j.measurement.2019.02.042

    Article  Google Scholar 

  16. Hassan MM, Hussein HI, Eesa AS, Mstafa RJ (2021) Face recognition based on Gabor feature extraction followed by FastICA and LDA. Comput, Mater Continua 68(2):1637–1659. https://doi.org/10.32604/cmc.2021.016467

    Article  Google Scholar 

  17. Hu M, Yang C, Zheng Y, Wang X, He L, Ren F (2019) Facial expression recognition based on fusion features of center-symmetric local signal magnitude pattern. IEEE Access 7:118435–118445. https://doi.org/10.1109/ACCESS.2019.2936976

    Article  Google Scholar 

  18. Jin X, Sun W, Jin Z (2020) A discriminative deep association learning for facial expression recognition. Int J Mach Learn Cybern 11:779–793. https://doi.org/10.1007/s13042-019-01024-2

    Article  Google Scholar 

  19. Kar NB, Babu KS, Jena SK (2017) Face expression recognition using histograms of oriented gradients with reduced features. Proceed Int Conf Comput Vision Image Process Advan Intell Syst Comput 460:209–219. https://doi.org/10.1007/978-981-10-2107-7_19

    Article  Google Scholar 

  20. Kas M, El Merabet Y, Ruichek Y, Messoussi R (2021) New framework for person-independent facial expression recognition combining textural and shape analysis through new feature extraction approach. Inf Sci 549:200–220. https://doi.org/10.1016/j.ins.2020.10.065

    Article  MathSciNet  Google Scholar 

  21. Langner O, Dotsch R, Bijlstra G, Wigboldus DHJ, Hawk ST, van Knippenberg A (2010) Presentation and validation of the radboud faces database. Cognit Emot 24(8):1377–1388. https://doi.org/10.1080/02699930903485076

    Article  Google Scholar 

  22. Lekdioui K, Ruichek Y, Messoussi R, Chaabi Y, Touahni R (2017) Facial expression recognition using face-regions. Proceed Int Conf Advan Technol Signal Image Process (ATSIP):1–6. https://doi.org/10.1109/ATSIP.2017.8075517

  23. Li K, Jin Y, Akram MW, Han R, Chen J (2020) Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy. Vis Comput 36:391–404. https://doi.org/10.1007/s00371-019-01627-4

    Article  Google Scholar 

  24. Li M, Xu H, Huang X, Song Z, Liu X, Li X (2018) Facial expression recognition with identity and emotion joint learning. IEEE Trans Affect Comput 14(8):1–8. https://doi.org/10.1109/TAFFC.2018.2880201

    Article  Google Scholar 

  25. Liu X, Kumar BVKV, Jia P, You J (2019) Hard negative generation for identity-disentangled facial expression recognition. Pattern Recogn 88:1–12. https://doi.org/10.1016/j.patcog.2018.11.001

    Article  Google Scholar 

  26. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis:1–28

  27. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. Proceed IEEE Comput Soc Conf Comp Vision Patt Recogn - Workshops:94–101. https://doi.org/10.1109/CVPRW.2010.5543262

  28. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. Proceed IEEE Int Conf Automatic Face Gesture Recogn:200–205. https://doi.org/10.1109/AFGR.1998.670949

  29. Meena HK, Joshi SD, Sharma KK (2019) Facial expression recognition using graph signal processing on HOG. IETE J Res:1–7. https://doi.org/10.1080/03772063.2019.1565952

  30. Mlakar U, Fister I, Brest J, Potocnik B (2017) Multi-objective differential evolution for feature selection in facial expression recognition systems. Expert Syst Appl J 89:129–137. https://doi.org/10.1016/j.eswa.2017.07.037

    Article  Google Scholar 

  31. Nigam S, Singh R, Misra AK (2018) Efficient facial expression recognition using histogram of oriented gradients in wavelet domain. Multimed Tools Appl 77:28725–28747. https://doi.org/10.1007/s11042-018-6040-3

    Article  Google Scholar 

  32. Ning X, Xu S, Zong Y, Tian W, Sun L, Dong X (2020) Emotiongan: facial expression synthesis based on pre-trained generator. J Phys: Conf Series 1518. https://doi.org/10.1088/1742-6596/1518/1/012031

  33. Ning X, Nan F, Xu S, Yu L, Zhang L (2020) Multi-view frontal face image generation: a survey. Concurrency Computation. https://doi.org/10.1002/cpe.6147

    Book  Google Scholar 

  34. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29(1):51–59. https://doi.org/10.1016/0031-3203(95)00067-4

    Article  Google Scholar 

  35. Rettkowski J, Boutros A, Göhringer D (2017) HW/SW co-design of the HOG algorithm on a Xilinx Zynq SoC. J Parall Distrib Comput 109:50–62. https://doi.org/10.1016/j.jpdc.2017.05.005

    Article  Google Scholar 

  36. Revina IM, Emmanuel WRS (2018) A survey on human face expression recognition techniques. J King Saud Univ - Comput Inform Sci. https://doi.org/10.1016/j.jksuci.2018.09.002

  37. Revina IM, Emmanuel WRS (2019) Face expression recognition with the optimization based multi-SVNN classifier and the modified LDP features. J Vis Commun Image Represent 62:43–55. https://doi.org/10.1016/j.jvcir.2019.04.013

    Article  Google Scholar 

  38. Sahu B, Mishra D (2012) A novel feature selection algorithm using particle swarm optimization for cancer microarray data. Procedia Eng 38:27–31. https://doi.org/10.1016/j.proeng.2012.06.005

    Article  Google Scholar 

  39. Salmam FZ, Madani A, Kissi M (2016) Facial expression recognition using decision trees. Proceed Int Conf Comput Graphics, Imaging Visualization (CGiV):125–130. https://doi.org/10.1109/CGiV.2016.33

  40. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650. https://doi.org/10.1109/TIP.2010.2042645

    Article  MathSciNet  MATH  Google Scholar 

  41. Tian Y, Kanade T, and Cohn JF (2011) “Facial expression recognition,” In: Li S., Jain A. (eds) Handbook of Face Recognition, pp. 487–519, https://doi.org/10.1007/978-0-85729-932-1_19.

  42. Vedantham R, Reddy ES (2020) A robust feature extraction with optimized DBN-SMO for facial expression recognition. Multimed Tools Appl 79:21487–21512. https://doi.org/10.1007/s11042-020-08901-x

    Article  Google Scholar 

  43. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proceed 2001 IEEE Comput Soc Conf Comput Vision Patt Recogn (CVPR):511–518. https://doi.org/10.1109/CVPR.2001.990517

  44. Wang F, Ying G, Zhang C, Lv J, Chen S (2019) Facial expression recognition from image based on hybrid features understanding. J Vis Commun Image Represent 59:84–88. https://doi.org/10.1016/j.jvcir.2018.11.010

    Article  Google Scholar 

  45. Wang H, Wei S, Fang B (2020) Facial expression recognition using iterative fusion of MO-HOG and deep features. J Supercomput 76:3211–3221. https://doi.org/10.1007/s11227-018-2554-8Facial

    Article  Google Scholar 

  46. Wei W, Jia Q, Feng Y, Chen G, Chu M (2020) Multi-modal facial expression feature based on deep-neural networks. J Multimodal User Interfaces 14:17–23. https://doi.org/10.1007/s12193-019-00308-9

    Article  Google Scholar 

  47. Yan Y, Zhang Z, Chen S, Wang H (2020) Low-resolution facial expression recognition: a filter learning perspective. Signal Process 169:1–12. https://doi.org/10.1016/j.sigpro.2019.107370

    Article  Google Scholar 

  48. Yang Z, Wang H, Han Y, Zhu X (2018) Discriminative multi-task multi-view feature selection and fusion for multimedia analysis. Multimed Tools Appl 77:3431–3453. https://doi.org/10.1007/s11042-017-5165-0

    Article  Google Scholar 

  49. Yolcu G, … Bunyak F (2019) Facial expression recognition for monitoring neurological disorders based on convolutional neural network. Multimed Tools Appl 78:31581–31603. https://doi.org/10.1007/s11042-019-07959-6

    Article  Google Scholar 

  50. Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie AM (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273:643–649. https://doi.org/10.1016/j.neucom.2017.08.043

    Article  Google Scholar 

  51. Zheng H, … Lv H (2020) Discriminative deep multi-task learning for facial expression recognition. Inf Sci 533:60–71. https://doi.org/10.1016/j.ins.2020.04.041

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Faculty of Science, University of Zakho, Duhok, Kurdistan Region, 42002, Iraq

    Haval I. Hussein, Hivi Ismat Dino, Ramadhan J. Mstafa & Masoud Muhammed Hassan

  2. Department of Computer Science, College of Science, Nawroz University, Duhok, Kurdistan Region, 42001, Iraq

    Ramadhan J. Mstafa

Authors
  1. Haval I. Hussein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hivi Ismat Dino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramadhan J. Mstafa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Masoud Muhammed Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramadhan J. Mstafa.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hussein, H.I., Dino, H.I., Mstafa, R.J. et al. Person-independent facial expression recognition based on the fusion of HOG descriptor and cuttlefish algorithm. Multimed Tools Appl 81, 11563–11586 (2022). https://doi.org/10.1007/s11042-022-12438-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12438-6

Keywords

Search

Navigation