Skip to main content
SpringerLink
Log in

Chess pattern with different weighting schemes for person independent facial expression recognition

  • 1169: Interdisciplinary Forensics: Government, Academia and Industry Interaction
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Facial expressions are an important form of non-verbal communication as they can depict the internal mood and emotions of an individual. In Automatic Facial Expression Recognition (AFER) system, the main task is to extract features that can best classify the expressions into various categories. The existing local based approaches fail in obtaining different feature values for edge, corner and flat image regions. In this work, Chess Pattern, a game based feature descriptor is proposed based on the movements of the chessmen such as Rook, Bishop and Knight and also the combinations of Rook_Knight, Rook_Bishop and Knight_Bishop are considered for feature extraction. Apart from using binary weights, new weighting schemes such as fibonacci weights, prime weights, natural weights, squares weights are also proposed for facial feature extraction. The Chess Pattern with different weights is applied on JAFFE, MUG, TFEID, KDEF, WSEFEP and ADFES datasets for six and seven expressions. Also, for SFEW, TFEID and ADFES datasets the experiments are conducted for seven, eight and ten expressions respectively. The experiments are conducted in person independent setup, in order to simulate a real world scenario. The comparison results shows the efficiency of the proposed approach when compared to other existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Agarwal S, Santra B, Mukherjee DP (2018) Anubhav: recognizing emotions through facial expression. Vis Comput 34(2):177–191

    Article  Google Scholar 

  2. Aghamaleki JA, Chenarlogh VA (2019) Multi-stream cnn for facial expression recognition in limited training data. Multimed Tools Appl 78(16):22861–22882

    Article  Google Scholar 

  3. Aifanti N, Papachristou C, Delopoulos A (2010) The mug facial expression database. In: 11th International Workshop on Image Analysis for Multimedia Interactive Services WIAMIS 10. IEEE, pp 1–4

  4. Alphonse AS, Shankar K, Rakkini MJ, Ananthakrishnan S, Athisayamani S, Singh AR, Gobi R (2020) A multi-scale and rotation-invariant phase pattern (mripp) and a stack of restricted boltzmann machine (rbm) with preprocessing for facial expression classification. J Ambient Intell Humaniz Comput 12(3):3447–3463

    Article  Google Scholar 

  5. Ashir AM, Eleyan A (2017) Facial expression recognition based on image pyramid and single-branch decision tree. SIViP 11(6):1017–1024

    Article  Google Scholar 

  6. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  7. Burges CJ (1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2(2):121–167

    Article  Google Scholar 

  8. Chan T-H, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2015) Pcanet: A simple deep learning baseline for image classification. IEEE Trans Image Process 24(12):5017–5032

    Article  MathSciNet  MATH  Google Scholar 

  9. Chen L-F, Yen Y-S (2007) Taiwanese facial expression image database. Institute of Brain Science, National Yang-Ming University, Taipei, Taiwan, Brain Mapping Laboratory

    Google Scholar 

  10. Dhall A, Goecke R, Lucey S, Gedeon T (2011) Static facial expressions in tough conditions: Data, evaluation protocol and benchmark. In: 1st IEEE International Workshop on Benchmarking Facial Image Analysis Technologies BeFIT, ICCV2011

  11. Etemad K, Chellappa R (1997) Discriminant analysis for recognition of human face images. Josa A 14(8):1724–1733

    Article  Google Scholar 

  12. Fussell SR (2002) The verbal communication of emotion: Introduction and overview. In: The verbal communication of emotions. Psychology Press, pp 9–24

  13. Goeleven E, De Raedt R, Leyman L, Verschuere B (2008) The karolinska directed emotional faces: a validation study. Cognit Emot 22(6):1094–1118

    Article  Google Scholar 

  14. Happy S, Routray A (2014) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6(1):1–12

    Article  Google Scholar 

  15. Hong H, Neven H, Von der Malsburg C (1998) Online facial expression recognition based on personalized galleries. In: Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, pp 354–359

  16. Hu M, Zheng Y, Yang C, Wang X, He L, Ren F (2019) Facial expression recognition using fusion features based on center-symmetric local octonary pattern. IEEE Access 7:29882–29890

    Article  Google Scholar 

  17. Iqbal MTB, Abdullah-Al-Wadud M, Ryu B, Makhmudkhujaev F, Chae O (2020) Facial expression recognition with neighborhood-aware edge directional pattern (nedp). IEEE Trans Affect Comput 11(1):125–137

    Article  Google Scholar 

  18. Iqbal MTB, Ryu B, Song G, Kim J, Makhmudkhujaev F, Chae O (2016) Exploring positional ternary pattern (ptp) for conventional facial expression recognition from static images. Korea Comput Congress pp 853–855

  19. Jabid T, Kabir MH, Chae O (2010) Robust facial expression recognition based on local directional pattern. ETRI J 32(5):784–794

    Article  Google Scholar 

  20. Jabid T, Kabir MH, Chae O (2010) Gender classification using local directional pattern (ldp). In: 2010 20th International Conference on Pattern Recognition. IEEE, pp 2162–2165

  21. Kartheek MN, Prasad MVNK, Bhukya R (2020) Local optimal oriented pattern for person independent facial expression recognition. In: Twelfth International Conference on Machine Vision (ICMV 2019), vol. 11433. International Society for Optics and Photonics, p 114330R1–8

  22. Kola DGR, Samayamantula SK (2021) Facial expression recognition using singular values and wavelet-based lgc-hd operator. IET Biom 10(2):207–218

    Article  Google Scholar 

  23. Kotsia I, Pitas I (2006) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187

    Article  MathSciNet  Google Scholar 

  24. Lai C-C, Ko C-H (2014) Facial expression recognition based on two-stage features extraction. Optik-International Journal for Light and Electron Optics 125(22):6678–6680

    Article  Google Scholar 

  25. Lee JRH, Wang L, Wong A (2020) Emotionnet nano: An efficient deep convolutional neural network design for real-time facial expression recognition. arXiv preprint arXiv:2006.15759

  26. Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition, pp 200–205

  27. Makhmudkhujaev F, Abdullah-Al-Wadud M, Iqbal MTB, Ryu B, Chae O (2019) Facial expression recognition with local prominent directional pattern. Signal Process Image Commun 74:1–12

    Article  Google Scholar 

  28. Makhmudkhujaev F, Iqbal MTB, Ryu B, Chae O (2019) Local directional-structural pattern for person-independent facial expression recognition. Turk J Electr Eng Comput Sci 27(1):516–531

    Article  Google Scholar 

  29. Mandal M, Verma M, Mathur S, Vipparthi SK, Murala S, Kumar DK (2019) Regional adaptive affinitive patterns (radap) with logical operators for facial expression recognition. IET Image Process 13(5):850–861

    Article  Google Scholar 

  30. Minaee S, Minaei M, Abdolrashidi A (2021) Deep-emotion: Facial expression recognition using attentional convolutional network. Sensors 21(9):3046

    Article  Google Scholar 

  31. Olszanowski M, Pochwatko G, Kuklinski K, Scibor-Rylski M, Lewinski P, Ohme RK (2015) Warsaw set of emotional facial expression pictures: a validation study of facial display photographs. Front Psychol 5:1–8

    Article  Google Scholar 

  32. Pantic M, Patras I (2006) Dynamics of facial expression: recognition of facial actions and their temporal segments from face profile image sequences. IEEE Trans Syst Man Cybern B Cybern 36(2):433–449

    Article  Google Scholar 

  33. Reddy PCS, Rao PVP, Reddy PKK, Sridhar M (2019) Motif shape primitives on fibonacci weighted neighborhood pattern for age classification. In: Soft Computing and Signal Processing. Springer, pp 273–280

  34. Revina IM, Emmanuel WS (2018) A survey on human face expression recognition techniques. Journal of King Saud University-Computer and Information Sciences 1(5):1–9

    Google Scholar 

  35. Revina IM, Emmanuel WS (2019) Mdtp: a novel multi-directional triangles pattern for face expression recognition. Multimed Tools Appl 78(18):26223–26238

    Article  Google Scholar 

  36. Rivera AR, Castillo JR, Chae O (2015) Local directional texture pattern image descriptor. Pattern Recogn Lett 51:94–100

    Article  Google Scholar 

  37. Rivera AR, Castillo JR, Chae OO (2012) Local directional number pattern for face analysis: Face and expression recognition. IEEE Trans Image Process 22(5):1740–1752

    Article  MathSciNet  MATH  Google Scholar 

  38. Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26(12):6006–6018

    Article  MathSciNet  Google Scholar 

  39. Sadeghi H, Raie A-A (2019) Human vision inspired feature extraction for facial expression recognition. Multimed Tools Appl 78(21):30335–30353

    Article  Google Scholar 

  40. Sebe N, Lew MS, Sun Y, Cohen I, Gevers T, Huang TS (2007) Authentic facial expression analysis. Image Vis Comput 25(12):1856–1863

    Article  Google Scholar 

  41. Sen D, Datta S, Balasubramanian R (2019) Facial emotion classification using concatenated geometric and textural features. Multimed Tools Appl 78(8):10287–10323

    Article  Google Scholar 

  42. Shabat AM, Tapamo J-R (2018) Angled local directional pattern for texture analysis with an application to facial expression recognition. IET Comput Vis 12(5):603–608

    Article  Google Scholar 

  43. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: A comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  44. Sun Z, Chiong R, Hu Z-P (2020) Self-adaptive feature learning based on a priori knowledge for facial expression recognition. Knowl-Based Syst 204:106124

  45. Sun Z, Hu Z-P, Wang M, Zhao S-H (2017) Individual-free representation-based classification for facial expression recognition. SIViP 11(4):597–604

    Article  Google Scholar 

  46. Sun Z, Hu Z-P, Wang M, Zhao S-H (2019) Dictionary learning feature space via sparse representation classification for facial expression recognition. Artif Intell Rev 51(1):1–18

    Article  Google Scholar 

  47. Tian YL, Kanade T, Cohn JF (2005) Facial expression analysis. In: Handbook of face recognition. Springer, pp 247–275

  48. Tuncer T, Dogan S, Abdar M, Plawiak P (2020) A novel facial image recognition method based on perceptual hash using quintet triple binary pattern. Multimed Tools Appl 79(39):29573–29593

    Article  Google Scholar 

  49. Tuncer T, Dogan S, Ataman V (2019) A novel and accurate chess pattern for automated texture classification. Physica A 536:122584

  50. Turk M, Pentland A (1991) Face recognition using eigenfaces. In: Proceedings. 1991 IEEE computer society conference on computer vision and pattern recognition, pp 586–587

  51. Van Der Schalk J, Hawk ST, Fischer AH, Doosje B (2011) Moving faces, looking places: validation of the amsterdam dynamic facial expression set (adfes). Emotion 11(4):907–920

    Article  Google Scholar 

  52. Verma M, Vipparthi SK, Singh G (2019) Hinet: Hybrid inherited feature learning network for facial expression recognition. IEEE Letters of the Computer Society 2(4):36–39

    Article  Google Scholar 

  53. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  54. Wang Y, Li M, Wan X, Zhang C, Wang Y (2020) Multiparameter space decision voting and fusion features for facial expression recognition. Comput Intell Neurosci

  55. Wu B-F, Lin C-H (2018) Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE Access 6:12451–12461

    Article  Google Scholar 

  56. Yan Y, Zhang Z, Chen S, Wang H (2020) Low-resolution facial expression recognition: a filter learning perspective. Signal Process 169:107370

  57. Yang S, Bhanu B (2011) Facial expression recognition using emotion avatar image. In: Face and Gesture 2011. IEEE, pp 866–871

  58. Zhou H, Wang R, Wang C (2008) A novel extended local-binary-pattern operator for texture analysis. Inf Sci 178(22):4314–4325

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Development and Research in Banking Technology, Hyderabad, India

    Mukku Nisanth Kartheek & Munaga V. N. K. Prasad

  2. National Institute of Technology, Warangal, India

    Mukku Nisanth Kartheek & Raju Bhukya

Authors
  1. Mukku Nisanth Kartheek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Munaga V. N. K. Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raju Bhukya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mukku Nisanth Kartheek.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kartheek, M.N., Prasad, M.V.N.K. & Bhukya, R. Chess pattern with different weighting schemes for person independent facial expression recognition. Multimed Tools Appl 81, 22833–22866 (2022). https://doi.org/10.1007/s11042-021-11270-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11270-8

Keywords

Search

Navigation