Skip to main content

Advertisement

SpringerLink
Log in

Facial expression recognition using modified Viola-John’s algorithm and KNN classifier

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the way of communication, facial expression act as non-verbal communication and play an important role in social interaction by providing some contextual information. Facial expressions also express human’s inner emotional state, which is very effective for communication with the actual emotions. In this paper, an algorithm has been proposed to detect the face and facial parts more accurate to the Viola – John’s algorithm, and a fast-tracking algorithm for face tracking in real-time scenarios. The fusion of the facial features is used for feature extraction and comparative work on the several classifiers has been presented. In this approach, the images were acquired and seven significant facial parts from the image were cropped, then extract and store the features of several facial expressions. Finally, the expressions in the images were recognized using the classifiers. The algorithm was tested on four kinds of database and achieved accurate performance through the designed system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Ahmed M, Laskar RH (2019) Eye detection and localization in a facial image based on the partial geometric shape of iris and eyelid under practical scenarios. J Electron Imaging 28(3):033009. https://doi.org/10.1117/1.JEI.28.3.033009

    Article  Google Scholar 

  2. Bartlett MS, Littleworth G, Fasel I, Movellan JR (2003) Real-time face detection and facial expression recognition: development and applications to human-computer interaction. In: 2003 conference on computer vision and pattern recognition workshop, vol. 5, pp. 53–53. IEEE

  3. Bartlett MS, Littleworth G, Fasel I, Movellan R (2003) Real-time face detection and facial expression recognition: development and application to human-computer interaction. In: Proceedings CVPR workshop on computer vision and pattern recognition for human-computer interaction, vol 5

  4. Buciu I, Pitas I (2004) Application of non-negative and local non negative matrix factorization to facial expression recognition. In: Proceeding of ICPR, pp 288–291, Cambridge, UK, Aug. 23–26, 2004

  5. Chen J, Chen Z, Chi Z, Hong F (2014) Facial expression recognition based on facial components detection and hog features. International workshops on electrical and computer engineering subfields, pp 884–888

  6. Chibelushi CC, Bourel F (2003) Facial expression recognition: a brief tutorial overview. CVonline: On-Line Compendium of Computer Vision 9

  7. CK+ database: http://www.consortium.ri.cmu.edu/ckagree/. Accessed 10 June 2017

  8. Cristinacce D, Cootes TF, Scott IM (2004) A multi-stage approach to facial feature detection. In BMVC, vol 1, pp 277–286

  9. Dumas M (2001) Emotional expression recognition using support vector machines. In: Proceedings of international conference on multimodal interfaces

  10. Ekman P, Sorenson ER, Friesen WV (1969) Pancultural elements in Facial displays of emotion. Science 164(3875):86–88

    Article  Google Scholar 

  11. Ekweariri AN, Yurtkan K (2017) Facial expression recognition using enhanced local binary patterns. In: 2017 9th international conference on Computational Intelligence and Communication Networks (CICN), pp 43–47. IEEE

  12. El Maghraby A, Abdalla M, Enany O, El Nahas MY (2013) Hybrid face detection system using a combination of Viola-Jones method and skin detection. Int J Comput Appl 71(6):15–22

    Google Scholar 

  13. Feng X, Pietikainen M, Hadid A (2005) Facial expression recognition with local binary patterns and linear programming. Pattern Recognition And Image Analysis C/C of Raspoznavaniye Obrazov I Analiz Izobrazhenii 15(2):546. https://zenodo.org/record/3451524

  14. JAFFE database: http://www.kasrl.org/jaffe.html/. Accessed 10 June 2017

  15. Joho H, Staiano J, Sebe N, Jose JM (2011) Looking at the viewer:analyzing facial activity to detect personal highlights of multimedia contents. Multimed Tools Appl 51(2):505–523

    Article  Google Scholar 

  16. Kasim S, Hassan R, Zaini NH, Ahmad AS, Ramli AA, Saedudin RR (2017) A study on facial expression recognition using local binary pattern. Int J Adv Sci EngInf Technol 7(5):1621–1626

    Article  Google Scholar 

  17. Kotsia I, Pitas I (2007) Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans Image Process 16(1):172–187

    Article  MathSciNet  Google Scholar 

  18. Kotsia I, Buciu I, Pitas I (July 2008) An analysis of facial expression recognition under partial facial image occlusion. Image Vis Comput 26(7):1052–1067

    Article  Google Scholar 

  19. Kumari J, Rajesh R, Pooja KM (2015) Facial expression recognition: a survey. Proc Comput Sci 58:486–491

    Article  Google Scholar 

  20. Lemaire P, Amor BB, Ardabilian M, Chen L, Daoudi M (2011) Fully automatic 3D facial expression recognition using a region-based approach. In: Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding, pp 53–58. ACM

  21. Marcolin F et al (2017) 3D geometry-based automatic landmark localization in the presence of facial occlusions. Multimed Tools Appl 77:1–29

    Google Scholar 

  22. Mehrabian A (1968) Communication without words. Psychol Today 2(4):53–56

    Google Scholar 

  23. Michel P, Kaliouby R (2003) Real-time facial expression recognition in video using support vector machines. In: Proceedings ogf 5th International Conference of Multimodal Interfaces, Vancouver, BC, Canada, pp 258–264

  24. MMI database: https://mmifacedb.eu/. Accessed 10 June 2017

  25. Moore S, Bowden R (2011) Local binary patterns for multi-view facial expression recognition. Comput Vis Image Underst 115(4):541–558

    Article  Google Scholar 

  26. Murthy GRS, Jadon RS (2009) Effectiveness of Eigenspaces for facial expressions recognition. Int J Comput Theory Eng 1(5):638–642

    Article  Google Scholar 

  27. Pantic M, Patras I (2006) Dynamics of facial expression: recognition of facial actions and their temporal segments form face profile image sequences. IEEE Trans Syst Man Cybern B 36(2):433–449

    Article  Google Scholar 

  28. Pantic M, Rothkrantz JM (2004) Facial action recognition for facial expression analysis from static face images. IEEE Trans Syst Man arid Cybern B 34(3):1449–1461

    Article  Google Scholar 

  29. Punitha A, Geetha MK (2013) Texture-based emotion recognition from facial expressions using a support vector machine. Int J Comput Appl

  30. Sandbach G, Zafeiriou S, Pantic M, Yin L (2012) Static and dynamic 3D facial expression recognition: a comprehensive survey. Image Vis Comput 30(10):683–697

    Article  Google Scholar 

  31. Sarode N, Bhatia S (2010) Facial expression recognition. Int J Comput Sci Eng 2(5):1552–1557

    Google Scholar 

  32. Sawardekara S, Prof. Sowmiya Raksha Naik (2008) Facial expression recognition using efficient LBP and CNN,e-ISSN: 2395-0056, Volume: 05(June 2018), p-ISSN: 2395-0072

  33. Sobottka K, Pitas I (1996) Face localization and facial feature extraction based on shape and color information. In: Proceedings of 3rd IEEE international conference on image processing, vol 3, pp 483–486. IEEE

  34. The LNMIIT database: http://anshyadav.cf/the-lnmiit-datast/. Accessed 10 June 2017

  35. Tian Y, Kanade T, Cohn J (2001) Recognizing action units for facial expression analysis. IEEE Trans Pattern Analysis Mach Intell 23(2):97–115

    Article  Google Scholar 

  36. Tornincasa S et al (2019) 3D facial action units and expression recognition using a crisp logic. Comput Aided Des Appl 16(2):256–268

    Article  Google Scholar 

  37. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, vol 1, pp I-I. IEEE

  38. Vupputuri A, Meher S (2015) Facial expression recognition using local binary patterns and kullback leibler divergence. In: International Conference on Communications and Signal Processing (ICCSP), pp 0349–0353. IEEE

  39. Wang J, Yin L (2007) Static topographic modeling for facial expression recognition and analysis. Comput Vis Image Underst 108:19–34

    Article  Google Scholar 

  40. Yadav KS, Singha J, Laskar RH (2019) Facial expression recognition using facial features detection using the fusion of classifiers in the real-time scenario. In: Proceedings of 4th IEEE international conference on Information Systems and Computer Networks (ISCON)

  41. Yang S, Luo P, Loy C-C, Tang X (2015) From facial parts responses to face detection: A deep learning approach. In: Proceedings of the IEEE international conference on computer vision, pp 3676–3684

  42. Yang X, Li M, Zhao S (2017) Facial expression pecognition algorithm based on CNN and LBP feature fusion. In: Proceedings of the 2017 international conference on robotics and artificial intelligence, pp 33–38. ACM

  43. Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Article  Google Scholar 

  44. Zhang Z, Lyons M, Schuster M, Akamatsu S (1998) Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron. In: Proceedings third IEEE international conference on automatic face and gesture recognition, pp 454–459. IEEE

Download references

Author information

Authors and Affiliations

  1. Department of ECE, National Institute of Technology, Silchar, Assam, India

    Kuldeep Singh Yadav

  2. Department of ECE, The LNM Institute of Information and Technology, Jaipur, India

    Joyeeta Singha

Authors
  1. Kuldeep Singh Yadav
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joyeeta Singha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kuldeep Singh Yadav.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yadav, K.S., Singha, J. Facial expression recognition using modified Viola-John’s algorithm and KNN classifier. Multimed Tools Appl 79, 13089–13107 (2020). https://doi.org/10.1007/s11042-019-08443-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08443-x

Keywords

Search

Navigation