Skip to main content
SpringerLink
Log in

Facial expression recognition using a combination of multiple facial features and support vector machine

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper presents a novel facial expression recognition (FER) technique based on support vector machine (SVM) for the FER. Here it is called the FERS technique. First, the FERS technique develops a face detection method that combines the Haar-like features method with the self-quotient image (SQI) filter. As a result, the FERS technique possesses better detection rate because the face detection method gets more accurate in locating face regions of an image. The main reason is that the SQI filter can overcome the insufficient light and shade light. Subsequently, three schemes, the angular radial transform (ART), the discrete cosine transform (DCT) and the Gabor filter (GF), are simultaneously employed in the design of the feature extraction for facial expression in the FERS technique. More specifically, they are employed in constructing a set of training patterns for the training of an SVM. The FERS technique then exploits the trained SVM to recognize the facial expression for a query face image. Finally, experimental results show that the recognition performance of the FERS technique can be better than that of other existing methods under consideration in the paper.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  • Al-Shabi M, Cheah WP, Connie T (2016) Facial expression recognition using a hybrid CNN–SIFT aggregator. https://arxiv.org/pdf/1608.02833

  • Burges CJC (1998) A Tutorial on support vector machines for pattern recognition. Data Min Knowl Disc 2:121–167

    Article  Google Scholar 

  • Chai Z, Vazquez MH, He R, Sun Z, Tan T (2014) Explore semantic pixel sets based local patterns with information entropy for face recognition. EURASIP J Image Video Process 26:1–11

    Google Scholar 

  • Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:1–27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm

  • Chen HY, Huang CL, Fu CM (2008) Hybrid-boost learning for multi-pose face detection and facial expression recognition. Pattern Recogn 41:1173–1185

    Article  MATH  Google Scholar 

  • Cheng K, Chen Y, Zhan Y (2008) A new approach for facial expression recognition based on Burial Markov model. In: Proceedings of the fifth international conference on fuzzy systems and knowledge discovery, pp 3–9

  • Fang J, Qiu G (2003) Human face detection using angular radial transform and support vector machines. In: Proceedings of international conference on image processing, pp 669–672

  • Fellenz WA, Taylor JG, Tsapatsoulis N, Kollias S (1999) Comparing template-based, feature-based and supervised classification of facial expressions from static images. In: Proceedings of circuits systems, communications and computers, pp 5331–5336

  • Haykin S (1995) Neural networks: a comprehensive foundation. Macmillan College Publishing Company, New York

    MATH  Google Scholar 

  • JAFFE (2008) The Japanese female facial expression (JAFFE) database. http://www.kasrl.org/jaffe.html

  • Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition, pp 46–53

  • Kharat GU, Dudul SV (2008) Human emotion recognition system using optimally designed SVM with different facial feature extraction techniques. WSEAS Trans Comput 7:650–659

    Google Scholar 

  • Kim DH, Jung SU, Chung MJ (2008) Extension of cascaded simple feature based face detection to facial expression recognition. Pattern Recogn Lett 29:1621–1631

    Article  Google Scholar 

  • Kotsia I, Zafeiriou S, Pitas I (2008) Texture and shape information fusion for facial expression and facial action unit recognition. Pattern Recogn 41:833–851

    Article  MATH  Google Scholar 

  • Lajevardi SM, Hussain ZM (2012) Automatic facial expression recognition: feature extraction and selection. SIViP 6:159–169

    Article  Google Scholar 

  • Lee JJ, Uddin MZ, Kim TS (2008) Spatiotemporal human facial expression recognition using fisher independent component analysis and Hidden Markov Model. In: Proceedings of the 30th annual international IEEE EMBS conference, Vancouver, British Columbia, pp 2546–2549

  • Liu W, Wang Z (2006) Facial expression recognition based on fusion of multiple Gabor features. In: Proceedings of the 18th international conference on pattern recognition, pp 536–539

  • Loh MP, Wong YP, Wong CO (2006) Facial expression recognition for e-Learning systems using Gabor wavelet & neural network. In: Proceedings of the sixth international conference on advanced learning technologies, pp 523–525

  • Mita T, Kaneko T, Hori O (2005) Joint haar-like features for face detection. In: Proceedings of the tenth IEEE international conference on computer vision, pp 1619–1626

  • OpenCV (2015) http://opencv.org/

  • Rowley HA, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20:23–38

    Article  Google Scholar 

  • Saeed I, Wang A, Senaratne R, Halgamuge S (2007) Using the active appearance model to detect driver fatigue. In: Proceedings of the third international conference on information and automation for sustainability, pp 124–128

  • Shih P, Liu C (2006) Face detection using discriminating feature analysis and support vector machine. Pattern Recogn 39:260–276

    Article  Google Scholar 

  • Sun X, Xu H, Zhao C, Yang J (2008) Facial expression recognition based on histogram sequence of local Gabor binary patterns. In: Proceedings of IEEE conference on cybernetics and intelligent systems, pp 158–163

  • Tsai HH, Lai YS, Lo SC (2013) A zero-watermark scheme with geometrical invariants using SVM and PSO against geometrical attacks for image protection. J Syst Softw 86:335–348

    Article  Google Scholar 

  • Tsai HH, Chang BM, Liu SH (2014) Rotation invariant image retrieval using PSO and SVR. Appl Soft Comput 17:127–139

    Article  Google Scholar 

  • Tsai HH, Cheng JS (2005) Adaptive signal-dependent audio watermarking based on human auditory system and neural networks. Appl Intell 23:191–206

    Article  Google Scholar 

  • VenkataRamiReddy Ch, Kishore KVK, Bhattacharyya D, Kim TH (2014) Multi-feature fusion based facial expression classification using DLBP and DCT. Int J Softw Eng Appl 8:55–68

    Google Scholar 

  • Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 511–518

  • Wallhoff F (2005) Facial expressions and emotion database. http://www.mmk.ei.tum.de/~waf/fgnet/

  • Wei W, Jia Q (2016) Weighted feature Gaussian kernel SVM for emotion recognition. Comput Intell Neurosci 2016:1–7

    Article  Google Scholar 

  • Wong JJ, Cho SY (2009) A local experts organization model with application to face emotion recognition. Expert Syst Appl 36:804–819

    Article  Google Scholar 

  • Yang MH, Kriegman D, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24:34–58

    Article  Google Scholar 

  • Yang P, Liu Q, Metaxas DN (2009) Boosting encoded dynamic features for facial expression recognition. Pattern Recogn Lett 30:132–139

    Article  Google Scholar 

  • Zhou G, Zhan Y, Zhang J (2006) Facial expression recognition based on selective feature extraction. In: Proceedings of the sixth international conference on intelligent systems design and applications, pp 412–417

Download references

Acknowledgements

Authors would like to thank Ministry of Science and Technology of Taiwan, ROC, for financially supporting this research under Contract Nos. NSC 98-2511-S-150-002 and MOST 104-2221-E-150-010.

Author information

Authors and Affiliations

  1. Department of Information Management, National Formosa University, Huwei, 632, Yulin, Taiwan, ROC

    Hung-Hsu Tsai & Yi-Cheng Chang

Authors
  1. Hung-Hsu Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Cheng Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hung-Hsu Tsai.

Ethics declarations

Conflict of interest

Author Prof. Tsai has received research grants from Company Taiwan. OR if no conflict exists: Author Prof. Tsai declares that he has no conflict of interest. Author Mr. Chang declares that he has no conflict of interest.

Ethical approval

All applicable international, national, and/or institutional guidelines for the care and use of animals were followed.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsai, HH., Chang, YC. Facial expression recognition using a combination of multiple facial features and support vector machine. Soft Comput 22, 4389–4405 (2018). https://doi.org/10.1007/s00500-017-2634-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-017-2634-3

Keywords

Search

Navigation