Skip to main content

Advertisement

SpringerLink
Log in

Automated facial expression recognition based on histograms of oriented gradient feature vector differences

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

This article proposes an efficient automated method for facial expression recognition based on the histogram of oriented gradient (HOG) descriptor. This subject-independent method was designed for recognizing six prototyping emotions. It recognizes emotions by calculating differences on a level of feature descriptors between a neutral expression and a peak expression of an observed person. The parameters for the HOG descriptor were determined by using a genetic algorithm. Support vector machines (SVM) were applied during the recognition phase, whereat one SVM classifier was trained for one emotion. Each classifier was trained using difference vectors obtained by subtraction of HOG feature vectors calculated for the neutral and apex emotion subjects image. The proposed method was tested by using a leave-one-subject-out validation strategy for 106 subjects on 1232 images from the Cohn Kanade, and for 10 subjects on 192 images from the JAFFE database. A mean recognition rate of 95.64 % was obtained using the Cohn Kanade database, which is higher than the recognition rates for almost all other single-image- or video-based methods for facial emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bourel, F., Chibelushi, C.C., Low, A.A.: Recognition of facial expressions in the presence of occlusion. In: BMVC, pp. 1–10. Citeseer (2001)

  2. Buciu, I., Kotropoulos, C., Pitas, I.: Comparison of ica approaches for facial expression recognition. Signal Image Video Process. 3(4), 345–361 (2009)

    Article  MATH  Google Scholar 

  3. Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2:27:1–27:27 (2011). Software available at url http://www.csie.ntu.edu.tw/cjlin/libsvm

  4. Cohen, I., Sebe, N., Garg, A., Chen, L.S., Huang, T.S.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91(1), 160–187 (2003)

    Article  Google Scholar 

  5. Corcoran, A.L., Sen, S.: Using real-valued genetic algorithms to evolve rule sets for classification. In: Proceedings of the First IEEE Conference on Evolutionary Computation, 1994. IEEE World Congress on Computational Intelligence, pp. 120–124. IEEE (1994)

  6. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Schmid, C., Soatto, S., Tomasi, C., (eds.) International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 886–893. INRIA Rhône-Alpes, ZIRST-655, av. de l’Europe, Montbonnot-38334 (2005)

  7. Donia, M., Youssif, A., Hashad, A.: Spontaneous facial expression recognition based on histogram of oriented gradients descriptor. Comput. Inf. Sci. 7(3), 31–37 (2014)

    Google Scholar 

  8. Edwards, G.J., Cootes, T.F., Taylor, C.J.: Face recognition using active appearance models. In: Computer Vision ECCV98, pp. 581–595. Springer, Berlin (1998)

  9. Ekman, P., Friesen, W.V.: Measuring facial movement. Environ. Psychol. Nonverbal Behav. 1(1), 56–75 (1976)

    Article  Google Scholar 

  10. El Kaliouby, R., Robinson, P.: Real-time inference of complex mental states from facial expressions and head gestures. In: Real-Time Vision for Human-Computer Interaction, pp. 181–200. Springer, US (2005)

  11. Fanelli, G., Yao, A., Noel, P.-L., Gall, J., Van Gool, L.: Hough forest-based facial expression recognition from video sequences. In: Trends and Topics in Computer Vision, pp. 195–206. Springer (2012)

  12. Fang, H., Mac Parthaláin, N.M., Aubrey, A.J., Tam, G., Borgo, R., Rosin, P., Grant, P., David, M., Chen, M.: Facial expression recognition in dynamic sequences: an integrated approach. Pattern Recogn. 47(3), 1271–1281 (2014)

    Article  Google Scholar 

  13. Geetha, P., Narayanan, V.: Evolutionary computational method of facial expression analysis for content-based video retrieval using 2-dimensional cellular automata. (2010). arXiv preprint arXiv:1009.1983

  14. Gritti, T., Shan, C., Jeanne, V., Braspenning, R.: Local features based facial expression recognition with face registration errors. In: FG’08. 8th IEEE International Conference on Automatic Face and Gesture Recognition, 2008, pp. 1–8. IEEE (2008)

  15. Kahou, S.E., Froumenty, P., Pal, C.: Facial expression analysis based on high dimensional binary features. In: Computer Vision-ECCV 2014 Workshops, pp. 135–147. Springer, Switzerland (2015)

  16. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000, pp. 46–53. IEEE (2000)

  17. Kotsia, I., Buciu, I., Pitas, I.: An analysis of facial expression recognition under partial facial image occlusion. Image Vis. Comput. 26(7), 1052–1067 (2008)

    Article  Google Scholar 

  18. Lajevardi, S., Hussain, Z.: Automatic facial expression recognition: feature extraction and selection. Signal Image Video Process. 6(1), 159–169 (2012)

    Article  Google Scholar 

  19. Long, F., Wu, T., Movellan, J.R., Bartlett, M.S., Littlewort, G.: Learning spatiotemporal features by using independent component analysis with application to facial expression recognition. Neurocomputing 93, 126–132 (2012)

    Article  Google Scholar 

  20. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101. IEEE (2010)

  21. Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with gabor wavelets. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, 1998, pp. 200–205. IEEE (1998)

  22. Michel, P., Kaliouby, R.E.: Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th International Conference on Multimodal Interfaces, pp. 258–264. ACM (2003)

  23. Orrite, C., Gañán, A., Rogez, G.: Hog-based decision tree for facial expression classification. In: Pattern Recognition and Image Analysis, pp. 176–183. Springer, Berlin (2009)

  24. Padgett, C., Cottrell, G.W.: Representing face images for emotion classification. Adv. Neural Inf. Process. Syst. 9, 894– 900 (1997)

    Google Scholar 

  25. Pantic, M., Rothkrantz, L.: Case-based reasoning for user-profiled recognition of emotions from face images. In: ICME’04. 2004 IEEE International Conference on Multimedia and Expo, vol. 1, pp. 391–394. IEEE (2004)

  26. Saragih, J.M., Lucey, S., Cohn, J.F.: Face alignment through subspace constrained mean-shifts. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1034–1041. IEEE (2009)

  27. Sebe, N., Lew, M.S., Sun, Y., Cohen, I., Gevers, T., Huang, T.S.: Authentic facial expression analysis. Image Vis. Comput. 25(12), 1856–1863 (2007)

    Article  Google Scholar 

  28. Shan, C., Gong, S., McOwan, P.W.: Robust facial expression recognition using local binary patterns. In: ICIP 2005. IEEE International Conference on Image Processing, 2005, vol. 2, pp. II–370. IEEE (2005)

  29. Siddiqi, M., Ali, R., Khan, A., Kim, E., Kim, G., Lee, S.: Facial expression recognition using active contour-based face detection, facial movement-based feature extraction, and non-linear feature selection. Multimed. Syst. (2014). doi:10.1007/s00530-014-0400-2

  30. Tian, Y., Kanade, T., Cohn, J.F.: Facial expression recognition. In: Handbook of Face Recognition, pp. 487–519. Springer, London (2011)

  31. Tian, Y.-L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. Springer, Berlin (2005)

    Book  Google Scholar 

  32. Valstar, M., Pantic, M.: Fully automatic recognition of the temporal phases of facial actions. IEEE Trans Syst. Man Cybern. Part B Cybern. 42(1), 28–43 (2012)

  33. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, vol. 1, pp. I–511. IEEE (2001)

  34. Wang, X., Jin, C., Liu, W., Hu, M., Xu, L., Ren, F.: Feature fusion of hog and wld for facial expression recognition. In: 2013 IEEE/SICE International Symposium on System Integration (SII), pp. 227–232 (2013)

  35. Wu, C.-H., Tzeng, G.-H., Goo, Y.-J., Fang, W.-C.: A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy. Expert Syst. Appl. 32(2), 397–408 (2007)

    Article  Google Scholar 

  36. Yeasin, M., Bullot, B., Sharma, R.: Recognition of facial expressions and measurement of levels of interest from video. IEEE Trans. Multimed. 8(3), 500–508 (2006)

    Article  Google Scholar 

  37. Zhao, G., Pietikäinen, M.: Boosted multi-resolution spatiotemporal descriptors for facial expression recognition. Pattern Recognit. Lett. 30(12), 1117–1127 (2009)

    Article  Google Scholar 

  38. Zheng, W., Zhou, X., Zou, C., Zhao, L.: Facial expression recognition using kernel canonical correlation analysis (kcca). IEEE Trans. Neural Netw. 17(1), 233–238 (2006)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, University of Maribor, Smetanova 17, 2000, Maribor, Slovenia

    Uroš Mlakar & Božidar Potočnik

Authors
  1. Uroš Mlakar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Božidar Potočnik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uroš Mlakar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mlakar, U., Potočnik, B. Automated facial expression recognition based on histograms of oriented gradient feature vector differences. SIViP 9 (Suppl 1), 245–253 (2015). https://doi.org/10.1007/s11760-015-0810-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-015-0810-4

Keywords

Search

Navigation