Advertisement

Automatic facial expression recognition: feature extraction and selection

Abstract

In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 99

This is the net price. Taxes to be calculated in checkout.

References

  1. 1

    Battiti R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5, 537–550 (1994)

  2. 2

    Chibelushi, C.C., Bourel, F.: Facial expression recognition: A brief tutorial overview. CVonline: On-Line Compendium of Computer Vision, vol. 9, (2003)

  3. 3

    Duda R.O., Hart P.E., Stork D.G.: Pattern Classification. Wiley, New York (2001)

  4. 4

    Fasel B., Luettin J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36, 259–275 (2003)

  5. 5

    Field D.J.: Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. 4, 2379–2394 (1987)

  6. 6

    Guyon I., Gunn S., Nikravesh M., Zadeh L.: Feature extraction: foundations and applications. Springer, Verlag (2006)

  7. 7

    Hanchuan P., Fuhui L., Ding C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1226–1238 (2005)

  8. 8

    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53, Grenoble, France, March (2000)

  9. 9

    Kohavi R., John G.H.: Wrapper for feature subset selection. Artif. Intell. 97, 273–324 (1997)

  10. 10

    Lajevardi, S.M., Lech, M.: Averaged Gabor Filter Features for Facial Expression Recognition. In: Computing: Techniques and Applications, 2008. DICTA’08. Digital Image Canberra Canberra, Australia, Dec, pp. 71–76 (2008)

  11. 11

    Lajevardi, S.M., Lech, M.: Facial Expression Recognition from Image Sequences Using Optimised Feature Selection. 23rd International Conference on Image and Vision Computing (IVCNZ’08) Christchurch, New Zealand, Nov, pp. 1–6 (2008)

  12. 12

    Lajevardi S.M., Hussain Z.M.: Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition. IET Image Process. 4, 114–119 (2010)

  13. 13

    Lee T.S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Anal. Mach. Intell. 18, 959–971 (1996)

  14. 14

    Liu, F. et al.: Facial expression recognition using HLAC features and WPCA. In: Affective Computing and Intelligent Interaction, First International Conference (ACII 2005) Beijing, China, Oct 2005, pp. 88–94 (2005)

  15. 15

    Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets, in automatic face and gesture recognition, 1998. In: Proceedings. Third IEEE International Conference on Japan, pp. 200–205 (1998)

  16. 16

    Mehrabian A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)

  17. 17

    Ojala T., Pietikäinen M., Harwood D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29, 51–59 (1996)

  18. 18

    Otsu, N., Kurita, T.: A new scheme for practical flexible and intelligent vision systems. In: Proceedings of the IAPR Workshop on Computer Vision, pp. 431–435 (1988)

  19. 19

    Rish, I.: An empirical study of the naive Bayes classifier. In: IJCAI Workshop on Empirical Methods in Artificial Intelligence, pp. 41–46 (2001)

  20. 20

    Srinivas M., Patnik L.M.: Genetic algorithms: a Survey. IEEE Comput. Soc. Press 27, 17–26 (1994)

  21. 21

    Tian, Y.L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. Hand Book of Face Recognition, pp. 247–275 (2005)

  22. 22

    Toyoda, T., Hasegawa, O.: Texture classification using extended higher order local autocorrelation features. In: Proceedings of the 4th International Workshop on Texture Analysis and Synthesis, pp. 131–136 (2005)

  23. 23

    Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)

Download references

Author information

Correspondence to Seyed Mehdi Lajevardi.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lajevardi, S.M., Hussain, Z.M. Automatic facial expression recognition: feature extraction and selection. SIViP 6, 159–169 (2012). https://doi.org/10.1007/s11760-010-0177-5

Download citation

Keywords

  • Facial expression recognition
  • Emotion recognition
  • Mutual information
  • Higher order auto correlation
  • Gabor filters