Signal, Image and Video Processing

, Volume 6, Issue 1, pp 159–169 | Cite as

Automatic facial expression recognition: feature extraction and selection

  • Seyed Mehdi Lajevardi
  • Zahir M. Hussain
Original Paper


In this paper, we investigate feature extraction and feature selection methods as well as classification methods for automatic facial expression recognition (FER) system. The FER system is fully automatic and consists of the following modules: face detection, facial detection, feature extraction, selection of optimal features, and classification. Face detection is based on AdaBoost algorithm and is followed by the extraction of frame with the maximum intensity of emotion using the inter-frame mutual information criterion. The selected frames are then processed to generate characteristic features using different methods including: Gabor filters, log Gabor filter, local binary pattern (LBP) operator, higher-order local autocorrelation (HLAC) and a recent proposed method called HLAC-like features (HLACLF). The most informative features are selected based on both wrapper and filter feature selection methods. Experiments on several facial expression databases show comparisons of different methods.


Facial expression recognition Emotion recognition Mutual information Higher order auto correlation Gabor filters 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Battiti R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5, 537–550 (1994)CrossRefGoogle Scholar
  2. 2.
    Chibelushi, C.C., Bourel, F.: Facial expression recognition: A brief tutorial overview. CVonline: On-Line Compendium of Computer Vision, vol. 9, (2003)Google Scholar
  3. 3.
    Duda R.O., Hart P.E., Stork D.G.: Pattern Classification. Wiley, New York (2001)zbMATHGoogle Scholar
  4. 4.
    Fasel B., Luettin J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36, 259–275 (2003)zbMATHCrossRefGoogle Scholar
  5. 5.
    Field D.J.: Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. 4, 2379–2394 (1987)CrossRefGoogle Scholar
  6. 6.
    Guyon I., Gunn S., Nikravesh M., Zadeh L.: Feature extraction: foundations and applications. Springer, Verlag (2006)zbMATHGoogle Scholar
  7. 7.
    Hanchuan P., Fuhui L., Ding C.: Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1226–1238 (2005)CrossRefGoogle Scholar
  8. 8.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53, Grenoble, France, March (2000)Google Scholar
  9. 9.
    Kohavi R., John G.H.: Wrapper for feature subset selection. Artif. Intell. 97, 273–324 (1997)zbMATHCrossRefGoogle Scholar
  10. 10.
    Lajevardi, S.M., Lech, M.: Averaged Gabor Filter Features for Facial Expression Recognition. In: Computing: Techniques and Applications, 2008. DICTA’08. Digital Image Canberra Canberra, Australia, Dec, pp. 71–76 (2008)Google Scholar
  11. 11.
    Lajevardi, S.M., Lech, M.: Facial Expression Recognition from Image Sequences Using Optimised Feature Selection. 23rd International Conference on Image and Vision Computing (IVCNZ’08) Christchurch, New Zealand, Nov, pp. 1–6 (2008)Google Scholar
  12. 12.
    Lajevardi S.M., Hussain Z.M.: Novel higher-order local autocorrelation-like feature extraction methodology for facial expression recognition. IET Image Process. 4, 114–119 (2010)CrossRefGoogle Scholar
  13. 13.
    Lee T.S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Anal. Mach. Intell. 18, 959–971 (1996)CrossRefGoogle Scholar
  14. 14.
    Liu, F. et al.: Facial expression recognition using HLAC features and WPCA. In: Affective Computing and Intelligent Interaction, First International Conference (ACII 2005) Beijing, China, Oct 2005, pp. 88–94 (2005)Google Scholar
  15. 15.
    Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets, in automatic face and gesture recognition, 1998. In: Proceedings. Third IEEE International Conference on Japan, pp. 200–205 (1998)Google Scholar
  16. 16.
    Mehrabian A.: Communication without words. Psychol. Today 2(4), 53–56 (1968)Google Scholar
  17. 17.
    Ojala T., Pietikäinen M., Harwood D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. 29, 51–59 (1996)CrossRefGoogle Scholar
  18. 18.
    Otsu, N., Kurita, T.: A new scheme for practical flexible and intelligent vision systems. In: Proceedings of the IAPR Workshop on Computer Vision, pp. 431–435 (1988)Google Scholar
  19. 19.
    Rish, I.: An empirical study of the naive Bayes classifier. In: IJCAI Workshop on Empirical Methods in Artificial Intelligence, pp. 41–46 (2001)Google Scholar
  20. 20.
    Srinivas M., Patnik L.M.: Genetic algorithms: a Survey. IEEE Comput. Soc. Press 27, 17–26 (1994)Google Scholar
  21. 21.
    Tian, Y.L., Kanade, T., Cohn, J.F.: Facial Expression Analysis. Hand Book of Face Recognition, pp. 247–275 (2005)Google Scholar
  22. 22.
    Toyoda, T., Hasegawa, O.: Texture classification using extended higher order local autocorrelation features. In: Proceedings of the 4th International Workshop on Texture Analysis and Synthesis, pp. 131–136 (2005)Google Scholar
  23. 23.
    Viola P., Jones M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2010

Authors and Affiliations

  1. 1.School of Electrical and Computer EngineeringRMIT UniversityMelbourneAustralia

Personalised recommendations