Multiple Image Characterization Techniques for Enhanced Facial Expression Recognition

Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 384)


This paper describes an enhanced facial expression recognition system. In the first step, the face localization is done using a simplified method, then the facial components are extracted and described by three feature vectors: the Zernike moments, the spectral components’ distribution through the DCT transform and by LBP features. The different feature vectors are used separately then combined to train back-propagation neural networks which are used in the facial expression recognition step. A subset feature selection algorithm was applied to these combined feature vectors in order to make dimensionality reduction and to improve the facial expression recognition process. Experiments performed on the JAFFE database along with comparisons to other methods have affirmed the validity and the good performances of the proposed approach.


Face detection Expression recognition DCT transform Zernike moments LBP MIFS NMIFS 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Decety, J., Meyer, M.: From emotion resonance to empathic understanding: A social develomental neuroscience account. Development and Psychopathology 20, 1053–1080 (2008)CrossRefGoogle Scholar
  2. 2.
    Ekman, P., Friesen, W.V.: Facial Action Coding System: A technique for the Mesurement of Facial Movement. Consulting Psychologists Press, Palo Alto (1978)Google Scholar
  3. 3.
    Kilbride, J.E., Yarczower, M.: Ethnic bias in the recognition of facial expressions. Journal of Nonverbal Behavior FALL 8(1), 27–41 (1983)Google Scholar
  4. 4.
    Boucher, J.D., Carlson, G.E.: Recognition of facial expression in three cultures. Journal of Cross-Cultural Psychology 11, 263–280 (1980)CrossRefGoogle Scholar
  5. 5.
    Jung-Wei, H., Kai-Tai, S.: Facial expression recognition under illumination variation. In: IEEE Advanced Robotics and its Social Impacts, ARSO 2007, pp. 1–6 (2007)Google Scholar
  6. 6.
    Wu, S., Lin, W., Xie, S.: Skin heat transfer model of facial thermograms and its application in face recognition. Elsevier Pattern Recognition 41(8), 2718–2729 (2008)CrossRefGoogle Scholar
  7. 7.
    Graf, H., Cosatto, E., Ezzat, T.: Face analysis for the synthesis of photo-realistic talking heads. In: Proc. Fourth IEEE Int. Conf. Automatic Face and Gesture Recognition, Grenoble, France, pp. 189–194 (2000)Google Scholar
  8. 8.
    Reisfeld, D., Yeshurun, Y.: Preprocessing of face images: Detection of features and pose normalization. Comput. Vision Image Understanding 71(3), 413–430 (1998)CrossRefGoogle Scholar
  9. 9.
    Yokoyama, T., Yagi, Y., Yachida, M.: Facial contour extraction model. In: IEEE Proc. of 3rd Int. Conf. On Automatic Face and Gesture Recognition (1998)Google Scholar
  10. 10.
    Yuille, A.L., Hallinan, P.W., Cohen, D.S.: Feature extraction from faces using deformable templates. Int. J. Comput. Vision 8, 99–111 (1992)CrossRefGoogle Scholar
  11. 11.
    Smith, M.L., Cottrell, G.W., Gosselin, F., Schyns, P.G.: Transmitting and decoding facial expressions. Psychol. Sci. 16, 184–189 (2005)CrossRefGoogle Scholar
  12. 12.
    Hjelmas, E., Low, B.K.: Face detection: A survey. Computer Vision and Image Understanding 83(3), 236–274 (2001)CrossRefMATHGoogle Scholar
  13. 13.
    Yuille, A.L., Hallinan, P.W., Cohen, D.S.: Feature Extraction From Faces Using deformable Templates. Int. J. Comput. Vision 8, 99–111 (1992)Google Scholar
  14. 14.
    Shin, J., Kim, D.: Hybrid Approach for Facial Feature Detection and Tracking under Occlusion. IEEE Signal Processing Letters 21(12) (2014)Google Scholar
  15. 15.
    Viola, P., Jones, M.: Robust real-time face detection. International Journal of Computer Vision (IJCV) 57(2), 137–154 (2004)CrossRefMATHGoogle Scholar
  16. 16.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Vitányi, P. (ed.) EurocoLT 1995, pp. 23–37. Springer, Heidelberg (1995)Google Scholar
  17. 17.
    Saaidia, M., Chaari, A., Lelandais, S., Vigneron, V., Bedda, M.: Face localization by neural networks trained with Zernike moments and Eigenfaces feature vectors. A comparaison. In: AVSS 2007, pp. 377–382 (2007)Google Scholar
  18. 18.
    Kotropoulos, C., Pitas, I.: Rule-based face detection in frontal views. In: Proc. Int. Conf. on Acoustic, Speech and Signal Processing (1997)Google Scholar
  19. 19.
    Amayeh, G., Erol, A., Bebis, G., Nicolescu, M.: Accurate and efficient computation of high order zernike moments. In: First ISVC, Lake Tahoe, NV, USA, pp. 462–469 (2005)Google Scholar
  20. 20.
    Akkoca, B.S, Gokmen, M.: Facial expression recognition using local zernike moments. In: Signal Processing and Communications Applications Conference (SIU), pp. 1–4 (April 2013)Google Scholar
  21. 21.
    Alirezaee, S., Ahmadi, M., Aghaeinia, H., Faez, K.: A weighted pseudo-zernike feature extractor for face recognition. IEEE, ICSMC 3, 2128–2132 (2005)Google Scholar
  22. 22.
    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distribution. Pattern Recognition 29(1), 51–59 (1996)CrossRefGoogle Scholar
  23. 23.
    Feng, X.-Y., Hadid, A., Pietikäinen, M.: A coarse-to-fine classification scheme for facial expression recognition. In: Campilho, A.C., Kamel, M.S. (eds.) ICIAR 2004. LNCS, vol. 3212, pp. 668–675. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  24. 24.
    Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing 27, 803–816 (2009)CrossRefGoogle Scholar
  25. 25.
    Zhang, S., Zhao, X., Lei, B.: Facial Expression Recognition Based on Local Binary Patterns and Local Fisher Discriminant Analysis. WSEAS Transactions on Signal Processing 8(1), 21–31 (2012)Google Scholar
  26. 26.
    Luoa, Y., Wub, C.-M., Zhang, Y.: Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - International Journal for Light and Electron Optics 124(17), 2767–2770 (2013)CrossRefGoogle Scholar
  27. 27.
    Kharat, G.U., Dudul, S.V.: Neural Network Classifier for Human Emotion Recognition from Facial Expressions Using Discrete Cosine Transform. IEEE, 653–658 (2008)Google Scholar
  28. 28.
    Estévez, P.A., Tesmer, M., Perez, C.A., Zurada, J.M.: Normalized Mutual Information Feature Selection. IEEE Transactions on Neural Networks 20(2), 189–201 (2009)Google Scholar
  29. 29.
    Terrillon, J.-C., Shirazi, M.N., Fukamachi, H., Akamatsu, S.: Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in colour images. In: Proc. of the International Conference on Face and Gesture Recognition, pp. 54–61 (2000)Google Scholar
  30. 30.
    Kovac, J., Peer, P., Solina, F.: Human skin color clustering for face detection. In: EUROCON 2003. Computer as a Tool. The IEEE Region 8, vol. 2, pp. 144–148 (September 2003)Google Scholar
  31. 31.
    Brown, D., Craw, I., Lewthwait, J.: A SOM based approach to skin detection with application in real time systems. In: Proc. of the British Machine Vision Conference (2001)Google Scholar
  32. 32.
    Lyons, M.J., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with gabor wavelets. In: Proc. of the Third IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 200–205. IEEE Computer Society, Nara, Japan (1998)Google Scholar
  33. 33.
    El-Sayed, R.S., El-Kholy, A., El-Nahas, M.Y.: Robust Facial Expression Recognition via Sparse Representation and Multiple Gabor filters. International Journal of Advanced Computer Science and Applications 4(3), 82–87 (2013)Google Scholar
  34. 34.
    Zhang, S., Zhao, X., Lei, B.: Facial Expression Recognition Using Sparse Representation. Wseas Transactions On Systems 11(8), 440–452 (2012)Google Scholar
  35. 35.
    Hablani, R., Chaudhari, N., Tanwani, S.: Recognition of Facial Expressions using Local Binary Patterns of Important Facial Parts. International Journal of Image Processing (IJIP) 7(2), 163–170 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Mohammed Saaidia
    • 1
  • Narima Zermi
    • 2
  • Messaoud Ramdani
    • 2
  1. 1.Département de Génie-ElectriqueUniversité M.C.M Souk-AhrasSouk-AhrasAlgeria
  2. 2.Département d’ ElectroniqueUniversité Badji-Mokhtar de AnnabaAnnabaAlgeria

Personalised recommendations