Automatic Affect Analysis: From Children to Adults

  • Rizwan Ahmed KhanEmail author
  • Alexandre Meyer
  • Saida Bouakaz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9475)


This article presents novel and robust framework for automatic recognition of facial expressions for children. The proposed framework also achieved results better than state of the art methods for stimuli containing adult faces. The proposed framework extract features only from perceptual salient facial regions as it gets its inspiration from human visual system. In this study we are proposing novel shape descriptor, facial landmark points triangles ratio (LPTR). The framework was first tested on the “Dartmouth database of children’s faces” which contains photographs of children between 6 and 16 years of age and achieved promising results. Later we tested proposed framework on Cohn-Kanade (CK+) posed facial expression database (adult faces) and obtained results that exceeds state of the art.


Facial Expression Human Visual System Shape Descriptor Facial Region Facial Expression Recognition 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This study is financially supported by BPI France ( and FUI-KURIO EYE Project.


  1. 1.
    Ekman, P.: Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage, 3rd edn. W. W. Norton & Company, New York (2001)Google Scholar
  2. 2.
    Pantic, M., Pentland, A., Nijholt, A., Huang, T.: Human computing and machine understanding of human behavior: A survey. In: ACM International Conference on Multimodal Interfaces (2006)Google Scholar
  3. 3.
    Ekman, P.: Universals and cultural differences in facial expressions of emotion. In: Nebraska Symposium on Motivation, pp. 207–283 (1971)Google Scholar
  4. 4.
    Littlewort, G., Bartlett, M.S., Fasel, I., Susskind, J., Movellan, J.: Dynamics of facial expression extracted automatically from video. Image Vis. Comput. 24, 615–625 (2006)CrossRefGoogle Scholar
  5. 5.
    Khan, R., Meyer, A., Konik, H., Bouakaz, S.: Facial expression recognition using entropy and brightness features. In: 11th International Conference on Intelligent Systems Design and Applications (2011)Google Scholar
  6. 6.
    Tian, Y.: Evaluation of face resolution for expression analysis. Comput. Vis. Pattern Recogn. Workshop 68, 179–201 (2004)Google Scholar
  7. 7.
    Khan, R.A., Meyer, A., Konik, H., Bouakaz, S.: Human vision inspired framework for facial expressions recognition. In: IEEE International Conference on Image Processing (2012)Google Scholar
  8. 8.
    Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27, 699–714 (2005)CrossRefGoogle Scholar
  9. 9.
    Valstar, M., Patras, I., Pantic, M.: Facial action unit detection using probabilistic actively learned support vector machines on tracked facial point data. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, pp. 76–84 (2005)Google Scholar
  10. 10.
    Khan, R., Meyer, A., Konik, H., Bouakaz, S.: Pain detection through shape and appearance features. In: 2013 IEEE International Conference on Multimedia and Expo (ICME) (2013)Google Scholar
  11. 11.
    Bai, Y., Guo, L., Jin, L., Huang, Q.: A novel feature extraction method using pyramid histogram of orientation gradients for smile recognition. In: International Conference on Image Processing (2009)Google Scholar
  12. 12.
    Khan, R.A., Meyer, A., Konik, H., Bouakaz, S.: Framework for reliable, real-time facial expression recognition for low resolution images. Pattern Recogn. Lett. 34, 1159–1168 (2013)CrossRefGoogle Scholar
  13. 13.
    Zhaoping, L.: Theoretical understanding of the early visual processes by data compression and data selection. Netw. Comput. Neural Syst. 17, 301–334 (2006)CrossRefGoogle Scholar
  14. 14.
    Khan, R.A., Meyer, A., Konik, H., Bouakaz, S.: Exploring human visual system: study to aid the development of automatic facial expression recognition framework. In: Computer Vision and Pattern Recognition Workshop (2012)Google Scholar
  15. 15.
    Lucey, P., Cohn, J., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., Prkachin, K.: Automatically detecting pain in video through facial action units. IEEE Trans. Syst., Man, Cybern., Part B: Cybern. 41, 664–674 (2011)CrossRefGoogle Scholar
  16. 16.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Fourth IEEE International Conference on Automatic face and Gesture Recognition (FG 2000), pp. 46–53 (2000)Google Scholar
  17. 17.
    Yang, P., Liu, Q., Metaxas, D.N.: Exploring facial expressions with compositional features. In: Computer Vision Pattern Recognition, pp. 2638–2644 (2010)Google Scholar
  18. 18.
    Ekman, P., Friesen, W.: The facial action coding system: A technique for the measurement of facial movements. Consulting Psychologist, Palo Alto (1978)Google Scholar
  19. 19.
    Pantic, M., Valstar, M.F., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: International Conference on Multimedia and Expo (2005)Google Scholar
  20. 20.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kande dataset (CK+): A complete facial expression dataset for action unit and emotion-specified expression. In: Computer Vision and Pattern Recognition Workshops (2010)Google Scholar
  21. 21.
    Dalrymple, K.A., Gomez, J., Duchaine, B.: The Dartmouth database of children’s faces: Acquisition and validation of a new face stimulus set. PLoS ONE 8, e79131 (2013)CrossRefGoogle Scholar
  22. 22.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Conference on Computer Vision and Pattern Recognition (2001)Google Scholar
  23. 23.
    Silva, C., Schnitman, L., Oliveira, L.: Detection of facial landmarks using local-based information. In: Brazilian Conference on Automation (2012)Google Scholar
  24. 24.
    Egger, H., Pine, D., Nelson, E., Leibenluft, E., Ernst, M., K.E., T., Angold, A.: The NIMH child emotional faces picture set (NIMH-ChEFS): A new set of children’s facial emotion stimuli. Int. J. Methods Psychiatr. Res. 20(3), 145–56 (2011)Google Scholar
  25. 25.
    Zhao, G., Pietikäinen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29, 915–928 (2007)CrossRefGoogle Scholar
  26. 26.
    Kotsia, I., Zafeiriou, S., Pitas, I.: Texture and shape information fusion for facial expression and facial action unit recognition. Pattern Recogn. 41, 833–851 (2008)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Rizwan Ahmed Khan
    • 1
    Email author
  • Alexandre Meyer
    • 1
  • Saida Bouakaz
    • 1
  1. 1.Université Claude Bernard Lyon 1, CNRS, LIRIS, UMR5205LyonFrance

Personalised recommendations