Exaggeration Quantified: An Intensity-Based Analysis of Posed Facial Expressions



Posed facial expressions are characterized by deliberate and often exaggerated behaviours that usually fail to generalize to the complexity of expressive exhibition of human emotion in real-life. The detailed understanding of exaggeration and its quantification that defines each facial expression as a feature vector in the face space will provide deep insight to the growing interest in analysing posed facial expressions. In this paper, an attempt to quantify the intensity of facial expressions via estimating exaggeration as the deviation in the combined relationship between correlated landmark points that characterize the deformable face model, is made. Such relationships that underpin the discrimination of different facial expressions using exaggeration measurements is based on novel geometric morphometric inspired feature descriptors together with enhanced appearance models that seed a cascaded Support Vector Machine (SVM) classifier. In addition to demonstrating the superiority of the proposed method by applying it to images containing a wide range of expressions using standard datasets; results also distinguish different important facial landmarks for classifying that expression from every other expression.


Landmark Points Posed Expressions Appearance Descriptors Matrix Form Representation Indented Appearance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    D. Matsumoto, H.S. Hwang, Reading Facial Expressions of Emotion. Psychological Science Agenda Science Brief (American Psychological Association, 2011),
  2. 2.
    S. Tulyakov, T. Slowe, Z. Zhang, V. Govindaraju, Facial expression biometrics using tracker displacement features, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007), pp. 17–22Google Scholar
  3. 3.
    F. Dornaika, B. Raducanu, Facial expression recognition for HCI applications, in Encyclopedia of Artificial Intelligence, vol. II (IGI Global, London, 2008), pp. 625–631Google Scholar
  4. 4.
    V. Beek, D. Yolanda, J. Semon, Decoding basic and non-basic facial expressions and depressive symptoms in late childhood and adolescence. J. Nonverbal Behav. 32(1), 53–64 (2008)CrossRefGoogle Scholar
  5. 5.
    V. Bettadapura, Face expression recognition and analysis: the state of the art. Technical Report (2012),
  6. 6.
    A. Dhall, Context based facial expression analysis in the wild, in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII) (2013), pp. 636–641Google Scholar
  7. 7.
    Y. Tian, T. Kanade, J. Cohn, Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)CrossRefGoogle Scholar
  8. 8.
    P. Ekman, W.V. Friesen, The Facial Action Coding System: A Technique for the Measurement of Facial Movement (Consulting Psychologists Press, San Francisco, 1978)Google Scholar
  9. 9.
    M.A. Sayette, J.F. Cohn, J.M. Wertz, M.A. Perrott, D.J. Parrott, A psychometric evaluation of the facial action coding system for assessing spontaneous expression. J. Nonverbal Behav. 25, 167–186 (2001)CrossRefGoogle Scholar
  10. 10.
    F. De la Torre, J.F. Cohn, Facial expression analysis, in Guide to Visual Analysis of Humans: Looking at People (Springer, New York, 2011)Google Scholar
  11. 11.
    W-J. Yan, S-J. Wang, G. Zhao, X. Fu, Quantifying micro-expressions with constraint local model and local binary pattern, in Computer Vision - ECCV 2014 Workshops. Lecture Notes in Computer Science, vol. 8925 (Springer, Heidelberg, 2014), pp. 296–305Google Scholar
  12. 12.
    M. Shreve, S. Godavarthy, D. Goldgof, S. Sarkar, Macro- and micro-expression spotting in long videos using spatio-temporal strain, in Proceedings of the International Conference on Automatic Face and Gesture Recognition (FG) (2011)Google Scholar
  13. 13.
    M.A. Shreve, Automatic macro- and micro-facial expression spotting and applications, Ph.D. Thesis, University of Southern Florida, 2013Google Scholar
  14. 14.
    M. Shreve, S. Godavarthy, V. Manohar, D. Goldgof, S. Sarkar, Towards macro- and micro-expression spotting in video using strain patterns, in Workshop on Applications of Computer Vision (WACV) (2009), pp. 1–6Google Scholar
  15. 15.
    B. Fasel, J. Luettin, Recognition of asymmetric facial action unit activities and intensities, in Proceedings of the International Conference on Pattern Recognition (ICPR) (2000)Google Scholar
  16. 16.
    S. Mitra, Y. Liu, Local facial asymmetry for expression classification, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2 (2004), pp. 889–894Google Scholar
  17. 17.
    Y. Liu, K.L. Schmidt, J.F. Cohn, S. Mitra, Facial asymmetry quantification for expression invariant human identification. J. Comput. Vis. Image Underst. 91(1–2), 138–159 (2003)CrossRefGoogle Scholar
  18. 18.
    M. Kyperountas, A. Tefas, I. Pitas, Salient feature and reliable classifier selection for facial expression classification. Pattern Recogn. 43(3), 972–986 (2010)CrossRefzbMATHGoogle Scholar
  19. 19.
    L. Zhang, D.W. Tjondronegoro, V. Chandran, Geometry vs appearance for discriminating between posed and spontaneous emotions, in Neural Information Processing. Lecture Notes in Computer Science, vol. 7064 (Springer, Heidelberg, 2011), pp. 431–440Google Scholar
  20. 20.
    M.F. Valstar, H. Gunes, M. Pantic, How to distinguish posed from spontaneous smiles using geometric features, in Proceedings of the 9th International Conference on Multimodal Interfaces (ICMI) (2007)Google Scholar
  21. 21.
    H. Dibeklioglu, A.A. Salah, T. Gevers, Are you really smiling at me? Spontaneous versus posed enjoyment smiles, in Proceedings of the European Conference on Computer Vision (ECCV) (2012), pp. 525–538Google Scholar
  22. 22.
    M.S. Bartlett, G. Littlewort, C. Lainscsek, I. Fasel, J. Movellan, Machine learning methods for fully automatic recognition of facial expressions and facial actions, in IEEE International Conference on Systems, Men and Cybernetics (2004), pp. 592–597Google Scholar
  23. 23.
    M.F. Valstar, M. Pantic, Z. Ambadar, J.F. Cohn, Spontaneous vs. posed facial behavior: automatic analysis of brow actions, in Proceedings of the ACM International Conference on Multimodal Interfaces (2006), pp. 162–170Google Scholar
  24. 24.
    G.C. Littlewort, M.S. Bartlett, K. Lee, Automatic coding of facial expressions displayed during posed and genuine pain. Image Vis. Comput. 27(12), 1797–1803 (2009)CrossRefGoogle Scholar
  25. 25.
    M. Bartlett, G. Littlewort, E. Vural, K. Lee, M. Cetin, A. Ercil, J. Movellan, Data mining spontaneous facial behavior with automatic expression coding, in Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction (Springer, Heidelberg, 2008), pp. 1–20CrossRefGoogle Scholar
  26. 26.
    H. Dibeklioglu, R. Valenti, A. Salah, T. Gevers, Eyes do not lie: spontaneous versus posed smiles, in Proceedings of the ACM International Conference on Multimedia (2010), pp. 703–706Google Scholar
  27. 27.
    S.M. Lajevardi, M. Lech, Facial expression recognition using neural networks and log-gabor filters, in Proceedings of the Digital Image Computing: Techniques and Applications (DICTA) (2008), pp. 77–83Google Scholar
  28. 28.
    H. Bhaskar, M. Al-Mualla, Spontaneous Vs. posed facial expression analysis using deformable feature models and aggregated classifiers, in Proceedings of the International Conference on Information FUSION (2013)Google Scholar
  29. 29.
    H. Bhaskar, D. La Torre, M. Al-Mualla, Posed Facial Expression Detection using Reflection Symmetry and Structural Similarity (2015) - 218-228 - ICIAR 2015, ed. by A. Campilho, M. Kamel. Lecture Notes in Computer Science, vol. 9164 (Springer, 2015), pp 218–228Google Scholar
  30. 30.
    P. Viola, M.J. Jones, Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)CrossRefGoogle Scholar
  31. 31.
    M.J. Black, A.D. Jepson, Eigentracking: Robust matching and tracking of objects using view-based representation. Int. J. Comput. Vis. 26(1), 63–84 (1998)CrossRefGoogle Scholar
  32. 32.
    T.F. Cootes, D. Cooper, C.J. Taylor, J. Graham, Active shape models - their training and application. J. Comput. Vis. Image Underst. 61(1), 38–59 (1995)CrossRefGoogle Scholar
  33. 33.
    P.A. Tresadern, H. Bhaskar, S.A. Adeshina, J.C. Taylor, T.F. Cootes, Combining local and global shape models for deformable object matching, in Proceedings of the British Machine Vision Conference (BMVC) (2009), pp. 1–12Google Scholar
  34. 34.
    T.F. Cootes, G.J. Edwards, C.J. Taylor, Active appearance models, in Proceedings of the European Conference on Computer Vision (ECCV). LNCS, vol. 2 (Springer, Heidelberg, 1998), pp. 484–498Google Scholar
  35. 35.
    V. Blanz, T. Vetter, A morphable model for the synthesis of 3D faces, in Proceedings of SIGGRAPH (1999)Google Scholar
  36. 36.
    A.R. Rivera, J.A.R. Castillo, O. Chae, Recognition of face expressions using local principal texture pattern, in IEEE International Conference on Image Processing (ICIP) (2012), pp. 2609–2612Google Scholar
  37. 37.
    M.J. Lyons, S. Akemastu, M. Kamachi, J. Gyoba, Coding facial expressions with gabor wavelets, in 3rd IEEE International Conference on Automatic Face and Gesture Recognition (1998), pp. 200–205Google Scholar
  38. 38.
    S. Dongcheng, C. Fang, D. Guangyi, Facial expression recognition based on gabor wavelet phase features, in Proceedings of the International Conference on Image and Graphics (ICIG) (2013), pp. 520–523Google Scholar
  39. 39.
    S. Jain, H. Changbo, J.K. Aggarwal, Facial expression recognition with temporal modeling of shapes, in Proceedings of the IEEE International Conference on Computer Vision Workshop (2011), pp. 1642–1649Google Scholar
  40. 40.
    I. Buciu, C. Kotropoulos, I. Pitas, ICA and Gabor representation for facial expression recognition, in Proceedings of the International Conference on Image Processing, vol. 2 (2003), pp. 855–858Google Scholar
  41. 41.
    Z. Guoying, M. Pietikainen, Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)CrossRefGoogle Scholar
  42. 42.
    H. Soyel, H. Demirel, Improved SIFT matching for pose robust facial expression recognition, in Proceedings of the International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011) (2011), pp. 585–590Google Scholar
  43. 43.
    A.S. Md. Sohail, P. Bhattacharya, Classification of facial expressions using K-nearest neighbor classifier, in Computer Vision/Computer Graphics Collaboration Techniques. Lecture Notes in Computer Science, vol. 4418 (Springer, Heidelberg, 2007), pp. 555–566Google Scholar
  44. 44.
    A. Graves, C. Mayer, M. Wimmer, J. Schmidhuber, B. Radig, Facial expression recognition with recurrent neural networks, in Proceedings of the International Workshop on Cognition for Technical Systems (2008)Google Scholar
  45. 45.
    E. Owusu, Y. Zhan, Q.R. Mao, A neural-AdaBoost based facial expression recognition system. Expert Syst. Appl. 41(7), 3383–3390 (2014)CrossRefGoogle Scholar
  46. 46.
    S. Lele, J.T. Richtsmeier, Euclidean distance matrix analysis: confidence intervals for form and growth differences. Am. J. Phys. Anthropol. 98(1), 73–86 (1995)CrossRefGoogle Scholar
  47. 47.
    A.K.K. Bermani, A.Z. Ghalwash, A.A.A. Youssif, Automatic facial expression recognition based on hybrid approach. Int. J. Adv. Comput. Sci. Appl. 3(11), 102–107 (2012)Google Scholar
  48. 48.
    S.L. Happy, A. Routray, Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)CrossRefGoogle Scholar
  49. 49.
    Z. Wang, A.C. Bovik, Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)CrossRefGoogle Scholar
  50. 50.
    Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  51. 51.
    M.J. Wainwright, O. Schwartz, E.P. Simoncelli, Natural image statistics and divisive normalization: modeling non-linearity and adaptation in cortical neurons. in Probabilistic Models of the Brain: Perception and Neural Function (MIT Press, Cambridge, 2002), pp. 203–222Google Scholar
  52. 52.
    M. Ester, H-P. Kriegel, J. Sander, X. Xu, A density-based algorithm for discovering clusters in large spatial databases with noise, in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96) (AAAI Press, Portland, 1996), pp. 226–231Google Scholar
  53. 53.
    Y. Gao, A. Rehma, Z. Wang, CW-SSIM based image classification, in Proceedings of the IEEE International Conference on Image Processing (ICIP) (2011), pp. 1249–1252Google Scholar
  54. 54.
    N. Aifanti, C. Papachristou, A. Delopoulos, The MUG facial expression database, in Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) (2010)Google Scholar
  55. 55.
    T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2000), pp. 46–53Google Scholar
  56. 56.
    P. Lucey, J.F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews, The extended Cohn-Kande dataset (CK+): a complete facial expression dataset for action unit and emotion-specified expression, in Proceedings of the Third IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis (2010)Google Scholar
  57. 57.
    M.N. Dailey, C. Joyce, M.J. Lyons, M. Kamachi, H. Ishi, J. Gyoba, G.W. Cottrell, Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion 10(6), 874–893 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Visual Signal Analysis and Processing (VSAP) Research CenterKhalifa University of Science Technology and ResearchAbu DhabiUnited Arab Emirates
  2. 2.Department of Applied Mathematics and SciencesKhalifa University of Science, Technology and ResearchAbu DhabiUnited Arab Emirates
  3. 3.Department of Economics, Management and Quantitative MethodsUniversity of MilanMilanItaly
  4. 4.Khalifa University of Science Technology and ResearchAbu DhabiUnited Arab Emirates

Personalised recommendations