Abstract
Posed facial expressions are characterized by deliberate and often exaggerated behaviours that usually fail to generalize to the complexity of expressive exhibition of human emotion in real-life. The detailed understanding of exaggeration and its quantification that defines each facial expression as a feature vector in the face space will provide deep insight to the growing interest in analysing posed facial expressions. In this paper, an attempt to quantify the intensity of facial expressions via estimating exaggeration as the deviation in the combined relationship between correlated landmark points that characterize the deformable face model, is made. Such relationships that underpin the discrimination of different facial expressions using exaggeration measurements is based on novel geometric morphometric inspired feature descriptors together with enhanced appearance models that seed a cascaded Support Vector Machine (SVM) classifier. In addition to demonstrating the superiority of the proposed method by applying it to images containing a wide range of expressions using standard datasets; results also distinguish different important facial landmarks for classifying that expression from every other expression.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
D. Matsumoto, H.S. Hwang, Reading Facial Expressions of Emotion. Psychological Science Agenda Science Brief (American Psychological Association, 2011), http://www.apa.org/science/about/psa/2011/05/facial-expressions.aspx
S. Tulyakov, T. Slowe, Z. Zhang, V. Govindaraju, Facial expression biometrics using tracker displacement features, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2007), pp. 17–22
F. Dornaika, B. Raducanu, Facial expression recognition for HCI applications, in Encyclopedia of Artificial Intelligence, vol. II (IGI Global, London, 2008), pp. 625–631
V. Beek, D. Yolanda, J. Semon, Decoding basic and non-basic facial expressions and depressive symptoms in late childhood and adolescence. J. Nonverbal Behav. 32(1), 53–64 (2008)
V. Bettadapura, Face expression recognition and analysis: the state of the art. Technical Report (2012), http://arxiv.org/abs/1203.6722
A. Dhall, Context based facial expression analysis in the wild, in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII) (2013), pp. 636–641
Y. Tian, T. Kanade, J. Cohn, Recognizing action units for facial expression analysis. IEEE Trans. Pattern Anal. Mach. Intell. 23(2), 97–115 (2001)
P. Ekman, W.V. Friesen, The Facial Action Coding System: A Technique for the Measurement of Facial Movement (Consulting Psychologists Press, San Francisco, 1978)
M.A. Sayette, J.F. Cohn, J.M. Wertz, M.A. Perrott, D.J. Parrott, A psychometric evaluation of the facial action coding system for assessing spontaneous expression. J. Nonverbal Behav. 25, 167–186 (2001)
F. De la Torre, J.F. Cohn, Facial expression analysis, in Guide to Visual Analysis of Humans: Looking at People (Springer, New York, 2011)
W-J. Yan, S-J. Wang, G. Zhao, X. Fu, Quantifying micro-expressions with constraint local model and local binary pattern, in Computer Vision - ECCV 2014 Workshops. Lecture Notes in Computer Science, vol. 8925 (Springer, Heidelberg, 2014), pp. 296–305
M. Shreve, S. Godavarthy, D. Goldgof, S. Sarkar, Macro- and micro-expression spotting in long videos using spatio-temporal strain, in Proceedings of the International Conference on Automatic Face and Gesture Recognition (FG) (2011)
M.A. Shreve, Automatic macro- and micro-facial expression spotting and applications, Ph.D. Thesis, University of Southern Florida, 2013
M. Shreve, S. Godavarthy, V. Manohar, D. Goldgof, S. Sarkar, Towards macro- and micro-expression spotting in video using strain patterns, in Workshop on Applications of Computer Vision (WACV) (2009), pp. 1–6
B. Fasel, J. Luettin, Recognition of asymmetric facial action unit activities and intensities, in Proceedings of the International Conference on Pattern Recognition (ICPR) (2000)
S. Mitra, Y. Liu, Local facial asymmetry for expression classification, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2 (2004), pp. 889–894
Y. Liu, K.L. Schmidt, J.F. Cohn, S. Mitra, Facial asymmetry quantification for expression invariant human identification. J. Comput. Vis. Image Underst. 91(1–2), 138–159 (2003)
M. Kyperountas, A. Tefas, I. Pitas, Salient feature and reliable classifier selection for facial expression classification. Pattern Recogn. 43(3), 972–986 (2010)
L. Zhang, D.W. Tjondronegoro, V. Chandran, Geometry vs appearance for discriminating between posed and spontaneous emotions, in Neural Information Processing. Lecture Notes in Computer Science, vol. 7064 (Springer, Heidelberg, 2011), pp. 431–440
M.F. Valstar, H. Gunes, M. Pantic, How to distinguish posed from spontaneous smiles using geometric features, in Proceedings of the 9th International Conference on Multimodal Interfaces (ICMI) (2007)
H. Dibeklioglu, A.A. Salah, T. Gevers, Are you really smiling at me? Spontaneous versus posed enjoyment smiles, in Proceedings of the European Conference on Computer Vision (ECCV) (2012), pp. 525–538
M.S. Bartlett, G. Littlewort, C. Lainscsek, I. Fasel, J. Movellan, Machine learning methods for fully automatic recognition of facial expressions and facial actions, in IEEE International Conference on Systems, Men and Cybernetics (2004), pp. 592–597
M.F. Valstar, M. Pantic, Z. Ambadar, J.F. Cohn, Spontaneous vs. posed facial behavior: automatic analysis of brow actions, in Proceedings of the ACM International Conference on Multimodal Interfaces (2006), pp. 162–170
G.C. Littlewort, M.S. Bartlett, K. Lee, Automatic coding of facial expressions displayed during posed and genuine pain. Image Vis. Comput. 27(12), 1797–1803 (2009)
M. Bartlett, G. Littlewort, E. Vural, K. Lee, M. Cetin, A. Ercil, J. Movellan, Data mining spontaneous facial behavior with automatic expression coding, in Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction (Springer, Heidelberg, 2008), pp. 1–20
H. Dibeklioglu, R. Valenti, A. Salah, T. Gevers, Eyes do not lie: spontaneous versus posed smiles, in Proceedings of the ACM International Conference on Multimedia (2010), pp. 703–706
S.M. Lajevardi, M. Lech, Facial expression recognition using neural networks and log-gabor filters, in Proceedings of the Digital Image Computing: Techniques and Applications (DICTA) (2008), pp. 77–83
H. Bhaskar, M. Al-Mualla, Spontaneous Vs. posed facial expression analysis using deformable feature models and aggregated classifiers, in Proceedings of the International Conference on Information FUSION (2013)
H. Bhaskar, D. La Torre, M. Al-Mualla, Posed Facial Expression Detection using Reflection Symmetry and Structural Similarity (2015) - 218-228 - ICIAR 2015, ed. by A. Campilho, M. Kamel. Lecture Notes in Computer Science, vol. 9164 (Springer, 2015), pp 218–228
P. Viola, M.J. Jones, Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)
M.J. Black, A.D. Jepson, Eigentracking: Robust matching and tracking of objects using view-based representation. Int. J. Comput. Vis. 26(1), 63–84 (1998)
T.F. Cootes, D. Cooper, C.J. Taylor, J. Graham, Active shape models - their training and application. J. Comput. Vis. Image Underst. 61(1), 38–59 (1995)
P.A. Tresadern, H. Bhaskar, S.A. Adeshina, J.C. Taylor, T.F. Cootes, Combining local and global shape models for deformable object matching, in Proceedings of the British Machine Vision Conference (BMVC) (2009), pp. 1–12
T.F. Cootes, G.J. Edwards, C.J. Taylor, Active appearance models, in Proceedings of the European Conference on Computer Vision (ECCV). LNCS, vol. 2 (Springer, Heidelberg, 1998), pp. 484–498
V. Blanz, T. Vetter, A morphable model for the synthesis of 3D faces, in Proceedings of SIGGRAPH (1999)
A.R. Rivera, J.A.R. Castillo, O. Chae, Recognition of face expressions using local principal texture pattern, in IEEE International Conference on Image Processing (ICIP) (2012), pp. 2609–2612
M.J. Lyons, S. Akemastu, M. Kamachi, J. Gyoba, Coding facial expressions with gabor wavelets, in 3rd IEEE International Conference on Automatic Face and Gesture Recognition (1998), pp. 200–205
S. Dongcheng, C. Fang, D. Guangyi, Facial expression recognition based on gabor wavelet phase features, in Proceedings of the International Conference on Image and Graphics (ICIG) (2013), pp. 520–523
S. Jain, H. Changbo, J.K. Aggarwal, Facial expression recognition with temporal modeling of shapes, in Proceedings of the IEEE International Conference on Computer Vision Workshop (2011), pp. 1642–1649
I. Buciu, C. Kotropoulos, I. Pitas, ICA and Gabor representation for facial expression recognition, in Proceedings of the International Conference on Image Processing, vol. 2 (2003), pp. 855–858
Z. Guoying, M. Pietikainen, Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
H. Soyel, H. Demirel, Improved SIFT matching for pose robust facial expression recognition, in Proceedings of the International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011) (2011), pp. 585–590
A.S. Md. Sohail, P. Bhattacharya, Classification of facial expressions using K-nearest neighbor classifier, in Computer Vision/Computer Graphics Collaboration Techniques. Lecture Notes in Computer Science, vol. 4418 (Springer, Heidelberg, 2007), pp. 555–566
A. Graves, C. Mayer, M. Wimmer, J. Schmidhuber, B. Radig, Facial expression recognition with recurrent neural networks, in Proceedings of the International Workshop on Cognition for Technical Systems (2008)
E. Owusu, Y. Zhan, Q.R. Mao, A neural-AdaBoost based facial expression recognition system. Expert Syst. Appl. 41(7), 3383–3390 (2014)
S. Lele, J.T. Richtsmeier, Euclidean distance matrix analysis: confidence intervals for form and growth differences. Am. J. Phys. Anthropol. 98(1), 73–86 (1995)
A.K.K. Bermani, A.Z. Ghalwash, A.A.A. Youssif, Automatic facial expression recognition based on hybrid approach. Int. J. Adv. Comput. Sci. Appl. 3(11), 102–107 (2012)
S.L. Happy, A. Routray, Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)
Z. Wang, A.C. Bovik, Mean squared error: love it or leave it? A new look at signal fidelity measures. IEEE Signal Process. Mag. 26(1), 98–117 (2009)
Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
M.J. Wainwright, O. Schwartz, E.P. Simoncelli, Natural image statistics and divisive normalization: modeling non-linearity and adaptation in cortical neurons. in Probabilistic Models of the Brain: Perception and Neural Function (MIT Press, Cambridge, 2002), pp. 203–222
M. Ester, H-P. Kriegel, J. Sander, X. Xu, A density-based algorithm for discovering clusters in large spatial databases with noise, in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96) (AAAI Press, Portland, 1996), pp. 226–231
Y. Gao, A. Rehma, Z. Wang, CW-SSIM based image classification, in Proceedings of the IEEE International Conference on Image Processing (ICIP) (2011), pp. 1249–1252
N. Aifanti, C. Papachristou, A. Delopoulos, The MUG facial expression database, in Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) (2010)
T. Kanade, J.F. Cohn, Y. Tian, Comprehensive database for facial expression analysis, in Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG) (2000), pp. 46–53
P. Lucey, J.F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, I. Matthews, The extended Cohn-Kande dataset (CK+): a complete facial expression dataset for action unit and emotion-specified expression, in Proceedings of the Third IEEE Workshop on Computer Vision and Pattern Recognition for Human Communicative Behavior Analysis (2010)
M.N. Dailey, C. Joyce, M.J. Lyons, M. Kamachi, H. Ishi, J. Gyoba, G.W. Cottrell, Evidence and a computational explanation of cultural differences in facial expression recognition. Emotion 10(6), 874–893 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Bhaskar, H., Torre, D.L., Al-Mualla, M. (2016). Exaggeration Quantified: An Intensity-Based Analysis of Posed Facial Expressions. In: Kawulok, M., Celebi, M., Smolka, B. (eds) Advances in Face Detection and Facial Image Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-25958-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-25958-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25956-7
Online ISBN: 978-3-319-25958-1
eBook Packages: EngineeringEngineering (R0)