Skip to main content
SpringerLink
Log in
Book cover

Understanding Facial Expressions in Communication pp 143–172Cite as

Automatic Facial Expression Analysis

  • Chapter
  • First Online:

Abstract

Automatic Facial Expression Recognition systems have come a long way since the earliest approaches in the early 1970s. We are now at a point where the earliest systems are commercially applied, most notably the smile detectors in digital cameras. But although facial expression recognition is maturing as a research field, it is far from finished. New techniques continue to be developed on all aspects of the processing pipeline: from face detection, via feature extraction to machine learning. Nor is the field blind to the progress made in the social sciences with respect to emotion theory. Gone are the days that people only tried to detect six discrete expressions that were turned-on or off like the switching of lights. The theory of Social Signal Processing now complements classical emotion theory, and modern approaches dissect an expression into its temporal phases, analyse intensity, symmetry, micro-expressions and dynamic differences between morphologically similar expressions. Brave new worlds are opened up—Automatic Facial Expression Analysis is poised to revolutionalise medicine with the advent of behaviomedics, gaming with enriched player–non-player interactions, teleconference meetings with automatic trust and engagement analysis, and human–robot interaction with robots displaying actual empathy.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Ahmed, N., Natarajan, T., & Rao, K. R. (1974). Discrete cosine transform. IEEE Transactions on Computers, 23, 90–93.

    Article  Google Scholar 

  • Almaev, T., & Valstar, M. (2013). Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Proceedings of Affective Computing and Intelligent Interaction.

    Google Scholar 

  • Almaev, T., Yce, A., & Valstar, M. (2013). Distribution-based iterative pairwise classification of emotions in the wild using lgbp-top. In Proceedings of ACM International Conference Multimodal Interaction.

    Google Scholar 

  • Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5), 403–410.

    Article  PubMed  Google Scholar 

  • Ashraf, A. B., Lucey, S., & Chen, T. (2010). Reinterpreting the application of Gabor filters as a manipulation of the margin in linear support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(7), 1335–1341.

    Article  PubMed  Google Scholar 

  • Baltrusaitis, T., McDuf, D., Banda, N., Mahmoud, M., Kaliouby, R. E., Picard, C. et al. (2011). Real-time inference of mental states from facial expressions and upper body gestures (pp. 909–914). In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Banziger, T., & Scherer, K. R. (2010). Introducing the Geneva multimodal emotion portrayal (gemep) corpus. In K. R. Scherer, T. Banziger, & E. B. Roesch (Eds.), Blueprint for affective computing: A sourcebook (pp. 271–294). Oxford: Oxford University Press.

    Google Scholar 

  • Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2006). Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia, 1(6), 22–35.

    Article  Google Scholar 

  • Bazzo, J., & Lamar, M. (2004). Recognizing facial actions using Gabor wavelets with neutral face average difference (pp. 505–510). In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Calix, R., Khazaeli, M., Javadpour, L., & Knapp, G. (2011). Dimensionality reduction and classification analysis on the audio section of the semaine database. In S. D’Mello, A. Graesser, B. Schuller, & J. C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 323–331)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Cen, L., Yu, Z., & Dong, M. (2011). Speech emotion recognition system based on l1 regularized linear regression and decision fusion. In S. D’Mello, A. Graesser, B. Schuller, & J. C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 332–340)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1–27.

    Article  Google Scholar 

  • Chang, K., Liu, T., & Lai, S. (2009). Learning partially-observed hidden conditional random fields for facial expression recognition (pp. 533–540). In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Chen, J., Liu, X., Tu, P., & Aragones, A. (2013). Learning person-specific models for facial expressions and action unit recognition. Pattern Recognition Letters, 34(15), 1964–1970.

    Article  Google Scholar 

  • Chew, S. W., Lucey, P., Saragih, S., Cohn, J. F., & Sridharan, S. (2012). In the pursuit of effective affective computing: The relationship between features and registration. IEEE Trans. Systems, Man and Cybernetics, Part B, 42(4), 1006–1016.

    Google Scholar 

  • Chew, S. W., Lucey, P., Lucey, S., Saragih, J., Cohn, J. F., & Sridharan, S. (2011). Person-independent facial expression detection using constrained local models (pp. 915–920). In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Chu, S., De la Torre, F., & Cohn, J. F. (2013). Selective transfer machine for personalized facial action unit detection. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Cohn, J. F., Reed, L., Ambadar, Z., Xiao, J., & Moriyama, T. (2004). Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior (pp. 610–616). In Procedings of IEEE International Conference on Systems, Man and Cybernetics.

    Google Scholar 

  • Cohn, J. F., & Schmidt, K. L. (2004). The timing of facial motion in posed and spontaneous smiles. International Journal of Wavelets, Multiresolution and Information Processing, 2(2), 121–132.

    Article  Google Scholar 

  • Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., & Schroder, M. (2000). Feeltrace: An instrument for recording perceived emotion in real time (pp. 19–24). In Proceedings of ISCA Workshop on Speech and Emotion.

    Google Scholar 

  • Crabtree, A., Chamberlain, A., Davies, M., Glover, K., Reeves, S., Rodden, T., Jones, M. et al. (2013). Doing innovation in the wild. In Proceedings of CH Italy.

    Google Scholar 

  • Cruz, A., Bhanu, B., & Yang, S. (2011). A psychologically-inspired match-score fusion model for video-based facial expression recognition. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 341–350)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Dahmane, M., & Meunier, J. (2011a). Continuous emotion recognition using gabor energy filters. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 351–358)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Dahmane, M., & Meunier, J. (2011). Emotion recognition using dynamic grid-based hog features (pp. 884–888). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.

    Google Scholar 

  • De la Torre, F., Campoy, J., Ambadar, Z., & Cohn, J. F. (2007). Temporal segmentation of facial behavior (pp. 1–8). In Proceedings of IEEE International Conference on Computer Vision.

    Google Scholar 

  • Donato, G., Bartlett, M. S., Hager, V., Ekman, P., & Sejnowski, T. J. (1999). Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10), 974–989.

    Article  PubMed  PubMed Central  Google Scholar 

  • Douglas-Cowie, E., Cowie, R., Cox, C., Amier N., & Heylen, D. (2008). The sensitive artificial listener: an induction technique for generating emotionally coloured conversation (pp. 1–4). In LREC Workshop on Corpora for Research on Emotion and Affect.

    Google Scholar 

  • Ekman, P., Friesen, W. V., & Hager, J. C. (2002). FACS manual. Salt Lake City: Research Nexus.

    Google Scholar 

  • Fasel, B., & Luettin, J. (2000). Recognition of asymmetric facial action unit activities and intensities (pp. 1100–1103). In Proceedings of International Conference on Pattern Recognition.

    Google Scholar 

  • Fasel, B., & Luettin, J. (2003). Automatic facial expression analysis: a survey. Pattern Recognition, 36(1), 259–275.

    Article  Google Scholar 

  • Fewzee, P., & Karray, F. (2012). Elastic net for paralinguistic speech recognition (pp. 509–516). In Proceedings of the 14th ACM international conference on Multimodal interaction,lCMl ‘12, New York.

    Google Scholar 

  • Fontaine, J., Scherer, K., Roesch, E., & Ellsworth, P. (2007). The world of emotions is not two-dimensional. Psychological Science, 18(2), 1050–1057.

    Article  PubMed  Google Scholar 

  • Gehrig, T., & Ekenel, H. K. (2011). Facial action unit detection using kernel partial least squares. In Proceedings of IEEE International Conference on Computer Vision Workshop.

    Google Scholar 

  • Glodek, M., Schels, M., Palm, G., & Schwenker, F. (2012). Multiple classifier combination using reject options and markov fusion networks (pp. 465–472). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12, New York.

    Google Scholar 

  • Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S., chwenker, F. et al. (2011). Multiple classifier systems for the classification of audio-visual emotional states. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction. Lecture notes in computer science, (vol. 6975, pp. 359–368). Berlin: Springer.

    Google Scholar 

  • Gonzalez, I., Sahli, H., Enescu, V., & Verhelst, W. (2011). Context-independent facial action unit recognition using shape and Gabor phase information (pp. 548–557). In Proceedings of the International Conference on affective computing and intelligent interaction.

    Google Scholar 

  • Gunes, H., Schuller, B., Pantic, M., & Cowie, R. (2011). Emotion representation, analysis and synthesis in continuous space: A survey (pp. 827–834). In Proceedings International Workshop on Emotion Synthesis, representation, and Analysis in Continuous space, EmoSPACE 2011, held in conjunction with the 9th IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, FG 2011, Santa Barbara.

    Google Scholar 

  • Hamm, J., Kohler, C. G., Gur, R. C., & Verma, R. (2011). Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200(2), 237–256.

    Article  PubMed  PubMed Central  Google Scholar 

  • Huang, D., Shan, C., & Ardabilian, M. (2011). Local binary pattern and its application to facial image analysis: A survey. IEEE Transactions on Systems, Man and Cybernetics, Part C, 41, 1–17.

    Article  Google Scholar 

  • Jaiswal, S., Almaev, T., & Valstar, M. (2013). Semi-supervised learning for multi-pose facial point detection in the wild. In Proceedings of ACM International Conference on Computer Vision, in print.

    Google Scholar 

  • Jeni, L. A., Girard, J. M., Cohn, J., & De la Torre, F. (2013). Continuous au intensity estimation using localized, sparse facial feature space. In IEEE International Conference on Automatic Face and Gesture Recognition Workshop.

    Google Scholar 

  • Jeni, L. A., Lorincz, A., Nagy, T., Palotai, Z., Sebok, J., Szabo, Z., et al. (2012). 3d shape estimation in video sequences provides high precision evaluation of facial expressions. Image and Vision Computing, 30(10), 785–795.

    Article  Google Scholar 

  • Jiang, B., Valstar, M. F., Martinez, B., & Pantic, M. (2013). Dynamic appearance descriptor approach to facial actions temporal modelling. IEEE Transactions of Systems, Man and Cybernetics, Part B, Accepted.

    Google Scholar 

  • Jiang, B., Valstar, M., & Pantic, M. (2011). Action unit detection using sparse appearance descriptors in space-time video volumes (pp. 314–321). In Proceedings of IEEE Inernational. Conference on Automatic Face and Gesture Recognition, Santa Barbara.

    Google Scholar 

  • Kaltwang, S., Rudovic, O., & Pantic, M. (2012). Continuous pain intensity estimation from facial expressions. In R. Boyle, B. Parvin, D. Koracin, N. Paragios, & S. M. Tanveer (Eds.), Advances in visual computing. Lecture notes in computer science (vol. 7432, pp. 368–377). Heidelberg: Springer.

    Google Scholar 

  • Kanade, T., Cohn, J., & Tian, Y. (2000). Comprehensive database for facial expression analysis (pp. 46–53). In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Khademi, M., Manzuri-Shalmani, M. T., Kiapour, M. H., & Kiaei, A. A. (2010). Recognizing combinations of facial action units with diferent intensity using a mixture of hidden markov models and neural network (pp. 304–313). In International conference on Multiple Classifier Systems.

    Google Scholar 

  • Kim, J., Rao, H., & Clements, M. (2011). Investigating the use of formant based features for detection of affective dimensions in speech. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 369–377)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Koelstra, S., Pantic, M., & Patras, I. (2010). A dynamic texture based approach to recognition of facial actions and their temporal models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 1940–1954.

    Article  PubMed  Google Scholar 

  • Li, Y., Chen, J., Zhao, Y., & Ji, Q. (2013). Data-free prior model for facial action unit recognition. IEEE Transactions on Affective Computing, 4(2), 127–141.

    Article  Google Scholar 

  • Littlewort, G. C., Bartlett, M. S., & Lee, K. (2009). Automatic coding of facial expressions displayed during posed and genuine pain. Image and Vision Computing, 27, 1797–1803.

    Article  Google Scholar 

  • Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., et al. (2011). Automatically detecting pain in video through facial action units. IEEE Transactions on Systems, Man and Cybernetics, Part B, 41, 664–674.

    Article  Google Scholar 

  • Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (1998). Coding facial expressions with gabor wavelets (pp. 200–205). In Proceedings of Automatic Face and Gesture Recognition, Third IEEE International Conference.

    Google Scholar 

  • Mahoor, M. H., Cadavid, S., Messinger, D. S., & Cohn, J. F. (2009). A framework for automated measurement of the intensity of non-posed facial action units (pp. 74–80). In IEEE Conference on Computer Vision and Pattern Recognition Workshop.

    Google Scholar 

  • Mahoor, M. H., Zhou M., Veon K. L., Mavadati M., & Cohn J. F. (2011). Facial action unit recognition with sparse representation (pp. 336–342). In IEEE International Conference on Automatic Face and Gesture Recognition,.

    Google Scholar 

  • Martinez, B., Valstar, M., Binefa, X., & Pantic, M. (2013). Local evidence aggregation for regression based facial point detection. Transactions on Pattern Analysis and Machine Intelligence, 35(5), 1149–1163.

    Article  Google Scholar 

  • McCallum, A., Freitag, D., & Pereira, F. C. N. (2000). Maximum entropy markov models for information extraction and segmentation (pp. 591–598). In International Conference on Machine Learning.

    Google Scholar 

  • McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012). The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Transactions of Affective Computing, 3, 5–17.

    Article  Google Scholar 

  • Meng, V., & Bianchi-Berthouze, N. (2011). Naturalistic affective expression classification by a multi-stage approach based on hidden markov models. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 378–387)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Meng, H., Romera-Paredes, B., & Berthouze, N. (2011). Emotion recognition by two view svm_2 k classifier on dynamic facial expression features (pp. 854–859). In Proceedings of IEEE International Conference Automatic Face and Gesture Analysis.

    Google Scholar 

  • Mitchell, T. (1997). Machine learning. New York: McGraw Hill.

    Google Scholar 

  • Nicolle, J., Rapp, V., Bailly, K., Prevost, L., & Chetouani, M. (2012). Robust continuous prediction of human emotions using multiscale dynamic cues (pp. 501–508). In Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI ‘12, New York.

    Google Scholar 

  • Ojala, T., Pietikainen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on featured distribution. Pattern Recognition, 29(1), 51–59.

    Article  Google Scholar 

  • Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multi resolution grey-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.

    Article  Google Scholar 

  • Ojansivu, V., & Heikkila, J. (2008). Blur insensitive texture classification using local phase quantization (vol. 5099, pp. 236–243). In Intelligence Conference on Image and Signal Processing.

    Google Scholar 

  • Orozco, J., Martinez, B., & Pantic, M. (2013). Empirical analysis of cascade deformable models for multi-view face detection (pp. 1–5). In IEEE International Conference on Image Processing.

    Google Scholar 

  • Ozkan, D., Scherer, S., & Morency, L.-P. (2012). Step-wise emotion recognition using concatenated-hmm (pp. 477–484). In Proceedings of the 14th ACM International Conference on Multimodal interaction, ICMI ‘12, New York.

    Google Scholar 

  • Pan, S., Tao, J., & Li, Y. (2011). The casia audio emotion recognition method for audio/visual emotion challenge 2011. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 388–395)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.

    Article  Google Scholar 

  • Pantic, M., & Patras, I. (2004). Temporal modeling of facial actions from face profile image sequences (vol. 1, pp. 49–52). In Proceedings of Interanational Conference on Multimedia & Expo.

    Google Scholar 

  • Pantic, M., & Patras, I. (2005). Detecting facial actions and their temporal segments in nearly frontal-view face image sequences (pp. 3358–3363). In Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

    Google Scholar 

  • Pantic, M., & Patras, I. (2006). Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transaction on Systems, Man and Cybernetics, Part B, 36, 433–449.

    Article  Google Scholar 

  • Pantic, M., & Rothkrantz, L. (2000). Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1424–1445.

    Article  Google Scholar 

  • Papageorgiou, C. P., Oren, M., & Poggio, T. (1998). A general framework for object detection (pp. 555–562). In Proceedings of IEEE International Conference on Computer Vision.

    Google Scholar 

  • Ramirez, G., Baltruaitis, T., &. Morency, L.P. (2011). Modeling latent discriminative dynamic of multi-dimensional affective signals. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction. Lecture notes in computer science (vol. 6975, pp. 396–406). Berlin: Springer.

    Google Scholar 

  • Rogers, Y. (2011). Interaction design gone wild: Striving for wild theory. Interactions, 18(4), 58.

    Article  Google Scholar 

  • Rudovic, O., Pavlovic, V., & Pantic, M. (2012). Kernel conditional ordinal random fields for temporal segmentation of facial action units. In European Conference on Computer Vision Workshop.

    Google Scholar 

  • Samal, A., & Iyengar, P. A. (1992). Automatic recognition and analysis of human faces and facial expressions: A survey. Pattern Recognition, 25, 65–77.

    Article  Google Scholar 

  • Saragih, J. M., Lucey, S., & Cohn, J. F. (2011). Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2), 200–215.

    Article  Google Scholar 

  • Savran, A., Cao, H., Shah, M., Nenkova, A., & Verma, R. (2012). Combining video, audio and lexical indicators of affect in spontaneous conversation via particle filtering (pp. 485–492). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12, New York.

    Google Scholar 

  • Savran, A., Sankur, B., & Bilge, M. T. (2012b). Comparative evaluation of 3D versus 2D modality for automatic detection of facial action units. Pattern Recognition, 45(2), 767–782.

    Article  Google Scholar 

  • Savran, A., Sankur, B., & Bilge, M. T. (2012c). Regression-based intensity estimation of facial action units. Image and Vision Computing, 30(10), 774–784.

    Article  Google Scholar 

  • Sayedelahl, A., Fewzee, P., Kamel, M., & Karray, F. (2011). Audio-based emotion recognition from natural conversations based on co-occurrence matrix and frequency domain energy distribution features. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 407–414)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Schuller, B., Valstar, M., Eyben, F., Cowie, R., & Pantic. M. (2012). Avec 2012—the continuous audio/visual emotion challenge (pp. 449–456). In Proceedings ACM International Conference on Multimodal Interaction.

    Google Scholar 

  • Schuller, B., Valstar, M., Eyben, F., McKeown, G., Cowie, R. & Pantic. M. (2011). AVEC 2011—The first international audio/visual emotion challenge (vol. II, pp. 415–424). In Proceedings International Conference on Affective Computing and Intelligent Interaction 2011, ACII 2011, Memphis.

    Google Scholar 

  • Senechal, T., Rapp,V., Prevost, L., Salam, H., Seguier, R., & Bailly, K. (2011). Combining lgbp histograms with aam coefficients in the multi-kernel svm framework to detect facial action units (pp. 860–865). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.

    Google Scholar 

  • Shan, C., Gong, S., & McOwan, P. (2008). Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6), 803–816.

    Article  Google Scholar 

  • Smith, R. S., & Windeatt, T. (2011). Facial action unit recognition using filtered local binary pattern features with bootstrapped and weighted ECOC classifiers. Ensembles in Machine Learning Applications, 373, 1–20.

    Article  Google Scholar 

  • Soladie, C., Salam, H., Pelachaud, C., Stoiber, N., & Seguier, R. (2012). A multimodal fuzzy inference system using a continuous facial expression representation for emotion detection (pp. 493–500). In Proceedings of the 14th ACM International conference on Multimodal interaction, ICMI ‘12, New York.

    Google Scholar 

  • Sun, R., & Moore, E, I. I. (2011). Investigating glottal parameters and teager energy operators in emotion recognition. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 425–434)., Lecture notes in computer science Berlin: Springer.

    Chapter  Google Scholar 

  • Tariq, U., Lin, K. H., Li, Z., Zhou, X., Wang, Z., Le, V., Han, T. et al. (2011). Emotion recognition from an ensemble of features (pp. 872–877). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.

    Google Scholar 

  • Tian, Y., Kanade, T., & Cohn, J. (2001). Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2), 97–115.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tian, Y., Kanade, T., & Cohn. J. F. (2002). Evaluation of gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity (pp. 229–234). In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Tong, Y., Chen, J., & Ji, Q. (2010). A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2), 258–273.

    Article  PubMed  Google Scholar 

  • Tong, Y., Liao, W., & Ji, Q. (2007). Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10), 1683–1699.

    Article  PubMed  Google Scholar 

  • Valstar, M., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011). The first facial expression recognition and analysis challenge (pp. 921–926). In Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, Santa Barbara.

    Google Scholar 

  • Valstar, M., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012). Meta-analysis of the first facial expression recognition challenge. IEEE Transactions on Systems, Man, and Cybernetics-B, 42(4), 966–979.

    Article  Google Scholar 

  • Valstar, M., & Pantic, M. (2010) Induced disgust, happiness and surprise: an addition to the mmi facial expression database (pp. 65–70). In Proceedings of 3rd International Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect.

    Google Scholar 

  • Valstar, M. F., Gunes, H., & Pantic, M. (2007). How to distinguish posed from spontaneous smiles using geometric features (pp. 38–45). In Proceedings of ACM International Conference on Multimodal Interfaces (ICMI’07), Nagoya.

    Google Scholar 

  • Valstar, M. F., & Pantic, M. (2012). Fully automatic recognition of the temporal phases of facial actions. IEEE Transactions on Systems, Man and Cybernetics, 42, 28–43.

    Article  Google Scholar 

  • Valstar, M. F., Pantic, M. Ambadar, Z., & Cohn, J. (2006). Spontaneous vs. posed facial behavior: Automatic analysis of brow actions (pp. 162–170). In Proceedings of ACM International Conference on Multimodal Interfaces (ICMI’06), Banff.

    Google Scholar 

  • Valstar, M. F., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia,S., … Pantic, M. (2013). Avec 2013—The continuous audio/visual emotion and depression recognition challenge. In Procedings of International Conference ACM Multimedia, in print.

    Google Scholar 

  • Van der Maaten, L. (2012). Audio-visual emotion challenge 2012: A simple approach (pp. 473–476). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12. ACM: New York.

    Google Scholar 

  • Van der Maaten, L., & Hendriks, E. (2012). Action unit classification using active appearance models and conditional random field. Cognitive Processing, 13, 507–518.

    Article  PubMed  PubMed Central  Google Scholar 

  • Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D’ericco, F., et al. (2012). Bridging the gap between social animal and unsocial machine: A survey of social signal processing. IEEE Transactions on Afective Computing, 3(1), 69–87.

    Article  Google Scholar 

  • Viola, P., & Jones, M. (2002). Robust real-time object detection. International Jorunal on Computer Vision, 57(2), 137–154.

    Article  Google Scholar 

  • Whitehill, J., & Omlin, C. W. (2006). Haar features for FACS AU recognition. In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2011). Action unit recognition transfer across datasets (pp. 860–865). In IEEE International Conference on Automatic Face and Gesture Recognition.

    Google Scholar 

  • Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2012). Multilayer architectures of facial action unit recognition. IEEE Tranactions on Systems, Man and Cybernetics, Part B, (In print).

    Google Scholar 

  • Yang, P., Liua, Q., & Metaxas, D. N. (2009). Boosting encoded dynamic features for facial expression recognition. Pattern Recognition Letters, 30(2), 132–139.

    Article  Google Scholar 

  • Yang, P., Liua, Q., & Metaxas, D. N. (2011). Dynamic soft encoded patterns for facial event analysis. Computer Vision, and Image Understanding, 115(3), 456–465.

    Article  Google Scholar 

  • Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009). A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 39–58.

    Article  PubMed  Google Scholar 

  • Zhang, C., & Zhang, Z. (2010). A survey of recent advances in face detection. Technical Report MSR-TR-2010-66, Microsoft Research, 2010.

    Google Scholar 

  • Zhang, L., Tong, Y., & Ji, Q. (2008). Active image labeling and its application to facial action labeling (pp. 706–719). In European Conference on Computer Vision.

    Google Scholar 

  • Zhao, G. Y., & Pietikainen, M. (2007). Dynamic texture recognition using local binary pattern with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(6), 915–928.

    Article  Google Scholar 

  • Zhou, F., De la Torre, F., & Cohn, J. F. (2010). Unsupervised discovery of facial events. In IEEE Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation and landmark localization in the wild (pp. 2879–2886). In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.

    Google Scholar 

  • Zhu, Y., De la Torre, F., Cohn, J. F., & Zhang, Y. (2011). Dynamic cascades with bidirectional bootstrapping for action unit detection in spontaneous facial behavior. IEEE Transactions on Affective Computing, 2, 79–91.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, University of Nottingham, Nottingham, UK

    Michel Valstar

Authors
  1. Michel Valstar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michel Valstar .

Editor information

Editors and Affiliations

  1. Defence Institute of Psychological Research, Defence Research and Development Organization, New Delhi, India

    Manas K. Mandal

  2. Defence Institute of Psychological Research, Defence Research and Development Organization, New Delhi, India

    Avinash Awasthi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer India

About this chapter

Cite this chapter

Valstar, M. (2015). Automatic Facial Expression Analysis. In: Mandal, M., Awasthi, A. (eds) Understanding Facial Expressions in Communication. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1934-7_8

Download citation

Publish with us

Policies and ethics

Search

Navigation