Abstract
Automatic Facial Expression Recognition systems have come a long way since the earliest approaches in the early 1970s. We are now at a point where the earliest systems are commercially applied, most notably the smile detectors in digital cameras. But although facial expression recognition is maturing as a research field, it is far from finished. New techniques continue to be developed on all aspects of the processing pipeline: from face detection, via feature extraction to machine learning. Nor is the field blind to the progress made in the social sciences with respect to emotion theory. Gone are the days that people only tried to detect six discrete expressions that were turned-on or off like the switching of lights. The theory of Social Signal Processing now complements classical emotion theory, and modern approaches dissect an expression into its temporal phases, analyse intensity, symmetry, micro-expressions and dynamic differences between morphologically similar expressions. Brave new worlds are opened up—Automatic Facial Expression Analysis is poised to revolutionalise medicine with the advent of behaviomedics, gaming with enriched player–non-player interactions, teleconference meetings with automatic trust and engagement analysis, and human–robot interaction with robots displaying actual empathy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ahmed, N., Natarajan, T., & Rao, K. R. (1974). Discrete cosine transform. IEEE Transactions on Computers, 23, 90–93.
Almaev, T., & Valstar, M. (2013). Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Proceedings of Affective Computing and Intelligent Interaction.
Almaev, T., Yce, A., & Valstar, M. (2013). Distribution-based iterative pairwise classification of emotions in the wild using lgbp-top. In Proceedings of ACM International Conference Multimodal Interaction.
Ambadar, Z., Schooler, J. W., & Cohn, J. F. (2005). Deciphering the enigmatic face: The importance of facial dynamics in interpreting subtle facial expressions. Psychological Science, 16(5), 403–410.
Ashraf, A. B., Lucey, S., & Chen, T. (2010). Reinterpreting the application of Gabor filters as a manipulation of the margin in linear support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(7), 1335–1341.
Baltrusaitis, T., McDuf, D., Banda, N., Mahmoud, M., Kaliouby, R. E., Picard, C. et al. (2011). Real-time inference of mental states from facial expressions and upper body gestures (pp. 909–914). In IEEE International Conference on Automatic Face and Gesture Recognition.
Banziger, T., & Scherer, K. R. (2010). Introducing the Geneva multimodal emotion portrayal (gemep) corpus. In K. R. Scherer, T. Banziger, & E. B. Roesch (Eds.), Blueprint for affective computing: A sourcebook (pp. 271–294). Oxford: Oxford University Press.
Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2006). Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia, 1(6), 22–35.
Bazzo, J., & Lamar, M. (2004). Recognizing facial actions using Gabor wavelets with neutral face average difference (pp. 505–510). In IEEE International Conference on Automatic Face and Gesture Recognition.
Calix, R., Khazaeli, M., Javadpour, L., & Knapp, G. (2011). Dimensionality reduction and classification analysis on the audio section of the semaine database. In S. D’Mello, A. Graesser, B. Schuller, & J. C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 323–331)., Lecture notes in computer science Berlin: Springer.
Cen, L., Yu, Z., & Dong, M. (2011). Speech emotion recognition system based on l1 regularized linear regression and decision fusion. In S. D’Mello, A. Graesser, B. Schuller, & J. C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 332–340)., Lecture notes in computer science Berlin: Springer.
Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3), 1–27.
Chang, K., Liu, T., & Lai, S. (2009). Learning partially-observed hidden conditional random fields for facial expression recognition (pp. 533–540). In IEEE Conference on Computer Vision and Pattern Recognition.
Chen, J., Liu, X., Tu, P., & Aragones, A. (2013). Learning person-specific models for facial expressions and action unit recognition. Pattern Recognition Letters, 34(15), 1964–1970.
Chew, S. W., Lucey, P., Saragih, S., Cohn, J. F., & Sridharan, S. (2012). In the pursuit of effective affective computing: The relationship between features and registration. IEEE Trans. Systems, Man and Cybernetics, Part B, 42(4), 1006–1016.
Chew, S. W., Lucey, P., Lucey, S., Saragih, J., Cohn, J. F., & Sridharan, S. (2011). Person-independent facial expression detection using constrained local models (pp. 915–920). In IEEE International Conference on Automatic Face and Gesture Recognition.
Chu, S., De la Torre, F., & Cohn, J. F. (2013). Selective transfer machine for personalized facial action unit detection. In IEEE Conference on Computer Vision and Pattern Recognition.
Cohn, J. F., Reed, L., Ambadar, Z., Xiao, J., & Moriyama, T. (2004). Automatic analysis and recognition of brow actions and head motion in spontaneous facial behavior (pp. 610–616). In Procedings of IEEE International Conference on Systems, Man and Cybernetics.
Cohn, J. F., & Schmidt, K. L. (2004). The timing of facial motion in posed and spontaneous smiles. International Journal of Wavelets, Multiresolution and Information Processing, 2(2), 121–132.
Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., & Schroder, M. (2000). Feeltrace: An instrument for recording perceived emotion in real time (pp. 19–24). In Proceedings of ISCA Workshop on Speech and Emotion.
Crabtree, A., Chamberlain, A., Davies, M., Glover, K., Reeves, S., Rodden, T., Jones, M. et al. (2013). Doing innovation in the wild. In Proceedings of CH Italy.
Cruz, A., Bhanu, B., & Yang, S. (2011). A psychologically-inspired match-score fusion model for video-based facial expression recognition. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 341–350)., Lecture notes in computer science Berlin: Springer.
Dahmane, M., & Meunier, J. (2011a). Continuous emotion recognition using gabor energy filters. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 351–358)., Lecture notes in computer science Berlin: Springer.
Dahmane, M., & Meunier, J. (2011). Emotion recognition using dynamic grid-based hog features (pp. 884–888). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.
De la Torre, F., Campoy, J., Ambadar, Z., & Cohn, J. F. (2007). Temporal segmentation of facial behavior (pp. 1–8). In Proceedings of IEEE International Conference on Computer Vision.
Donato, G., Bartlett, M. S., Hager, V., Ekman, P., & Sejnowski, T. J. (1999). Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10), 974–989.
Douglas-Cowie, E., Cowie, R., Cox, C., Amier N., & Heylen, D. (2008). The sensitive artificial listener: an induction technique for generating emotionally coloured conversation (pp. 1–4). In LREC Workshop on Corpora for Research on Emotion and Affect.
Ekman, P., Friesen, W. V., & Hager, J. C. (2002). FACS manual. Salt Lake City: Research Nexus.
Fasel, B., & Luettin, J. (2000). Recognition of asymmetric facial action unit activities and intensities (pp. 1100–1103). In Proceedings of International Conference on Pattern Recognition.
Fasel, B., & Luettin, J. (2003). Automatic facial expression analysis: a survey. Pattern Recognition, 36(1), 259–275.
Fewzee, P., & Karray, F. (2012). Elastic net for paralinguistic speech recognition (pp. 509–516). In Proceedings of the 14th ACM international conference on Multimodal interaction,lCMl ‘12, New York.
Fontaine, J., Scherer, K., Roesch, E., & Ellsworth, P. (2007). The world of emotions is not two-dimensional. Psychological Science, 18(2), 1050–1057.
Gehrig, T., & Ekenel, H. K. (2011). Facial action unit detection using kernel partial least squares. In Proceedings of IEEE International Conference on Computer Vision Workshop.
Glodek, M., Schels, M., Palm, G., & Schwenker, F. (2012). Multiple classifier combination using reject options and markov fusion networks (pp. 465–472). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12, New York.
Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S., chwenker, F. et al. (2011). Multiple classifier systems for the classification of audio-visual emotional states. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction. Lecture notes in computer science, (vol. 6975, pp. 359–368). Berlin: Springer.
Gonzalez, I., Sahli, H., Enescu, V., & Verhelst, W. (2011). Context-independent facial action unit recognition using shape and Gabor phase information (pp. 548–557). In Proceedings of the International Conference on affective computing and intelligent interaction.
Gunes, H., Schuller, B., Pantic, M., & Cowie, R. (2011). Emotion representation, analysis and synthesis in continuous space: A survey (pp. 827–834). In Proceedings International Workshop on Emotion Synthesis, representation, and Analysis in Continuous space, EmoSPACE 2011, held in conjunction with the 9th IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, FG 2011, Santa Barbara.
Hamm, J., Kohler, C. G., Gur, R. C., & Verma, R. (2011). Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200(2), 237–256.
Huang, D., Shan, C., & Ardabilian, M. (2011). Local binary pattern and its application to facial image analysis: A survey. IEEE Transactions on Systems, Man and Cybernetics, Part C, 41, 1–17.
Jaiswal, S., Almaev, T., & Valstar, M. (2013). Semi-supervised learning for multi-pose facial point detection in the wild. In Proceedings of ACM International Conference on Computer Vision, in print.
Jeni, L. A., Girard, J. M., Cohn, J., & De la Torre, F. (2013). Continuous au intensity estimation using localized, sparse facial feature space. In IEEE International Conference on Automatic Face and Gesture Recognition Workshop.
Jeni, L. A., Lorincz, A., Nagy, T., Palotai, Z., Sebok, J., Szabo, Z., et al. (2012). 3d shape estimation in video sequences provides high precision evaluation of facial expressions. Image and Vision Computing, 30(10), 785–795.
Jiang, B., Valstar, M. F., Martinez, B., & Pantic, M. (2013). Dynamic appearance descriptor approach to facial actions temporal modelling. IEEE Transactions of Systems, Man and Cybernetics, Part B, Accepted.
Jiang, B., Valstar, M., & Pantic, M. (2011). Action unit detection using sparse appearance descriptors in space-time video volumes (pp. 314–321). In Proceedings of IEEE Inernational. Conference on Automatic Face and Gesture Recognition, Santa Barbara.
Kaltwang, S., Rudovic, O., & Pantic, M. (2012). Continuous pain intensity estimation from facial expressions. In R. Boyle, B. Parvin, D. Koracin, N. Paragios, & S. M. Tanveer (Eds.), Advances in visual computing. Lecture notes in computer science (vol. 7432, pp. 368–377). Heidelberg: Springer.
Kanade, T., Cohn, J., & Tian, Y. (2000). Comprehensive database for facial expression analysis (pp. 46–53). In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition.
Khademi, M., Manzuri-Shalmani, M. T., Kiapour, M. H., & Kiaei, A. A. (2010). Recognizing combinations of facial action units with diferent intensity using a mixture of hidden markov models and neural network (pp. 304–313). In International conference on Multiple Classifier Systems.
Kim, J., Rao, H., & Clements, M. (2011). Investigating the use of formant based features for detection of affective dimensions in speech. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 369–377)., Lecture notes in computer science Berlin: Springer.
Koelstra, S., Pantic, M., & Patras, I. (2010). A dynamic texture based approach to recognition of facial actions and their temporal models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(11), 1940–1954.
Li, Y., Chen, J., Zhao, Y., & Ji, Q. (2013). Data-free prior model for facial action unit recognition. IEEE Transactions on Affective Computing, 4(2), 127–141.
Littlewort, G. C., Bartlett, M. S., & Lee, K. (2009). Automatic coding of facial expressions displayed during posed and genuine pain. Image and Vision Computing, 27, 1797–1803.
Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., et al. (2011). Automatically detecting pain in video through facial action units. IEEE Transactions on Systems, Man and Cybernetics, Part B, 41, 664–674.
Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (1998). Coding facial expressions with gabor wavelets (pp. 200–205). In Proceedings of Automatic Face and Gesture Recognition, Third IEEE International Conference.
Mahoor, M. H., Cadavid, S., Messinger, D. S., & Cohn, J. F. (2009). A framework for automated measurement of the intensity of non-posed facial action units (pp. 74–80). In IEEE Conference on Computer Vision and Pattern Recognition Workshop.
Mahoor, M. H., Zhou M., Veon K. L., Mavadati M., & Cohn J. F. (2011). Facial action unit recognition with sparse representation (pp. 336–342). In IEEE International Conference on Automatic Face and Gesture Recognition,.
Martinez, B., Valstar, M., Binefa, X., & Pantic, M. (2013). Local evidence aggregation for regression based facial point detection. Transactions on Pattern Analysis and Machine Intelligence, 35(5), 1149–1163.
McCallum, A., Freitag, D., & Pereira, F. C. N. (2000). Maximum entropy markov models for information extraction and segmentation (pp. 591–598). In International Conference on Machine Learning.
McKeown, G., Valstar, M., Cowie, R., Pantic, M., & Schroder, M. (2012). The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent. IEEE Transactions of Affective Computing, 3, 5–17.
Meng, V., & Bianchi-Berthouze, N. (2011). Naturalistic affective expression classification by a multi-stage approach based on hidden markov models. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 378–387)., Lecture notes in computer science Berlin: Springer.
Meng, H., Romera-Paredes, B., & Berthouze, N. (2011). Emotion recognition by two view svm_2 k classifier on dynamic facial expression features (pp. 854–859). In Proceedings of IEEE International Conference Automatic Face and Gesture Analysis.
Mitchell, T. (1997). Machine learning. New York: McGraw Hill.
Nicolle, J., Rapp, V., Bailly, K., Prevost, L., & Chetouani, M. (2012). Robust continuous prediction of human emotions using multiscale dynamic cues (pp. 501–508). In Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI ‘12, New York.
Ojala, T., Pietikainen, M., & Harwood, D. (1996). A comparative study of texture measures with classification based on featured distribution. Pattern Recognition, 29(1), 51–59.
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002). Multi resolution grey-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 971–987.
Ojansivu, V., & Heikkila, J. (2008). Blur insensitive texture classification using local phase quantization (vol. 5099, pp. 236–243). In Intelligence Conference on Image and Signal Processing.
Orozco, J., Martinez, B., & Pantic, M. (2013). Empirical analysis of cascade deformable models for multi-view face detection (pp. 1–5). In IEEE International Conference on Image Processing.
Ozkan, D., Scherer, S., & Morency, L.-P. (2012). Step-wise emotion recognition using concatenated-hmm (pp. 477–484). In Proceedings of the 14th ACM International Conference on Multimodal interaction, ICMI ‘12, New York.
Pan, S., Tao, J., & Li, Y. (2011). The casia audio emotion recognition method for audio/visual emotion challenge 2011. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 388–395)., Lecture notes in computer science Berlin: Springer.
Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.
Pantic, M., & Patras, I. (2004). Temporal modeling of facial actions from face profile image sequences (vol. 1, pp. 49–52). In Proceedings of Interanational Conference on Multimedia & Expo.
Pantic, M., & Patras, I. (2005). Detecting facial actions and their temporal segments in nearly frontal-view face image sequences (pp. 3358–3363). In Proceedings of IEEE International Conference on Systems, Man and Cybernetics.
Pantic, M., & Patras, I. (2006). Dynamics of facial expression: Recognition of facial actions and their temporal segments from face profile image sequences. IEEE Transaction on Systems, Man and Cybernetics, Part B, 36, 433–449.
Pantic, M., & Rothkrantz, L. (2000). Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1424–1445.
Papageorgiou, C. P., Oren, M., & Poggio, T. (1998). A general framework for object detection (pp. 555–562). In Proceedings of IEEE International Conference on Computer Vision.
Ramirez, G., Baltruaitis, T., &. Morency, L.P. (2011). Modeling latent discriminative dynamic of multi-dimensional affective signals. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction. Lecture notes in computer science (vol. 6975, pp. 396–406). Berlin: Springer.
Rogers, Y. (2011). Interaction design gone wild: Striving for wild theory. Interactions, 18(4), 58.
Rudovic, O., Pavlovic, V., & Pantic, M. (2012). Kernel conditional ordinal random fields for temporal segmentation of facial action units. In European Conference on Computer Vision Workshop.
Samal, A., & Iyengar, P. A. (1992). Automatic recognition and analysis of human faces and facial expressions: A survey. Pattern Recognition, 25, 65–77.
Saragih, J. M., Lucey, S., & Cohn, J. F. (2011). Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision, 91(2), 200–215.
Savran, A., Cao, H., Shah, M., Nenkova, A., & Verma, R. (2012). Combining video, audio and lexical indicators of affect in spontaneous conversation via particle filtering (pp. 485–492). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12, New York.
Savran, A., Sankur, B., & Bilge, M. T. (2012b). Comparative evaluation of 3D versus 2D modality for automatic detection of facial action units. Pattern Recognition, 45(2), 767–782.
Savran, A., Sankur, B., & Bilge, M. T. (2012c). Regression-based intensity estimation of facial action units. Image and Vision Computing, 30(10), 774–784.
Sayedelahl, A., Fewzee, P., Kamel, M., & Karray, F. (2011). Audio-based emotion recognition from natural conversations based on co-occurrence matrix and frequency domain energy distribution features. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 407–414)., Lecture notes in computer science Berlin: Springer.
Schuller, B., Valstar, M., Eyben, F., Cowie, R., & Pantic. M. (2012). Avec 2012—the continuous audio/visual emotion challenge (pp. 449–456). In Proceedings ACM International Conference on Multimodal Interaction.
Schuller, B., Valstar, M., Eyben, F., McKeown, G., Cowie, R. & Pantic. M. (2011). AVEC 2011—The first international audio/visual emotion challenge (vol. II, pp. 415–424). In Proceedings International Conference on Affective Computing and Intelligent Interaction 2011, ACII 2011, Memphis.
Senechal, T., Rapp,V., Prevost, L., Salam, H., Seguier, R., & Bailly, K. (2011). Combining lgbp histograms with aam coefficients in the multi-kernel svm framework to detect facial action units (pp. 860–865). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.
Shan, C., Gong, S., & McOwan, P. (2008). Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6), 803–816.
Smith, R. S., & Windeatt, T. (2011). Facial action unit recognition using filtered local binary pattern features with bootstrapped and weighted ECOC classifiers. Ensembles in Machine Learning Applications, 373, 1–20.
Soladie, C., Salam, H., Pelachaud, C., Stoiber, N., & Seguier, R. (2012). A multimodal fuzzy inference system using a continuous facial expression representation for emotion detection (pp. 493–500). In Proceedings of the 14th ACM International conference on Multimodal interaction, ICMI ‘12, New York.
Sun, R., & Moore, E, I. I. (2011). Investigating glottal parameters and teager energy operators in emotion recognition. In S. D’Mello, A. Graesser, B. Schuller, & J.-C. Martin (Eds.), Affective computing and intelligent interaction (Vol. 6975, pp. 425–434)., Lecture notes in computer science Berlin: Springer.
Tariq, U., Lin, K. H., Li, Z., Zhou, X., Wang, Z., Le, V., Han, T. et al. (2011). Emotion recognition from an ensemble of features (pp. 872–877). In Proceedings of IEEE International Conference on Automatic Face and Gesture Analysis.
Tian, Y., Kanade, T., & Cohn, J. (2001). Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2), 97–115.
Tian, Y., Kanade, T., & Cohn. J. F. (2002). Evaluation of gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity (pp. 229–234). In IEEE International Conference on Automatic Face and Gesture Recognition.
Tong, Y., Chen, J., & Ji, Q. (2010). A unified probabilistic framework for spontaneous facial action modeling and understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2), 258–273.
Tong, Y., Liao, W., & Ji, Q. (2007). Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(10), 1683–1699.
Valstar, M., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011). The first facial expression recognition and analysis challenge (pp. 921–926). In Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, Santa Barbara.
Valstar, M., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012). Meta-analysis of the first facial expression recognition challenge. IEEE Transactions on Systems, Man, and Cybernetics-B, 42(4), 966–979.
Valstar, M., & Pantic, M. (2010) Induced disgust, happiness and surprise: an addition to the mmi facial expression database (pp. 65–70). In Proceedings of 3rd International Workshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect.
Valstar, M. F., Gunes, H., & Pantic, M. (2007). How to distinguish posed from spontaneous smiles using geometric features (pp. 38–45). In Proceedings of ACM International Conference on Multimodal Interfaces (ICMI’07), Nagoya.
Valstar, M. F., & Pantic, M. (2012). Fully automatic recognition of the temporal phases of facial actions. IEEE Transactions on Systems, Man and Cybernetics, 42, 28–43.
Valstar, M. F., Pantic, M. Ambadar, Z., & Cohn, J. (2006). Spontaneous vs. posed facial behavior: Automatic analysis of brow actions (pp. 162–170). In Proceedings of ACM International Conference on Multimodal Interfaces (ICMI’06), Banff.
Valstar, M. F., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia,S., … Pantic, M. (2013). Avec 2013—The continuous audio/visual emotion and depression recognition challenge. In Procedings of International Conference ACM Multimedia, in print.
Van der Maaten, L. (2012). Audio-visual emotion challenge 2012: A simple approach (pp. 473–476). In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI ‘12. ACM: New York.
Van der Maaten, L., & Hendriks, E. (2012). Action unit classification using active appearance models and conditional random field. Cognitive Processing, 13, 507–518.
Vinciarelli, A., Pantic, M., Heylen, D., Pelachaud, C., Poggi, I., D’ericco, F., et al. (2012). Bridging the gap between social animal and unsocial machine: A survey of social signal processing. IEEE Transactions on Afective Computing, 3(1), 69–87.
Viola, P., & Jones, M. (2002). Robust real-time object detection. International Jorunal on Computer Vision, 57(2), 137–154.
Whitehill, J., & Omlin, C. W. (2006). Haar features for FACS AU recognition. In IEEE International Conference on Automatic Face and Gesture Recognition.
Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2011). Action unit recognition transfer across datasets (pp. 860–865). In IEEE International Conference on Automatic Face and Gesture Recognition.
Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2012). Multilayer architectures of facial action unit recognition. IEEE Tranactions on Systems, Man and Cybernetics, Part B, (In print).
Yang, P., Liua, Q., & Metaxas, D. N. (2009). Boosting encoded dynamic features for facial expression recognition. Pattern Recognition Letters, 30(2), 132–139.
Yang, P., Liua, Q., & Metaxas, D. N. (2011). Dynamic soft encoded patterns for facial event analysis. Computer Vision, and Image Understanding, 115(3), 456–465.
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009). A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 39–58.
Zhang, C., & Zhang, Z. (2010). A survey of recent advances in face detection. Technical Report MSR-TR-2010-66, Microsoft Research, 2010.
Zhang, L., Tong, Y., & Ji, Q. (2008). Active image labeling and its application to facial action labeling (pp. 706–719). In European Conference on Computer Vision.
Zhao, G. Y., & Pietikainen, M. (2007). Dynamic texture recognition using local binary pattern with an application to facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(6), 915–928.
Zhou, F., De la Torre, F., & Cohn, J. F. (2010). Unsupervised discovery of facial events. In IEEE Conference on Computer Vision and Pattern Recognition.
Zhu, X., & Ramanan, D. (2012). Face detection, pose estimation and landmark localization in the wild (pp. 2879–2886). In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition.
Zhu, Y., De la Torre, F., Cohn, J. F., & Zhang, Y. (2011). Dynamic cascades with bidirectional bootstrapping for action unit detection in spontaneous facial behavior. IEEE Transactions on Affective Computing, 2, 79–91.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer India
About this chapter
Cite this chapter
Valstar, M. (2015). Automatic Facial Expression Analysis. In: Mandal, M., Awasthi, A. (eds) Understanding Facial Expressions in Communication. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1934-7_8
Download citation
DOI: https://doi.org/10.1007/978-81-322-1934-7_8
Published:
Publisher Name: Springer, New Delhi
Print ISBN: 978-81-322-1933-0
Online ISBN: 978-81-322-1934-7
eBook Packages: Behavioral ScienceBehavioral Science and Psychology (R0)