Abstract
FACS (Facial Action Coding System) coding is the state of the art in manual measurement of facial actions. FACS coding, however, is labor intensive and difficult to standardize. A goal of automated FACS coding is to eliminate the need for manual coding and realize automatic recognition and analysis of facial actions. Success of this effort depends in part on access to reliably coded corpora; however, manual FACS coding remains expensive and slow. This paper proposes Fast-FACS, a computer vision aided system that improves speed and reliability of FACS coding. Three are the main novelties of the system: (1) to the best of our knowledge, this is the first paper to predict onsets and offsets from peaks, (2) use Active Appearance Models for computer assisted FACS coding, (3) learn an optimal metric to predict onsets and offsets from peaks. The system was tested in the RU-FACS database, which consists of natural facial behavior during a two-person interview. Fast-FACS reduced manual coding time by nearly 50% and demonstrated strong concurrent validity with manual FACS coding.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ekman, P., Friesen, W.: Facial Action Coding System: A technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)
Cohn, J.F., Kanade, T.: Use of automated facial image analysis for measurement of emotion expression. In: The Handbook of Emotion Elicitation and Assessment. Series in Affective Science. Oxford University Press, Oxford (2007)
Bartlett, M., Littlewort, G., Lainscsek, C., Fasel, I., Frank, M., Movellan, J.: Fully automatic facial action recognition in spontaneous behavior. In: 7th International Conference on Automatic Face and Gesture Recognition (2006)
Pantic, M., Patras, I.: Dynamics of Facial Expression: Recognition of Facial Actions and their Temporal Segments from Face Profile Image Sequences. IEEE Transactions on Systems, Man, and Cybernetics - Part B 36(2), 433–449 (2006)
Cohn, J.F., Ekman, P.: Measuring facial action by manual coding, facial EMG, and automatic facial image analysis. In: Harrigan, J.A., Rosenthal, R., Scherer, K. (eds.) Handbook of Nonverbal Behavior Research Methods in the Affective Sciences, NY, Oxford (2005)
Valstar, M., Pantic, M.: Combined support vector machines and hidden Markov models for modeling facial action temporal dynamics. In: Proceedings of IEEE Workshop on Human Computer Interaction. In conjunction with IEEE ICCV 2007, pp. 118–127 (2007)
Zhang, L., Tong, Y., Ji, Q.: Active image labeling and its application to facial action labeling. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 706–719. Springer, Heidelberg (2008)
De la Torre, F., Campoy, J., Ambadar, Z., Cohn, J.F.: Temporal segmentation of facial behavior. In: International Conference on Computer Vision (2007)
De la Torre, F., Nguyen, M.: Parameterized kernel principal component analysis: Theory and applications to supervised and unsupervised image alignment. In: IEEE Computer Vision and Pattern Recognition (2008)
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998, Part II. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)
Matthews, I., Baker, S.: Active appearance models revisited. International Journal of Computer Vision 60(2), 135–164 (2004)
Xing, E., Ng, A., Jordan, M., Russell, S.: Distance metric learning, with application to clustering with side-information. In: Advances in Neural Information Processing Systems, vol. 15, pp. 505–512. MIT Press, Cambridge (2003)
Grant, M., Boyd, S.: Cvx: Matlab software for disciplined convex programming (2008), http://stanford.edu/~boyd/cvx
Frank, M., Movellan, J., Bartlett, M., Littleworth, G.: RU-FACS-1 database, Machine Perception Laboratory, U.C. San Diego
Noldus information technology. Observer XT 10, The Netherlands (2011)
Sayette, M.A., Cohn, J.F., Wertz, J.M., Perrott, M., Parrott, D.J.: A psychometric evaluation of the facial action coding system for assessing spontaneous expression. Journal of Nonverbal Behavior (25), 167–186 (2007)
Cohn, J.F., Ambadar, Z., Ekman, P.: Observer-based measurement of facial expression with the Facial Action Coding System. In: Coan, J.A., Allen, J.J.B. (eds.) The Handbook of Emotion Elicitation and Assessment. Series in Affective Science. Oxford University Press, Oxford (2007)
Ekman, P., Friesen, W., Hager, J.: Facial action coding system: Research nexus. Network Research Information, Salt Lake City, UT (2002)
Fleiss, J.: Statistical methods for rates and proportions. Wiley, New York (1981)
Ambadar, Z., Cohn, J.F., Reed, L.: All smiles are not created equal: Morphology and timing of smiles perceived as amused, polite, and embarrassed/nervous. Journal of Nonverbal Behavior 33(1), 17–34 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
De la Torre, F., Simon, T., Ambadar, Z., Cohn, J.F. (2011). Fast-FACS: A Computer-Assisted System to Increase Speed and Reliability of Manual FACS Coding. In: D’Mello, S., Graesser, A., Schuller, B., Martin, JC. (eds) Affective Computing and Intelligent Interaction. ACII 2011. Lecture Notes in Computer Science, vol 6974. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24600-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-24600-5_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24599-2
Online ISBN: 978-3-642-24600-5
eBook Packages: Computer ScienceComputer Science (R0)