Abstract
A method was developed for automated coding of facial behavior in computer-aided test or game situations. Facial behavior is registered automatically with the aid of small plastic dots which are affixed to pre-defined regions of the subject's face. During a task, the subject's face is videotaped, and the picture is digitized. A special pattern-recognition algorithm identifies the dot pattern, and an artificial neural network classifies the dot pattern according to the Facial Action Coding System (FACS; Ekman & Friesen, 1978). The method was tested in coding the posed facial expressions of three subjects, themselves FACS experts. Results show that it is possible to identify and differentiate facial expressions by their corresponding dot patterns. The method is independent of individual differences in physiognorny.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albers, G., Cellerier, G., & Schreter, Z. (1986).Some ideas on connectionism. Geneva, Switzerland: University of Geneva, Faculty of Psychology and Educational Science, Genetic Artificial Intelligence and Epistemics Laboratory.
Ancoli, S. (1978).Psychophysiological response patterns to emotions. Doctoral Dissertation, University of California, San Francisco.
Bänninger-Huber, E. (1985).FACS (Facial Action Coding System) als Instrument zur Erfassung therapeutischer Veränderungen: Analyse von Daten eines Therapieexperimentes mit Stotterern. Unveröff. Dissertation an der Abteilung Klinische Psychologie der Universität Zürich (Unpublished doctoral dissertation, Department of Psychology, University of Zürich, Switzerland).
Bänninger-Huber, E., & Steiner, F. (1985).Methodological problems in assessing changes in psychotherapy with FACS. Paper presented at the 2nd European Conference on Facial Measurement, Saarbrücken.
Bassili, J. N. (1979). Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face.Journal of Personality and Social Psychology, 37, 2049–2058.
Ekman, P. (1982). Methods for measuring facial action. In K. R. Scherer & P. Ekman (Eds.),Handbook of methods in nonverbal behavior research (pp. 45–90). Cambridge: Cambridge University Press.
Ekman, P. (1985).Telling lies: Clues to deceit in the marketplace, politics, and marriage. New York: Norton.
Ekman, P., & Friesen, W. V. (1978).Manual for the Facial Action Coding System. Palo Alto: Consulting Psychologists Press.
Ekman, P., & Friesen, W. V. (1982). Felt, false, and miserable smiles.Journal of Nonverbal Behavior, 6 238–252.
Ekman, P., Hager, J. C., & Friesen, W. V. (1981). The symmetry of emotional and deliberate facial actions.Journal of Psychophysiology, 18 101–106.
Heimann, H., & Lukàcs, G. (1966). Eine Methode zur quantitativen Analyse der mimischen Bewegung.Archiv für die gesamte Psychologie, 118 1–17.
Hirsig, R., & von Burg, J. (1990).Interactive computer games: A new psychodiagnostic instrument. Paper presented at the 8th International Congress of Cybernetics and Systems, New York.
Kaiser, S. (1990).Möglichkeiten und Grenzen der automatischen Kodierung mimischen Verhaltens. Unveröff. Dissertation an der Abteilung Angewandte Psychologie der Universität Zürich (Unpublished doctoral dissertation, Department of Psychology, University of Zürich, Switzerland).
Kaiser, S., & Wehrle, T. (in preparation). Situated action and the display of emotion.
Katsikitis, M., Pilowsky, I., & Innes, J. M. (1990). The quantification of smiling using a micro-computer-based approach.Journal of Nonverbal Behavior, 14 3–17.
Le Cun, Y. (1988).Tutorial 3: Multi-layer neural networks. Paper presented at the International Conference Connectionism in Perspective, Zürich, Switzerland.
Leonard, C. M., Voeller, K. K., & Kuldau, J. M. (1991). When's a smile a smile? Or how to detect a message by digitizing the signal.Psychological Science, 2 166–172.
Lindsay, P. H., & Norman, D. A. (1977).Human information processing. New York: Academic Press.
McClelland, J. L., & Rumelhart, D. E. (1986).Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 2: Psychological and biological models. Cambridge, MA: MIT Press.
Rumelhart, D. E., & McClelland, J. L. (1986).Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations. Cambridge, MA: MIT Press.
Schenker, B. (1987).Entwicklung einer Methode zur Erfassung mimischer Bewegungen. Unveröff. Lizentiatsarbeit an der Abteilung Angewandte Psychologie der Universität Zürich (Unpublished master's thesis, Department of Psychology, University of Zürich).
Schleidt, W. M., Bucher, U., & Gubler, H. (1982).Pattern analysis of human facial expression movement. Paper presented at the 6th International Conference on Pattern Recognition, München.
Thornton, M., & Pilowsky, I. (1982). Facial expression can be modelled mathematically. British Journal of Psychiatry,140 61–63.
Unzner, L., & Schneider, K. (1990). Facial reactions in preschoolers: A descriptive study.Journal of Nonverbal Behavior, 14 19–31.
Wehrle, T. (1988).Kodierung mimischen Verhaltens durch künstliche neuronale Netzwerke. Unveröff. Lizentiatsarbeit an der Abteilung Angewandte Psychologie der Universität Zürich (Unpublished master's thesis, Department of Psychology, University of Zürich).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kaiser, S., Wehrle, T. Automated coding of facial behavior in human-computer interactions with facs. J Nonverbal Behav 16, 67–84 (1992). https://doi.org/10.1007/BF00990323
Issue Date:
DOI: https://doi.org/10.1007/BF00990323