Abstract
Attention is drawn to three interrelated types of error that are committed with high frequencies in the description and analysis of studies of nonverbal behavior. The errors involve the calculation of inappropriate measures of accuracy, the use in statistical analyses of inappropriate chance levels, and misapplications ofX 2 and binomial statistical tests. Almost all papers published between 1979 and 1991 that reported performance separately for different stimulus and response classes suffer from one or more of these errors. The potential consequences of these errors are described, and a variety of proposed measures of performance is examined. Since all measures formerly proposed have weaknesses, a new and easily calculated measure, an unbiased hit rate (H u ), is proposed. This measure is the joint probability that a stimulus category is correctly identified given that it is presented at all and that a response is correctly used given that it is used at all. Two available data sets are reanalyzed using this measure, and the differences in the conclusions reached compared to those reached with an analysis of hit rates are described.
Similar content being viewed by others
Studies Included in Summary
Allen, V. L., & Atkinson, M. L. (1981). Identification of spontaneous and deliberate behavior.Journal of Nonverbal Behavior, 5, 224–237.
Apple, W., & Hecht, K. (1982). Speaking emotionally: the relation between verbal and vocal communication of affect.Journal of Personality and Social Psychology, 42, 864–875.
Beck, L., & Feldman, R. S. (1989). Enhancing children's decoding of facial expression.Journal of Nonverbal Behavior, 13, 269–278.
Boucher, J. D., & Carlson, G. E. (1980). Recognition of facial expression in three cultures.Journal of Cross-Cultural Psychology, 11, 263–280.
Bowers, D., Bauer, R. M., Coslett, H. B., & Hellman, K. M. (1985). Processing of faces by patients with unilateral hemisphere lesions.Brain and Cognition, 4, 258–272.
Brosgole, L., Kurucz, J., Plahovinsak, T. J., Sprotte, C., & Haveliwala, Y. A. (1983). Facial- and postural-affect recognition in senile elderly persons.International Journal of Neuroscience, 22, 37–46.
Buck, R., Baron, R., Goodman, N., & Shapiro, B. (1980). Unitization of spontaneous nonverbal behavior in the study of emotion communication.Journal of Personality and Social Psychology, 39, 522–529.
Buck, R. W., & Duffy, R. J. (1980). Nonverbal communication of affect in brain-damaged patients.Cortex, 16, 351–362.
Camras, L. A. (1980). Children's understanding of facial expressions used during conflict encounters.Child Development, 51, 879–885.
Camras, L. A., Grow, J. G., & Ribordy, S. C. (1983). Recognition of emotional expression by abused children.Journal of Clinical Child Psychology, 12, 325–328.
Camras, L. A., & Allison, K. (1985). Children's understanding of emotional facial expressions and verbal labels.Journal of Nonverbal Behavior, 9, 84–94.
Camras, L. A., Ribordy, S., Hill, J., Martino, S., Spaccarelli, S., & Stefani, R. (1988). Recognition and posing of emotional expressions by abused children and their mothers.Developmental Psychology, 24, 776–781.
Camras, L. A., Ribordy, S., Hill, J., Martino, S., Sachs, V., Spaccarelli, S., & Stefani, R. (1990). Maternal facial behavior and the recognition and production of emotional expression by maltreated and nonmaltreated children.Developmental Psychology, 26, 304–312.
Carlson, C. R., Felleman, E. S., & Masters, J. C. (1983). Influence of children's emotional states on the recognition of emotion in peers and social motives to change another's emotional state.Motivation and Emotion, 7, 61–79. (a)
Carlson, C. R., Gantz, F., & Masters, J. C. (1983). Adults' emotional states and their recognition of emotion in young children.Motivation and Emotion, 7, 81–101. (b)
Chan, D. W. (1985). Perception and judgement of facial expressions among the Chinese.International Journal of Psychology, 20, 681–692.
Crozier, W. R. (1981). Do photographs of facial displays provide a sound basis for classifying the primary emotions?Current Psychological Research, 1, 199–202.
Custrini, R. J., & Feldman, R. S. (1989). Children's social competence and nonverbal encoding and decoding of emotions.Journal of Clinical Child Psychology, 18, 336–342.
Denham, S. A., & Couchoud, E. A. (1990). Young preschoolers' understanding of emotions.Child Study Journal, 20, 171–192.
Ducci, L., Arcuri, L., W/Georgis, T., & Sineshaw, T. (1982). Emotion recognition in Ethiopia: The effect of familiarity with Western culture on accuracy of recognition.Journal of Cross-Cultural Psychology, 13, 340–351.
Duda, P. D., & Brown, J. (1984). Lateral asymmetry of positive and negative emotions.Cortex, 20, 253–261.
Edelmann, R. J., & Hampson, S. E. (1981). The recognition of embarrassment.Personality and Social Psychology Bulletin, 7, 109–116.
Ehlers, L., & Dalby, M. (1987). Appreciation of emotional expressions in the visual and auditory modality in normal and brain damaged patients.Acta Neurologica Scandinavica, 76, 251–256.
Ekman, P., & Friesen, W. V. (1986). A new pan-cultural facial expression of emotion.Motivation and Emotion, 10, 159–168.
Ekman, P., & Heider, K. G. (1988). The universality of a contempt expression: A replication.Motivation and Emotion, 12, 303–308.
Feinman, J. A., & Feldman, R. S. (1982). Decoding children's expressions of affect.Child Development, 53, 710–716.
Felleman, E. S., Barden, R. C., Carlson, C. R., Rosenberg, L., & Masters, J. C. (1983). Children's and adults' recognition of spontaneous and posed emotional expressions in young children.Developmental Psychology, 19, 405–413.
Field, T. M., & Walden, T. A. (1982). Production and discrimination of facial expressions by preschool children.Child Development, 53, 1299–1311.
Friedman, H. S., Prince, L. M., Riggio, R. E., & DiMatteo, M. R. (1980). Understanding and assessing nonverbal expressiveness: The Affective Communication Test.Journal of Personality and Social Psychology, 39, 333–351.
Friedman, H. S., Riggio, R. E., & Segall, D. O. (1980). Personality and the enactment of emotion.Journal of Nonverbal Behavior, 5, 35–48.
Gallagher, D., & Shuntich, R. J. (1981). Encoding and decoding of nonverbal behavior through facial expressions.Journal of Research in Personality, 15, 241–252.
Goldenthal, P. (1985). Posing and judging facial expressions of emotion: The effects of social skills.Journal of Social and Clinical Psychology, 3, 325–338.
Gray, J. M., Fraser, W. L., & Leudar, I. (1983). Recognition of emotion from facial expression in mental handicap.British Journal of Psychiatry, 142, 566–571.
Guthrie, P. T., & Smouse, A. D. (1981). Perception of emotions and attribution of acceptance by normal and emotionally disturbed children.Journal of Nonverbal Behavior, 5, 253–263.
Hall, J. A., & Levin, S. (1980). Affect and verbal-nonverbal discrepancy in schizophrenic and non-schizophrenic family communication.British Journal of Psychiatry, 137, 78–92.
Harrigan, J. A. (1984). The effects of task order on children's identification of facial expressions.Motivation and Emotion, 8, 157–169.
Hobson, R. P., Ouston, J., & Lee, A. (1988). Emotion recognition in autism: Coordinating faces and voices.Psychological Medicine, 18, 911–923.
Jackson, H. F., & Moffat, N. J. (1987). Impaired emotional recognition following severe head injury.Cortex, 23, 293–300.
Kilbride, J. E., & Yarczower, M. (1980). Recognition and imitation of facial expressions: A cross-cultural comparison between Zambia and the United States.Journal of Cross-Cultural Psychology, 11, 281–296.
Kilbride, J. E., & Yarczower, M. (1983). Ethnic bias in the recognition of facial expressions.Journal of Nonverbal Behavior, 8, 27–41.
Kirouac, G., & Doré, F. Y. (1982). Identification des expressions faciales émotionnelles par un échantillon québécois francophone.International Journal of Psychology, 17, 1–7.
Kirouac, G., & Doré, F. Y. (1983). Accuracy and latency of judgment of facial expressions of emotions.Perceptual and Motor Skills, 57, 683–686.
Kirouac, G., & Doré, F. Y. (1985). Accuracy of the judgement of facial expression of emotions as a function of sex and level of education.Journal of Nonverbal Behavior, 9, 3–7.
Kleck, R. E., & Mendolia, M. (1990). Decoding of profile versus full-face expressions of affect.Journal of Nonverbal Behavior, 14, 35–49.
Kropp, J. P., & Haynes, O. M. (1987). Abusive and nonabusive mothers' ability to identify general and specific emotion signals of infants.Child Development, 58, 187–190.
Larrance, D. T., & Zuckerman, M. (1981). Facial attractiveness and vocal likeability as determinants of nonverbal sending skills.Journal of Personality, 49, 349–362.
Leathers, D. G., & Emigh, T. H. (1980). Decoding facial expressions: A new test with decoding norms.Quarterly Journal of Speech, 66, 418–436.
Levin, S., Hall, J. A., Knight, R. A., & Alpert, M. (1985). Verbal and nonverbal expression of affect in speech of schizophrenic and depressed patients.Journal of Abnormal Psychology, 94, 487–497.
Macdonald, H., Rutter, M., Howlin, P., Rios, P., Le Conteur, A., Evered, C., & Folstein, S. (1989). Recognition and expression of emotional cues by autistic and normal adults.Journal of Child Psychology and Psychiatry, 30, 865–877.
Mandal, M. K., & Palchoudhury, S. (1985). Decoding of facial affect in schizophrenia.Psychological Reports, 56, 651–652.
Mandal, M. K., & Palchoudhury, S. (1985). Perceptual skill in decoding facial affect.Perceptual and Motor Skills, 60, 96–98.
Maurer, H., & Newbrough, J. R. (1987). Facial expressions of mentally retarded and nonretarded children: I. Recognition by mentally retarded and nonretarded adults.American Journal of Mental Deficiency, 91, 505–510. (a)
Maurer, H., & Newbrough, J. R. (1987). Facial expressions of mentally retarded and nonretarded children: II. Recognition by nonretarded adults with varying experience with mental retardation.American Journal of Mental Deficiency, 91, 511–515. (b)
McAndrew, F. T. (1986). A cross-cultural study of recognition thresholds for facial expressions of emotion.Journal of Cross-Cultural Psychology, 17, 211–224.
McCown, W., Johnson, J., & Austin, S. (1986). Inability of delinquents to recognize facial affects.Journal of Social Behavior and Personality, 1, 489–496.
McDonald, P. W., & Prkachin, K. M. (1990). The expression and perception of facial emotion in alexithymia: A pilot study.Psychosomatic Medicine, 52, 199–210.
Montepare, J. M., Goldstein, S. B., & Clausen, A. (1987). The identification of emotions from gait information.Journal of Nonverbal Behavior, 11, 33–42.
Motley, M. T., & Camden, C. T. (1988). Facial expression of emotion: A comparison of posed expressions versus spontaneous expressions in an interpersonal communication setting.Western Journal of Speech Communication, 52, 1–22.
Nowicki, S., & Hartigan, M. (1988). Accuracy of facial affect recognition as a function of locus of control orientation and anticipated interpersonal interaction.Journal of Social Psychology, 128, 363–372.
Nummenmaa, T. (1988). The recognition of pure and blended facial expressions of emotion from still photographs.Scandinavian Journal of Psychology, 29, 33–47.
Nummenmaa, T. (1990). Sender repertoires of pure and blended facial expressions of emotion.Scandinavian Journal of Psychology, 31, 161–180.
Philippot, P., & Feldman, R. S. (1990). Age and social competence in preschoolers' decoding of facial expression.British Journal of Social Psychology, 29, 43–54.
Reichenbach, L., & Masters, J. C. (1983). Children's use of expressive and contextual cues in judgments of emotion.Child Development, 54, 993–1004.
Riggio, R. E., Widaman, K. F., & Friedman, H. S. (1985). Actual and perceived emotional sending and personality correlates.Journal of Nonverbal Behavior, 9, 69–83.
Rosenstein, D., & Oster, H. (1988). Differential facial responses to four basic tastes in newborns.Child Development, 59, 1555–1568.
Rotter, N. G., & Rotter, G. S. (1988). Sex differences in the encoding and decoding of negative facial emotions.Journal of Nonverbal Behavior, 12, 139–148.
Schneider, B. (1981). L'Évolution génétique du décodage des émotions faciales chez les jeunes sourds: Comparison avec l'entendant.Revue de Psychologie Appliquée, 31, 415–435.
Sincoff, J. B., & Rosenthal, R. (1985). Content-masking methods as determinants of results of studies of nonverbal communication.Journal of Nonverbal Behavior, 9, 121–129.
Sogon, S., & Izard, C. E. (1985). Ability to recognize emotion in normal and mentally retarded Japanese children.Japanese Psychological Research, 27, 125–132.
Stanners, R. F., Byrd, D. M., & Gabriel, R. (1985). The time it takes to identify facial expressions: Effects of age, gender of subject, sex of sender, and type of expression.Journal of Nonverbal Behavior, 9, 201–213.
Stifter, C. A., & Fox, N. A. (1987). Preschool children's ability to identify and label emotions.Journal of Nonverbal Behavior, 11, 43–54.
Toner, H. L., & Gates, G. R. (1985). Emotional traits and recognition of facial expression of emotion.Journal of Nonverbal Behavior, 9, 48–66.
Tremblay, C., Kirouac, G., & Doré, F. Y. (1987). The recognition of adults' and children's facial expressions of emotions.Journal of Psychology, 121, 341–350.
Tucker, J. S., & Riggio, R. E. (1988). The role of social skills in encoding posed and spontaneous facial expressions.Journal of Nonverbal Behavior, 12, 87–97.
Van Bezooijen, R., Otto, S. A., & Heenan, T. A. (1983). Recognition of vocal expressions of emotion: A three-nation study to identify universal characteristics.Journal of Cross-Cultural Psychology, 14, 387–406.
Wagner, H. L. (1990). The spontaneous facial expression of differential positive and negative emotions.Motivation and Emotion, 14, 27–43.
Wagner, H. L., MacDonald, C. J., & Manstead, A. S. R. (1986). Communication of individual emotions by spontaneous facial expressions.Journal of Personality and Social Psychology, 50, 737–743.
Wagner, H. L., & Smith, J. (1991). Facial expressions in the presence of friends and strangers.Journal of Nonverbal Behavior, 15, 201–214.
Walden, T. A., & Field, T. M. (1982). Discrimination of facial expressions by preschool children.Child Development, 53, 1312–1319.
Walker, E. (1981). Emotion recognition in disturbed and normal children: A research note.Journal of Child Psychology and Psychiatry, 22, 263–268.
Walker, E., Marwit, S. J., & Emory, E. (1980). A cross-sectional study of emotion recognition in schizophrenics.Journal of Abnormal Psychology, 89, 428–436.
Wallbott, H. G. (1988). Big girls don't frown, big boys don't cry—gender differences of professional actors in communicating emotion via facial expression.Journal of Nonverbal Behavior, 12, 98–106.
Wallbott, H. G. (1991). The robustness of communication of emotion via facial expression: Emotion recognition from photographs with deteriorated pictorial quality.European Journal of Social Psychology, 21, 89–98.
Wallbott, H. G., & Scherer, K. R. (1986). Cues and channels in emotion recognition.Journal of Personality and Social Psychology, 51, 690–699.
Wiggers, M., & van Lieshout, C. F. M. (1985). Development of recognition of emotions: children's reliance on situational and facial expressive cues.Developmental Psychology, 21, 338–349.
Yarczower, M., & Daruns, L. (1982). Social inhibition of spontaneous facial expressions in children.Journal of Personality and Social Psychology, 43, 831–837.
General References
Baird, J. C., & Noma, E. (1978).Fundamentals of Scaling and Psychophysics. New York: John Wiley.
Beck, L., & Feldman, R. S. (1989). Enhancing children's decoding of facial expression.Journal of Nonverbal Behavior, 13, 269–278.
Buck, R. W., Savin, V. J., Miller, R. E., & Caul, W. F. (1972). Communication of affect through facial expressions in humans.Journal of Personality and Social Psychology, 23, 362–371.
Carlson, C. R., Felleman, E. S., & Masters, J. C. (1983). Influence of children's emotional states on the recognition of emotion in peers and social motives to change another's emotional state.Motivation and Emotion, 7, 61–79.
Carlson, C. R., Gantz, F., & Masters, J. C. (1983). Adults' emotional states and their recognition of emotion in young children.Motivation and Emotion, 7, 81–101.
Cohen, J. (1960). A coefficient of agreement for nominal scales.Educational and Psychological Measurement, 20, 37–46.
Ekman, P., & Friesen, W. V. (1986). A new pan-cultural facial expression of emotion.Motivation and Emotion, 10, 159–168.
Felleman, E. S., Barden, R. C., Carlson, C. R., Rosenberg, L., & Masters, J. C. (1983). Children's and adults' recognition of spontaneous and posed emotional expressions in young children.Developmental Psychology, 19, 405–413.
Gray, J. M., Fraser, W. L., & Leudar, I. (1983). Recognition of emotion from facial expression in mental handicap.British Journal of Psychiatry, 142, 566–571.
Hall, J. A., & Levin, S. (1980). Affect and verbal-nonverbal discrepancy in schizophrenic and non-schizophrenic family communication.British Journal of Psychiatry, 137, 78–92.
Levin, S., Hall, J. A., Knight, R. A., & Alpert, M. (1985). Verbal and nonverbal expression of affect in speech of schizophrenic and depressed patients.Journal of Abnormal Psychology, 94, 487–497.
Lewis, D., & Burke, C. J. (1949). The use and misuse of the chi-square test.Psychological Bulletin, 46, 433–489.
Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush & E. Galanter (Eds.),Handbook of Mathematical Psychology (pp. 103–189). New York/London: John Wiley.
Macdonald, H., Rutter, M., Howlin, P., Rios, P., Le Conteur, A., Evered, C., & Folstein, S. (1989). Recognition and expression of emotional cues by autistic and normal adults.Journal of Child Psychology and Psychiatry, 30, 865–877.
Macmillan, N. A., & Creelman, C. D. (1991).Detection theory: A user's guide. Cambridge, U.K.: Cambridge University Press.
Martin, W., & Rovira, M. (1982). Response biases in eye-gaze perception.Journal of Psychology, 110, 203–209.
Martin, W. W., & Rovira, M. (1981). Signal detection theory: Its implications for social psychology.Personality and Social Psychology Bulletin, 7, 232–239. (a)
Martin, W. W., & Rovira, M. L. (1981). An experimental analysis of discriminability and bias in eye-gaze judgment.Journal of Nonverbal Behavior, 5, 155–163. (b)
Maurer, H., & Newbrough, J. R. (1987). Facial expressions of mentally retarded and nonretarded children: I. Recognition by mentally retarded and nonretarded adults.American Journal of Mental Deficiency, 91, 505–510. (a)
Maurer, H., & Newbrough, J. R. (1987). Facial expressions of mentally retarded and nonretarded children: II. Recognition by nonretarded adults with varying experience with mental retardation.American Journal of Mental Deficiency, 91, 511–515. (b)
Mosteller, F. (1968). Association and estimation in contingency tables.Journal of the American Statistical Association, 63, 1–28.
Nummenmaa, T. (1990). Sender repertoires of pure and blended facial expressions of emotion.Scandinavian Journal of Psychology, 31, 161–180.
Rosenthal, R. (1982). Conducting judgment studies. In K. R. Scherer and P. Ekman (Eds.),Handbook of Methods in Nonverbal Behavior Research (pp. 287–361). Cambridge, MA: Cambridge University Press.
Rosenthal, R. (1987).Judgment studies: Design, analysis, and meta-analysis. New York: Cambridge University Press.
Rosenthal, R., & Rubin, D. B. (1989). Effect size estimation for one-sample multiple-choicetype data: Design, analysis, and meta-analysis.Psychological Bulletin, 106, 332–337.
Russell, J. A. (1992).Is there universal recognition of emotion from facial expression? A review of methods and studies. Unpublished manuscript, University of British Columbia.
Scherer, K. R., Banse, R., Wallbott, H. G., & Goldbeck, T. (1991). Vocal cues in emotion encoding and decoding.Motivation and Emotion, 15, 123–148.
Smith, J. E. K. (1982). Recognition models evaluated: a commentary on Keren and Baggen.Perception and Psychophysics, 31, 183–189.
Swets, J. A. (1986). Form of ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance.Psychological Bulletin, 99, 181–198. (a)
Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: their ROCs and implied models.Psychological Bulletin, 99, 100–117. (b)
Wagner, H. L. (1990). The spontaneous facial expression of differential positive and negative emotions.Motivation and Emotion, 14, 27–43.
Wagner, H. L., Buck, R., & Winterbotham, M. (1993). Communication of specific emotions: Gender differences in sending accuracy and communication measures.Journal of Nonverbal Behavior, 17, 29–53.
Wagner, H. L., MacDonald, C. J., & Manstead, A. S. R. (1986). Communication of individual emotions by spontaneous facial expressions.Journal of Personality and Social Psychology, 50, 737–743.
Wagner, H. L., & Smith, J. (1991). Facial expressions in the presence of friends and strangers.Journal of Nonverbal Behavior, 15, 201–214.
Woodworth, R. S. (1938).Experimental psychology. New York: Henry Holt.
Author information
Authors and Affiliations
Additional information
I wish to thank Rainer Banse for helpful discussions in the course of development of the arguments presented here, and Bob Kleck, Tony Manstead, and John Wearden for their comments on an earlier version of the manuscript.
Rights and permissions
About this article
Cite this article
Wagner, H.L. On measuring performance in category judgment studies of nonverbal behavior. J Nonverbal Behav 17, 3–28 (1993). https://doi.org/10.1007/BF00987006
Issue Date:
DOI: https://doi.org/10.1007/BF00987006