Self-report studies have found evidence that cultures differ in the display rules they have for facial expressions (i.e., for what is appropriate for different people at different times). However, observational studies of actual patterns of facial behavior have been rare and typically limited to the analysis of dozens of participants from two or three regions. We present the first large-scale evidence of cultural differences in observed facial behavior, including 740,984 participants from 12 countries around the world. We used an Internet-based framework to collect video data of participants in two different settings: in their homes and in market research facilities. Using computer vision algorithms designed for this dataset, we measured smiling and brow furrowing expressions as participants watched television ads. Our results reveal novel findings and provide empirical evidence to support theories about cultural and gender differences in display rules. Participants from more individualist cultures displayed more brow furrowing overall, whereas smiling depended on both culture and setting. Specifically, participants from more individualist countries were more expressive in the facility setting, while participants from more collectivist countries were more expressive in the home setting. Female participants displayed more smiling and less brow furrowing than male participants overall, with the latter difference being more pronounced in more individualist countries. This is the first study to leverage advances in computer science to enable large-scale observational research that would not have been possible using traditional methods.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
The base rate is the proportion of time that a participant displays a facial action.
Ansfield, M. E. (2007). Smiling when distressed: When a smile is a frown turned upside down. Personality and Social Psychology Bulletin, 33(6), 763–775.
Argyle, M., Henderson, M., Bond, M., Iizuka, Y., & Contarello, A. (1986). Cross-cultural variations in relationship rules. International Journal of Psychology, 21(1–4), 287–315.
Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41(3), 687–699.
Brewer, M. B., & Kramer, R. M. (1985). The psychology of intergroup attitudes and behavior. Annual Review of Psychology, 36(1), 219–243.
Brody, L. R., & Hall, J. A. (2008). Gender and emotion in context. In M. Lewis, J. M. Haviland-Jones, & L. F. Barrett (Eds.), Handbook of emotions (3rd ed., pp. 395–408). New York, NY: The Guilford Press.
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., & Robinson, E. S. J. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376.
Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50(2), 260.
Cacioppo, J. T., & Tassinary, L. G. (1990). Inferring psychological significance from physiological signals. The American Psychologist, 45(1), 16–28.
Camras, L. A., Bakeman, R., Chen, Y., Norris, K., & Cain, T. R. (2006). Culture, ethnicity, and children’s facial expressions: A study of European American, Mainland Chinese, Chinese American, and adopted Chinese girls. Emotion, 6(1), 103–114.
Camras, L. A., Oster, H., Campos, J., Campos, R., Ujiie, T., Miyake, K., et al. (1998). Production of emotional facial expressions in European American, Japanese, and Chinese infants. Developmental Psychology, 34(4), 616–628.
Casella, G., & George, E. I. (1992). Explaining the gibbs sampler. The American Statistician, 46(3), 167–174.
Cohn, J. F., & Ekman, P. (2005). Measuring facial action by manual coding, facial EMG, and automatic facial image analysis. In J. A. Harrigan, R. Rosenthal, & K. R. Scherer (Eds.), The new handbook of nonverbal behavior research (pp. 9–64). New York, NY: Oxford University Press.
Darwin, C. (1872). The expression of emotions in man and animals (3rd ed.). New York, NY: Oxford University.
Ekman, P. (1977). Biological and cultural contributions to body and facial movement. In J. Blacking (Ed.), The anthropology of the body. London: Academic Press.
Ekman, P. (1979). About brows: Emotional and conversational signals. In M. V. Cranach, K. Foppa, W. Lepenies, & D. Ploog (Eds.), Human ethology (pp. 169–202). Cambridge: Cambridge University Press.
Ekman, P. (1982). Methods for measuring facial action. In K. R. Scherer & P. Ekman (Eds.), Handbook of methods in nonverbal behavior research (pp. 45–90). Cambridge: Cambridge University Press.
Ekman, P., Davidson, R. J., & Friesen, W. V. (1990). The Duchenne smile: Emotional expression and brain physiology: II. Journal of Personality and Social Psychology, 58(2), 342–353.
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17, 124–129.
Ekman, P., Friesen, W. V., & Ellsworth, P. (1972). Emotion in the human face: Guidelines for research and an integration of findings. New York, NY: Pergamon Press.
Ekman, P., Friesen, W. V., & Hager, J. (2002). Facial action coding system: A technique for the measurement of facial movement. Salt Lake City, UT: Research Nexus.
Ekman, P., Friesen, W. V., O’Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., et al. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53(4), 712–717.
Ekman, P., & Rosenberg, E. L. (2005). What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (FACS) (2nd ed.). New York, NY: Oxford University Press.
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86–88.
Evans, E. C. (1969). Physiognomies in the ancient world. Transactions of the American Philosophical Society, 59, 1–101.
Fischer, A. H., & Manstead, A. S. R. (2000). The relation between gender and emotions in different cultures. In A. H. Fischer (Ed.), Gender and emotion: Social psychological perspectives (pp. 71–94). New York, NY: Cambridge University Press.
Fridlund, A. J. (1991). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60(2), 229–240.
Friesen, W. V. (1973). Cultural differences in facial expressions in a social situation: An experimental test on the concept of display rules (Doctoral dissertation). University of California San Francisco.
Frith, H., & Kitzinger, C. (1998). “Emotion work” as a participant re-source: A feminist analysis of young women’s talk-in-interaction. Sociology, 32, 299–320.
Girard, J. M., & Cohn, J. F. (2016). A primer on observational measurement. Assessment, 23(4), 404–413.
Girard, J. M., Cohn, J. F., Jeni, L. A., Sayette, M. A., & De la Torre, F. (2015). Spontaneous facial expression in unscripted social interactions can be measured automatically. Behavior Research Methods, 47(4), 1136–1147.
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014). Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses. Image and Vision Computing, 32(10), 641–647.
Gosling, S. D., John, O. E., Craik, K. H., & Robins, R. W. (1998). Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. Journal of Personality and Social Psychology, 74(5), 1337–1349.
Hall, J. A., & Halberstadt, A. G. (1986). Smiling and gazing. In J. S. Hyde & M. C. Inn (Eds.), The psychology of gender: Advances through meta-analysis (pp. 136–185). Baltimore, MD: Johns Hopkins University Press.
Hecht, M. A., & LaFrance, M. (1998). License or obligation to smile: The effect of power and sex on amount and type of smiling. Personality and Social Psychology Bulletin, 24(12), 1332–1342.
Heck, R. H., & Thomas, S. L. (2015). An introduction to multilevel modeling techniques: MLM and SEM approaches using Mplus (3rd ed.). New York, NY: Routledge.
Hess, U., Adams, R. B., Jr., & Kleck, R. E. (2005). Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger. Cognition and Emotion, 19(4), 515–536.
Hofstede, G. H. (1980). Culture’s consequences: International differences in work-related values. Beverly Hills, CA: Sage Publications.
Hofstede, G. H. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations (2nd ed.). Thousand Oaks: Sage Publications.
Hofstede, G. H., & McCrae, R. R. (2004). Personality and culture revisited: Linking traits and dimensions of culture. Cross-Cultural Research, 38(1), 52–88.
Kagan, J., Arcus, D., Snidman, N., Feng, W. Y., Hendler, J., & Greene, S. (1994). Reactivity in infants: A cross-national comparison. Developmental Psychology, 30(3), 342–345.
Kisilevsky, B. S., Hains, S. M. J., Lee, K., Muir, D. W., Xu, F., Fu, G., et al. (1998). The still-face effect in Chinese and Canadian 3- to 6-month-old infants. Developmental Psychology, 34(4), 629–639.
Klineberg, O. (1940). Social psychology. New York, NY: Holt.
LaFrance, M., Hecht, M. A., & Paluck, E. L. (2003). The contingent smile: A meta-analysis of sex differences in smiling. Psychological Bulletin, 129(2), 305–334.
Markus, H. R., & Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98(2), 224–253.
Matsumoto, D. (1990). Cultural similarities and differences in display rules. Motivation and Emotion, 14(3), 195–214.
Matsumoto, D. (2006). Culture and nonverbal behavior. In Handbook of nonverbal communication (pp. 219–236).
Matsumoto, D., & Kupperbusch, C. (2001). Idiocentric and allocentric differences in emotional expression, experience, and the coherence between expression and experience. Asian Journal of Social Psychology, 4(2), 113–131.
Matsumoto, D., & Willingham, B. (2006). The thrill of victory and the agony of defeat: Spontaneous expressions of medal winners of the 2004 Athens Olympic games. Journal of Personality and Social Psychology, 91(3), 568–581.
Matsumoto, D., Willingham, B., & Olide, A. (2009). Sequential dynamics and culturally-moderated facial expressions of emotion. Psychological Science, 20(10), 1269–1274.
Matsumoto, D., Yoo, S. H., & Fontaine, J. (2008). Mapping expressive differences around the world: The relationship between emotional display rules and individualism versus collectivism. Journal of Cross-Cultural Psychology, 39(1), 55–74.
McCrae, R. R., & Costa, P. T. (1999). A five-factor theory of personality. In L. A. Pervin & O. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp. 139–153). New York, NY: Guilford Press.
McDuff, D. (2014). Crowdsourcing affective responses for predicting media effectiveness (Doctoral dissertation). Massachusetts Institute of Technology.
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.
Messick, D. M., & Mackie, D. M. (1989). Intergroup relations. Annual Review of Psychology, 40, 45–81.
Muthén, L. K., & Muthén, B. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
Oster, H. (1978). Facial expression and affect development. In M. Lewis & L. A. Rosenblum (Eds.), The development of affect. New York, NY: Plenum Press.
Oster, H., Daily, L., & Goldenthal, P. (1989). Processing facial affect. In A. W. Young & H. D. Ellis (Eds.), Handbook of research on face processing (pp. 101–161). Amsterdam: Elsevier.
Oyserman, D., Coon, H. M., & Kemmelmeier, M. (2002). Rethinking individualism and collectivism: Evaluation of theoretical assumptions and meta-analyses. Psychological Bulletin, 128(1), 3.
Russell, J. A. (1994). Is there universal recognition of emotion from facial expressions? A review of the cross-cultural studies. Psychological Bulletin, 115(1), 102.
Rychlowska, M., Miyamoto, Y., Matsumoto, D., Hess, U., Gilboa-Schechtman, E., Kamble, S., et al. (2015). Heterogeneity of long-history migration explains cultural differences in reports of emotional expressivity and the functions of smiles. Proceedings of the National Academy of Sciences, 112(19), E2429–E2436.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.
Scherer, K. R., Wallbott, H. F., & Summerfield, A. B. (1986). Experiencing emotion: A cross-cultural study. New York, NY: Cambridge University Press.
Schwartz, G. E., Ahern, G. L., & Brown, S. L. (1979). Lateralized facial muscle response to positive and negative emotional stimuli. Psychophysiology, 16(6), 561–571.
Senechal, T., McDuff, D., & Kaliouby, R. (2015). Facial action unit detection using active learning and an efficient non-linear kernel approximation. In Proceedings of the IEEE International Conference on Computer Vision Workshops (pp. 10–18).
Singelis, T. M. (1994). The measurement of independent and interdependent self-construals. Personality and Social Psychology Bulletin, 20(5), 580–591.
Suh, E., Diener, E., Oishi, S., & Triandis, H. C. (1998). The shifting basis of life satisfaction judgments across cultures: Emotions versus norms. Journal of Personality and Social Psychology, 74(2), 482–493.
Tajfel, H. (1982). Social psychology of intergroup relations. Annual Review of Psychology, 33(1), 1–39.
Waxer, P. H. (1985). Video ethology: Television as a data base for cross-cultural studies in nonverbal displays. Journal of Nonverbal Behavior, 9(2), 111–120.
Daniel McDuff and Jeffrey M. Girard have contributed equally to this study.
About this article
Cite this article
McDuff, D., Girard, J.M. & Kaliouby, R.e. Large-Scale Observational Evidence of Cross-Cultural Differences in Facial Behavior. J Nonverbal Behav 41, 1–19 (2017). https://doi.org/10.1007/s10919-016-0244-x
- Facial expression
- Computer vision