Abstract
For some time, the relationship between processing of facial expression and facial identity has been in dispute. Using realistic synthetic faces, we reexamined this relationship for both perception and short-term memory. In Experiment 1, subjects tried to identify whether the emotional expression on a probe stimulus face matched the emotional expression on either of two remembered faces that they had just seen. The results showed that identity strongly influenced recognition short-term memory for emotional expression. In Experiment 2, subjects’ similarity/ dissimilarity judgments were transformed by multidimensional scaling (MDS) into a 2-D description of the faces’ perceptual representations. Distances among stimuli in the MDS representation, which showed a strong linkage of emotional expression and facial identity, were good predictors of correct and false recognitions obtained previously in Experiment 1. The convergence of the results from Experiments 1 and 2 suggests that the overall structure and configuration of faces’ perceptual representations may parallel their representation in short-term memory and that facial identity modulates the representation of facial emotion, both in perception and in memory. The stimuli from this study may be downloaded from http://cabn.psychonomic-journals.org/content/supplemental.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion in Neurobiology, 12, 169–177.
Agam, Y., & Sekuler, R. (2007). Interactions between working memory and visual perception: An ERP/EEG study. NeuroImage, 36, 933–942.
Ashby, F. G., & Maddox, W. T. (1990). Integrating information from separable psychological dimensions. Journal of Experimental Psychology: Human Perception & Performance, 16, 598–612.
Baudouin, J.-Y., Martin, F., Tiberghien, G., Verlut, I., & Franck, N. (2002). Selective attention to facial emotion and identity in schizophrenia. Neuropsychologia, 40, 503–511.
Becker, D. V., Kenrick, D. T., Neuberg, S. L., Blackwell, K. C., & Smith, D. M. (2007). The confounded nature of angry men and happy women. Journal of Personality & Social Psychology, 92, 179–190.
Borg, I., & Groenen, P. J. F. (2005). Modern multidimensional scaling: Theory and applications (2nd ed.). New York: Springer.
Bosten, J. M., Robinson, J. D., Jordan, G., & Mollon, J. D. (2005). Multidimensional scaling reveals a color dimension unique to “colordeficient” observers. Current Biology, 15, R950-R952.
Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 443–446.
Bredart, S., & Devue, C. (2006). The accuracy of memory for faces of personally known individuals. Perception, 35, 101–106.
Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77, 305–327.
Clark, S. E., & Gronlund, S. D. (1996). Global matching models of recognition memory: How the models match the data. Psychonomic Bulletin & Review, 3, 37–60.
D’Argembeau, A., & van der Linden, M. (2004). Identity but not expression memory for unfamiliar faces is affected by ageing. Memory, 12, 644–654.
Darwin, C. (1872). The expression of the emotions in man and animals. London: Murray.
de Gelder, B., Pourtois, G., & Weiskrantz, L. (2002). Fear recognition in the voice is modulated by unconsciously recognized facial expressions but not by unconsciously recognized affective pictures. Proceedings of the National Academy of Sciences, 99, 4121–4126.
de Gelder, B., Vroomen, J., Pourtois, G., & Weiskrantz, L. (1999). Non-conscious recognition of affect in the absence of striate cortex. NeuroReport, 10, 3759–3763.
Duchaine, B. C., Parker, H., & Nakayama, K. (2003). Normal recognition of emotion in a prosopagnosic. Perception, 32, 827–838.
Duchaine, B. C., & Weidenfeld, A. (2003). An evaluation of two commonly used tests of unfamiliar face recognition. Neuropsychologia, 41, 713–720.
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48, 384–392.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face: A guide to recognizing emotions from facial clues. Englewood Cliffs, NJ: Prentice Hall.
Elfenbein, H. A., & Ambady, N. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128, 203–235.
Ellamil, M., Susskind, J. M., & Anderson, A. K. (2008). Examinations of identity invariance in facial expression adaptation. Cognitive, Affective, & Behavioral Neuroscience, 8, 273–281.
Estes, W. K. (1994). Classification and cognition. Oxford: Oxford University Press.
Fernberger, S. W. (1928). False suggestion and the Piderit model. American Journal of Psychology, 40, 562–568.
Fisher, R. A., & Yates, F. (1963). Statistical tables for biological, agricultural, and medical research (6th ed.). New York: Hafner.
Fox, C. J., Moon, S. Y., Iaria, G., & Barton, J. J. S. (2009). The correlates of subjective perception of identity and expression in the face network: An fMRI adaptation study. NeuroImage, 44, 569–580.
Fox, C. J., Oruc, I., & Barton, J. J. S. (2008). It doesn’t matter how you feel: The facial identity aftereffect is invariant to changes in facial expression. Journal of Vision, 8, 11.1–11.13.
Fung, H. H., Lu, A. Y., Goren, D., Isaacowitz, D. M., Wadlinger, H. A., & Wilson, H. R. (2008). Age-related positivity enhancement is not universal: Older Chinese look away from positive stimuli. Psychology & Aging, 23, 440–446.
Ganel, T., & Goshen-Gottstein, Y. (2004). Effects of familiarity on the perceptual integrality of the identity and expression of faces: The parallel-route hypothesis revisited. Journal of Experimental Psychology: Human Perception & Performance, 30,583–5977.
Garner, W. R. (1974). The processing of information and structure. Potomac, MD: Erlbaum.
Ge, L., Luo, J., Nishimura, M., & Lee, K. (2003). The lasting impression of chairman Mao: Hyperfidelity of familiar-face memory. Perception, 32, 601–614.
Goren, D., & Wilson, H. R. (2006). Quantifying facial expression recognition across viewing conditions. Vision Research, 46, 1253–1262.
Gosselin, P., & Kirouac, G. (1995). Decoding facial emotional prototypes. Canadian Journal of Experimental Psychology, 49, 313–329.
Habak, C., Wilkinson, F., & Wilson, H. R. (2008). Aging disrupts the neural transformations that link facial identity across views. Vision Research, 48, 9–15.
Hasselmo, M. E., Rolls, E. T., & Baylis, G. C. (1989). The role of expression and identity in the face-selective responses of neurons in the temporal visual cortex of the monkey. Behavioural Brain Research, 32, 203–218.
Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4, 223–233.
Hefter, R. L., Manoach, D. S., & Barton, J. J. S. (2005). Perception of facial expression and facial identity in subjects with social developmental disorders. Neurology, 65, 1620–1625.
Hollingworth, A., & Henderson, J. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception & Performance, 28, 113–136.
Howard, M. W., & Kahana, M. J. (2002). A distributed representation of temporal context. Journal of Mathematical Psychology, 46, 269–299.
Humphreys, G. W., Donnelly, N., & Riddoch, M. J. (1993). Expression is computed separately from facial identity, and it is computed separately for moving and static faces: Neuropsychological evidence. Neuropsychologia, 31, 173–181.
Humphreys, M. S., Pike, R., Bain, J. D., & Tehan, G. (1989). Global matching: A comparison of the SAM, Minerva II, Matrix, and TODAM models. Journal of Mathematical Psychology, 33, 36–67.
Isaacowitz, D. M., Toner, K., Goren, D., & Wilson, H. R. (2008). Looking while unhappy: Mood-congruent gaze in young adults, positive gaze in older adults. Psychological Science, 19, 848–853.
Isaacowitz, D. M., Wadlinger, H. A., Goren, D., & Wilson, H. R. (2006a). Is there an age-related positivity effect in visual attention? A comparison of two methodologies. Emotion, 6, 511–516.
Isaacowitz, D. M., Wadlinger, H. A., Goren, D., & Wilson, H. R. (2006b). Selective preference in visual fixation away from negative images in old age? An eye-tracking study. Psychology & Aging, 21, 40–48.
Jonides, J., Lacey, S., & Nee, D. (2005). Processes of working memory in mind and brain. Current Directions in Psychological Science, 14, 2–5.
Jonides, J., Lewis, R. L., Nee, D. E., Lustig, C. A., Berman, M. G., & Moore, K. S. (2008). The mind and brain of short-term memory. Annual Review of Psychology, 59, 193–224.
Kahana, M. J., Zhou, F., Geller, A. S., & Sekuler, R. (2007). Luresimilarity affects visual episodic recognition: Detailed tests of a noisy exemplar model. Memory & Cognition, 35, 1222–1232.
Kaufmann, J. M., & Schweinberger, S. R. (2004). Expression influences the recognition of familiar faces. Perception, 33, 399–408.
Kosslyn, S. M., Thompson, W. L., Kim, I. J., & Alpert, N. M. (1995). Topographical representations of mental images in primary visual cortex. Nature, 378, 496–498.
Laeng, B., & Rouw, R. (2001). Canonical views of faces and the cerebral hemispheres. Laterality, 6, 193–224.
Loffler, G., Gordon, G. E., Wilkinson, F., Goren, D., & Wilson, H. R. (2005). Configural masking of faces: Evidence for high-level interactions in face perception. Vision Research, 45, 2287–2297.
Loffler, G., Yourganov, G., Wilkinson, F., & Wilson, H. R. (2005). fMRI evidence for the neural representation of faces. Nature Neuroscience, 8, 1386–1391.
Neth, D., & Martinez, A. M. (2009). Emotion perception in emotionless face images suggests a norm-based representation. Journal of Vision, 9, 5.1–5.11.
Nosofsky, R. M., & Kantner, J. (2006). Exemplar similarity, study list homogeneity, and short-term perceptual recognition. Memory & Cognition, 34, 112–124.
Pantelis, P. C., van Vugt, M. K., Sekuler, R., Wilson, H. R., & Kahana, M. J. (2008). Why are some people’s names easier to learn than others? The effects of face similarity on memory for face-name associations. Memory & Cognition, 36, 1182–1195.
Pollick, F. E., Hill, H., Calder, A., & Paterson, H. (2003). Recognising facial expression from spatially and temporally modified movements. Perception, 32, 813–826.
Posamentier, M. T., & Abdi, H. (2003). Processing faces and facial expressions. Neuropsychology Review, 13, 113–143.
Posner, J., Russell, J. A., & Peterson, B. S. (2005). The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development & Psychopathology, 17, 715–734.
Postle, B. R. (2006). Working memory as an emergent property of the mind and brain. Neuroscience, 139, 23–38.
Revelle, W., & Acton, G. S. (2004). Evaluation of ten psychometric criteria for circumplex structure. Methods of Psychological Research Online, 9, 1–27.
Riby, D. M., Doherty-Sneddon, G., & Bruce, V. (2008). Exploring face perception in disorders of development: Evidence from Williams syndrome and autism. Journal of Neuropsychology, 2, 47–64.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality & Social Psychology, 39, 1161–1178.
Russell, R., Biederman, I., Nederhouser, M., & Sinha, P. (2007). The utility of surface reflectance for the recognition of upright and inverted faces. Vision Research, 47, 157–165.
Sacks, O. (1985). The man who mistook his wife for a hat. New York: Harper-Collins.
Sadr, J., Jarudi, I., & Sinha, P. (2003). The role of eyebrows in face recognition. Perception, 32, 285–293.
Schlosberg, H. (1952). The description of facial emotion in terms of two dimensions. Journal of Experimental Psychology, 44, 229–237.
Sekuler, R., & Kahana, M. J. (2007). A stimulus-oriented approach to memory. Current Directions in Psychological Science, 16, 305–310.
Sekuler, R., McLaughlin, C., Kahana, M. J., Wingfield, A., & Yotsumoto, Y. (2006). Visual recognition and temporal order information are both well-preserved in aging. Psychology & Aging, 21, 632–637.
Sergent, J., Ohta, S., MacDonald, B., & Zuck, E. (1994). Segregated processing of facial identity and emotion in the human brain: A PET study. Visual Cognition, 1, 349–369.
Smith, M. L., Cottrell, G. W., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions. Psychological Science, 16, 184–189.
Stevens, S. S. (ED.) (1951). Handbook of experimental psychology. New York: Wiley.
Torgerson, W. S. (1958). Theory and methods of scaling. New York: Wiley.
Treisman, A., & Zhang, W. (2006). Location and binding in visual working memory. Memory & Cognition, 34, 1704–1719.
Troje, N. F., & Kersten, D. (1999). Viewpoint-dependent recognition of familiar faces. Perception, 28, 483–487.
Tsao, D. Y., & Livingstone, M. S. (2008). Mechanisms of face perception. Annual Review of Neuroscience, 31, 411–437.
van Vugt, M. K., Schulze-Bonhage, A., Sekuler, R., Litt, B., Brandt, A., Baltuch, G., & Kahana, M. J. (2009). Intracranial electroencephalography reveals two distinct similarity effects during item recognition. Brain Research, 1299, 33–44.
Visscher, K. M., Kaplan, E., Kahana, M. J., & Sekuler, R. (2007). Auditory short-term memory behaves like visual short-term memory. Public Library of Science-Biology, 5, e56.
Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45, 174–194.
Wallace, D. F. (1997).A supposedly fun thing I’ll never do again: Essays and arguments. Boston: Little, Brown.
Weller, S. C., & Romney, A. K. (1988). Systematic data collection (Vol. 10). Newbury Park, CA: Sage.
Wells, G. L. (1993). What do we know about eyewitness identification? American Psychologist, 48, 553–571.
Whalen, P. J., Kagan, J., Cook, R. G., Davis, F. C., Kim, H., Polis, S., et al. (2004). Human amygdala responsivity to masked fearful eye whites. Science, 306, 2061.
Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131, 48–64.
Wilson, H. R., Loffler, G., & Wilkinson, F. (2002). Synthetic faces, face cubes, and the geometry of face space. Vision Research, 42, 2909–2923.
Wilson, H. R., & Wilkinson, F. (2002). Symmetry perception: A novel approach for biological shapes. Vision Research, 42, 589–597.
Woodworth, R. S., & Schlosberg, H. (1954). Experimental psychology (2nd ed.). New York: Holt.
Yotsumoto, Y., Kahana, M. J., Wilson, H. R., & Sekuler, R. (2007). Recognition memory for realistic synthetic faces. Memory & Cognition, 35, 1233–1244.
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was supported by National Institutes of Health Grants MH068404, MH55687, MH61975, and EY002158, and by Canadian Institute for Health Research Grant 172103.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Galster, M., Kahana, M.J., Wilson, H.R. et al. Identity modulates short-term memory for facial emotion. Cognitive, Affective, & Behavioral Neuroscience 9, 412–426 (2009). https://doi.org/10.3758/CABN.9.4.412
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/CABN.9.4.412