Introduction

Face recognition is a key competence in social cognition. Studies on post-traumatic impairments of face recognition provided substantial knowledge of brain regions involved in this highly specialized cognitive function [1, 2]. In order to assess the genetic and developmental foundations of face recognition, current research interests extend to cases of prosopagnosia in the absence of any lesion, i.e., congenital or developmental prosopagnosia. The prevalence of this condition is about 2% of either Caucasian or Hong Kong Chinese population [3–6] and it often runs in families. We will henceforth use the term “congenital prosopagnosia (CP)” as it is assumed to be present from birth [7]. In addition to lesion studies, cases of CP have the potential to unravel fine tuned changes in face processing anatomy and network interaction [8, 9]. Exploration of this condition covered behavioral, social, anatomical, and fMRI data (for review see [7, 10]). A major concern is a varying degree of heterogeneity in most of the studies reported. While some specific tests have proven useful [11], no single test is able to provide sufficient discriminatory power for the entire entity of prosopagnosia yet.

In the light of this heterogeneity, one obvious solution is to concentrate on single cases with very precise behavioral testing and scrutinize specific functions, e.g., emotion and object processing [12, 13]. An intermediate option is the testing of small groups of CP subjects, a strategy which has been very fruitful for anatomical and electrophysiological changes [14]. Behrmann et al. [15] have laid a foundation of anatomical evidence when showing a decrease in volume of the anterior fusiform gyrus correlating with face recognition impairment. In an intriguing diffusion tensor imaging (DTI) study, Thomas et al. [8] found an impairment of major ventral occipito-temporal white matter tracts. Although heterogeneous within the CP group, the apparent changes were clearly correlated with behavioral measures. Finally, Garrido et al. [16] undertook a large group study of anatomical changes in developmental prosopagnosic subjects. They found decreased gray matter volume in right anterior inferior temporal lobe and superior temporal sulcus/middle temporal gyrus bilaterally.

Single case and small group studies, however, pose obvious problems for the detection of fMRI activation changes. In a first fMRI study, Avidan and Behrmann [17] found no significant group difference when comparing four prosopagnosic subjects with six control subjects. In a recent report on six subjects [18], they postulated that CP subjects evinced normal adaptation effects in ventral occipito-temporal cortex, while face familiarity effects in prefrontal cortex were absent. The latter finding was not, however, validated by a direct group comparison.

In the light of the methodological constraints depicted above, we opted for a large group size approach based on diagnostic tools developed previously [19, 20]. This large sample size, similar to Garrido et al. [16], is likely to level out subtle differences in CP subjects which account for a significant degree of heterogeneity in small group studies. Furthermore, it enables us to screen for widespread changes in cortical density and functional activation, not limited to an a priori hypothesis of occipito-temporal areas known to be involved in CP.

In the current study, we were specifically interested in the processing of facial stimuli with different emotional valence and complex non-face stimuli (buildings). Akin to the anamnestic complaint of prosopagnosic subjects not recognizing people recently encountered, we placed special emphasis on long-term memory of faces and its modulation by emotional valence as we had an a priori knowledge that the emotional valence of a face is correctly recognized [23]. Building facades served as control condition as it was assumed that their recognition would depend more on complex visual analysis akin to face recognition as opposed to nameable object stimuli. A robust blocked design was used to stimulate face processing circuitry and tested long-term memory and categorization of emotional valence after the scanning session.

It was hypothesized that a large group size study would be able to detect alterations in the temporo-occipital core system of face circuitry. In line with recent studies [18], we further conjectured that the extended network of face processing, notably the prefrontal cortex, would be implicated in the disrupted information processing and impair the long-term memory performance. Likewise, it was anticipated that emotional valence, which differentially activates the core and extended system [21, 22], could represent a preserved modality of face related information processing in prosopagnosic subjects [23]. Thus, this study was intended to probe the impact of emotional facial expression on long-term memory success.

Methods

Subjects

A total of 49 subjects were included in the present study: 24 of these were congenital prosopagnosic subjects as defined by the criteria described by Grüter et al. [20]. In short, diagnosis was established via a standardized questionnaire and an in-depth semistructured interview the content of which is summarized in Table 1. These criteria are employed in recent literature on CP [3, 19, 20, 24, 25] as there is currently no single diagnostic test with sufficient sensitivity and specificity. See Gruter and Gruter [26] for a complete version of the questionnaire with only minor differences to the one used in our study. As four additional CP subjects had incomplete behavioral testing (encoding or memory), only their anatomical scans were included in the analysis.

Table 1 Characteristics of congenital prosopagnosic (CP) subjects. List of the most discriminative symptoms for the diagnosis of congenital prosopagnosia in the group of 24 prosopagnosic subjects. Symptoms were assessed based on a written questionnaire and an in-depth interview by an experienced physician (MG)
Full size table

Control subjects (n = 31) were chosen among university staff and students and underwent the standardized questionnaire in order to exclude prosopagnosia. Three recruited subjects had a dubious questionnaire. They received the semi-structured interview for internal control of the questionnaire (one being a clear CP subject) and were excluded from the study. Three additional subjects were excluded for technical reasons. Subjects with ophthalmologic impairments, neurological or psychiatric diseases capable of explaining the observed symptoms were excluded from participation in either group. Thus, 25 control subjects remained in the study. The control group was matched to the CP group for gender, age, education (in years of formal education) and handedness (Edinburgh handedness inventory, see supplementary data). The mean age was 35.6 ± 8.5 (range 23–56 years) for prosopagnosic subjects and 34.4 ± 10.6 (20–54 years) for control subjects.

Both groups comprised 14 women, 1 left-handed and 2 ambidextrous persons. All participants were Caucasian, native German speakers with normal or corrected-to-normal vision. All gave informed consent before participating in the study, which was performed according to the guidelines of the Institutional Ethics Committee. Control subjects were paid 10 €/hour, while prosopagnosic subjects were compensated for their travel expenses.

Experimental stimuli

We tested six different categories of visual stimuli: separate conditions of faces with neutral, negative and positive facial expressions (72 pictures for each category), building facades, scrambled face ellipses, and a fixation cross. Face stimuli were selected from NimStim Face Stimulus Set (http://www.macbrain.org), Pictures of facial affect [27], International Affective Picture Set [28], Karolinska Directed Emotional Faces [29] and freely available internet sources. All pictures represented adult faces with direct eye contact, formatted to a uniform standard (see Fig. 1), half of them were female. The negative condition comprised equal numbers of faces with angry and fearful expressions. The positive condition showed smiling faces. Building facades were selected from freely available photographs (e.g., http://www.photo.net), formatted to the same standard and divided into private (n = 36) and public buildings (n = 36; offices, factories, etc.). Of the face stimuli, 72 were grossly coarsened with the mosaic function of Adobe Photoshop and half of them rotated into horizontal orientation (Fig. 1). These stimuli were designed to preserve the outline and luminance of a face in the absence of any facial feature, see e.g., [30, 31], in contrast to “scrambled faces” with preserved featural information (see [32] as an example). A white fixation cross served as the resting condition and was continuously presented for one block length (24.12 s).

Fig. 1
figure 1

Behavioral data. Performance of congenital prosopagnosic (CP) subjects was specifically impaired in the gender decision task (a). They showed pronounced impairment for face recognition and for recognition of buildings (b). CP subjects had more difficulties than control subjects when recognizing individuals with neutral and positive facial expressions (c). No difference was found between CP and control subjects for recognition of individuals with negative facial expressions (p = 0.24). Significance level: *p < 0.05, **p < 0.0005 (see text)

Full size image

Experimental procedure

A blocked design with 12 cycles of each of the following conditions was used: neutral faces, positive faces, negative faces, building facades, scrambled face ellipses, and the resting condition without stimuli. Each cycle consisted of six pictures, with each picture being presented for 3.82 s (interstimulus interval: 200 ms). Stimuli were presented using the Experimental Run-Time System (ERTS, http://www.berisoft.com) and back-projected onto a translucent screen, visible via a mirror mounted on the head coil. Before entering the scanner, subjects had a practice session with 24 additional stimuli. They performed the following tasks via right hand key presses: gender identification for faces, public versus private building for the facades, and horizontal or vertical orientation for scrambled face ellipses. No task was required during resting condition (fixation cross).

After scanning, subjects had a break of 20 min before continuing with a surprise memory test. On a laptop computer, participants reviewed the pictures they had seen in the scanner (216 faces and 72 buildings), randomly intermixed with 108 new faces and 36 new buildings. During recognition, subjects expressed via self-paced button press whether they had already seen the face with high confidence (category “old”), whether they had not encountered it before (high confidence, category “new”) or whether they were uncertain. Uncertain answers were discarded from further analysis. Two response categories determined the accuracy of recognition: the category “hit”, a correct “old” response, given with high confidence, and the “false alarm”, an erroneous high confidence “old” response for a new item. The probability “hit” is the number of correct “old” responses over all “old” responses required, the probability “false alarm” is the fraction of “old” responses of all “new” responses required. A subject performed at chance level if his or her response “old” was as likely for the old than for the new items (probability hit minus probability false alarm = 0). If this equation yielded a positive result, the subject performed above chance level. After the recognition memory test, subjects were asked to rate the emotional facial expression in a forced choice between happy, neutral, angry, and fearful facial expression. All behavioral data were recorded using ERTS (http://www.erts.de/).

For multiple response possibilities, repeated measures ANOVAs were performed to test for main effects and interactions. In case of significant interactions for the factors group and response category, individual categories were compared by post hoc Student t tests. All statistics beyond imaging data were performed using SPSS. Values are given as mean ± standard deviation.

fMRI data acquisition

All scans were performed on a 1.5 T scanner (Symphony, Siemens, Erlangen, Germany) using standard gradients and a circular polarized phase array head coil. Initially, T1-weighted 3D-FLASH scans were acquired for anatomical localization and voxel-based morphometry (number of slices: 120; slice thickness: 1.5 mm; interslice gap: none; matrix size: 256 × 256; field of view: 230 mm; echo time: 4 ms; repetition time: 11 s). For fMRI, two series of T2*-weighted axial EPI scans (including 8 initial dummy scans) were acquired for each subject with the following parameters: number of slices: 30; slice thickness: 4 mm; interslice gap: 0.4 mm; matrix size: 64 × 64; field of view: 220 mm; echo time: 50 ms; repetition time: 2.52 s. The slice grid for EPI scans was adjusted onto the individual anatomical 3D series in order to ensure complete integration of inferior temporal areas.

Voxel-based morphometry

Anatomical scans were processed according to previously described protocols [33, 34] using statistical parametric mapping (SPM2, http://www.fil.ion.ucl.ac.uk/spm). This procedure optimizes the normalization for gray matter by the use of a group-specific gray matter template. The pre-processing of the data consists of two major processes: (1) the creation of a template, including new prior-probability maps for gray and white matter, and (2) the normalization and segmentation of the original data for statistical comparisons. As a first step, the anterior commissure was set in each image as the origin of the individual stereotaxic space, followed by a segmentation of the T1-weighted images into gray and white matter probability maps. Only the gray matter maps were normalized to a gray matter template, representing the stereotactical reference space, defined by the Montreal Neurological Institute (MNI). The thereby estimated transformations were applied to the original, non-segmented T1 dataset, which was then normalized to a voxel size of 2.0 mm3. This normalized T1 dataset was resegmented and the resulting probability maps for gray matter, white matter, and cerebrospinal fluid were averaged across all subjects and finally smoothed with a Gaussian kernel (full width at half maximum 8 mm). These images now served as new group-specific templates and as prior probability maps for gray matter, white matter, and cerebral spinal fluid, which are needed during the segmentation process. These new reference images still represent the MNI space by including individual variations for a better normalization of high-resolution T1-weighted images. The following procedure is very similar to the one described previously with the exception that now the new reference images are used instead of the standard templates, provided by SPM.

Therefore, we resegmented the original T1 images, normalized the derived gray matter probability maps to the new gray matter template, applied the thereby estimated transformation to the original T1 images, resampled it with a voxel-size of 1.0 mm3, segmented the normalized image again, and smoothed the resulting gray matter map with 12 mm. For technical reasons, only 38 subjects were included in this analysis (21 CP subjects (4 with incomplete behavioral testing), 17 control subjects). Their mean image served as a template for Figs. 2, 3, 4 and 5.

Fig. 2
figure 2

Widespread areas of decreased cortical density in CP subjects. In a voxel-based morphometry analysis, CP subjects showed diminished cortical density as compared to control subjects in bilateral lingual gyrus as well as right middle temporal gyrus and left prefrontal cortex, right cerebellar culmen and left declive (p < 0.001 unc., see also Table 2)

Full size image
Fig. 3
figure 3

Deficiencies in classical face and object processing circuitry in CP subjects. Enhanced activation for control subjects for face (a) and building processing (b). Face processing (faces versus scrambled faces): left fusiform gyrus, right lateral occipital complex (Brodmann area 18/19, inferior occipital gyrus) and right dorsolateral prefrontal cortex (Brodmann area 6). Building processing (buildings versus scrambled faces): left middle temporal gyrus, two areas of ventral prefrontal cortex and a fourth region of the left precentral dorsolateral prefrontal cortex (two sample t tests, peak voxels p < 0.001 unc., see “Methods” section and Table 3)

Full size image
Fig. 4
figure 4

Enhanced activation for CP subjects in the medial prefrontal cortex. CP subjects showed enhanced BOLD signal in the left medial frontal gyrus and anterior cingulate, Brodmann area 10 (faces versus scrambled faces, two sample t test, peak voxels p < 0.001 unc.)

Full size image
Fig. 5
figure 5

Different patterns for negative facial expressions in control and CP subjects. In control subjects (a), the contrast negative versus neutral facial expressions showed pronounced BOLD signal in bilateral lateral occipital complex, left posterior STS, right middle temporal gyrus; amygdala below threshold (p < 0.001 unc.). In CP subjects (b), the same contrast elicited a general enhancement of activation in bilateral temporo-occipital cortex, right ventrolateral prefrontal cortex, and left amygdala

Full size image

The group comparisons were done via two-sample t test. Differences are reported on an uncorrected peak threshold of p < 0.001 (minimum cluster size 10 voxels). All MNI coordinates were non-linearly transformed to Talairach space via the mni2tal function (http://www.mrc-cbu.cam.ac.uk/Imaging/mnispace.html).

The clusters at the peak threshold level of p < 0.001 uncorrected (see Fig. 2, right lingual gyrus, left V4v, right middle temporal gyrus, and left dorsolateral prefrontal cortex) were defined as regions of interest via the marsbar function implemented in Matlab (http://www.marsbar.sourceforge.net). For each subject, contrast estimates of enhanced cortical density in control subjects versus prosopagnosic subjects were obtained in the regions of interest in marsbar and exported to SPSS. These values were then tested in SPSS for bivariate correlation (Spearman’s) with the marker of long-term memory, i.e., hits minus false alarms in the face or building recognition test.

fMRI data analysis

MR images were analyzed using SPM2 implemented in MATLAB (Mathworks Inc., Sherborn, MA, USA). All images were realigned to the first image and normalized into standard stereotaxic anatomical MNI space by using the transformation matrix calculated from the first EPI scan of each subject and the EPI template. Afterwards, the normalized data with a resliced voxel size of 4 mm3 were smoothed with an 8-mm FWHM isotropic Gaussian kernel. Proportional scaling with high pass filtering was used to eliminate confounding effects of differences in global activity within and between subjects. The data were analyzed using a general linear model where the different experimental conditions were modeled as boxcar functions, convolved with the canonical hemodynamic response function (HRF) and its temporal derivative. Subsequently, parameter estimates of the HRF regressor for each of the different conditions were calculated from the least mean squares fit of the model to the time series. Parameter estimates for the temporal derivative were not further considered in any contrast. Estimation of the model included correction for non-sphericity induced to the design by the non-independent and non-identically distributed errors of the parameter estimates [35].

One way ANOVAs for the group of prosopagnosic subjects or for the group of control subjects were used when contrasting emotional and neutral facial expressions, at a peak threshold level of p < 0.001 uncorrected (Fig. 5). When comparing group differences between prosopagnosic subjects and control subjects, a random effects analysis was performed using two sample t tests. Here, the reported significance of the peak threshold is p < 0.005, cluster size 5 voxels, all including at least two peak activity voxels with a significance threshold of p < 0.001 uncorrected unless otherwise noted. Activations are shown overlaid onto a canonical brain without cerebellum rendered in three dimensions as well as projected onto coronal slices of the anatomic mean of 38 subjects. The reported voxel coordinates of activation peaks are given in Talairach coordinates in Table 3.

Results

Behavioral data

Face and building discrimination tasks

During fMRI scanning, subjects performed a gender decision task for all faces. For building facades, they assessed whether they saw a private or a public building. For scrambled face ellipses an orientation decision (horizontal or vertical) was required. This orientation task was designed to be feasible for both groups; thus, all performances ranged close to ceiling levels (see Fig. 1a).

Differences in task difficulty still caused a main effect for stimulus category (F 2,94 = 35.63; p < 0.001) in a repeated measures ANOVA, which also revealed a significant group effect (F 1,47 = 5.17; p = 0.028). The latter is difficult to interpret in the light of ceiling effects and in the absence of interaction with stimulus category (p > 0.05). However, it seems to be primarily mediated by a deficit in gender processing: prosopagnosic participants identified 93 ± 4% of faces correctly as female/male as compared to 96 ± 2% in control subjects (a t test would yield t = 2.91, degrees of freedom (df) = 35.9, p < 0.005 but no difference for buildings or ellipses, Fig. 1a).

Analysis of reaction times showed a significant main effect of task (F 2,46 = 217.4; p < 0.0001) as well as an interaction of group with stimulus category (F 2,46 = 4.56; p = 0.016), but no main effect of group. Post hoc t tests revealed that prosopagnosic subjects were faster for the orientation decision of ellipses (1,086 ± 150 ms vs. 1,175 ± 195 ms, t = 1.80, df = 44.8, p < 0.05) and in tendency faster for buildings (1,598 ± 232 ms vs. 1,701 ± 262 ms, p = 0.08), without noticeable differences for the gender decision (p = 0.38). Differences in face recognition memory should, therefore, not be biased by the reaction times during implicit encoding.

In summary, correct answers in the orientation task were close to ceiling levels, but detailed analysis suggests a subtle overall performance deficit in CP subjects, possibly driven by difficulties in the gender decision. For the non-face stimuli, CP subjects were faster than control subjects.

Impaired memory for faces and buildings in CP subjects

Subjects were confronted with a surprise recognition memory test 20 min after scanning. Memory performance was determined by the probability of a correct recognition (“hit”) versus the probability of a false recognition (see “Methods”). Here, we found a main effect for stimulus category (F 1,47 = 258, p < 0.001) and a significant group effect (F 1,47 = 7.67, p = 0.008) revealing a memory impairment for both faces and building facades in CP subjects (Fig. 1b). Yet, there was no significant interaction between stimulus category and group. For faces, prosopagnosic subjects scored 10 ± 7% above chance level. Control subjects attained 17 ± 10%, a difference which was significant in a planned t test (t = 2.73, df = 43.3, p < 0.005, Fig. 1b, buildings: t = 2.39, df = 43.4, p < 0.05). In addition, CP subjects were more likely to answer “uncertain” when asked to recognize faces (41 ± 2% of all answers vs. 31 ± 2%, t = 2.31, df = 40.8, p < 0.05), which also held in tendency when recognizing buildings (28 ± 2% vs. 21 ± 1% in control subjects, p = 0.06). Note that uncertain answers were not included in the above behavioral analyses. Hence, even when CP subjects felt confident, their recognition memory was significantly worse compared to control subjects.

Normal memory for faces with negative facial expression in CP subjects

The memory success for happy, neutral, and negative facial expressions were also determined. A repeated measure ANOVA revealed a significant interaction between the factors memory success per emotional valence and group (F 2,46 = 3.37; p = 0.043) and a group effect (F 1,47 = 7.48; p = 0.009), but there was no significant effect of stimulus category. Post hoc t tests showed that face recognition was deficient in CP subjects both for neutral and positive facial expressions (t = 2.65, df = 44.0, p < 0.01 and t = 3.52, df = 47.0, p < 0.0005, respectively, see Fig. 1c). However, CP subjects performed equally well when it came to recognizing individuals with negative facial expressions (p = 0.24). Within the control group, repeated measures ANOVA revealed no significant effect of memory success per emotional category. This was, however, in tendency the case for CP subjects (F 2,22 = 3.34; p = 0.054). Prosopagnosic subjects remembered faces with negative (and neutral) valence in tendency better than faces with positive valence (post hoc t tests would be p < 0.05 significant, Fig. 1c, white bars). For the recognition memory task, no main effect of group or interaction with group was noted.

Normal rating of the emotional valence in CP subjects

Concomitant to the self-paced face recognition memory test, subjects had to decide for each face whether its expression was neutral, happy, angry or fearful. CP subjects accomplished this task equally well as the control subjects. They attained 94 ± 4% correct emotional classification as compared to 92 ± 10% in control subjects (repeated measures ANOVA non significant). On a behavioral level, the emotional valence of a face is, thus, correctly categorized and, if negative, contributes to a recognition memory performance comparable to control subjects.

Anatomical differences and their relation to memory success

Voxel-based morphometry revealed diminished gray matter density in CP subjects in a distributed network. Main clusters span bilateral lingual gyri, extending in the left hemisphere into a posterior area compatible with V4v (Fig. 2; Table 2). Notably, parts of these clusters are projected onto both cerebellar hemispheres (see Table 2). Further areas of decreased gray matter density include the right middle temporal gyrus, the left precentral gyrus (Brodmann area 6), and a left prefrontal region (medial frontal gyrus, adjacent to Brodmann area 6, Fig. 2; Table 2). Note that not all of the subjects were included in this analysis (see “Methods”). There is no difference in overall gray and white matter volume between the two groups.

Table 2 Voxel-based morphometry data. Areas of diminished gray matter density in CP as compared to control subjects with t and p values (unc. = uncorrected) as well as Talairach coordinates (x, y, z in mm)
Full size table

How do the anatomical differences between CP subjects and control subjects relate to the behavioral outcome? Our most robust measure of impaired face recognition is the long-term memory, i.e. the percentage hits minus false alarms for each subject collapsed over the different emotional valences. In a bivariate correlation analysis (Spearman’s) between contrast estimate and memory outcome over all subjects, two areas of decreased cortical density were closely linked to face memory success, namely the left lingual gyrus (Spearman’s correlation coefficient ρ = 0.312; p < 0.05) and the left dorsolateral prefrontal cortex (DLPFC, ρ = 0.432, p < 0.01). In the latter area, this correlation was also significant within the prosopagnosia group itself (p < 0.05), indicating that the DLPFC is important for their face memory. A further anatomical cluster, the right middle temporal gyrus, was correlated with the recognition performance for buildings over all subjects (ρ = 0.502; p < 0.005), which held in tendency when calculated only in CP subjects (p = 0.062).

fMRI data

Enhanced activation for control subjects in landmarks of visual circuitry

The comparison of face processing between CP subjects and control subjects yielded three distinct areas of decreased activity in CP subjects: the left fusiform gyrus and the right lateral occipital complex in the core face processing areas of the ventral occipito-temporal cortex, and the right DLPFC. These areas are illustrated in Fig. 3a (faces versus scrambled faces, Brodmann area 18/19; for all Talairach coordinates see Table 2). The activation in the fusiform gyrus also encompasses more caudal areas like the left cerebellar hemisphere, which is not an uncommon finding in face processing (Table 3, see “Discussion”). The right DLPC represented the largest cluster, its coordinates being very much akin to the contralateral decrease in cortical density (Brodmann area 6, Fig. 3a).

Table 3 fMRI data. Activation differences between CP and control subjects with t and p values (unc. = uncorrected) as well as Talairach coordinates (x, y, z in mm) for fMRI data
Full size table

What were the differences for object stimuli? When contrasting buildings with visual control stimuli, our data are again most compatible with a dysfunctional network. CP subjects showed diminished BOLD signals in left middle temporal gyrus, in two areas of ventral prefrontal cortex and a fourth region of the left precentral DLPFC (Fig. 3b). This pattern of functional differences in the processing of buildings mirrors two locations of the anatomical changes in CP subjects: the middle temporal gyrus, where anatomical changes correlated with memory for buildings and the precentral DLPFC. The latter region emerges in the four axes of our data analysis: the study of cortical density, in the correlation with face and building memory, and here in functional face and building processing.

Enhanced activation for CP subjects in medial prefrontal cortex

When regarding enhanced activation in participants with CP (faces versus scrambled faces), only one brain region came into play: the left medial prefrontal cortex (Fig. 4, contrast faces vs. scrambled faces, Brodmann area 10). This area is commonly associated with emotional processing, notably of negative facial expressions [36]. Accordingly, this enhanced activation in CP subjects appeared even more pronounced for negative faces (see Table 2). The contrast negative versus neutral faces did not yield significant group differences. It was suggestive, though, of a substantial overall enhancement of face responsive activity in CP subjects by negative facial expression (Fig. 5b). The activation in the CP group included the left amygdala and extensive ventral occipito-temporal cortex in line with an overall enhancement of face sensitive areas by fearful faces [62]. Behavioral and functional data in CP subjects, therefore, suggest a preserved processing of emotional information. Control subjects showed a weaker effect, most obvious in the occipital cortex, the right middle temporal gyrus, and left posterior STS (Fig. 5a). Note that CP subjects did not show any preferentially engaged area in the processing of buildings.

Discussion

Our behavioral and imaging data evince that people with CP are impaired in long-term memory for faces and, to a lesser extent, other complex visual stimuli such as buildings. However, their memory for faces with negative valence and the ability to categorize emotional expressions are preserved. The comparison of cortical density revealed structural deficits in medial inferior temporal areas, notably in the lingual gyrus, in right middle temporal cortex, and DLPFC. In most of these areas, gray matter density correlated with memory success. CP subjects showed decreased BOLD signal during face processing in the fusiform gyrus, lateral occipital complex, and right DLPFC. Processing of buildings was accompanied by decreased brain activity in middle temporal gyrus, ventral and precentral DLPFC. Thus, both structural and functional deficits in the DLPFC emerged in different lines of analysis. Prosopagnosic subjects, on the other hand, showed enhanced activation in the medial prefrontal cortex.

Impaired memory for complex neutral stimuli but not for negative faces

The behavioral results revealed diminished long-term memory for faces with neutral and happy expressions, in close keeping with the pivotal complaint of CP subjects in everyday life. Although face memory deterioration is the most prominent behavioral impairment, the memory for complex objects such as buildings is not spared [37]. Here we could show a clear performance deficit in CP subjects both for faces and building facades. Data on CP subjects regarding deficits in object recognition is not homogeneous [38–40]. However, in a family with ten prosopagnosic subjects, recognition impairment was found to extend to categories such as cars and guns [41].

To our knowledge, this is the first study to probe long-term memory for emotional faces. Interestingly, recognition of faces with negative expressions is preserved in CP subjects. Emotional categorization [42] and fear related memory is obviously intact in CP subjects. This finding will be discussed in conjunction with the functional data below.

In the orientation task, people with CP have a discreet overall disadvantage in the face and object tasks, which was most obvious for the gender decision. This is in line with earlier study results [38]. Note, however, that reaction times were faster for CP subjects in object-related tasks. Hence, their deficits are best explained by a circumscribed impairment in the processing and recognition memory for complex visual stimuli [41].

Voxel-based morphometry: anatomical changes in a distributed visual network

Lingual gyrus

Voxel-based morphometry revealed widespread areas of diminished gray matter density. Deficits were most prominent in bilateral lingual gyrus and were correlated with face memory success, suggesting that these cortical zones are actively engaged in face processing tasks. In terms of Talairach coordinates (right: 13, −50, −11; left: −15, −54, −7), they are well compatible with the building-specific region in the right lingual gyrus (20.6 ± 5.0; 53.8 ± 6.7; −9.2 ± 3.9 [43]) and with the parahippocampal place area. Note that the topography of face and building sensitive areas is not only a product of object class but also markedly biased by peripheral visual processing. Levy et al. [44, 45] concluded that the processing of buildings relies more on peripheral vision. With controlled stimuli, they demonstrated that central perception of both building and face information activates lateral portions of inferior temporal cortex (often implicated in face processing), whereas peripheral vision engages the more medial areas, topographically close to the diminished cortical clusters of lingual gyrus in CP subjects. So we may speculate that decreased cortical density in this region is related to general perceptual differences in CP subjects. The gray matter deficits both in lingual and in middle temporal gyrus are well in line with a recent large study on gray matter volume in developmental prosopagnosic subjects, where the right lingual gyrus was deficient at the uncorrected level, yet it was not included in small volume correction masks (Garrido et al. [16], supplementary data).

Middle temporal gyrus

A further cluster of decreased cortical density lay in right middle temporal gyrus and correlated with long-term memory for buildings. This area, which also showed diminished BOLD signal in building processing in CP subjects (Fig. 3b), has multimodal functional tasks [46] and was implicated in naming famous faces and buildings [47]. Possibly, its role in configural analysis, as demonstrated for faces by Lobmaier et al. [32], may add to its importance in building and face identification. This finding fits nicely to the study by Garrido et al. [16]: the gray matter volume of the right superior temporal sulcus/middle temporal gyrus emerged as the most significant difference between developmental prosopagnosic subjects and controls. They found it to be associated with a behavioral component related to face identity, in keeping with its role as a face sensitive area.

Dorsolateral prefrontal cortex

In the current study, bilateral involvement of precentral DLPFC emerged in anatomical, behavioral, and functional lines of analysis. This finding supports the hypothesis that the disruption of information propagation to the extended system of face processing has a crucial pathophysiological role in CP [8, 18]. Thomas et al. [8] found alterations in major inferior temporal fiber connections, namely the inferior fronto-occipital fasciculus in a study with six CP subjects. Interestingly, this fasciculus connects most of the deficient areas we found (e.g., the lingual, fusiform, and inferior temporal gyri) with the inferolateral frontal cortex [48]. It is a part of the fronto-occipital fasciculus which projects to areas 6, 8 and 9 in DLPFC [49]. We may, therefore, speculate that prosopagnosic subjects suffer from an impairment of face processing circuitry that affects the integrity of both white matter tracts and dependant cortical areas. This disruption also compromises the activation of major network nodes.

Differential fMRI activation reveals a putative alternative route

Inferior temporal cortex

Face and building processing in CP subjects was accompanied by decreased functional activity in visual circuitry. During the processing of faces of all valences, the left fusiform gyrus was less active in CP subjects. This activation is topographically compatible with the fusiform face area (FFA), which was not separately determined in our study. The region of decreased activity comprises more caudal areas extending into the cerebellum, which is not an uncommon finding in previously described FFA activations (Table 2, see [50, 51] for comparison). Implication of this region in CP is, hence, in agreement with the pivotal role of the FFA in face-related tasks [21, 52, 53]. Diminished FFA activation has been reported in small samples of CP subjects [54, 55], but its significance is a matter of debate. Avidan and Behrmann [18] reported a weaker signal for right and most notably left FFA in CP subjects, but as its modulation by repetition of faces was normal, they considered the disruption of information propagation to the prefrontal extended system as more significant.

Dorsolateral prefrontal cortex

The prefrontal cortex is differentially activated in CP subjects: BOLD signal is diminished in DLPFC and increased in medial prefrontal cortex. The DLPFC and the adjacent premotor cortex have a role in facial working memory [56–58] and contribute via complex mechanisms to long-term memory [59], while bolstering rather a feeling of familiarity than conscious recollection [60]. The DLPFC has been associated with relational working memory encoding of visual items [61], a task which draws heavily on mental imagery know to be deficient in CP subjects [37, 62]. As the cortical density in a left DLPFC area correlated with recognition memory performance and DLPFC activation was decreased in CP subjects, we suggest that DLPFC impairment contributes to the long-term memory deficits in CP subjects. The altered gaze behavior towards faces in CP subjects [67] may also indicate an executive dysfunction of the visual system mediated by the DLPFC and adjacent premotor cortex [63, 64]. However, the exact nature of the DLPFC malfunction in CP remains unclear. It may be the primary cause of the condition or a consequence of deficits in other parts of the visual system.

Medial prefrontal cortex

The medial prefrontal cortex is significantly more engaged in CP, both during general face processing and when processing negative facial expressions. This discrepancy to the study by Avidan and Behrmann [18], who showed anterior cingulate activation in control subjects but not in CP subjects, may be due to our larger group size, which allows for direct group comparison. In addition, we used unfamiliar and emotional faces, which elicit more amygdala and orbitofrontal activation [21, 65]. Both areas have been shown to be strongly interconnected [65, 66]. Implication of the extended system of face processing [18] in CP subjects would, hence, depend on the emotional valence of the visual input. As several prefrontal regions, namely the anterior cingulate and rostral prefrontal cortex, are important for successful recognition memory for emotional faces [67], we may speculate that prosopagnosic subjects rely on extraction of emotional cues via the medial prefrontal cortex, which may help to preserve their long-term memory for emotional faces. Yet, our study cannot tap further into this question due to methodological constraints. The fact that we allowed an initial response option “uncertain” in the memory task resulted in a considerable limitation of the analysis. The number of discarded trials varied between subjects (see supplementary data), thus, hampering the detailed analysis of memory performance and the subsequent memory effect in fMRI. Still we feel inclined to hypothesize that a concurrent network of emotional processing via the medial prefrontal cortex is intact in CP subjects.

Conclusion

The symptoms of congenital prosopagnosia are best explained by a dysfunctional network of face processing which likely comprises both the core and the extended system of visual circuitry. In the core system, anatomical changes are most apparent in the lingual gyrus and in the middle temporal gyrus, two areas commonly involved in face and building processing. Functional deficits further extend to major landmarks of face circuitry, such as fusiform gyrus, lateral occipital complex, and middle temporal gyrus. Furthermore, our data unravels the differential implication of the extended system in prefrontal cortex. Thus, the long-term memory impairment for neutral material may be explained by insufficient involvement of DLPFC. Conversely, prosopagnosic subjects are able to compensate for these deficits in emotion related task, possibly via an alternative route to the medial prefrontal cortex. This alternative route warrants further scrutiny.