Brain Activations During Correct and False Recognitions of Visual Stimuli: Implications for Eyewitness Decisions on an fMRI Study Using a Film Paradigm
Abstract
Human behavior strongly relies on the visual storage of events. Unfortunately, the sense of sight is susceptible to numerous distortions and misidentifications. Our study investigated the possibility of applying neuroimaging methods for verification of eyewitness reports. We developed a film paradigm and investigated three related picture sets in a recognition task using functional magnetic resonance imaging (fMRI). For each picture subjects were instructed to make a known–unknown decision. Behavioral results showed false recognitions for nearly half of the presented stimuli. The fMRI results revealed distinct activations for correct and false recognitions. The orbitofrontal cortex could be distinguished as a key region for processing imagined and distorted information resulting in correct rejection of unknown material. Otherwise, false recognitions of unknown material highlighted surprising activation within the posterior cingulate gyrus indicating the subjects’ strain to match unknown information to that known. The results are discussed in terms of current false memory research.
Keywords
False memory Eyewitness Neuroimaging Frontal lobe Posterior cingulateIntroduction
Our memories define who we are and how we behave in new situations. Decisions and social relationships are strongly based on previous experiences that inform rules, beliefs, and preconceptions throughout our lives. But what happens when our memories betray us? Conscious, semi-conscious, and unconscious portions of our memory may become mixed and distorted by proactive and retroactive interference effects. The term false memories was created to circumscribe phenomena in which we remember event-like information that never happened or that we experienced in a different way (Schacter and Curran 2000). It is nearly impossible to differentiate between memories of truly experienced events and memories of imagined events which still can be deeply integrated in our general memory. In general, three different forms of false memories are distinguished in the literature: false recognitions, intrusions, and confabulations (Schacter et al. 1998).
In this article we will focus on false recognitions that endorse making an inaccurate decision regarding specific facts of an event. The most well-known test for analyzing false recognitions is the Deese–Roediger–McDermott (DRM), or word-list paradigm (Deese 1959; Roediger and McDermott 1995). This paradigm enabled the induction of surprisingly high values of false recognitions for related but previously not presented critical words (e.g. bread) by using lists of related words (e.g. butter, food, eat, sandwich) as learning material. The study of Roediger and McDermott showed that false recognitions can be evaluated easily under controlled laboratory conditions. This made it possible to find answers to the questions ‘why’ and ‘how’ these false recognitions evolve. During the processes of encoding and retrieval, information additional to the currently activated information is manipulated mentally. This can lead to distortions resulting from the connection between the new information and older memories, which are related to each other through content or emotion (Buckner et al. 2001). As a consequence false recognitions may occur, which can have serious implications, especially in the context of eyewitness testimonies (Ihlebæk et al. 2003; Lindsay et al. 2004). Gonsalves and Paller (2000) introduced a new procedure that causes false recognition, which resulted from reality-monitoring failures. They presented object names, half of which were followed by a picture of the corresponding object. During a memory test, subjects heard some of the presented object names together with new object names. For each object name, they were instructed to state whether or not they had seen not only the word but also a picture of the object before. For 30% of the stimuli, the subjects made false recognitions, demonstrating that they had imagined the objects’ pictures, after having only read their names.
Over the last few decades an increased number of studies have been conducted that employ neuroimaging techniques in order to investigate neural correlates of false memories (e.g., Gonsalves and Paller 2002; Johnson and Raye 1998; Kim and Cabeza 2007; Kopelman 2002). A remarkable finding is that correct and false memories often share a neural network reflecting the comparable effort to activate sensory information (Garoff-Eaton et al. 2006; Slotnick and Schacter 2004). Another main result is that correct recognitions activate a wider neural network than false ones (Schacter et al. 1996). In a study by Okado and Stark (2003) greater neural activity was observed in occipital regions and the posterior parahippocampal region for correct retrieval. This pointed to the richer sensory content of actually perceived information. In the same study, the right anterior cingulate gyrus was described to be more involved in false recognitions. The activation of this region mirrors the stronger attempt as well as the resulting conflict monitoring to retrieve presumably known material (cf. Botvinick et al. 2004). Aside from the medial temporal lobe, the prefrontal region is also of high relevance in false memory research because of its involvement in monitoring processes and inhibition of inappropriate information (e.g. Shimamura 1995; Wiese and Daum 2006). In general, it can be concluded that even though several regions have been found to be involved in the production of false memories no specific structure or network turned out to be involved ubiquitously.
We have developed a film in order to assay the induction of false recognitions while mimicking everyday experiences. Our hypothesis is that this film paradigm could be used to provoke false recognitions that could be analyzed using functional magnetic resonance imaging (fMRI). We created the film paradigm that could be used under controlled laboratory conditions and still produce results that could be compared to problems with eyewitness reports. We assumed that subjects would make few mistakes in recognizing fixed pictures taken from the original film, called actuals. Additionally, we employed two unknown types of stimuli: similars and unknowns. These two types of unknown stimuli corresponded to failures made in everyday situations. The similars were related to situations in which eyewitnesses may confuse details of an incident, like confusing the color of a car (Gonsalves and Paller 2002; Loftus 2002). We expected that subjects should have more problems correctly rejecting similars than actuals. The reason is that when the subjects have only encoded a schema of a presented object in the film, but not the detailed parameters of it, this set of stimuli should induce false recognitions. The other type of unknown stimuli was called unknowns. This set was developed with respect to experimental situations, in which subjects might falsely recall unknown words after studying word lists (Dodd and MacLeod 2004). We assumed that the unknowns would produce an even higher false recognition rate as actuals or similars, as they represented the difficulties with filling in the gaps. An example of a filling in the gap situation occurs when we watch a basketball match and at the very moment a player takes a shot that results in a basket we have looked away. We only see the player land with his arms outstretched and the ball going through the basket. Afterwards we might create the whole scene in our mind, seeing how the player runs up to the basket, jumps and throws the ball through the hoop.
By investigating these differently induced correct and false recognitions with fMRI, we should be able to discriminate the neural bases for them. Considering all three sets, two possible outcomes can be hypothesized. First, neural activity of known stimuli (actuals) should be stronger than of unknown stimuli (similars and unknowns) because only actuals can reactivate the truly perceived and stored sensory information. The second hypothesis is that the decision making process of the unknown stimuli, regardless of their classification to one of the two unknown sets, might need a more sophisticated evaluation associated with a wider network of brain areas. The possibility of applying the results of this method to ambiguous eyewitness cases is discussed.
Method and materials
Subjects
Twelve male, right-handed, native German speakers participated in this study. The subjects varied in age from 34 to 54 years (mean age = 42.75 years, SD = 6.21). Duration of education varied from 9 to 13 years (one subject 9, four 10, seven 13 years). None of the subjects had a history of psychiatric or neurological problems. At the beginning of the test, subjects were informed only about participating in a memory study. They knew that they could terminate the test at anytime. They also signed a letter of agreement for their participation and gave consent that their data could be used later for publication. Before the subjects were tested with the stimulus material, they were screened for MRI contraindication. After the fMRI study, they each received 40 euros for their inconvenience. They were not given an opportunity to talk to each other about the study.
Film paradigm
The film paradigm consisted of a film that was 19:44 min long and three sets of 42 pictures each. The film material was produced using a Sony digital video camera, the DCR-PC9E, and was edited with Adobe Premiere 6.5 (Rockford, Adobe System, Inc.). Half of the overall 42 scenes of the film contained a female as the main character and the other half contained a male as the main character, which prevented a possible gender-specific memorization effect. A further fundamental requirement was to create an unemotional film, in order to minimize the effects of internal arousal that might result in changes in the neuronal response (Kensinger and Schacter 2005; Markowitsch et al. 2003; Piefke et al. 2003). In this way, we created a film with a close relationship to everyday events carried out by a man or a woman without any emotive actions. For example, the second scene of the film shows the woman walking into a perfumery, looking around, walking to a shelf, and picking up a bottle. She opens the bottle and sprays some perfume onto her right wrist, sniffs the scent, puts the bottle back, and leaves the store. The next scene shows the man getting up in the morning, stretching his back and arms while sitting still on the bed, and pulling up the roller blind. Each scene in the film represents a self-contained activity. The scenes were presented alternately, so that a scene with the woman was followed by one with the man and so on. These alternations were made to obtain an equal probability of memorization for both films. The perpetuation of the two stories was preserved. The two characters did not meet in any of the scenes and they did not appear in identical locations. The film was presented without sound to ensure that only visual memory was tested and no auditory information interfered with the memorization process. The film was presented on a computer screen in an isolated room.
The two components of the film paradigm: the film and the recognition material. Of each of the three sets, pictures from two different scenes of the film are shown exemplarily for the total of 126 pictures. The reference pictures are also presented
Additionally, reference pictures were used, showing either a train or a plane with their heads pointing to the left or to the right. The subjects were instructed to indicate by pressing the respective button into which direction the head showed (i.e., if the head of the train pointed to the left, subjects had to press the left button with their left thumb). These reference pictures were necessary because they enabled the calculation of two important factors. First, we were able to filter out brain activation related specifically to the motor response during pressing a button and, secondly, brain regions which were associated with the processing of visual stimuli in general could also be filtered out. Throughout the recognition task four reference pictures were presented during the initial familiarization phase and 40 more during the main recognition test. They were intermingled and randomized together with the recognition material from the three sets. The program ‘Presentation Version 081 Build 04.28.04’ (Neurobehavioral Systems) was used to present the recognition stimuli and to automatically randomize anew the material for each subject.
Data acquisition
Instructions for the learning phase contained the information that the film would be roughly twenty minutes long and that it would be important to follow it very closely, because a test would follow, based on the film scenes. The recognition test occurred in a 1.5 T scanner (Siemens Magnetom Symphony, Erlangen, Germany) equipped with a standard head coil and with echoplanar imaging capability. To position the axial T2*-weighted images along the AC-PC line, scout and sagittal T1-weighted images were obtained from each subject. To provide an anatomical reference and to exclude gross brain pathology, a T1-weighted 3D-sequence (MPRAGE, TR = 11.1 ms, TE = 4.3 ms, slice thickness 1.5 mm, FOV 201 × 230 mm, matrix 224 × 256) was obtained from all subjects.
Stimuli were presented using a mirror on the head coil. Subjects were equipped with a response box for each hand and were instructed to press with their left thumb when the image was known to them and with their right thumb when it was unknown. The sequence of the recognition phase started with two instruction slides, followed by six example trials to familiarize subjects with the procedure, followed by two recapitulating instruction slides. Each trial consisted of one picture presented for 3 s and an intermediate pause of 6 s showing a black screen. Volume scans were taken at intervals of 3 s using a standard EPI sequence (TR = 3,000 ms, TE = 50 ms, flip angle 90°, FOV 192 mm, matrix 64 × 64). Each volume scan comprised 16 axial T2*-weighted MR-slices with a slice thickness of 7 mm. Using a thickness of 7 mm we were sure to include the total volume of the brain (length 11.2 cm) in our analysis. Each picture was response-connected and vanished as soon as the subject made a choice by pressing a button during presentation. Subjects could not influence the duration of the intermediate black screen. If a subject did not respond during the image presentation, he had time to decide during the subsequent pause. Finally, after the instruction slides and the first six pictures intended for familiarization with the procedure, the recognition trial with 120 pictures started. To ensure equal likeliness of known/unknown judgments for all three sets of stimuli, the sequence was randomized for each subject.
Response analysis
After the recognition test we had a total of six different sets, each composed of a combination of one of the three picture sets—actuals, similars, and unknowns—and one of the two possible response types—known and unknown. Since a correct known or unknown response depended on the picture, we extended the set description to actuals-correct, actuals-false, similars-correct, similars-false, unknowns-correct, and unknowns-false. These terms are applied in the following analysis and discussion.
Image analysis
The first two images of each session were discarded to allow for T1 saturation effects during the first scans. The remaining images were processed using SPM99. To correct for head movements, the images were first realigned using the SPM99 default algorithm. Prior to smoothing and group comparisons, anatomical differences were compensated by spatial normalization and reslicing, aligning the voxel size to 3 × 3 × 3 mm, using the SPM99 default settings and the standard stereotactic space that is the MNI (Montreal Neurological Institute) brain. Then spatial smoothing was done with a Gaussian kernel of 10 mm full-width at half-maximum (FWHM), to increase signal and anatomical conformity (10 mm approximates the triple side length of a voxel). A fixed-effects statistical analysis was executed on a voxel-by-voxel basis using the General Linear Model (GLM). For the analysis, maps of t-statistics were corrected for multiple comparisons at p < 0.05, and the minimum size of displayed clusters was 10 voxels. The MNI coordinates of the major activation clusters were transformed into the Talairach and Tournoux space (Talairach and Tournoux 1988) using a correction procedure (Brett 1999) and then transferred into the Talairach Daemon (Lancaster et al. 2000) to obtain anatomical projections of maximum activation.
Results
Behavioral data
The results of the recognition task showed that at 44.2%, the subjects falsely recognized nearly half of the 120 items entering the evaluation (mean = 66.75; SD = 7.78). Furthermore, descriptive analysis of each set showed that most of the false recognitions were made for unknowns (mean = 25.33; SD = 5.42) and similars (mean = 21.75; SD = 6.25), and only a few for actuals (mean = 6.17; SD = 3.79). In regard to the correct responses, most of the actuals (mean = 33.83; SD = 3.79) were correctly recognized, followed by nearly half of the similars (mean = 18.25; SD = 6.25) and finally, with a mean of 14.67 (SD = 5.42), the fewest stimuli were correctly rejected in the set unknowns.
For a more robust estimation and interpretation, discriminability indices (d′) and response biases (c) were calculated according to signal-detection theory (Green and Swets 1966; Stanislaw and Todorov 1999). The response bias describes the criterion for which a subject makes a response decision. The criterion is defined as a flexible point on a subject’s internal response axis. When the internal response is above this criterion, the subject responds with ‘yes’/‘known’ and when it is lower than the criterion, the response is ‘no’/‘unknown’. The discriminability index, d′, is defined by the separation and spread between signal (here: actuals) and noise (here: similars and unknowns) curves. It is important to note that d′ and c must be interpreted together because they are interdependent. For a correct interpretation of d′ and c, it is also fundamental to bear in mind that of a total of 120 pictures presented during the recognition test, correct responses would consist of 80 unknown (similars and unknowns) and 40 known (actuals). The results for both pairs revealed the following positive values for d′: actuals-correct & similars-false: mean = 1.31, SD = 0.96; actuals-correct & unknowns-false: mean = 1.52, SD = 0.88. On the other hand, the response bias for both combinations showed negative values: actuals-correct & similars-false: mean = −0.50, SD = 0.24; actuals-correct & unknowns-false: mean = −0.39, SD = 0.20. The discriminability index and the response bias scores did not differ significantly between the comparisons, but the positive values of d′ indicated that the subjects differentiated between signal (actuals) and noise (unknowns and similars). The response bias, c, which was negative for both comparisons, pointed to a criterion located on the internal response axis in such a way that the subjects answered mostly with a ‘known’ response, not only to actuals (resulting in correct responses), but also to similars and unknowns (resulting in false recognitions).
Neuroimaging data
False recognitions
Relative increase in neural activity associated with similars-false (a), unknowns-false (b), actuals-false (c), and all false recognitions across all three sets (d). Areas of significant relative increase in neural activity are shown as through-projections onto representations of standard stereotaxic space (“glass brain”) as defined by Talairach and Tournoux (1988)
Local maxima of significantly activated regions associated with comparisons between falsely recognized items of the three sets across all subjects (first sorted by pattern, then by level of significance [Z-scores])
| Side | Region | BA | Voxels in cluster | Z | X | y | z |
|---|---|---|---|---|---|---|---|
| Similars-false | |||||||
| L | Middle temporal gyrus | 19 | 1,956 | 6.76 | −36 | −78 | 20 |
| L | SMA | 6 | 359 | 5.13 | −30 | 11 | 46 |
| R | Middle occipital region | 19 | 978 | 6.04 | 45 | −78 | 12 |
| R | Posterior cingulate gyrus | 29 | 245 | 5.43 | 15 | −46 | 11 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 173 | 4.65 | 54 | 24 | 15 |
| R | SMA | 6 | 78 | 3.95 | 36 | 5 | 47 |
| Unknowns-false | |||||||
| L | Dorsolateral frontal cortex | 8 | 341 | 5.57 | −3 | 23 | 46 |
| L | Dorsolateral/ventrolateral frontal cortex | 46/45 | 459 | 5.49 | −42 | 24 | 15 |
| L | Orbitofrontal cortex | 10 | 25 | 4.09 | −27 | 62 | 11 |
| L | Hypothalamus | 34 | 3.66 | −9 | −3 | −7 | |
| R | Middle occipital gyrus | 19 | 6,665 | Inf. | 45 | −78 | 9 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 336 | 5.44 | 54 | 24 | 21 |
| R | SMA | 6 | 355 | 4.95 | 33 | −15 | 56 |
| R | Midbrain | 24 | 4.05 | 9 | −21 | −12 | |
| R | Dorsolateral frontal cortex/SMA | 8/6 | 15 | 3.69 | 33 | 17 | 43 |
| R | Posterior cingulate gyrus | 31 | 15 | 3.68 | 12 | −39 | 38 |
| R | Inferior parietal gyrus | 40 | 17 | 3.66 | 36 | −44 | 44 |
| Actuals-false | |||||||
| L | Middle occipital gyrus | 19 | 24 | 3.89 | −39 | −87 | 7 |
| L | SMA | 6 | 10 | 3.60 | −39 | −6 | 56 |
| R | dorsolateral frontal cortex | 46 | 13 | 3.40 | 56 | 30 | 18 |
| R | Middle occipital gyrus | 19 | 16 | 3.33 | 48 | −73 | 6 |
Correct recognitions
Relative increase in neural activity associated with similars-correct (a), unknowns-correct (b), actuals-correct (c), and all correct recognitions across all three sets (d). Areas of significant relative increase in neural activity are shown as through-projections onto representations of standard stereotaxic space (“glass brain”) as defined by Talairach and Tournoux (1988)
Local maxima of significantly activated regions associated with comparisons between correctly recognized items of the three sets across all subjects (first sorted by pattern, then by level of significance [Z-scores])
| Side | Region | BA | Voxels in cluster | Z | X | y | z |
|---|---|---|---|---|---|---|---|
| Similars-correct | |||||||
| L | Middle temporal gyrus | 19 | 4,122 | 6.86 | −36 | −81 | 21 |
| L | Orbitofrontal cortex | 47 | 204 | 5.43 | −33 | 23 | −1 |
| L | Dorsolateral frontal cortex | 9 | 408 | 5.18 | −48 | 8 | 36 |
| L | Subthalamic nucleus | 81 | 3.84 | −12 | −15 | −4 | |
| R | Orbitofrontal cortex | 47 | 346 | 5.77 | 36 | 26 | −4 |
| R | Dorsolateral frontal cortex | 9 | 490 | 4.96 | 6 | 31 | 32 |
| R | Dorsolateral frontal cortex | 9 | 39 | 4.60 | 59 | 16 | 30 |
| R | Brainstem, pons | 63 | 4.28 | 0 | −24 | −19 | |
| Unknowns-correct | |||||||
| L | Middle occipital gyrus | 19 | 5,421 | Inf. | −36 | −84 | 18 |
| L | Precentral gyrus | 4 | 879 | 6.34 | −36 | −12 | 53 |
| L | Anterior cingulate gyrus | 32 | 932 | 5.89 | −9 | 22 | 32 |
| L | Dorsolateral frontal cortex | 46 | 83 | 4.16 | −45 | 24 | 15 |
| L | Insula | 57 | 4.11 | −30 | 21 | 5 | |
| L | Precuneus | 7 | 18 | 3.40 | 0 | −49 | 61 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 431 | 6.66 | 56 | 24 | 18 |
| R | Brainstem, pons | 52 | 4.48 | 0 | −21 | −22 | |
| R | Orbitofrontal cortex | 47 | 62 | 4.30 | 33 | 23 | −4 |
| R | SMA | 6 | 29 | 4.01 | 42 | 2 | 47 |
| R | Inferior parietal gyrus | 40 | 12 | 3.58 | 33 | −47 | 47 |
| Actuals-correct | |||||||
| L | Dorsolateral frontal cortex | 9 | 241 | 4.45 | −48 | 8 | 36 |
| L | SMA | 6 | 29 | 3.80 | −30 | 11 | 46 |
| R | Middle occipital gyrus | 19 | 4,701 | 7.35 | 50 | −69 | 9 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 159 | 5.19 | 42 | 16 | 19 |
| R | Precentral gyrus | 4 | 43 | 3.92 | 36 | −12 | 48 |
| R | SMA | 6 | 35 | 3.59 | 3 | 28 | 37 |
Local maxima of significantly activated regions associated with false as well as correct recognitions across all three sets across all subjects (first sorted by pattern, then by level of significance [Z-scores])
| Side | Region | BA | Voxels in cluster | Z | X | y | z |
|---|---|---|---|---|---|---|---|
| All false recognitions | |||||||
| L | Middle occipital gyrus | 19 | 773 | 7.30 | −36 | −81 | 18 |
| L | Posterior cingulate gyrus | 31 | 86 | 6.40 | −18 | −60 | 22 |
| L | Dorsolateral frontal cortex | 9 | 17 | 5.22 | −45 | 7 | 33 |
| R | Middle temporal gyrus | 19 | 457 | 7.67 | 45 | −78 | 12 |
| R | Fusiform gyrus | 37 | 100 | 6.92 | 42 | −50 | −15 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 83 | 6.19 | 54 | 27 | 21 |
| R | Posterior cingulate gyrus | 23/30 | 128 | 6.11 | 15 | −52 | 14 |
| All correct recognitions | |||||||
| L | Middle temporal gyrus | 19 | 4,818 | Inf. | −36 | −81 | 21 |
| L | Dorsolateral frontal cortex | 9 | 375 | 7.22 | −48 | 8 | 36 |
| L | Anterior cingulate gyrus/SMA | 32/6 | 453 | 7.06 | −6 | 25 | 35 |
| L | Orbitofrontal cortex | 47 | 81 | 6.53 | −33 | 23 | −1 |
| R | Ventrolateral/dorsolateral frontal cortex | 45/46 | 288 | 6.84 | 56 | 27 | 18 |
| R | Orbitofrontal cortex | 47 | 91 | 6.80 | 36 | 26 | −6 |
| R | Brainstem/pons | 121 | 5.20 | 0 | −24 | −19 | |
Discussion
The main goal of this study was to demonstrate that a complex visual stimulus simulating the real world can be used to explore the underlying brain activity of false recognitions considering the difficulties eyewitness reports can imply. First, we discuss the behavioral results highlighting the functionality of the film paradigm. Secondly, we look into the neuroimaging results which presented surprising activations for known as well as unknown material.
Behavioral data
The behavioral data clearly shows that the subjects falsely recognized nearly half of the presented stimuli. By calculating the discriminability indices, which were positive, we were able to show that the subjects did not react by chance, but that they made their decisions deliberately. In addition, the negative values for the response bias underlined that these intentional responses really produced false memories for at least half of the presented stimuli. The stimuli of the set unknowns proved to be particularly prone to false recognitions, indicating that the subjects had used their imaginations to fill the gaps in many of the film scenes. A possible explanation for this very common phenomenon is the theory of cognitive dissonance (Festinger 1957), which deals with the urge to avoid discrepancies between knowledge and behavior. The most famous example is the smoker who consciously engages in something he knows will harm his health. One way to solve this conflict is by unconsciously referring to consistency and change biases (Schacter 2001). When we embed perceived information into imagined parts, the end result will be consistent, like the example presented in the Introduction of remembering the complete play in the basketball game. Furthermore, if something is imagined over and over again, we create a powerful false recollection which can lead to the belief that an imagined event was actually experienced (Goff and Roediger 1998). These processes are practical in everyday life, but not when a crime is witnessed. In this case, it is important to remember correctly what was actually witnessed and what was not because of some kind of distraction. Following this line of thought, it becomes apparent that the set of unknowns represents a form of false recognitions that might result out of source monitoring errors (Lindsay and Johnson 2000). Thus unknowns provoked more false recognitions than similars or actuals, because explicit memory of how something happened in the film is necessary to correctly reject them. Our results reveal that to memorize everyday experiences in such an explicit way usually exceeds our memory capacity.
The stimuli of the set similars were developed as schema-consistent to the film. A recent study by Silvia et al. (2006) found that for subjects to correctly recognize schema-consistent objects later, it is important to focus their attention on them during the instruction phase. Our subjects were not instructed before they saw the film. The high amount of false recognitions that we found for similars together with the results of Silvia et al. outline the problem with schema-consistent stimuli. Without any focus of our attention before we experience an event every person memorizes a slightly different situation. Schema-consistent stimuli answer, at least partly, the question why witness reports often differ between several people. These results further demonstrate the importance of investigating the similarities and differences between truly perceived and schema-consistent stimuli on the neural level.
It is also relevant to briefly discuss the differences between the present film paradigm and changed blindness paradigms, which have also used short film sequences. Changed blindness paradigms might superficially appear quite similar to our method (Levin and Simons 1997), especially with regard to the stimuli of the set similars. However, there are two important differences between our film paradigm and changed blindness paradigms. The first is that the subjects’ attention is actively focused on specific details of a changed blindness film (e.g., like counting shots of a ball) which leads to inattention to further changes within the scene (e.g., a man costumed as a gorilla walking through the scene [Simons and Chabris 1999]). The subjects who participated in our film paradigm task were not at any time influenced by instructions, such as suggestive questions (Bernstein et al. 2005) or other devices, that might draw their attention to specific details. The film paradigm was not developed to investigate changes in detection abilities, but to find out how accurately memory works with everyday experiences. Thus we did not change anything in the presented film. Instead, we showed the stimuli of similars during the recognition task. The other difference is that if our film paradigm was comparable to change blindness paradigms, the correct recognition rate should have been above chance level (see, e.g., the study by Varakin and Levin [2006]). That was not the case. The film paradigm was produced to investigate subjects’ memories under laboratory conditions, so that the results might give an idea of what could happen when, for example, someone witnesses a robbery. This witness would not be instructed prior to the incident, and his or her attention would not automatically focus on details that might be important later, e.g., color of the robber’s jacket.
In order to obtain evidence of what kinds of memory failures are of interest for an investigation, we created the two different unknown stimulus sets: similars and unknowns. Both sets produced false recognition that justified further analysis of the underlying neural activations.
Neuroimaging data
Our neuroimaging results clearly showed that the three stimuli sets invoked different neural networks for correct and false recognitions. The results were surprising insofar as unknown stimuli generally resulted in stronger and larger activations than studied ones. This suggests that the processing and evaluation of the sets similars and unknowns required wider neural resources than the set actuals did. Furthermore, actuals-false activated only a few small clusters whereas actuals-correct were associated with more and larger activations. This difference might be due to the small number of false recognitions of actuals which resulted in too little data to show more of the related activations.
Two regions were detected that seem to mirror a special involvement in the known/unknown decision making process. The orbitofrontal cortex, which was activated only in contrasts involving unknown material, and the posterior cingulate cortex, which was related to false recognitions of unknown stimuli. Activation within the right hemisphere of the orbitofrontal cortex was associated with the correct rejection of similars and unknowns. Interestingly, false recognition of unknowns activated an area within the left orbitofrontal cortex. The orbitofrontal cortex was reported to be involved in verification processes during the retrieval of information (Cabeza et al. 2001). Examining the unknown stimuli of our study might have induced a familiar feeling, but contrary to the known stimuli, they lacked truly perceived information. Work with primates has shown that neurons in the orbitofrontal area respond specifically to novel visual stimuli (Rolls et al. 2005). This suggests that, in particular, the orbitofrontal cortex evaluates the new contents of unknown material. Thus the orbitofrontal cortex might function as a mediator between memories and current demand by monitoring the ongoing performance during the retrieval (Stern et al. 2000). Interestingly, a recent study by Kensinger and Schacter (2007) found that, together with the amygdala, the orbitofrontal cortex was only activated during correct recognition of visual details of negative stimuli. The authors interpreted this result as an indication of a strong relationship between these (para-) limbic regions and the effective retrieval of information. This would also explain our finding that the right orbitofrontal cortex solely correlated with correct rejections of unknown stimuli. It can be concluded that, although we used unemotional material, the evaluation of schema-consistent and content-related unknown material seemed to evoke a negative value in the subjects. However, the activation within the left hemisphere for the falsely recognized unknowns underlines the challenge such stimuli present—pointing to the common memory failure of unconsciously filling gaps in memories. A surprising finding was the activation of the posterior cingulate gyrus, bilateral for all false recognitions versus baseline and only right hemispheric for similars-false and unknowns-false, and the right fusiform gyrus that was solely associated with false recognitions of similars and unknowns. Vogt and colleagues linked the posterior cingulate region to several processes: monitoring sensory input, evaluating one’s behavior, spatial orientation, and autobiographical memory processes (1992). Furthermore, it was found to be associated with the attentional control during the detection of previously studied information (Small et al. 2003). The fusiform gyrus was found to be involved in the perception of visual information and the processing of specific details about the form of presented objects (Garoff et al. 2005; Simons et al. 2003). The findings of these earlier studies make our results even more puzzling. There are two possible explanations for the involvement of the posterior cingulate as well as the fusiform gyrus during false recognition of unknown material. One is that the subjects tried hard to compare the imagined parts with those known from the film and thus retrieved elements of the previous studied film which they connected to the unconsciously fabricated information. This explanation is supported by studies that describe the involvement of both areas in successful memory retrieval (Garoff et al. 2005; Maddock et al. 2003) and especially the posterior cingulate gyrus also in mental imagery (Fletcher et al. 1995). Considering our study design the activity in these regions indicate that the subjects were convinced of the correctness of their responses. Thus the neural activity corresponds to the behavioral data discussed earlier underlying the generation of false memories. An alternative explanation is that during the process of looking at the unknown pictures the subjects tried hard to find details matching those of the studied film. This explanation refers to the described involvement of the posterior cingulate gyrus in spatial orientation tasks and would also explain the involvement of the right fusiform gyrus as a region that is connected to retrieve specific details of visually encoded information.
The evaluation of the pictures, known and unknown alike, was further associated with the dorsolateral frontal cortex and the ventrolateral frontal cortex (Rahm et al. 2006). The activity in the dorsolateral frontal cortex especially shows the subjects’ effort to select and manipulate the information presented in the recognition stimuli, and to evaluate this information together with the studied film, all the while monitoring this entire process (Fletcher and Henson 2001). We can assume that the subjects had to monitor their own performance during the recognition task. The randomized presentation of the recognition stimuli demanded a high level of concentration to remember which stimuli had already been presented and which decision had been made. This kind of monitoring might also be comparable to presenting several pictures of possible suspects to a witness. The bilateral activation of the dorsolateral frontal cortex, especially for the stimuli of the set unknowns, underscores the high demand for processing them thoroughly. The nature of the study meant that subjects had to focus their attention and were forced to make clear decisions, rather like those that could occur during the interrogation of a witness. These decisions involve situations in which response conflict and even mistakes might happen. The activation of the left dorsal anterior cingulate cortex for correct recognition of unknowns, and across all three sets, confirms this interpretation as it was found to be associated to the process of avoiding failures (Magno et al. 2006).
Bilateral activation of the middle occipital gyrus and the inferior temporal gyrus was reported to correlate with correct and false recognitions respectively (Slotnick and Schacter 2004). Furthermore, several studies came to the conclusion that false memories are related to lesser neural activity than correct ones (e.g. Okado and Stark 2003). This leads to the conclusion that truly perceived information contains richer sensory details, which should result in stronger and larger activations than false memories resulting only, or primarily, from imagined information. The results of our study did not confirm these earlier findings. The strong and large activation in the right middle occipital gyrus, found for actuals-correct, seems to be related to the reactivation of truly perceived information during seeing the film. For the two unknown picture sets that elicit even larger bilateral activations in this area a different explanation has to be found. The middle occipital gyrus was reported to be involved not only in visual perception, but also in imagery processes (Ganis et al. 2004). Furthermore, it is known that lesions in this area result in the inability to imagine visual scenarios (Ogden 1993). This leads to the interpretation that in our study the occipital-temporal area was highly related to the process of comparing reactivated information from the studied film with unconsciously imagined information.
In summary, the neuroimaging results of our study showed activation of a frontal network, consisting primarily of the orbitofrontal cortex, the ventrolateral frontal cortex, the dorsolateral frontal cortex, and the anterior cingulate cortex. These regions seem to be associated with difficult decision processes that often result in correct responses. Additionally, the posterior cingulate and the fusiform gyri could be identified as regions that are contrary to previous findings associated with false recognitions of visual stimuli. Their activation might mirror the degree of inner certainty with which a given response was considered to be correct, independent of the impartial rightness of it. Considering the two formulated hypotheses from the beginning of this paper it can be stated that our study clearly demonstrated that in contrast to previous findings the film paradigm induced a stronger neural network during the processing of unknown stimuli. Our design meant that we closely mimicked a situation where a witness is asked to give testimony at a trial. The results show that our visual memory of an event is often flawed. It seems that when gaps in our memories are unconsciously filled several brain areas are involved in the process. Later on these fabricated parts are associated with even wider and stronger neural network. Additionally, we were able to show that it is not necessarily the reactivation of the truly encoded sensory material which activates more and stronger clusters in the brain but that secondary processes as imagining not seen parts or confusing schema-consistent elements are associated with stronger neural networks.
Few studies have used functional neuroimaging techniques to assess putative underlying neural structures of false recognitions while considering their effect on eyewitness testimonies. Okado and Stark (2005) investigated the misinformation effect, which can lead to false recognitions, by using fMRI during the encoding of pictorial and misinformation material. They found that differential activations in medial temporal and prefrontal regions predict memory errors during the recognition phase outside the scanner. Unfortunately, the occurrence of encoding processes during a recognition task like the one used in our study are unavoidable. However, we were able to minimize this effect through randomization of the recognition stimuli during the task. For example, the encoding during the decision process of a similar of scene 14 had an effect on one subject’s subsequent decisions in regard to the actual and unknown related to the same scene. This effect should be diminished across the group analysis of the neuroimaging data because the other subjects had seen these exemplary three pictures of scene 14 in a different order. Therefore, we still believe that the film paradigm elicited valid data that can be interpreted meaningfully in regard to false recognition research, as well as to the difficulties of eyewitness testimonies. Another study that looked into false memories and compared them to eyewitness report found larger and stronger neuronal activity changes for real than for imagined memories (Conway et al. 2003). Conway and colleagues instructed their subjects to visualize internally at a given cue a real or an imagined memory. For the latter they were to use places and people they knew. The important difference between their study and ours is that our subjects used studied information from the film to create — especially in the case of the unknowns — the imagined parts, which they later falsely recognized as known ones.
Our study very closely resembles what might happen during the production of false memories after witnessing a crime. Eyewitnesses see and process an overwhelming amount of information at the time of the incident. During and after the event they might transform the real occurrence into something that might fit better with their beliefs and previous knowledge. Thus true and false information can easily be mixed up to the point that a witness can no longer distinguish between what he experienced and what he only imagined. This study produced results which show that neuroimaging techniques can be used to investigate false recognitions produced for everyday events. Still, the final implication of our study is that before these methods can be used in real life situations, for example during a hearing in court, further detailed research should be carried out.
Notes
Acknowledgements
Research was supported by DFG (GK-518), EU (“Eyewitness Memory”, FP6-043460), and the Köhler Stiftung.
References
- Bernstein, D. M., Laney, C., Morris, E. K., & Loftus, E. F. (2005). False beliefs about fattening foods can have healthy consequences. Proceedings of the National Academy of Sciences of the United States of America, 102, 13724–13731.PubMedCrossRefGoogle Scholar
- Botvinick, M. M., Cohen, J. D., & Carter, C. S. (2004). Conflict monitoring and anterior cingulate cortex: An update. Trends in Cognitive Sciences, 8, 539–546.PubMedCrossRefGoogle Scholar
- Brett, C. (1999). The MNI brain and the Talairach atlas. http://www.mrc-cbu.cam.ac.uk/imaging/mnispace.html. Accessed 26 Sept 2003.
- Buckner, R. L., Wheeler, M. E., & Sheridan, M. A. (2001). Encoding processes during retrieval tasks. Journal of Cognitive Neuroscience, 13, 460–415.CrossRefGoogle Scholar
- Cabeza, R., Rao, S. M., Wagner, A. D., Mayer, A. R., & Schacter, D. L. (2001). Can medial temporal lobe regions distinguish true from false? An event-related functional MRI study of veridical and illusory recognition memory. Proceedings of the National Academy of Sciences of the United States of America, 98, 4805–4810.PubMedCrossRefGoogle Scholar
- Conway, M. A., Pleydell-Pearce, C. W., Whitecross, S. E., & Sharpe, H. (2003). Neurophysiological correlates of memory for experienced and imagined events. Neuropsychologia, 41, 334–340.PubMedCrossRefGoogle Scholar
- Deese, J. (1959). On the prediction of occurrence of particular verbal intrusions in immediate recall. Journal of Experimental Psychology, 58, 17–22.PubMedCrossRefGoogle Scholar
- Dodd, M. D., & MacLeod, C. M. (2004). False recognition without intentional learning. Psychonomic Bulletin & Review, 11, 137–142.Google Scholar
- Festinger, L. (1957). A theory of cognitive dissonance. Stanford, CA: Stanford University Press.Google Scholar
- Fletcher, P. C., Frith, C. D., Baker, S. C., Shallice, T., Frackowiak, R. S. J., & Dolan, R. J. (1995). The mind’s eye—Precuneus activation in memory-related imagery. Neuroimage, 2, 195–200.PubMedCrossRefGoogle Scholar
- Fletcher, P. C., & Henson, R. N. A. (2001). Frontal lobes and human memory: Insights from functional neuroimaging. Brain, 124, 849–881.PubMedCrossRefGoogle Scholar
- Ganis, G., Thompson, W. L., & Kosslyn, S. M. (2004). Brain areas underlying visual imagery and visual perception: An fMRI study. Cognitive Brain Research, 20, 226–241.PubMedCrossRefGoogle Scholar
- Garoff, R. J., Slotnick, S. D., & Schacter, D. L. (2005). The neural origins of specific and general memory: The role of the fusiform cortex. Neuropsychologia, 43, 847–859.PubMedCrossRefGoogle Scholar
- Garoff-Eaton, R. J., Slotnick, S. D., & Schacter, D. L. (2006). Not all false memories are created equal: The neural basis of false recognition. Cerebral Cortex, 16, 1645–1652.PubMedCrossRefGoogle Scholar
- Goff, L., & Roediger Jr., H. L. (1998). Imagination inflation for action events: Repeated imaginings lead to illusory recollections. Memory and Cognition, 26, 20–33.Google Scholar
- Gonsalves, B., & Paller, K. A. (2000). Neural events that underlie remembering something that never happened. Nature Neuroscience, 3, 1316–1321.PubMedCrossRefGoogle Scholar
- Gonsalves, B., & Paller, K. A. (2002). Mistaken memories: Remembering events that never happened. The Neuroscientist, 8, 391–395.PubMedGoogle Scholar
- Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: Wiley.Google Scholar
- Ihlebæk, C., Løve, T., Eilertsen, D. E., & Magnussen, S. (2003). Memory for a staged criminal event witnessed live on video. Memory, 11, 319–327.PubMedCrossRefGoogle Scholar
- Johnson, M. K., & Raye, C. L. (1998). False memories and confabulation. Trends in Cognitive Sciences, 2, 137–145.CrossRefGoogle Scholar
- Kensinger, E. A., & Schacter, D. L. (2005). Emotional content and reality-monitoring ability: fMRI evidence for the influences of encoding processes. Neuropsychologia, 43, 1429–1443.PubMedCrossRefGoogle Scholar
- Kensinger, E. A., & Schacter, D. L. (2007). Remembering the specific visual details of presented objects: Neuroimaging evidence for effects of emotion. Neuropsychologia, 45, 2951–2962.PubMedCrossRefGoogle Scholar
- Kim, H., & Cabeza, R. (2007). Trusting our memories: Dissociating the neural correlates of confidence in veridical versus illusory memories. Journal of Neuroscience, 27, 12190–12197.PubMedCrossRefGoogle Scholar
- Kopelman, M. D. (2002). Disorders of memory. Brain, 125, 2152–2190.PubMedCrossRefGoogle Scholar
- Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., et al. (2000). Automated Talairach Atlas labels for functional brain mapping. Human Brain Mapping,, 10, 120-131.PubMedCrossRefGoogle Scholar
- Levin, D. T., & Simons, D. J. (1997). Failure to detect changes to attended objects in motion pictures. Psychonomic Bulletin & Review, 4, 501–506.Google Scholar
- Lindsay, D. S., Allen, B. P., Chan, J. C. K., & Dahl, L. C. (2004). Eyewitness suggestibility and source similarity: Intrusions of details from one event into memory reports of another event. Journal of Memory and Language, 50, 96–111.CrossRefGoogle Scholar
- Lindsay, D. S., & Johnson, M. K. (2000). False memories and the source memory framework—Reply to Reyna and Lloyd (1997). Learning and Individual Differences, 12, 145–161.CrossRefGoogle Scholar
- Loftus, E. (2002). Memory faults and fixes. Issues in Science and Technology, 18, 41–50.Google Scholar
- Loftus, E. (2003). Our changeable memories: Legal and practical implications. Nature Reviews Neuroscience, 4, 232–233.CrossRefGoogle Scholar
- Maddock, R. J., Garrett, A. S., & Buonocore, M. H. (2003). Posterior cingulate cortex activation by emotional words: fMRI evidence from a valence decision task. Human Brain Mapping, 18, 30–41.PubMedCrossRefGoogle Scholar
- Magno, E., Foxe, J. J., Molholm, S., Robertson, I. H., & Garavan, H. (2006). The anterior cingulate and error avoidance. The Journal of Neuroscience, 26, 4769–4773.PubMedCrossRefGoogle Scholar
- Markowitsch, H. J., Vandekerckhove, M. M. P., Lanfermann, H., & Russ, M. O. (2003). Engagement of lateral and medial prefrontal areas in the ecphory of sad and happy autobiographical memories. Cortex, 39, 643–665.PubMedCrossRefGoogle Scholar
- Ogden, J. (1993). Visual object agnosia, prosopagnosia, chromatopsia, loss of visual imagery, and autobiographical amnesia following recovery from optical blindness: Case M.H. Neuropsychologia, 31, 571–589.PubMedCrossRefGoogle Scholar
- Okado, Y., & Stark, C. E. L. (2003). Neural processing associated with true and false memory retrieval. Cognitive, Affective, & Behavioral Neuroscience, 3, 323–334.CrossRefGoogle Scholar
- Okado, Y., & Stark, C. E. L. (2005). Neural processing during encoding predicts false memories created by misinformation. Learning & Memory, 12, 3–11.CrossRefGoogle Scholar
- Piefke, M., Weiss, P. H., Zilles, K., Markowitsch, H. J., & Fink, G. R. (2003). Differential remoteness and emotional tone modulate the neural correlates of autobiographical memory. Brain, 126, 650–668.PubMedCrossRefGoogle Scholar
- Rahm, B., Opwis, K., Kaller, C. P., Spreer, J., Schwarzwald, R., Seifritz, E., et al. (2006). Tracking the subprocesses of decision-based action in the human frontal lobes. Neuroimage, 30, 656–667.PubMedCrossRefGoogle Scholar
- Roediger III, H. L., & McDermott, K. B. (1995). Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 803–814.CrossRefGoogle Scholar
- Rolls, E. T., Browning, A. S., Inoue, K., & Hernadi, I. (2005). Novel visual stimuli activate a population of neurons in the primate orbitofrontal cortex. Neurobiology of Learning and Memory, 84, 111–123.PubMedCrossRefGoogle Scholar
- Schacter, D. L. (2001). The seven sins of memory: How the mind forgets and remembers. New York: Houghton Mifflin Company.Google Scholar
- Schacter, D. L., & Curran, T. (2000). Memory without remembering and remembering without memory: Implicit and false memory. In M. S. Gazzaniga (Ed.), The New Cognitive Neuroscience (pp. 829–840, 2nd ed.). Cambridge, MA: MIT.Google Scholar
- Schacter, D. L., Norman, K. A., & Koutstaal, W. (1998). The cognitive neuroscience of constructive memory. Annual Review of Psychology, 49, 289–318.PubMedCrossRefGoogle Scholar
- Schacter, D. L., Reiman, E., Curran, T., Yun, L. S., Bandy, D., McDermott, K. B., et al. (1996). Neuroanatomical correlates of veridical and illusory recognition memory: Evidence from positron emission tomography. Neuron, 17, 267–274.PubMedCrossRefGoogle Scholar
- Shimamura, A. P. (1995). Memory and frontal lobe function. In M. S. Gazzaniga (Ed.), The Cognitive Neurosciences (pp. 803–813). Cambridge, MA: MIT.Google Scholar
- Silvia, M. M., Groeger, J. A., & Bradshaw, M. F. (2006). Attention–memory interactions in scene perception. Spatial Vision, 19, 9–19.CrossRefGoogle Scholar
- Simons, D. J., & Chabris, C. F. (1999). Gorillas in our midst: Sustained inattentional blindness for dynamic events. Perception, 28, 1059–1074.PubMedCrossRefGoogle Scholar
- Simons, J. S., Koutstaal, W., Prince, S., Wagner, A. D., & Schacter, D. L. (2003). Neural mechanisms of visual object priming: Evidence for perceptual and semantic distinctions in fusiform cortex. Neuroimage, 19, 613–626.PubMedCrossRefGoogle Scholar
- Slotnick, S. D., & Schacter, D. L. (2004). A sensory signature that distinguishes true from false memories. Nature Neuroscience, 7, 664–672.PubMedCrossRefGoogle Scholar
- Small, D. M., Gitelman, D. R., Gregory, M. D., Nobre, A. C., Parrish, T. B., & Mesulam, M. -M. (2003). The posterior cingulate and medial prefrontal cortex mediate the anticipatory allocation of spatial attention. Neuroimage, 18, 633–641.PubMedCrossRefGoogle Scholar
- Stanislaw, H., & Todorov, N. (1999). Calculation of signal detection theory measures. Behavior Research Methods, Instruments, & Computers, 31, 137–149.Google Scholar
- Stern, C. E., Owen, A. M., Tracey, I., Look, R. B., Rosen, B. R., & Petrides, M. (2000). Activity in ventrolateral and mid-dorsolateral prefrontal cortex during nonspatial visual working memory processing: Evidence from functional magnetic resonance imaging. Neuroimage, 11, 392–399.PubMedCrossRefGoogle Scholar
- Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain. 3-dimensional proportional system: An approach to cerebral imaging. Stuttgart: Thieme.Google Scholar
- Varakin, D. A., & Levin, D. T. (2006). Change blindness and visual memory: Visual representations get rich and act poor. British Journal of Psychology, 97, 51–77.PubMedCrossRefGoogle Scholar
- Vogt, B. A., Finch, D. M., & Olson, C. R. (1992). Functional heterogeneity in cingulate cortex: The anterior executive and posterior evaluative regions. Cerebral Cortex, 2, 435–443.PubMedGoogle Scholar
- Wiese, H., & Daum, I. (2006). Frontal positivity discriminates true from false recognition. Brain Research, 1075, 183–192.PubMedCrossRefGoogle Scholar