In and out of consciousness: Sustained electrophysiological activity reflects individual differences in perceptual awareness
- 865 Downloads
Although significant advances in our understanding of the cognitive and neural processes involved in conscious awareness have occurred in recent years, the precise mechanisms that support consciousness remain elusive. Examining the neural correlates associated with the moment a stimulus enters or exits conscious awareness is one way to potentially identify the neural mechanisms that give rise to consciousness. In the present study, we recorded neural activity using electroencephalography (EEG) while participants observed a bilateral shape-from-motion (SFM) display. While the display is in motion, the observer perceives an object that is immediately segregated from a noisy background. After the motion stops, the observer’s experience of the object remains momentarily in awareness, before it eventually fades out of consciousness back into the noisy background. Consistent with subjective reports of perceptual experience, we observed a prominent sustained posterior contralateral negativity known as the contralateral delay activity (CDA). This activity was sustained only in conditions associated with sustained awareness. Interestingly, the amplitude of the CDA was correlated with individual differences in visual awareness, suggesting that this activity plays a significant role in the maintenance of objects in consciousness. The CDA is typically associated with visual short-term memory (VSTM), suggesting that conscious visual awareness may be mediated by the same neural and cognitive mechanisms that support VSTM. Our results demonstrate that the CDA may reflect the contents of conscious awareness, and therefore can provide a measure to track when information moves in and out of consciousness.
KeywordsVisual awareness Event-related potentials Attention Visual short-term memory Visual perception
Understanding how conscious experience arises from neural activity has been described as one of the most important questions facing science this century (Miller, 2005). Currently, however, we know very little about the neural processes that give rise to conscious experience. One of the problems facing consciousness research is the methodological challenge inherent to objectively measuring subjective phenomena. One approach has been to examine how a visual stimulus transitions from conscious awareness to “unawareness,” while holding the physical stimulus constant (Tong, Nakayama, Vaughan, & Kanwisher, 1998). While such studies describe the neural processes associated with conscious relative to unconscious perception, the mechanisms that enable the maintenance of awareness remain less clear.
In the present study, we attempted to capture the transition between states of conscious awareness using a shape-from-motion (SFM) paradigm (Regan, 1986, 2000). In this paradigm, fragmented line drawings of object stimuli are embedded among a background of randomly distributed line segments, making it initially impossible to distinguish the object from the background. Only when the object begins to move in a direction counter to that of the background does it become possible to perceive and recognize the object. Interestingly, when the motion stops, there is a brief period of time when the object can still be perceived, as its line fragments quickly seem to “fade” into the background (for a demo, see www.psych.utoronto.ca/~ferber/demos/sfmdemo.html). In other words, the awareness of the object percept is present where previously no awareness had existed (i.e., prior to the motion onset). This persistence of awareness, however, does not occur when the line fragments are removed from the display at the instant when motion stops. Moreover, the length of persistence has been shown to be independent of attentional manipulations (Ferber & Emrich, 2007) and to improve performance in objective sensitivity measures (Wallis, Williams, & Arnold, 2009), indicating that persistence represents a measurable change in perceptual awareness rather than a change in either response biases or attentional states.
A number of functional neuroimaging studies have attempted to identify the neural substrates subserving persistence during SFM (Ferber, Humphrey, & Vilis, 2005; Large, Aldcroft, & Vilis, 2005; Strother et al., 2011; Wong, Aldcroft, Large, Culham, & Vilis, 2009). These studies have revealed that perceptual persistence is associated with sustained activation in the ventral visual pathway, particularly in the object-sensitive lateral occipital complex (LOC; Ferber et al., 2005; Large et al., 2005), as well as in the primary visual cortex (Strother et al., 2011). However, it remains unclear what precise neurobiological mechanisms mediate the persistence of perceptual awareness. That is, how does the visual system maintain awareness of a stimulus when the features supporting that percept are no longer present? According to the global workspace theory of consciousness, sensory percepts become conscious by being broadcast within a global workspace, which functions as a temporary storage system (Baars, 1993, 2005). Short-term memory has been proposed as a likely contributor to the cognitive architecture of this workspace (Block, 2007; Crick & Koch, 1990). As such, perceptual persistence may represent a type of visual short-term memory (VSTM) for segregated perceptual forms (Ferber & Emrich, 2007).
If the persistence of awareness in SFM reflects the maintenance of object information in VSTM, then the event-related potential (ERP) associated with VSTM should be observed during perceptual persistence. Specifically, VSTM is associated with the contralateral delay activity (CDA) that is present throughout the duration of the delay period of change detection tasks. The amplitude of the CDA scales with the number of objects maintained in memory and predicts individual differences in VSTM capacity (Vogel & Machizawa, 2004), suggesting that this activity closely reflects the contents of memory. Moreover, this activity has also been associated with maintaining task-relevant information in awareness (e.g., distractors during a visual search task: Emrich, Al-Aidroos, Pratt, & Ferber, 2009), and it corresponds to the number of attended moving targets (Drew & Vogel, 2008). Thus, the CDA not only appears to reflect the activity during the delay period of a typical VSTM task, but it may also reflect the contents of visual awareness and perception.
Therefore, in order to capture the neural signature of an object that enters and exits conscious awareness, we examined the ERPs elicited during perceptual persistence generated in a lateralized SFM task. If VSTM-like neural mechanisms mediate conscious awareness, then the CDA should be elicited during perceptual persistence. Moreover, if this activity truly reflects perceptual awareness, it should be associated with individual differences in subjective reports of persistence.
A group of 16 individuals with normal or corrected-to-normal vision participated in this study. Three of these participants were excluded from the analyses due to high trial rejection rates (see below), resulting in a total of 13 participants (11 female, 2 male; ages 18–27 years, M = 20, SD = 2; all but 2 were right-handed). All participants provided written informed consent, and all procedures were approved by the University of Toronto Research Ethics Board.
Stimuli and procedure
Memory capacity estimate
Prior to the SFM task, behavioral VSTM capacity was estimated using a change detection task similar to that of Vogel, Woodman, and Luck (2001). Participants had to maintain one to four or six uniquely colored squares in memory for a 1,200-ms delay period and to indicate whether any changes occurred between the sample and probe arrays. There were 30 trials presented per set size, for a total of 150 trials. An electroencephalogram (EEG) was not recorded during this task. Memory capacity (k) was estimated by applying the Pashler–Cowan formula—k = set-size × (hits – false alarms) (Cowan, 2001; Pashler, 1988)—and we chose the participants’ highest score among the set sizes as their memory capacity estimate.
Electrophysiological recording and analysis
EEG recordings were obtained using a BioSemi ActiveTwo system (www.biosemi.com) with 64 active Ag/AgCl electrodes, digitally recorded at 512 Hz, mounted on an elastic cap with channel locations configured according to the international 10/20 layout. The electrophysiological data were analyzed using the EEGLAB software (http://sccn.ucsd.edu/eeglab/). The data were downsampled to 256 Hz, and a band-pass filter of 0.01–30 Hz was applied offline to the EEG. Trials containing large horizontal eye movements and/or eye blinks (>30-μV horizontal electrooculogram [HEOG] step-like artifacts and >80-μV vertical electrooculogram [VEOG] peak-to-peak threshold) were excluded from the analysis. The data of 2 participants with trial rejection rates over 40% in any condition were rejected; the remaining participants had an average rejection rate of 4% per condition.
ERPs were computed for a window of 3,000 ms and were time-locked to the onset of the static image, with a baseline correction of 200 ms preceding motion onset, which coincided with the delay after the arrow cue. Difference waves were calculated by subtracting ipsilateral from contralateral channels and then collapsing across presentation sides, resulting in a CDA measure for the persistence and vanish conditions. For all ERPs, the analysis was restricted to the 12 posterior channels that demonstrated maximal CDA activity: CP5/CP6, P3/P4, P5/P6, P7/P8, P9/P10, and PO3/PO4. Mean amplitudes were computed for four windows of the ERP: The first window (w1) was from 1,100–1,300 ms, which corresponded to the very end of the motion phase; the second window (w2), from 1,700–1,900 ms, corresponded to 300 ms post-motion-offset; the third (w3) and fourth (w4) windows corresponded to immediately before and after the average subjective estimation persistence (2,500–2,700 ms and 2,700–2,900 ms). The mean amplitudes were submitted to a repeated measures ANOVA with the factors Condition and Site. Pearson correlations were also computed between individual ERPs and the behavioral RTs. In all of the windows of interest, we found no significant interactions between electrode site and condition, so only main effects of amplitude are reported.
Consistent with previous behavioral results, perceptual persistence judgments were significantly longer for the persistence condition (M = 1,275 ms) than for the vanish condition (M = 511),F(1, 12) = 71.46,p < .001. Thus, the subjective awareness of the object was maintained for longer in the persistence condition than in the vanish condition. The visual short-term memory capacity estimates for all 13 participants had an average of 3.5 items, with a range of 2.8–4.4.
Finally, we attempted to examine whether the CDA measures the transition between states of awareness. That is, given the temporal resolution of EEG, it should be possible to demonstrate that the amplitude of the CDA decreases as the percept fades from awareness. To examine this question, we compared the amplitudes of the CDA in two 200-ms windows immediately before and after the average subjective reports of awareness (Fig. 2a, w3 and w4 results). This analysis revealed that the significant difference in amplitudes between the persistence and vanish conditions was sustained through the window immediately prior to the average perceptual persistence (2,500–2,700 ms; M = –1.171 and M = –0.329, respectively), F(1, 12) = 5.765, MSE = 27.602, p = .033. Critically, looking at the later window of 2,700–2,900 ms, we found no significant differences in mean amplitudes between the persistence (M = –0.944) and vanish (M = –0.423) conditions, F(1, 12) = 1.870, MSE = 10.594, p = .197. In other words, changes in CDA over time corresponded to participants’ subjective reports of when the percept faded from awareness, providing further evidence that the CDA is associated with the persistence of awareness during an SFM task.
The fact that the participants withheld their responses for a longer period during the persistence condition may have contributed to the present pattern of the CDA, as other components relating to the response (such as the lateralized readiness potential) could be present during this period. Although a comparison of the CDA amplitudes across hemispheres during w2 in the persistence condition revealed no significant difference, F(1, 12) = 3.524, MSE = 144.793, p = .085, we conducted a second experiment to address this possible alternative explanation of our findings.
A group of 10 individuals with normal or corrected-to-normal vision participated in this study. Two of the participants were excluded from the analyses due to high trial rejection rates (see below), resulting in a total of 8 participants (4 female, 4 male; ages 19–28 years, M = 21, SD = 3; 7 right-handed).
Stimuli and procedure
The procedure was identical to that of Experiment 1A, with the exception that we modified the design of the SFM task by instructing participants to withhold any response during the postmotion display. After the postmotion display, which we also changed to 2,000 ms, participants were given three response options—“short,” “medium,” or “long”—and were asked to select which response corresponded best to their subjective judgment of how long the object had persisted after the motion offset (Fig. 1b).
During the persistence condition, participants on average rated 144 of 160 trials as either “medium” or “long,” while in the vanish condition they rated only 16 of 160 trials as either “medium” or “long,” t(15) = –19.514, p < .001. These results are consistent with those of Experiment 1A, confirming that the subjective awareness of the object was maintained for longer in the persistence condition than in the vanish condition. The visual short-term memory capacity estimates for all 8 participants had an average of 3.6 items, with a range of 2.4–5.2.
The data from 2 participants with trial rejection rates over 25% in any condition were rejected; the remaining participants had an average rejection rate of 7.4% per condition. The same time windows were examined, with the exception of w3, as the responses contained no RT information. We did, however, include w4 in the analysis, based on the assumption that the persistence effect would likely have ended by that time period for the majority of participants.
The same pattern of results was observed in this experiment, replicating our initial findings. At w1, the mean CDA amplitudes between the persistence (M = –0.660) and vanish (M = –0.694) conditions did not differ significantly, F(1, 7) = 0.025, MSE = 0.027, p = .879 (Fig. 2b, w1 results). After motion stopped, we did see a difference in the mean CDA amplitudes between the persistence (M = –1.121) and vanish (M = –0.115) conditions, F(1, 7) = 6.435, MSE = 24.430, p = .039 (Fig. 2b, w2 results). Finally, at w4, presumably after the duration of the persistence effect, the mean CDA amplitude of the persistence condition (M = –0.917) no longer differed from that of the vanish condition (M = –0.380), F(1, 7) = 1.601, MSE = 6.924, p = .246 (Fig. 2b, w4 results).
Our results demonstrate that a sustained negative ERP contralateral to the attended display (i.e., the CDA) was observed during SFM perception. Importantly, immediately after the offset of motion, this activity was only sustained when the subjective awareness of the object persisted (i.e., in the persistence condition but not the vanish condition), and the decrease of the CDA corresponded to the disappearance of the percept from awareness. Moreover, the amplitude of the CDA was significantly correlated with individual differences in the duration of perceptual persistence. Thus, these results demonstrate that the CDA may be an electrophysiological marker of perceptual awareness. This study adds to the growing number of ERPs that have been associated with states of consciousness, such as the visual awareness negativity and the late positivity components (Railo, Koivisto, & Revonsuo, 2011). Importantly, however, the findings observed here revealed that the CDA is sensitive to the fairly quick transition between states of visual awareness. In other words, while other ERPs have been shown to be associated with the presence or absence of awareness, the CDA appears to reflect the state of sustained awareness, as well as capturing the time course of information as it transitions out of consciousness.
One alternative possibility is that the decrease in the amplitude of the CDA observed in the vanish condition does not reflect a change in subjective awareness, but instead reflects the loss of information from the display. We can rule out this alternative, however, given the gradual decrease in the CDA in the persistence condition. That is, even though the physical stimulus remains constant, the CDA in the persistence condition eventually becomes statistically indistinguishable from that of the vanish condition. Instead, this decrease likely reflects the loss of perceptual awareness experienced by the observers.
Overall, these findings are consistent with evidence from studies of memory (McCollough, Machizawa, & Vogel, 2007), perception (Robitaille & Jolicœur, 2006), visual search (Emrich et al., 2009), and object tracking (Drew & Vogel, 2008) that show that the CDA is associated with the focus of conscious processing. Importantly, this study provides the first direct evidence that the CDA reflects the contents and time course of conscious awareness. In other words, the CDA may in part reflect a neuronal global workspace for conscious awareness (Dehaene, Changeux, & Naccache, 2011), as this component is evident whenever visual information is made the object of conscious awareness.
In addition, the finding that the CDA is present during perceptual persistence supports the proposal that persistence is mediated by VSTM (Ferber & Emrich, 2007). Moreover, these findings also support theories positing that VSTM is critical to visual awareness (Baars, 1993; Churchland, 2007; Crick & Koch, 1990). The mnemonic process subserving persistence, however, is unlikely to be identical to the process that supports VSTM for simple shapes and features, as VSTM capacity was not correlated with the duration of perceptual persistence, nor with the amplitude of the persistence CDA (r = –.113, p = .714, and r = –.225, p = .460, respectively), indicating partially independent cognitive and neural processes. That is, whereas simple colored shapes may be maintained by areas or networks specialized for processing color information, the maintenance of object information during SFM is likely mediated primarily by object-sensitive LOC (Ferber et al., 2005). In both cases, however, the CDA may reflect the neural mechanism that mediates the sustained online processing of relevant region-specific information. This proposal is consistent with theories positing that short-term memory is an emergent property of perceptual and motor processes (Postle, 2006). Alternatively, the CDA in the present context may reflect the synchronization between higher visual areas and primary visual cortex, as both VSTM (Harrison & Tong, 2009; Serences, Ester, Vogel, & Awh, 2009; Super, Spekreijse, & Lamme, 2001) and perceptual persistence (Large et al., 2005; Strother et al., 2011) are associated with sustained activity in early visual cortex. In other words, the CDA may reflect reentrant processing and synchronization between primary and secondary visual cortices that enables information to be maintained online (Lamme, 2010).
In summary, the results of the present study demonstrate that the CDA can be used as a reliable measure of perceptual awareness. In addition, the sustained nature and temporal precision of the CDA enable us to measure the neural correlates associated with the awareness of an object as it fades in and out of consciousness. Thus, the results suggest that perceptual awareness may be mediated by neural mechanisms similar to those that support VSTM. Consequently, these findings may have significant implications for the relationship between the CDA, VSTM, and conscious awareness.
The authors thank Trafton Drew for useful feedback. This research was conducted at the Centre for Biological Timing and Cognition at the University of Toronto and was supported by grants from the Natural Sciences and Engineering Research Council of Canada and the Canadian Institutes of Health Research, and by an Early Researcher Award from the Province of Ontario.
- Baars, B. J. (1993). A cognitive theory of consciousness. Cambridge: Cambridge University Press.Google Scholar
- Churchland, P. M. (2007). The engine of reason, the seat of the soul: A philosophical journey into the brain. Cambridge: MIT Press.Google Scholar
- Crick, F., & Koch, C. (1990). Towards a neurobiological theory of consciousness. Seminars in the Neurosciences, 2, 263–275.Google Scholar
- Dehaene, S., Changeux, J. P., & Naccache, L. (2011). The global neuronal workspace model of conscious access: from neuronal architectures to clinical applications. In S. Dehaene & Y. Christen (Eds.), Characterizing consciousness: From cognition to the clinic? (pp. 55–84). Berlin: Springer.CrossRefGoogle Scholar
- Regan, D. (2000). Human perception of objects: Early visual processing of spatial form defined by luminance, color, texture, motion, and binocular disparity. Sunderland: Sinauer.Google Scholar
- Robitaille, N., & Jolicœur, P. (2006). Fundamental properties of the N2pc as an index of spatial attention: Effects of masking. Canadian Journal of Experimental Psychology, 60, 79–89.Google Scholar