Despite our experience of a rich and complete world, the amount of information that becomes consciously accessible to us is only a fraction of the information available to us at any given moment (Marois & Ivanoff, 2005). What information reaches the level of consciousness has therefore been a topic of much inquiry, particularly in the area of visual awareness (Crick & Koch, 1990; Lamme et al., 2000; Milner, 1995), as it has been demonstrated that awareness of a visual stimulus can be manipulated through various experimental paradigms, such as binocular rivalry, continuous flash suppression (CFS), and visual masking, among others (see Kim & Blake, 2005, for a review).

Previous studies of visual awareness have tended to use coarse measures of awareness, such as forced-choice judgments of shape or pattern (e.g., Olivers, Meijer, & Theeuwes, 2006; Pan, Cheng, & Luo, 2012). That is, most studies of visual perception have tended to focus on the question of sensitivity. It is possible, however, that items that reach the threshold for awareness may also vary in their fidelity or perceptual quality. For example, it has been shown that information stored in visual working memory (VWM) varies dramatically in its fidelity, in addition to (Zhang & Luck, 2008) or instead of (Bays & Husain, 2008) varying only in the likelihood that an item is encoded or maintained in memory.

To address the question of perceptual fidelity, Harrison, Rajsic, & Wilson (2015) examined how masking influences the fidelity of perceptual representations. They used object-substitution masking (OSM) (Enns & Di Lollo, 1997), in which the visibility of a target item is impaired by a sparse four-dot mask that has a delayed offset from the target (for a review, see Goodhew et al., 2013). Harrison and colleagues (2015) found that the precision of a visual representation was affected by the presence and timing of a mask. This suggests that, contrary to our naïve experience, our conscious experience of visual objects is not all-or-none, but rather comes with varying degrees of perceptual fidelity.

If visual awareness comes with varying degrees of fidelity, then it is possible that factors that alter the threshold of awareness could also affect the fidelity of those representations. For example, it has been shown that the contents of visual awareness can be biased towards task-relevant information. Gaillard and colleagues (2006) manipulated this threshold of awareness by consciously priming subsequently presented masked words, significantly increasing the rate of target detection. It has also been demonstrated that the threshold of awareness can be altered via the contents of visual working memory. Using CFS, Gayet, Paffen, and Van der Stigchel (2013) found that VWM contents matching the color of a suppressed stimulus allowed the memory-matching stimulus to reach awareness faster, indicating that visual awareness is prioritized for information matching the contents of VWM.

The results of Gayet et al. (2013) suggest that the contents of VWM could have two different (and potentially complementary) effects on subsequently perceived stimuli. On the one hand, the threshold for perceiving items that match the contents of VWM might be lowered, leading to increased sensitivity; alternatively, the perceived representation might be biased towards the contents of VWM, resulting in an increase in the quality or fidelity of perception. Consequently, in the current study, we examined to what extent information stored in VWM could bias whether a stimulus reached the threshold of conscious perception as opposed (or in addition) to whether it affected the fidelity of the visual representation. To examine this question, we used a combined VWM-OSM paradigm, and a mixture-model analysis of a continuous-report task.

Methods

Participants

A total of 35 participants completed the study and 29 participants were included in the final analysis [13 male, mean age (M age) = 20 years]. Sample size was determined a priori based on the assumption of a relatively small effect size compared to other studies using similar analytical approaches (Emrich & Ferber, 2012). The rejection criteria are defined below in the Data analysis section.

All participants self-reported normal or corrected-to-normal vision. Informed consent was provided before participation, as approved by the Research Ethics Board of Brock University. Participants received course credit or monetary remuneration for their participation.

Apparatus

The task was completed on a 41-inch NEC MultiSync LCD 2090UXi computer monitor (1024 × 768 pixels, 60 Hz refresh rate) in a private testing room. All stimuli were presented using Psychopy version 2 (Peirce, 2009). Viewing distance was set to approximately 57 cm.

Stimuli and procedure

An example trial sequence can be found in Fig. 1. All stimuli were presented against a uniform grey background. Each trial began with a fixation dot for 500 ms followed by two randomly oriented Landolt-Cs (radius of 2.75°) presented in the center of the screen for 500 ms each (with a 500 ms fixation between them). The gap of each Landolt-C was approximately one-quarter of the size of the stimulus, and was in a randomly selected orientation from 0° to 359°, with the restriction that the orientations of both Cs could not be less than 30° from one another. Next, the number 1 or 2 appeared on the screen for 500 ms, which post-cued participants on which Landolt-C to remember for the subsequent memory task (1 = the first Landolt-C, 2 = the second Landolt-C), followed by a 100 ms fixation.

Fig. 1
figure 1

Example trial sequence (matching condition). Participants saw two large Landolt-Cs and a number which post-cued which one to remember. Next, they were presented with a perceptual task wherein they localized a small masked Landolt-C (target). Participants were then probed to report the orientation of one of the two Landolt-Cs. Blank fixation screens are omitted

Full size image

Next, a search screen of six Landolt-Cs (radius of 0.95°) was presented for 30 ms. The stimuli were presented equally spaced around an imaginary circle with a radius of 5° from the fixation point. One of the Landolt-Cs was pseudo-randomly chosen to be surrounded by a four-dot mask (2.5° × 2.5°), which disappeared 80 ms following the stimuli offset (delayed mask offset). In the matching condition, the orientation of the target Landolt-C matched the orientation of the cued Landolt-C. In the non-matching condition, the orientation of the target Landolt-C matched the orientation of the non-cued Landolt-C.

Next, participants were probed to either report the orientation of the Landolt-C presented during the memory task, or the orientation of the target Landolt-C (perceptual task). Each of the conditions occurred on one-half of the trials and were intermixed pseudo-randomly. The VWM task was cued by a Landolt-C of the same size and central position of the larger to-be-memorized Landolt-C. This task ensured that participants were successfully holding the item in VWM on every trial. Participants responded to the perceptual task when they were presented with a Landolt-C the same size and non-central position of the target Landolt-C. Both of the response Landolt-C orientations were jittered on the response screen. To make their response, participants used the mouse to move the orientation of the Landolt-C on the screen to the orientation of either the to-be-memorized or target Landolt-C. Responses were recorded in degrees with the mouse click. Positive feedback was provided on-screen when participants were within 20° of the correct orientation (with the correct Landolt-C depending on the task).

Participants were given eight practice trials before beginning the experiment, with additional practice if necessary. All participants completed a total of 500 trials. These trials were divided evenly by task (VWM or OSM), and whether or not the item held in VWM matched the OSM target.

Data analysis

Analyses were completed in MATLAB version R2014a using the three-component swap model as developed by Bays et al. (2009) (http://www.paulbays.com/code/JV10/) using the MemToolBox 1.0 (Suchow, Brady, Fougnie, & Alvarez, 2013).

Response error was calculated on each trial as the difference between the target’s actual orientation and the participant’s response in degrees. Response error was then broken down into a mixture of three types of responses according to the methods of Bays et al. (2009): target responses, non-target responses, and guess responses. Target responses occur when the target orientation is correctly reported, non-target responses occur when the orientation of a distractor (non-target) is reported, and guess responses occur when a random orientation response is made. Target and non-target responses were drawn from a circular Gaussian (von Mises) distribution, and guess responses were drawn from a uniform distribution. The standard deviation (SD) of the circular von Mises distribution for non-guess responses was used to estimate the precision of responses. Note that SD is inversely related to precision. A fourth parameter (target responses) was defined as 1 – guess response – non-target error responses, and reflects the probability of correctly reporting the target orientation.

For both the perceptual and memory tasks, a total of six non-targets were included in the mixture model, five of which were defined as the orientations of the distractors that were presented during the OSM display. One additional non-target was defined for each task according to the memory-matching condition. This was done to ensure that participants were reporting the orientation of the ‘correct’ Landolt-C for each task and therefore served as a check for task compliance. In the matching condition of the perceptual task and both the matching and non-matching conditions of the VWM task, the additional non-target was defined as the orientation of the non-cued memory item. In the non-matching condition of the perceptual task, the additional non-target was defined as the cued memory item (i.e., not the target of the perceptual task).

A high non-target error rate in the perceptual task would suggest that the participant was instead reporting the orientation of the VWM item. Similarly, a high non-target error rate in the VWM task would suggest that the participant was reporting the orientation of the perceptual target. Five participants’ data were excluded for non-target error rates greater than three SDs from the mean in any condition in either the perceptual or memory task. Data from one additional participant were excluded due to guess rates that were greater than three SDs from the mean in both conditions of the perceptual (OSM) task.

Results

As the memory task was not the main focus of the study, and served only as a check for task compliance, the results from the memory task can be found in the supporting information available on-line.

For the perceptual task, a one-tailed paired-samples t-test was conducted for each parameter estimate (guess rate, non-target error rate, SD, and target responses) between the matching and non-matching conditions (see Fig. 2), based on the prediction that performance in each measure would improve in the memory-matching condition (lower guess rate, non-target error rate, and SD; greater target rate). When examining the guess rate and target rate, no significant improvement was observed in the memory-matching trials compared to the non-matching trials (p values > 0.25, d values < 0.12). Confirming this, the Bayes Factor (JASP version 0.7.5.5, 2016) calculation supported the null over the alternative hypotheses (guess rate: BF 01 = 4.59, target rate: BF 01 = 2.79). Non-target errors, which were defined as the proportion of trials in which participants incorrectly reported one of the non-masked stimuli in the perceptual task, also did not decrease in the matching compared to the non-matching condition, t(28) = .601, p = .724, d = .112, BF 01 = 7.54.

Fig. 2
figure 2

a Mean response proportions and b mean and individual standard deviations (SD) (greater SD = less precision) for each memory-matching condition. Within-subjects error bars represent the 95 % CI. (N = 29)

Full size image

In contrast to the effect of memory-matching condition on the measures of perceptual threshold, we observed a significant reduction in the SD of the reported target in the memory-matching condition (M = 22.98, SD = 5.35) relative to the non-matching condition (M = 25.49, SD = 8.18), t(28) = –1.914, p = .033, d = –.355. This effect is further supported by the calculation of the Bayes Factor, which indicated that a smaller SD in the matching as compared to the non-matching condition was 1.876 times more likely than the null. Examining individual subjects’ data, 19 out of 29 participants were more precise in the matching than in the non-matching condition (see Fig. 2b).

Given the relatively small evidence (BF < 3) for a change in SD, as well as the potential that our initial rejection criteria could not remove those subjects who might be employing a biased response strategy, we examined the possibility that outliers were driving the observed effects by performing a second rejection step on the difference scores between memory-matching conditions for each parameter estimate. Participants with scores greater than 1.5 times the interquartile range outside of the first or third quartile of scores were excluded (Tukey, 1977), resulting in the exclusion of four additional participants (N = 25). Once again, SD was significantly different between the matching (M = 22.02, SD = 4.67) and non-matching (M = 24.47, SD = 6.46) conditions, t(24) = –2.70, p = .006, d = –0.539, BF10 = 7.79, with no significant differences observed between conditions for guess rate, non-target errors, or target rate, p values > 0.26, d values < 0.13, BFs01 > 2.7. Thus, removing outliers from the analysis actually increased the effect size of the observed difference in SD, consistent with the interpretation that the contents of VWM increase the fidelity of subsequently perceived stimuli, without affecting the threshold of awareness.

Is it possible that participants were simply reporting the orientation of the item held in VWM on the perceptual task, leading to the appearance of increased precision on the memory-matching trials? To test this question, we included the item held in memory in the calculation of non-target errors for non-matching trials (see Data analysis section). Critically, there was no increase in the proportion of non-target errors in the non-matching condition compared to the matching condition, t(28) = –.381, p = .353, d = –.071, BF 01 = 3.693. Thus, participants did not have a response bias towards the item held in memory that would account for the observed increase in precision.

Discussion

In the current study, we sought to answer whether the contents of VWM affected the fidelity of perceived items, in addition to or instead of affecting the threshold of perceptual awareness. The results provide evidence that the contents of VWM solely affect the fidelity of subsequent memory-matching representations, without having an effect on the threshold of awareness.

How might the contents of VWM affect the fidelity of subsequently perceived stimuli? These findings may be explained partially by overlapping neural substrates in which perceptual and VWM representations are maintained. It has been found that neural activity associated with VWM maintenance occurs in part through sensory recruitment (Postle, 2006), such that patterns of activation in sensory areas (i.e., V1–V5) store stimulus-specific information (Ester, Serences, & Awh, 2009; Harrison & Tong, 2009; Serences et al., 2009; but see, Christophel, Hebart, & Haynes, 2012; Ester, Sprague, & Serences, 2015). Moreover, representations are similar enough that patterns of activation from one task (e.g., VWM) can be used to train and test patterns from another task (e.g., perception; Harrison & Tong, 2009). In particular, the fidelity of memory representations (Emrich et al., 2013; Ester, Anderson, Serences, & Awh, 2013), but not the likelihood of memory recall (Emrich et al., 2013), seems to be coded by early visual cortex, suggesting that VWM and perception share overlapping neural substrates in early visual cortex, and that this activity is associated with the fidelity of representations, more so than the likelihood of memory encoding/retrieval. Consequently, early visual cortex may serve as a locus for perceptual and VWM representations to interact, resulting in neural responses to percepts that are biased by those coding for VWM contents.

In accordance with these findings, there is evidence that top-down expectations influence visual cortex, such that our earliest visual representations are biased by higher-order information (Kok et al., 2013). Using functional magnetic resonance imaging (fMRI), Kok, Failing, and Lange (2014) found that expectation of a visual stimulus evoked a pattern of activity in V1 that was similar to the activity evoked by the stimulus itself. Prior expectations have also been found to specifically affect the resolution of visual representations in V1, even though these expectations reduce the amplitude of neural responses in these areas (Kok, Jehee, & de Lange, 2012). The authors suggested that the prior expectation of a stimulus provides a greater signal-to-noise ratio (SNR), allowing for a sharpening of encoded representations. If so, this evidence corroborates our finding that VWM—a higher-level process similar to expectation—influences the fidelity of early visual representations, and that this effect may be mediated by an increase in the SNR of early visual activity. This may also explain why a change in perceptual fidelity may lead to a reduced RT for detecting a stimulus (Gayet et al., 2013) without affecting the threshold of awareness: an increase in SNR resulting from a change in perceptual fidelity may affect response decisions (Scolari & Serences, 2010), thereby speeding reaction times (Mazurek et al., 2003).

Our observed change in perceptual fidelity is in conflict with Asplund and colleagues (2014), wherein it was suggested that attention does not affect the precision of visual representations in an attentional blink task, only the threshold of awareness. Unlike the current study, Asplund and colleagues (2014) used a mixture model analysis that does not take into account the influence of non-target errors (Zhang & Luck, 2008). Bays et al. (2009) have demonstrated that a significant number of responses within the uniform (guess) component can be accounted for by these non-target responses, which also affects the SD of the target responses. Consequently, it is possible that accounting for non-target errors in the Asplund et al. (2014) study would reveal a change in response precision rather than a change in the likelihood of reporting the target. Alternatively, it is possible that there may be two independent stages of processing that affect the threshold and precision of perceptual representations, respectively; an item may first pass the threshold of awareness and then be affected by higher-level processes (such as prior expectation or VWM) that could tune the fidelity of the initial representation.

In sum, our findings suggest that the contents of VWM alter the fidelity of perceptual representations at the earliest stages of visual awareness. This finding has significant implications for theories of visual awareness. For example, a number of theories of awareness emphasize the role of feedforward and re-entrant processing (i.e., Lamme & Roelfsema, 2000), which have been previously implicated in OSM (Di Lollo, Enns, & Rensink, 2000). These theories posit that recurrent interactions between and within multiple levels of the brain are needed for consciousness (e.g., Lamme & Roelfsema, 2000). Although this explains the differences between conscious and unconscious processes, it does not elaborate upon the effects of re-entrant processes on the quality of conscious representations. Consequently, future studies should investigate the effect of VWM contents on feedforward and re-entrant activity, and how these processes affect the fidelity of visual representations. Indeed, our results further emphasize the importance of examining perceptual quality, in addition to measures of sensitivity, in future studies of visual awareness.