Object-substitution masking degrades the quality of conscious object representations
Object-substitution masking (OSM) is a unique paradigm for the examination of object updating processes. However, existing models of OSM are underspecified with respect to the impact of object updating on the quality of target representations. Using two paradigms of OSM combined with a mixture model analysis we examine the impact of post-perceptual processes on a target’s representational quality within conscious awareness. We conclude that object updating processes responsible for OSM cause degradation in the precision of object representations. These findings contribute to a growing body of research advocating for the application of mixture model analysis to the study of how cognitive processes impact the quality (i.e., precision) of object representations.
KeywordsObject-substitution masking Object-updating Consciousness
A fundamental goal in the study of perception is the delineation of the processes that transform sensory information into objects of consciousness (Dehaene & Naccache, 2001). To that end, visual masking has been an influential tool in probing the boundaries between conscious and unconscious vision (Breitmeyer, 2007). Typically, masking refers to the disruption in processing of a target object by a temporally proximate second object (the mask) that either overlaps or shares a contiguous spatial boundary with the target (Breitmeyer & Öğmen, 2006). Due to the spatial contiguity between target and mask, these standard masking procedures disrupt low level or early visual processes by merging the target and mask into a single indecipherable percept. This feature makes standard masking procedures unsuitable for addressing the later perceptual processing mechanisms that underlie conscious awareness.
Unlike standard masking procedures, object substitution masking (OSM), has properties that make it ideal for studying the later perceptual processing stages contributing to conscious awareness (Di Lollo, Enns, & Rensink, 2000; Goodhew, Pratt, Dux, & Ferber, 2013). Typically, an OSM paradigm involves the brief presentation of one or more objects with one of the objects (the target) surrounded by four small dots (the mask). On simultaneous mask trials, the target (and all other objects) and mask onset and offset together. On delayed mask trials, they onset together but the mask offset is delayed relative to the target and other objects. Effective masking is evidenced by reduced accuracy in reporting either the presence of, or some characteristic of, the target object in the delayed mask condition. Unlike other forms of masking whose results are explained primarily by disruptions to object formation mechanisms (see Enns, 2004, for a review), several unique characteristics of OSM (e.g., common onset of the target and mask, sparseness of the mask) support the notion that its effects result from conflict between object-based representations (Di Lollo et al., 2000; Lleras & Moore, 2003; Moore & Lleras, 2005; but see Põder, 2013). This object substitution account of masking describes a communicative process that resolves discrepancies between late and early stages of visual processing in an attempt to determine the appropriate conscious representation. In the case of OSM, this results in the target’s representation being updated with that of the mask, and the latter reflecting conscious experience.
To date, studies examining OSM have used only coarse measures of target awareness (e.g., present versus absent; oriented left versus right), an approach that has resulted in theories of OSM being underspecified with respect to the impact of OSM on the quality of target representations. In fact, object-mediated accounts of OSM contend that either the target representation is eliminated before a conscious representation is produced (e.g., Di Lollo et al., 2000) or a conscious target representation is overwritten by the mask’s representation (e.g., Lleras & Moore, 2003). An outstanding question then remains regarding whether OSM results in a complete loss of target information, or whether there is a partial loss such that the target is degraded but still accessible to consciousness.
In the working memory (WM) literature, a novel method for measuring the quality of representation of object properties has gained traction recently (Bays, Catalao, & Husain, 2009; Zhang & Luck, 2008). This method requires participants to retrieve a particular feature of a memory item and to respond using a continuous scale. On a typical trial, participants might be instructed to remember the orientation of several memory items and, after a short delay, reproduce the orientation of one item. This means that, unlike standard WM tasks, responses are not dichotomized as correct or incorrect. By fitting a mixture model, one can estimate the proportion of trials where responses came from a circular Gaussian distribution centred on target’s orientation, guess trials where responses came from a uniform distribution, and non-target trials in which responses came from a circular Gaussian centered on a different item. Critically, the standard deviation for the target responses’ distribution provides a quantifiable measure of the quality (i.e., precision) of the memory target feature.
To our knowledge, the mixture modelling approach has been applied to conscious perception in just one study. Asplund and colleagues (2014) showed that, during an attentional blink, discrete representations are lost, but the precision of object representations is unaffected. This led to the conclusion that conscious perception may be all-or-none. However, the attentional blink and OSM may rely on distinct mechanisms of conscious perception (Giesbrecht, Bischof, & Kingstone, 2003). The goal of this paper then was to adapt the mixture model analysis to the OSM paradigm and, by doing so, elucidate the impact of OSM on the quality of target representations. If OSM eliminates conscious object representations, as reported for the attentional blink, then the delayed mask should have no effect on target precision, but reduce the proportion of target responses and an increase the proportion of guess responses. If, however, OSM operates by degrading conscious representations, then there should be a reduction in target precision.
Twenty-three undergraduate students from Queen’s University participated in exchange for credit in an introductory psychology course. All had normal or corrected-to-normal vision, and were naïve to the experiment’s purpose.
The experiment was conducted on a personal computer in a dimly lit room. Stimuli were presented with Psychophysics Toolbox version 3.0.8 (Brainard, 1997) in MATLAB version 7.04 on a 16-inch CRT monitor. A chin-rest kept viewing distance constant at 50 cm. Responding was done with a Logitech t650 touchpad and keyboard.
Design and procedure
For each trial, orientation error was the difference between the target’s actual and reported orientation in degrees. Orientation errors for each of the nine conditions (three set sizes and three mask durations) were fitted to the three-component model of Bays et al. (2009). This model assumes a weighted mixture of three response types: (1) target responses, defined as correctly reporting the target orientation; (2) guess responses, defined as randomly guessing an orientation value; and (3) non-target responses, defined as reporting the orientation of a non-target object. Target and non-target responses were modelled with circular Gaussian distributions termed von Mises distributions, whereas guess responses were modeled with a uniform distribution. The concentration parameter (measure of spread) of each von Mises distribution was fit to the same parameter and then converted to a standard deviation using a mathematical transformation described by Bays et al. (2009). Thus, four parameter estimates were derived for each condition for each participant: (1) target responses (PTarget)—proportion of trials in which there was memory of the target’s orientation, (2) non-target responses (PNon-target)—proportion of trials in which a non-target’s orientation was reported, (3) guess responses (PGuess)—proportion of trials in which there was no memory and orientation was guessed, and (4) gap variance (SD)—standard deviation of the circular Gaussian distribution component for the non-guess responses.2 Note that SD is inversely related to precision.
Results and discussion
For PNon-target there was a main effect of mask duration, F(2,34) = 37.65, P < .001, partial η2 = .69; however, there was no main effect of set size and no interaction, all Fs < 2.16, all Ps > .13, all partial η2s < .11. Contrast analyses revealed that PNon-target increased linearly across mask durations, P = .003, partial η2s = .42.
For SD (see Fig. 2b), there was a main effect of set size, F(2,34) = 23.08, P < .001, partial η2 = .58, and mask duration, F(2,34) = 25.93, P < .001, partial η2 = .60; however, the interaction was not significant, F(1,280.07) = 2.69, P = .12, partial η2 = .14. Contrast analyses revealed a linear increase in variability (i.e., decrease in precision) across set sizes and mask durations, all Ps < .001, all partial η2 > .68. However, post-hoc tests showed that there were no differences in SD between 150 and 300 ms delayed mask durations for any set sizes, all ts < .96, all Ps > .35.
Importantly, our PTarget estimates mirror those from the OSM literature using either forced choice or presence/absence discrimination procedures (e.g., Di Lollo et al., 2000) in showing both decreases in PTarget with increasing set size, and reduced PTarget for delayed relative to simultaneous offset mask conditions (and vice-versa for PGuess). Furthermore, the mixture model demonstrated that both of these reductions in PTarget (and increases in PGuess) were accompanied by an increase in SD (i.e. reduced precision). These results speak to the strength of applying a mixture modeling approach, as we were able to demonstrate that SD increased by over 25 % from the simultaneous (22.28°) to the delayed mask offset conditions (28.97° and 28.43°).
Surprisingly, PNon-target was both non-zero and increased as a result of OSM. Whether this represents true location-feature misbinding, or whether when participants had no target memory, they adopted a strategy of responding with a non-masked item’s orientation rather than guessing, is beyond the scope of this report. Importantly, less than 10 % of trials were non-target trials and excluding this component from the model (i.e., using a two component model like the one proposed by Zhang and Luck, 2008) did not change the main finding that delayed mask conditions in OSM produce degradation in the precision of a target’s orientation.
That OSM can degrade a target’s representation without it being ‘substituted’ or ‘updated’ out of conscious awareness seems to be inconsistent with object-mediated accounts of OSM and is perhaps more in line with the feedforward account proposed by Põder (2013). Põder’s model shows that failures in target report in some OSM paradigms can be modeled by target degradation resulting from both signal decay and by the addition of mask noise to the target representation.
To test Põder’s (2013) account Jannati, Spalek, and Di Lollo (2013) developed a novel OSM procedure that separated an initial “target plus mask” stimulus from a subsequent “mask only” stimulus. Because the contribution of noise by the mask is equated across conditions, and signal/target decay is assumed to decrease with increasing ISI, Põder’s model predicts that OSM should increase with increasing ISI. However, Jannati et al. (2013) found that OSM peaked at an ISI of approximately 80 ms—an ISI close to the theorized time needed to execute communication between low- and high-level visual processes (Fahrenfort, Scholte, & Lamme, 2007).
The purpose was to first replicate the OSM effect on PTarget, PGuess, PNon-target, and SD using the procedure developed by Jannati et al. (2013), and second to determine whether the vitiating effects of OSM on target precision observed in Experiment 1 would generalize to a paradigm designed to isolate object updating mechanisms from possible feedforward mechanisms.
Twenty-three undergraduate students from Queen’s University participated in exchange for credit in an introductory psychology course. All participants had normal or corrected-to-normal vision, and were naïve to the experiment’s purpose.
Apparatus, design, and procedure
The apparatus, design and procedure were identical to Experiment 1 with the following exceptions (see Fig. 1b). We replaced the three mask durations with three ISI conditions (0, 80, 320 ms). ISI referred to the duration between the ‘target and mask’ offset and the mask-only onset, and were the same as those used in Experiment 3 of Jannatti et al. (2013). The duration of both the ‘target and mask’ display and mask-only display was fixed at 17 ms.
Results and discussion
Results for PNon-target mirrored those of Experiment 1 in that there was a main effect of ISI, F(2,42) = 12.34, P < .001, partial η2 = .37, such that there was no difference between the 0 (1.1 %) and 80 ms (1.3 %) ISI conditions, t(21) = 0.38, P = .71, but PNon-target increased from the 80 to 320 ms ISI condition (3.2 %), t(21) = 3.51, P = .002. There was no effect of set size and no interaction, all Fs < 2.82, all Ps > .07, all partial η2s < .12.
For SD (see Fig. 3b), there was a main effect of set size, F(2,42) = 8.26, P = .001, partial η2 = .28, and mask duration, F(2,42) = 39.60, P < .001, partial η2 = .65, but no interaction, F(4,84) = 0.65, P = .63, partial η2 = .03. Contrast analyses revealed a linear increase in SD across set sizes, and a quadratic trend for ISI, all Ps < .001, all partial η2 > .50. Critically, variance was greatest in the 80 ms ISI condition (28.74°) relative to the 0 ms (24.58°) and 320 ms (23.14°) conditions, all Ps < .001.
As with Experiment 1, re-analysis with a two-component model did not alter the interpretation of the results. Crucially, these results corroborate the conclusion of Experiment 1 that object-updating mechanisms resulting from OSM cause perceptual degradation of target representations. Furthermore, by eliminating the role that a sustained mask may play in adding noise to a target’s representation (as in the canonical OSM paradigm used in Experiment 1), Experiment 2 isolated an object-mediated process as cause of the vitiating effect.
Standard forced-choice methodologies have limited our understanding of the effect of OSM on the quality of target representations. By using a continuous report measure and a mixture modelling approach, we demonstrated that canonical impairments in accuracy during OSM tasks are the result of both the target’s elimination from consciousness as well as its deterioration within consciousness. We then replicated this result using a procedure that ruled out a feedforward account that could have accounted for the deterioration independent of object oriented processing. The two experiments converge on the conclusion that object-updating mechanisms degrade target representations within conscious awareness.
Our method of analysis revealed the now contentious interaction between set size and mask duration. Recently, Argyropoulos, Gellatly, Pilling, and Carter (2013) showed that the interaction—normally attributed to attention—could be eliminated by applying a guessing correction. Our findings were thus surprising as the mixture model estimates the independent contribution of guess responses and in essence then corrects for guesses. However, converging evidence suggests that ceiling level performance drives the interaction, something our design did not adequately account for (Filmer, Mattingley, & Dux, 2014; Pilling, Gellatly, Argyropoulos, & Skarratt, 2014). Importantly, it was not the purpose of this paper to address the existence of this contentious aspect of OSM but to characterize the effect of OSM on the quality of target representation.
Our results are consistent with previous results that showed an offset mask can cut off or “trim” a target without it being eliminated entirely from conscious (Kahan & Enns, 2010). Our results extend this partial alteration of a conscious object by showing that object updating can also occur via a process of degradation. Notably, no theories of OSM (either object mediated or feedforward) in their current form describe how OSM might degrade the representation of a target. Thus, our findings call for elaboration of extant models of how object representations are modified when faced with discrepant perceptual input.
Given that precision decrements can result from inter-item competition (Ma, Husain, & Bays, 2014), our finding that a mask can deteriorate a conscious object’s representation may mean that, in the fight between target and mask for representation, OSM reflects mask prioritization due to its persistent perceptual input in the standard paradigm (Experiment 1), and its presentation during a hypothesized window for communication between early and late visual processing areas (Experiment 2). Either way, mask prioritization leads to reduced sampling of the target object representation and a consequent reduction in representational quality.
In conclusion, by applying a mixture model approach to two OSM paradigms, we have shown that the updating of object representations does not function in a discrete all-or-none fashion, but instead impacts the quality of object representations. This novel methodological approach and demonstration that object updating impacts the precision of object representations should provide new research avenues for the study of representation in conscious and unconscious vision.
We also collected pilot data where participants used a mouse to produce their gap orientation response and clicked the left mouse button to confirm their response. Overall data patterns and statistical results did not differ across the two approaches but response variance estimates across conditions were more reliable and the total experiment time was reduced by approximately 15 % by using the touchpad.
We thank an anonymous reviewer for raising the possibility that an OSM paradigm may be better described with a model that does not assume equal precision for target and non-target (distractor) responses. When including a fifth parameter for the precision of non-target responses (SDNon-target) in the model, the impact of set size and mask duration on the other four estimates (PTarget, PNon-target, PGuess, SDTarget) was the same as observed in the original four parameter model. Furthermore, no effects of set size and mask duration were observed on SDNon-target. Because PNon-target was near zero for many participants, making estimating non-target precision problematic (typically, a minimum of 50 trials is required for reliable precision estimates), and because application of the five parameter model, with separate parameters to estimate the precision of target and non-target responses, did not alter any of the effects in our results, we opted to focus on the four-parameter model.
This research was supported in part by the National Sciences and Engineering Research Council of Canada Award RGPIN-341580-07 to D.E.W.
All research was approved by the Queen’s University graduate research ethics board and all participants provided informed consent before completing either experiment.
Conflict of interest
The authors declare no conflict of interest with respect to the authorship or publication of this article.
G.W.H. came up with the study concept. All authors contributed to the study design. Testing and data collection were performed by G.W.H. G.W.H. and J.R. performed the data analysis and interpretation under the supervision of D.E.W. G.W.H. drafted the manuscript, and J.R. and D.E.W. provided critical revisions. All authors approved the final version of the manuscript for submission.