Low attention impairs optimal incorporation of prior knowledge in perceptual decisions
When visual attention is directed away from a stimulus, neural processing is weak and strength and precision of sensory data decreases. From a computational perspective, in such situations observers should give more weight to prior expectations in order to behave optimally during a discrimination task. Here we test a signal detection theoretic model that counter-intuitively predicts subjects will do just the opposite in a discrimination task with two stimuli, one attended and one unattended: when subjects are probed to discriminate the unattended stimulus, they rely less on prior information about the probed stimulus’ identity. The model is in part inspired by recent findings that attention reduces trial-by-trial variability of the neuronal population response and that they use a common criterion for attended and unattended trials. In five different visual discrimination experiments, when attention was directed away from the target stimulus, subjects did not adjust their response bias in reaction to a change in stimulus presentation frequency despite being fully informed and despite the presence of performance feedback and monetary and social incentives. This indicates that subjects did not rely more on the priors under conditions of inattention as would be predicted by a Bayes-optimal observer model. These results inform and constrain future models of Bayesian inference in the human brain.
KeywordsAttention: divided attention and inattention Signal detection theory Cognitive and attentional control Ideal observer Bayesian models
According to ideal observer Bayesian principles, when perceptual information or perceptual processing are weak, optimal observers should rely more on prior expectations in order to behave optimally (Feldman & Friston, 2010; Friston, 2005; Knill & Pouget, 2004; Kouider, de Gardelle, Sackur, & Dupoux, 2010; Summerfield & de Lange, 2014; Summerfield & Egner, 2009; Summerfield & Koechlin, 2008). For instance, in a discrimination task, Bayes-optimal subjects capitalize on prior knowledge about the stimulus presentation probability to maximize the proportion of correct responses (Maddox & Bohil, 1998; Wyart, Nobre, & Summerfield, 2012). Such a strategy should be manifested in terms of a shift in subjects’ response criteria, e.g., as strength or precision of sensory data decreases, such as for unattended stimuli (Carrasco, 2011), when stimulus presentation probability is biased the optimal discrimination criterion should become more liberal.
Here we test a signal detection theoretic model (Rahnev et al., 2011) that makes empirical predictions that may seem implausible from this commonsensical Bayesian perspective. According to our Common Criterion Model, in addition to increasing the strength of the perceptual signal (possibly by increasing activity in visual cortical areas that represent the attended location and suppressing cortical activity that represents unattended locations (Desimone & Duncan, 1995)), attention also reduces the trial-by-trial variability of the internal perceptual response (Bressler & Silver, 2010; Pestilli, Carrasco, Heeger, & Gardner, 2011; Wyart et al., 2012). Importantly, motivated by previous empirical evidence within a signal detection theoretic framework (Gorea & Sagi, 2000; Zak, Katkov, Gorea, & Sagi, 2012), the model also presupposes that subjects use a single common decision criterion for discriminating both attended and unattended stimuli. This model has gathered empirical support from studies using transcranial magnetic stimulation and neuroimaging (Rahnev, Bahdo, de Lange, & Lau, 2012a; Rahnev, Maniscalco, Luber, Lau, & Lisanby, 2012b; Rounis, Maniscalco, Rothwell, Passingham, & Lau, 2010). The model makes the counterintuitive prediction that as long as there is uncertainty, when faced with a biased prior expectation of stimulus likelihood, human subjects will not be able to adjust their response bias in unattended trials as effectively as they will for attended trials.
We pit this model against the predictions of an Optimal Observer Model in a series of experiments in which subjects discriminated between left- and right-tilted gratings that were presented in cued (attended) or uncued (unattended or less attended) locations, one of which was probed in each trial. To anticipate, we confirmed the a priori prediction of our detection theoretic model repeatedly in five different experiments (plus one control experiment).
Materials and methods
Participants signed an informed-consent statement approved by the local institutional review board. Subjects had normal or corrected-to-normal vision and were naïve to the purpose of the experiments. They received a base compensation of US$10 per session plus a US$5 bonus if they had a higher percentage correct than the previous participant (whose performance was disclosed only at the end of the experiment). The first subject of each experiment competed against a mock subject who was given an arbitrary percentage of 80 % correct responses. This reward structure provided subjects with both monetary and social incentives to perform well in the task. A total of 79 subjects took part in this study. The mean age of participants was 23 years (range = 18–41 years).
Stimuli were generated using Psychophysics Toolbox (Brainard, 1997) in MATLAB (MathWorks, Natick, MA, USA) and were shown on an iMac monitor (24-in monitor size, 1920 × 1200 pixels, 60 Hz refresh rate). Stimuli were presented inside circular locations (5° diameter) on a gray background (111 cd/m2), with circles’ centers 5° away from fixation. Stimuli consisted of a noisy background composed of uniformly distributed intensity values with gratings on top (two cycles/degree) that could be tilted 45° (right) or 135° (left). Total contrast of stimuli was fixed at 30 % but the grating-to-noise ratio varied, thus making the gratings more or less visible.
General task design
Here we present the general design of our experiments. For details and variations, see the Experiment and task design section of each experiment.
Seated in a dimmed room (60 cm from the monitor), subjects were instructed to fixate at a central red square (0.4°) for 500 ms at the beginning of each trial. Subjects were instructed to attend to one of two possible locations pointed by a white (200 cd/m2) pre-cue double-headed arrow presented for 500 ms, which randomly changed orientation every 40 trials. This blocking of attentional cueing reduced the cognitive demands on subjects and made it less likely that they could confuse the identity of cued stimulus. Stimuli were presented for 50 ms inside four circular locations (5° diameter) on a gray background (111 cd/m2) on the four quadrants of the screen, with the center of each circle 5° away from fixation. This configuration was chosen in order to minimize eye movements and enhance central fixation, as in previous studies (Bahrami, Lavie, & Rees, 2007; Rahnev et al., 2011).
The tilt of the gratings was always the same in the two locations of each diagonal, thus the gratings in one diagonal can be considered as a single compound stimulus. The tilts of the gratings on different diagonals were always independent. After stimulus presentation, a response cue consisting of two white circles at the same location of the stimuli of one of the two diagonals indicated the target stimulus for which subjects had to provide a perceptual judgment regarding the tilt (left or right) of the gratings. The response cue coincided with the pre-cue on 70 % of the trials (cued) and it selected the stimulus that was not pre-cued in the remaining 30 % of trials (uncued). After providing a perceptual judgment, subjects rated confidence on a 4-point scale (1 = not confident at all; 2 = barely confident; 3 = somewhat confident; 4 = highly confident). Perceptual judgments were provided with two fingers from the left hand on a normal keyboard, pressing one key for right tilt and a different key for left tilt. Confidence ratings were provided with four different keys with a different finger of the right hand for each key. Subjects received auditory feedback (a brief low-pitch beep) after an incorrect answer at the end of the trial and their overall performance (percentage correct) was shown to them every 40 trials. We note that the aim of collecting confidence ratings was to assess subjects’ metacognitive capacity, but since they are irrelevant for the scope of this report regarding integration of prior information in perceptual decisions, we will not analyze them.
A session took approximately 1 hour to complete and consisted of 240 practice and 800 experimental trials. Two conditions were tested: in the unbiased condition, left and right tilts on the target stimuli were equally likely; in the biased condition, 70 % of the target stimuli were tilted to one side and 30 % to the other (i.e., 70 % of cued and 70 % of uncued trials were biased to one side and the remaining 30 % of each to the other side). The biased tilt was constant throughout the biased blocks in each experiment. Subjects were explicitly informed of the attentional and bias conditions, were reminded of them before every block, and had the chance to become familiar with the structure of the experiment during practice trials. The experimental trials were split into four blocks (two interleaved blocks for the unbiased and biased conditions, respectively; presentation order was counterbalanced across subjects). Participants had 15-second forced pauses every 40 trials, after which they were informed of their accumulated percentage correct. Subjects had self-paced pauses between blocks. Before the experiment, subjects completed 240 practice trials that followed the same structure of the main experiment except that they were split into six blocks: three per each type of bias. In the first four practice blocks stimuli were presented at higher grating-to-noise contrast ratio than in the actual experiment and in the last two blocks they had the same contrast ratio as in the actual experiment. Practice results were not analyzed.
Rationale for theoretical predictions
Figure 1a shows P(x|S 1), P(x|S 2) and the location of the optimal criterion given by Eq. 2 (dashed vertical lines) assuming that S 1 is more probable than S 2 (P(S 1) : P(S 2) = 70 : 30). In each panel, the distributions have different σ but the same μ, therefore sensitivity (d ′ = μ/σ) is different. Note that optimal observers shift the decision criterion to account for biased prior probabilities. Also, the criterion is larger when sensitivity decreases. This makes sense, as an optimal observer should capitalize on prior knowledge about what stimulus is more probable, especially when sensory information is weak.
Figure 1b illustrates the relationship between optimal response bias and sensitivity. Whereas c opt increases when sensitivity d′ decreases, log β opt is constant. The location of the optimal decision criterion when stimulus presentation prior probabilities of S 1 and S 2 are not equal, which is the case in our experiments, depends on sensitivity d′ (as illustrated in Fig. 1a) and hence it varies across subjects and across attentional conditions. However, optimal log β is constant for all subjects regardless of sensitivity: it only depends on the stimuli frequencies. In our study, then, log β is a preferable measure for subjects’ response bias because we can easily compare it to an optimal log β that is independent of d′ and constant across subjects when we bias the stimulus presentation frequencies.
Subjects setting a criterion different to the optimal given by Eq. 2 will perform worse than they potentially could. One suboptimal strategy worth considering is the one that our Common Criterion Model assumes: subjects use the same common decision criterion (x c fixed ) for discriminating both attended and unattended stimuli, despite differences in sensitivity. An intuitive understanding of the model’s predictions can be gained considering the fixed decision criterion in Fig. 1a (solid vertical lines). Using this suboptimal strategy, c and log β both increase when d′ increases (Fig. 1b, solid line). More importantly, the worse the sensitivity is (weak visual information), the larger the divergence between a suboptimal and an ideal observer. In other words, there is a less efficient use of prior knowledge when visual evidence is weak. This suboptimal behavior is what our Common Criterion Model predicts will happen in unattended trials in our experiments (see Experiment 1 below for details on the model). Also note that with equal stimulus presentation probabilities (i.e., P(S 1) : P(S 2) = 50 : 50), Eq. 4 implies that optimal log β = 0, meaning that optimal observers place their criterion at the intersection of the S 1 and S 2 distributions (x c = 0), regardless of their sensitivity d′.
Signal detection theoretic measures
Experiment 1: Smaller response bias shift in uncued trials compared to cued trials during biased stimulus presentation
Experiment and task design
Optimal observer and common criterion model predictions
In our experiments, on each trial, stimuli are presented at a cued and at an uncued location. Consequently, in the models, there are two internal response axes, one for the cued location and one for the uncued location.
In contrast, the ‘Common Criterion Model’ assumes that observers use a common criterion for cued and uncued stimuli (Gorea & Sagi, 2000; Zak et al., 2012). As a consequence, it makes very different predictions (Fig. 3b). This assumption is informed by previous empirical psychophysical evidence (Gorea & Sagi, 2000). In this case, because d′ is not the same for attended and unattended distributions, a common criterion implies that response bias cannot be optimal for both locations when stimuli presentation frequencies are biased. The prediction is that in attended trials, the response bias will be closer to optimal and, hence, further away from optimal in the unattended trials. This is the opposite of what Bayes-optimal observers would do: when vision is impaired (as during inattention), they shift response bias by a larger amount. In Fig. 3c we present the model predictions for the unbiased condition, which are the same for both the Optimal Observer and the Common Criterion Models. When stimulus presentation is unbiased, both models predict optimal response bias.
Response bias measured by c, as expected by our model’s prediction (Fig. 3b, c), was not significantly different between attentional conditions in either bias condition (paired t tests between cued and uncued conditions, unbiased: T(17) = 0.93, P = 0.36, CI = [−0.27,0.10]; biased: T(17) = 0.08, P = 0.93, CI = [−0.28,0.30]; Fig. 4b).
In the unbiased condition, log β was not different from optimality for cued and uncued trials, i.e., optimal log β = 0 for a left-tilted and right-tilted stimulus presentation ratio of 50:50 (see Eq. 4 in Materials and methods section; t tests against value zero, cued: T(17) = 0.55; P = 0.59, CI = [−0.17,0.29]; uncued: T(17) =−0.03, P = 0.97, CI = [−0.14,0.14]). When the stimulus probability changed from unbiased (i.e., the ratio of left-tilted to right-tilted stimuli being 50:50) to biased (in all our experiments the same ratio now being 70:30), the shift in subjects’ response bias as measured by log β was farther from optimality in unattended trials than in attended trials (Fig. 4c). Critically, there was a significant interaction between attention (cued vs. uncued) and bias (biased vs. unbiased) (2 × 2 repeated measures ANOVA, F(1,17) = 7.26, P = 0.01)
The shift in response bias measured by log β due to biased stimulus presentation was not numerically optimal on average in attended trials (as it is not unexpected based on previous literature (Gorea & Sagi, 2000; Green & Swets, 1966; Macmillan & Creelman, 2005; Rahnev et al., 2011)), but it was significant and in the right direction. On the other hand, the shift in uncued trials lagged behind the shift in cued trials, as if subjects were unable to adjust their response bias during inattention.
Note that the shift in response bias as measured by c is similar in the biased cued and uncued conditions (Fig. 4b). This, however, does not imply that subjects were equally close to optimality, since optimal c (not shown) is different for cued and uncued because d′ was significantly different and it varied for every subject (see Eq. 3). These results, then, support a model in which the variance of the internal signal distributions is modulated by attention and subjects use a common criterion such that they are closer to optimality in cued trials than in uncued trials.
Experiment 2: Multiple sessions and higher contrast
Worried that the results in Experiment 1 were obtained only due to subjects’ lack of familiarity with the task or extreme difficulty during unattended trials, in Experiment 2 we tested each subject for at least two 1-hour-long sessions (three subjects had four sessions), giving them ample opportunity to learn to behave more optimally. We also made the task somewhat easier during the unattended trials by increasing the mean contrast of both stimuli.
Experiment and task design
Each session was run in the exact same fashion as in Experiment 1. Participants (13 subjects; seven women), however, had two sessions, one with low and one with high contrast-contributing gratings (except for three participants who had four sessions, two with low and two with high contrasts). Statistics are presented for the subjects’ average across sessions. Different sessions took place on different days within 1 week. Contrast levels were higher overall than in Experiment 1 to allow an increase in sensitivity for uncued trials. In Experiment 2 contribution of the grating contrast to total contrast was 9 % (low) and 12.3 % (high), instead of the lower levels in Experiment 1 of 5.7 % (low) and 9 % (high).
Response bias as measured by c was different between attentional conditions in both bias conditions (paired t tests between cued and uncued conditions, unbiased: T(12) = 4.50, P = 0.001, CI = [0.18,0.51]; biased: T(12) = 2.33, P = 0.04, CI = [0.01,0.43]; Fig. 5b). Importantly, in the unbiased condition, log β was not different from optimality (0) for cued and uncued trials (t tests against value zero, T(12) =−0.06, P = 0.96, CI = [−0.48,0.46] and T(12) = 1.79, P = 0.10, CI = [−0.05,0.47], respectively). In the biased condition, despite the extra training, the higher overall sensitivity, and response bias c being higher on average for the unattended condition, log β shifted towards optimality (dashed line) in cued trials much more than it did in uncued trials. Importantly, as in the previous experiment, there was a significant interaction between attention and bias (2 × 2 repeated measures ANOVA, F(1,12) = 8.71, P = 0.01).
Higher sensitivity and extended training did not change significantly the pattern of results we found in Experiment 1. The mean response bias as measured by c was higher in uncued trials (in both the unbiased and biased conditions). Note, however, that the shift was not enough for making log β approach optimality in the same proportion as in the cued trials. This indicates that, despite the larger average shift in c, they were further away from optimality in uncued trials. The Optimal Observer Model predicted that a higher response bias c during inattention should have led to an equally optimal log β for cued and uncued trials if subjects were being optimal. We did not see this in our data. Rather, as we explained above (Materials and methods; Figs. 1b and 3), response bias c, when distributions have different variances, is not particularly informative about how the two values of c compare to each other. Log β, on the other hand, is a much better measure for determining how close to optimality subjects’ response bias is because optimal log β is the same regardless of variance or sensitivity. The results of this experiment, then, add evidence to the results from Experiment 1 to support our Common Criterion Model insofar as our model predicts that subjects would be able to shift their log β more in cued than in uncued trials.
Experiment 3: Low sensitivity and general inattention alone are not enough to prevent response bias shifting
The fact that discrimination sensitivity was relatively low for unattended stimuli in the first two experiments could be a source of concern. Subjects might have been frustrated and resolved to respond more or less randomly during unattended trials, i.e., in a 50/50 fashion. We note that sensitivity (d′) for uncued trials was significantly positive in both Experiments 1 and 2, and in Experiment 2 it was within a range that is not atypical in psychophysics experiments (mean uncued d′ = 0.71). Nevertheless, we performed Experiment 3 to test the hypothesis that it was the combination of attended and unattended trials in the same task rather than low d′ or just a general lack of attention that prevented subjects from shifting their response bias adequately in reaction to changes in their prior expectation in the uncued trials. Recall, our model predicted that a common criterion for cued and uncued trials would prevent subjects’ response bias from being equally close to optimality in both types of trial. If it were just inattention or just low sensitivity that drives the inability to shift response bias that we saw in Experiments 1 and 2, subjects should be unable to shift their criterion towards optimality in Experiment 3 as well.
Experiment and task design
Response bias as measured by c was different between conditions (paired t test: T(9) = 6.9, P = 0.0001, CI = [0.38,0.76]; Fig. 7b). Like in the unbiased conditions from previous experiments, in this experiment subjects displayed an optimal response bias measured by log β during the unbiased condition (t test against value zero, T(9) = 0.47, P = 0.65, CI = [−0.11,0.07]; Fig. 7c). However, unlike the uncued conditions in Experiments 1 and 2, in this experiment subjects were able to shift their response bias significantly towards optimality (dashed line) in the biased condition (Fig. 7c). Importantly, response bias in the biased condition was significantly different from the unbiased condition (paired t test, T(9) = 3.65, P = 0.005, CI = [0.10,0.42]).
The shift in log β was achieved even when discrimination sensitivity was at a low level, comparable to that in the uncued condition in Experiments 1 and 2. This finding is critical because a significant shift in log β despite such a low d′ implies that subjects were free to incorporate the priors about stimulus identity into their response bias. This suggests that, consistent with our Common Criterion Model (Fig. 3b), it is a common criterion for an attended and an unattended sets of distributions, rather than low sensitivity overall, that led to suboptimal behavior.
Experiment 4: Subjects fail to shift their response bias during inattention even when performance is matched
Another concern about the results in the first two experiments is that the different response bias shift was driven by the different performance in cued and uncued trials. Furthermore, despite the results of Experiment 3, a remaining concern could be that subjects behave erratically during uncued trials when these are mixed with cued trials, like in the tasks in Experiments 1 and 2. To rule out these possibilities we conducted Experiments 4A and 4B, in which we tried to match sensitivity between the cued and uncued trials by presenting a higher grating contrast in the uncued location.
Experiment and task design
Experiments 4A and 4B were similar in structure and design to Experiment 1. Eleven participants in Experiment 4A (three women) and sixteen in Experiment 4B (seven women) had one session only. Like in previous experiments, the biased tilt was constant throughout the biased blocks and it was counterbalanced across subjects. The only difference was that contrast for cued and uncued trials was different. In Experiments 4A and 4B we aimed to match sensitivity (d′) between cued and uncued trials using the QUEST threshold determination procedure (Watson & Pelli, 1983) to find for each subject the grating contribution to total contrast that would produce about 75 % correct responses for both the cued and uncued stimuli. In Experiment 4A we used 120 practice trials for QUEST; 70 % of these were cued and were used to determine the grating contrast contribution to total contrast for cued stimuli, while the remaining 30 % of trials (uncued) were used to determine the grating contrast contribution of uncued stimuli. Halfway through the experiment, i.e., after one block of each condition, we updated this ratio for the cued or uncued stimuli (or both) to force d′ for both type of trials to be as similar as possible. We lowered or raised the grating-to-noise contrast ratio using the psychometric function obtained during QUEST in order to decrease d′ if it was higher than 1 or raise it if it was lower than 1. Mean grating contrast contribution during the first half were: cued = 15.1 % ±9.2; uncued = 23.7 % ± 11.3, and during the second half: cued = 11.9 % ±8.1; uncued = 23.3 % ± 13.1.
In Experiment 4B, we used 80 practice trials for QUEST, 70 % of which were cued and were used to determine the grating contrast contribution for cued stimuli. The uncued grating contrast contribution was equal to e (log(x)+k), where x is the grating contrast contribution obtained with QUEST for cued stimuli and the constant k = 0.7 was determined by obtaining the mean difference between the natural logarithm of the grating contrast contributions to total contrast for cued and uncued stimuli of participants in Experiment 4A. We did not readjust the ratios halfway through the experiment. Mean grating contrast contribution for cued stimuli were: cued = 35.9 % ± 28.3; uncued = 46.9 % ± 31.
In Experiments 4A and 4B we observed a similar pattern of results. In the biased condition, log β shifted towards optimality only in the cued trials (paired t tests, Experiment 4A: T(10) = 2.63, P = 0.02, CI = [0.07,0.91]; Experiment 4B: T(15) = 2.85, P = 0.01, CI = [0.08,0.54]). Overall, sensitivity was not significantly different between the cued and uncued conditions in either Experiment 4A (paired t test, unbiased: P = 0.22; biased: P = 0.11) or 4B (paired t test, unbiased: P = 0.39; biased: P = 0.41). However, d′ matching was not always perfect for every subject, most likely because subjects had more opportunity to learn how to improve their sensitivity in the attended trials, which were presented more frequently.
Even when we imposed these conservative constraints to the data analysis, ensuring that we only considered subjects whose d′ was reasonably matched (unbiased condition mean difference between cued and uncued sensitivity = 0.16, biased condition mean difference = 0.15), we still found the same response patterns. Response bias c was not different between attentional conditions in either bias condition (paired t tests between cued and uncued conditions, unbiased: T(13) = 0.19, P = 0.85, CI = [−0.19,0.23]; biased: T(13) = 1.91, P = 0.08, CI = [−0.01,0.22]; Fig. 9b). In the unbiased condition, cued and uncued trials were near optimal (t test against value zero, T(13) = 0.56, P = 0.59, CI = [−0.19,0.32]; T(13) = 1.21, P = 0.25, CI = [−0.12,0.43], respectively), and subjects still struggled to shift their response bias significantly towards optimality only in uncued trials (Fig. 9c). Importantly, as in all the previous experiments, we found an interaction between attention and bias: 2×2 repeated measures ANOVA, F(1,13) = 4.5, P = 0.05.
In these experiments we found that subjects did not fail to shift their log β due to low sensitivity or due to being frustrated during uncued trials, since their sensitivity was equally good as in cued trials and the constant feedback makes it implausible to think they were not aware of their good performance. On the other hand, our model’s assumptions (i.e., a common criterion for cued and uncued trials and the reduction of variance by attention), account for the observed results as predicted a priori (Fig. 8).
Experiment 5: The effect is preserved with a simpler display
While the diagonal arrangement of the stimuli has been successfully used before with the purpose of minimizing eye movements (Bahrami et al., 2007; Rahnev et al., 2011), having multiple items in the display could have made the task unnecessarily difficult. In order to dispel this worry, in Experiment 5 we used a simpler display.
Experiment and task design
Response bias c was not significantly different between attentional conditions in either bias condition (paired t tests between cued and uncued conditions, unbiased: T(10) = 0.73, P = 0.48, CI = [−0.31,0.15]; biased: T(10) = 1.6, P = 0.14, CI = [−0.69,0.11]; Fig. 11b). In the unbiased condition, log β was optimal for cued and uncued trials as predicted by our model (t tests against value zero, T(10) = −0.29, P = 0.77, CI = [−0.60,0.46] and T(10) = −0.67, P = 0.52, CI = [−0.40,0.21], respectively). In the biased condition, log β was optimal only in cued but not in the uncued trials and, importantly, we were able to find an interaction between attention and bias: 2×2 repeated measures ANOVA, F(1,10) = 10.59, P = 0.02; Fig. 11c. See Figure S5 in the Supplemental Information for hit and false alarm rates.
Further data analysis
A potential worry about all the experimental results is that stimulus adaptation could be a confound. In the biased condition, one stimulus orientation appears more frequently than the other. Adaptation reduces stimulus salience, but attention can overcome adaptation restoring contrast sensitivity (Pestilli, Viera, & Carrasco, 2007). Uncued trials, then, would have been more affected by adaptation and possibly prevented subjects from incorporating priors into their responses in a way not related to our model. A similar worry is that subjects could have tried to balance their responses in uncued trials, thereby avoiding the most frequent response in the most frequent and salient trials (i.e., cued trials) and subsequently reducing the (more optimal) shift in their response bias (Akaishi, Umeda, Nagase, & Sakai, 2014).
We first note that the structure of our experiments makes both possibilities unlikely to account for our results. Briefly presented stimuli, like the ones we used, usually do not generate substantial adaptation (Patterson, Wissig, & Kohn, 2013). Further, the orientation of the gratings changed randomly every trial in cued and uncued trials alike, minimizing the probability of stimulus adaptation. Also, cued and uncued trials were interleaved, making it unlikely that subjects would avoid the more frequent response (in the cued trials) trying to somehow balance their responses in the uncued trials. The more frequent tilt was proportionally split between cued and uncued targets such that 70 % of cued and 70 % of uncued targets were tilted to the same side, making “the most frequent response” proportionally the same in both conditions. Furthermore, subjects received trial-by-trial feedback and the same results were obtained when the uncued stimuli were discriminated just as well as the cued ones in Experiments 4A and 4B, completely ruling out the possibility of explaining our results by subjects responding randomly during uncued trials.
If there was no adaptation or if there was no response compensation, the difference of the two relevant conditional probabilities should equal zero. A positive Adaptation Index value would mean there is positive priming and a negative value would mean there is repulsion. As expected for this kind of briefly presented stimuli, the Adaptation Index was not significantly different from zero in the biased condition of any of the experiments (P > 0.05 in all Experiments) and, importantly, it did not differ between the cued and uncued conditions (P > 0.05 in all Experiments) (Fig. 12, top). Similarly, a positive Compensation Index would mean subjects were biased to repeat the response equal to the most frequent stimulus, while a negative value would mean there was some tendency to avoid the response equal to the most frequent stimulus. Again, as expected, subjects were not biased in any direction in all the uncued conditions, ruling out alternative explanations of our results based on subjects just guessing in uncued trials (Fig. 12, bottom). For the cued condition in Experiment 1 subjects deviated from zero bias (P = 0.1), sticking to the most frequent response in a mild way (Compensation Index mean was less than 0.1 and, in any case, positive). Further, the difference with the uncued condition was not significant (P = 0.09). Summarizing, in all the experiments the cued and uncued conditions were not significantly different from each other by either the Adaptation or the Compensation Index measures, eliminating the worry that the lack of shift in response bias during uncued trials can be explained by stimulus adaptation or response compensation.
Taken together, the results from these six experiments confirm the counter-intuitive predictions of our Common Criterion Model. These predictions (Figs. 3 and 8) are counter-intuitive because at first glance they appear to defy the commonsensical Bayesian principle that during uncertainty, prior stimulus probability should have a bigger impact on optimal observers’ perceptual decisions when the sensory signal is weak. We by no means claim that all specific Bayesian models are falsified by the findings we present here—rather, we hope our results can inform the arbitration among these models.
Within the scope of perception and action during uncertainty, many studies have shown that human subjects can perform psychophysical tasks in a near-optimal fashion (Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006; Ernst & Banks, 2002; Knill & Pouget, 2004; Körding & Wolpert, 2006; Trommershäuser, Maloney, & Landy, 2008), particularly in the context of combining prior expectation with sensory information (Körding & Wolpert, 2004). Importantly, however, it also has been shown that when subjects had to learn the category structure and the base-rates of a task through continuous sampling, they started the task behaving suboptimally (Maddox & Bohil, 1998). Yet, they became optimal after a few hundred trials (possibly due to the lack of attentional manipulations and to stimulus presentation above threshold), which is something our subjects were unable to do, in some cases even after thousands of trials. This points to the possibility that asking subjects to keep track during a single task of both attended and unattended targets may pose a unique constraint to the perceptual decision system.
Our predictions rest on a simple signal detection theoretic model with two assumptions: that attention reduces the variability of internal perceptual response, and that subjects use a common decision criterion to classify both the attended and unattended stimuli. The assumption of variance reduction is supported by recent research (Bressler & Silver, 2010; Pestilli et al., 2011; Wyart et al., 2012). From a computational point of view, attention has recently been interpreted as facilitating inferring the level of uncertainty or precision during hierarchical perception, thus minimizing the error prediction signal (Feldman & Friston, 2010). At a neuronal circuitry level, it has recently been found that attention reduces the pairwise noise correlation between neurons (Cohen & Maunsell, 2009; Mitchell, Sundberg, & Reynolds, 2009). If we assume that the population response is somewhat monotonically related to the average spike rate of individual neurons, reduction of pairwise noise correlation would mean a less variable population response, because reduced pairwise noise correlation could mean noise is averaged out more effectively (Zohary, Shadlen, & Newsome, 1994).
The other assumption of a common criterion is also supported by previous empirical work within signal detection theoretic (Gorea & Sagi, 2000) and neurophysiological frameworks (Fetsch, Kiani, Newsome, & Shadlen, 2014). Nevertheless, in a recent study, Qamar et al. (2013) found that subjects are able to incorporate trial-to-trial knowledge of different levels of sensory uncertainty when setting their decision boundaries, with some subjects even performing almost optimally. However, it is important to note that in our experiments subjects experienced two stimuli with different degrees of noise (attention) in the same trial, while in Qamar et al.’s experiments subjects saw a single stimulus in every trial and uncertainty changed from trial-to-trial (due to contrast variations). The simultaneous presentation of an attended and an unattended stimulus is a crucial part of our task design, as our model’s assumption concerns a common criterion of two simultaneous sets of distributions of which subjects have to keep track.
In terms of neural mechanisms, one way to conceptualize how decision criteria are implemented is to consider the feed-forward connectivity from sensory to decisional regions in the frontal and parietal cortices. For instance, in a discrimination task, a biased criterion towards stimulus S 1 means that for the same level of sensory activation, the decision area is more likely to produce an S 1 response than an S 2 response, which in turn would mean the feed-forward connectivity is strong. This could be implemented by means of more excitable neurons in the decision areas when the discrimination criterion is biased. Alternatively, sensory information processing could be biased by altering the starting point of the evidence accumulation process (Mulder, Wagenmakers, Ratcliff, Boekel, & Forstmann, 2012) or its drift rate (van Ravenzwaaij, Mulder, Tuerlinckx, & Wagenmakers, 2012). If discrimination criterion is to be understood in one of these ways, in a context where the stimuli are mixed along a single experiment such that the decision circuitry has to read out potentially from either the cued or the uncued sensory representations, the same criterion (or two very similar criteria) will be applied to both. This is because the decision circuitry may not have the resolution to distinguish between the cued and uncued representations by significantly changing the respective feed-forward connectivity by the time subjects are asked to respond. When decision neurons are highly excitable, they may be excitable at very similar rates for both cued and uncued stimuli.
There are several limitations to our model, as it is intended to be simple. The notion of a single, common decision criterion can be further specified in order to avoid potential oversimplifications. It is perhaps more biologically realistic to say that the two decision criteria for the cued and uncued stimuli are very similar, rather than being constrained to be identical (Zak et al., 2012). The identity assumption should be considered a computational convenience of the model and also a tool of higher theoretical value since it allowed us to make empirical predictions that would have been impossible assuming two independent criteria. While it is possible that the brain applies different criteria for cued and uncued trials, it is biologically unrealistic that the firing thresholds implementing the criteria are extremely different during the same trial.
A related question arises regarding the factors that determine the common criterion’s value. In our experiments, the common criterion was placed on or short of the optimal response bias value for attended trials. This agrees with previous evidence that response bias tends to be conservative (Gorea & Sagi, 2000; Green & Swets, 1966; Macmillan & Creelman, 2005; Rahnev et al., 2011). However, it is still open why the criterion was closer to the attended optimal response bias value and not, for example, midway between the optimal attended and unattended optimal response biases. One possibility is that since attended stimuli were probed more often than unattended ones they were given preference when setting the common criterion. This pre-eminence of attended stimuli is ecologically valid too, since humans and other animals tend to solve problems and control their behavior guided by attention, and only infrequently they have to react to unattended stimuli in the environment. We emphasize that this is a speculative account that requires further evidential support.
Another concern is that our model’s assumption that the internal response distributions are Gaussian might not hold. Recent reports show that in attentional tasks with multiple items the precision of the internal representation itself could be a random variable, perhaps due to fluctuations in attention (Mazyar, van den Berg, & Ma, 2012; Mazyar, van den Berg, Seilheimer, & Ma, 2013; van den Berg, Shin, Chou, George, & Ma, 2012). Nevertheless, as can be inferred from Fig. 1, as long as the distributions remain roughly similar for cued and uncued stimuli, the same qualitative predictions would continue to hold. Our model’s predictions depend critically on a reduction in variance by attention paired with a common decision criterion but not on the specific shape of the distributions. Further, Experiment 5 suggests that our main finding cannot be solely attributable to alterations in the shape of the distributions of the internal representations because reducing the number of displayed stimuli (which should have lead to changes in the internal representations) did not affect the response bias patterns seen throughout our experiments.
Here we put to a test a signal detection theoretic model that parsimoniously predicted a priori that subjects would not optimally incorporate the priors into their perceptual judgments in unattended trials during a detection task where attended and unattended trials were interleaved. These results suggest that the commonsensical notion of optimal criterion setting may not apply to situations as common as inattention. Future modeling efforts should shed light on the cognitive constraints faced by the observer in dealing with situations where stimuli have different statistical properties or where there is different variability of internal responses, which may account for this interesting pattern of behavior that may seem “suboptimal” from the outset.
This work is partially supported by a grant from the Templeton Foundation (6–40689). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We thank Ai Koizumi, Megan Peters, and Alex White for comments on an earlier version of this manuscript, and David Heeger for suggesting that we reward the subjects financially to motivate them to behave optimally.
Conflict of interests
The authors declare no competing financial interests.
- Brewer, C. A. (Ed.). (2013). Color Brewer 2.0. Retrieved January 30, 2015, from http://colorbrewer2.org
- Feldman, H., & Friston, K. J. (2010). Attention, Uncertainty, and Free-Energy. Frontiers in Human Neuroscience, 4. doi: 10.3389/fnhum.2010.00215
- Green, D., & Swets, J. A. (1966). Signal detection theory and psychophysics (pp. 1–455). New York: Wiley.Google Scholar
- Macmillan, N. A., & Creelman, C. D. (2005). Detection theory (Secondth ed.). Mahwah: Lawrence Erlbaum.Google Scholar
- Mazyar, H., van den Berg, R., Seilheimer, R. L., & Ma, W. J. (2013). Independence is elusive: set size effects on encoding precision in visual search. Journal of Vision, 13(5). doi: 10.1167/13.5.8
- Qamar, A. T., Cotton, R. J., George, R. G., Beck, J. M., Prezhdo, E., Laudano, A., Tolias, A. S., & Ma, W. J. (2013). Trial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization. Proceedings of the National Academy of Sciences of the United States of America, 110(50), 20332–20337.PubMedCentralPubMedCrossRefGoogle Scholar
- Summerfield, C., & de Lange, F. P. (2014). Expectation in perceptual decision making: neural and computational mechanisms. Nature Reviews Neuroscience, 1–12. doi: 10.1038/nrn3838
- van den Berg, R., Shin, H., Chou, W.-C., George, R., & Ma, W. J. (2012). Variability in encoding precision accounts for visual short-term memory limitations. Proceedings of the National Academy of Sciences of the United States of America, 109(22), 8780–8785. doi: 10.1073/pnas.1117465109 PubMedCentralPubMedCrossRefGoogle Scholar
- van Ravenzwaaij, D., Mulder, M. J., Tuerlinckx, F., & Wagenmakers, E.-J. (2012). Do the Dynamics of Prior Information Depend on Task Context? An Analysis of Optimal Performance and an Empirical Test. Frontiers in Psychology, 3. doi: 10.3389/fpsyg.2012.00132
- Wyart, V., Nobre, A. C., & Summerfield, C. (2012). Dissociable prior influences of signal probability and relevance on visual contrast sensitivity. Proceedings of the National Academy of Sciences of the United States of America, 109(9), 3593–3598. doi: 10.1073/pnas.1120118109 PubMedCentralPubMedCrossRefGoogle Scholar