Active Listening Delays Attentional Disengagement and Saccadic Eye Movements
Successful goal-directed visual behavior depends on efficient disengagement of attention. Attention must be withdrawn from its current focus before being redeployed to a new object or internal process. Previous research has demonstrated that occupying cognitive processes with a secondary cellular phone conversation impairs attentional functioning and driving behavior. For example, attentional processing is significantly impacted by concurrent cell phone use, resulting in decreased explicit memory for on-road information. Here, we examined the impact of a critical component of cell-phone use—active listening—on the effectiveness of attentional disengagement. In the gap task—a saccadic manipulation of attentional disengagement—we measured saccade latencies while participants performed a secondary active listening task. Saccadic latencies significantly increased under an active listening load only when attention needed to be disengaged, indicating that active listening delays a disengagement operation. Simple dual-task interference did not account for the observed results. Rather, active cognitive engagement is required for measurable disengagement slowing to be observed. These results have implications for investigations of attention, gaze behavior, and distracted driving. Secondary tasks such as active listening or cell-phone conversations can have wide-ranging impacts on cognitive functioning, potentially impairing relatively elementary operations of attentional function, including disengagement.
KeywordsVisual attention Attentional disengagement Cognitive load Dual task Mental workload Distracted driving
Selective attention depends on the availability of cognitive control resources. For example, under a working memory load, behavioral interference from salient visual distractors increases (Lavie & de Fockert, 2005, 2006). Under a cognitive load, observers are less likely to detect the sudden appearance of a new object (Boot et al. 2005; Cosman & Vecera, 2009; Matsukura et al. 2011), and visual search rates increase (Han & Kim, 2004). Cognitive control processes thus appear critical for maintaining attentional priorities (e.g., Norman & Shallice, 1986).
Most of the previous studies on cognitive load and attentional control rely on non-naturalistic secondary tasks, such as digit maintenance or mental math (e.g., Han & Kim, 2004). More naturalistic secondary tasks, such as engaging in a conversation, can impact complex behaviors that heavily rely on attention, such as driving. For example, the cognitive load imposed by cell-phone negatively affects driving, including maintaining lane position (e.g., Strayer & Johnston, 2001) and explicit memory for on-road objects and events (Strayer et al. 2003; see Strayer & Drews, 2007). A cognitive load also reduces in-vehicle gaze behaviors, such as checking the rearview mirror and speedometer (Nunes & Recarte, 2002). Active listening appears to be the central component of cell-phone-induced performance declines in driving; Just et al. (2008) demonstrated that active listening—answering simple ‘true/false’ questions—impaired virtual driving ability.
Despite the evidence that more naturalistic secondary tasks impact complex behavior, the impact on specific attentional operations is less well understood. The deployment of visual attention depends on several component processes. Attention must be disengaged from its current locus before being moved and engaged at a new location (Posner et al. 1984; Posner & Petersen, 1990). Any of these components could be affected by conversation. Critically, the movement and (re) engagement processes are dependent on the initial disengagement or releasing of attention from its current focus; saccadic eye movements are not executed without first disengaging attention (Brockmole & Boot, 2009; Boot & Brockmole, 2010; Hoffman & Subramaniam, 1995; Fischer & Weber, 1993; Wright et al. 2015).
The purpose of the current research is to ask if naturalistic, active listening influences the control of visual attention, specifically the disengagement of visual attention. This issue is relevant for pinpointing the attentional operations affected by a cognitive load, but also for bridging basic attention research with applied issues, such as understanding the attentional operations responsible for the decline of driving performance during cell-phone use.
In two experiments, participants performed a laboratory measure of oculomotor disengagement during active listening or no active listening. Participants executed saccadic eye movements to peripheral targets when the central fixation either remained visible (“overlap” trials), or was removed prior to target onset (“gap” trials). Saccadic latencies are slower on overlap trials than on gap trials, a finding termed the “gap effect” (Saslow, 1967; also see Kingstone & Klein, 1993; Pratt et al. 1997; Reuter-Lorenz et al. 1991, 1995). One proposed mechanism for faster responses on gap trials is that fixation offset disengages attention prior to saccade execution (Fisher & Breitmeyer, 1987; Pratt et al. 2006; but see Kingstone & Klein, 1993; Reuter-Lorenz et al. 1991). Prior to an eye movement, active fixation must be suppressed—that is, disengaged—before saccade execution (e.g., Dorris & Munoz, 1995). Importantly, in our overlap trials, the fixation changed color with the same timing as fixation offset to equate the alerting associated with fixation offset on gap trials, allowing for a measure of disengagement uncontaminated by alerting (Reuter-Lorenz et al. 1995).
In Experiment 1a, participants performed the gap task while either actively listening to and answering true/false questions or in a no-question control condition. If attentional disengagement is slowed by a secondary active listening task, saccadic reaction times (SRTs) should be disproportionately slowed for overlap trials, because these trials require a release of attention prior to an eye movement (Hoffman & Subramaniam, 1995; Fischer & Weber, 1993). In contrast, if other attentional operations (e.g., the time to move attention to a peripheral target) are slowed under active listening, SRTs would be slowed for both gap and overlap trials under a concurrent active listening task.
In Experiment 1b, we ruled out peripheral interference from the mere presence of the auditory questions. Participants performed the gap task while listening to true/false questions, but participants were not required to answer the questions. If active listening slows attentional disengagement, then SRTs should not differ between the no-load and passive listening conditions in Experiment 1b. If any auditory distraction slows disengagement, then SRTs should be similar between Experiments 1a and 1b.
Twenty-six college-aged adults were recruited from the University of Iowa research participation pool. All had normal or corrected-to-normal vision. Twelve participants performed the gap task with active listening in Experiment 1a; 14 participants performed the gap task with passive listening in Experiment 1b.
Stimuli appeared on a CRT monitor (1024 × 768 resolution, 60 Hz refresh rate) positioned at the participant’s eye-level in a darkened room. Participants sat with their heads steadied by chin and forehead rests, approximately 60 cm from the monitor. Eye position was monitored on-line using an EyeLink 1000 system (SR Research) in a desktop-mounted configuration, operating at a 1000 Hz sampling rate.
Participants completed a brief 9-point calibration procedure. Recalibrations occurred as needed.
Participants performed two blocks of trials, counterbalanced across participants. During the no-question block, participants performed the gap task, making speeded saccades toward peripheral targets when the fixation point persisted (overlap trials), or when the fixation point disappeared (gap trials). During the question block, participants performed the gap task in conjunction with a secondary active listening manipulation described below. Participants performed a minimum of eight practice trials before beginning experimental trials. Each block contained 32 trials. Participants rested briefly between blocks.
Participants initiated each trial by pressing spacebar. In the no-question block, the words “eyes only” then appeared at fixation for 1 s, followed by fixation reappearing for 3000 ms. Next, two white square placeholders (0.25°) appeared simultaneously on the left and right of fixation for 2500 ms. These placeholders always appeared in opposite hemifields, and at the same eccentricity (4.7° or 9.2°). Simultaneously with placeholder onset, central fixation changed color, to either purple (RGB 255, 0, 255) or orange (255, 90, 20).
In overlap trials, central fixation changed color 200 ms prior to target onset. On trials where the central placeholder initially changed to orange at placeholder onset, it changed to purple, and vice versa. In gap trials, the central placeholder disappeared 200 ms prior to target onset. Then, a target object, a 1° white unfilled square, appeared for 1250 ms around one placeholder. Participants moved their eyes to the target as quickly and accurately as possible.
Target hemifield (left or right), saccade magnitude (4.7° or 9.2°), and gap condition varied unpredictably. Saccadic eye movements were detected in real-time as having an amplitude of 0.2° or greater, a velocity threshold of 30°/s or greater, and acceleration of 9000°/s or greater. Following target offset, a central placeholder appeared, marking the end of a trial. Participants pressed spacebar again to advance to the next trial.
The active listening block in Experiment 1a was identical to the standard gap described above. First, the words “please listen carefully” appeared at fixation. Following the reappearance of fixation, an experimenter began reading a true/false statement during the delay interval, subsequent target onset, and eye movement execution phases. True/false statements were created using a subset of items selected from a previously published investigation of listening and driving (Just et al. 2008), resulting in a total of 54 possible items read by the experimenter in a predetermined order.
After saccade execution, the text “respond true or false” appeared in the center of the screen. Participants responded verbally whether they believed the statement was true or false. No feedback was given regarding answer accuracy. All questions were approximately the same length, lasting 4–5 s when read by a trained experimenter, who was blind to the trial type (gap versus overlap). Importantly, all statements spanned the saccade planning and execution phases, requiring participants to actively process the spoken material throughout a trial.
When an anticipatory eye movement occurred, the experimenter discontinued the current question. These questions were not reused in subsequent trials. Participants were instructed to move their eyes as soon as the target was detected, and to not wait until the experimenter finished a statement. Finally, note that the number of questions exceeds the number of trials. We included additional questions to avoid resampling items on aborted trials.
The same questions were used in the passive listening task in Experiment 1b, except that participants were informed they did not need to answer the questions and only needed to move their eyes to the target as quickly as possible following its onset.
Results and Discussion
In Experiment 1a, participants answered the true/false statements correctly on 91.4% (SD = 6.23) of trials, which was significantly above chance performance, t(11) = 22.89, P < .001, indicating that participants were engaged in the active listening task.
Raw saccadic reaction times (SRTs) were first trimmed for each participant. All SRTs less than 100 ms, or greater than 2.5 SD from the participant’s mean, were discarded from subsequent analyses. This trimming removed less than 2% of all trials in Experiment 1a, and less than 1% of all trials in Experiment 1b. We conducted a repeated-measures ANOVA including within-subjects factors of gap condition (gap versus overlap trials) and listening (no-question versus active or passive listening load) and experiment as between-subjects factors. All analyses collapsed across target side and eccentricity.
Most important was the significant three-way interaction, F(1, 24) = 12.72, P = .002, ƞ2 = .35, which resulted because the gap effect was larger during active listening (98.24 ms) than in all other conditions (53.22 ms), all ts > 2.76, Ps < .02. One concern, however, is that these findings may be driven by a floor effect on the gap trials, in which eye movements are near their physical minimum and cannot become any faster. Nevertheless, SRTs were slower in the overlap condition during active listening (349.24 ms) than during passive listening (271.02 ms), t(24) = 3.42, P = .002, d = 1.51, demonstrating a delayed disengagement under active but not passive listening that does not depend on gap trial 3results of Experiment 1 indicate that active, but not passive, listening delays attentional disengagement as assayed by the gap task. Active listening slowed saccadic latencies on overlap trials, where attention had to be disengaged prior to saccade execution. We next explored the source of this delayed disengagement. One possibility is that accessing semantic memory to answer a question is effortful and delays attentional disengagement. Another possibility is that the working memory load associated with answering questions delays attentional disengagement. To test between these plausible alternatives, in Experiment 2 we manipulated question difficulty, using easy questions (from Experiment 1) and difficult questions. Difficult questions would require more effortful access of semantic memory, but not necessarily a greater working memory demand. If semantic memory access produced the disengage delay, then answering difficult questions should delay disengagement beyond answering easy questions. However, if the disengagement delay is due to working memory encoding and maintenance, then question difficulty should have little, if any effect.
Participants were 24 college-aged adults recruited from the University of Iowa research participation pool. All participants had normal or corrected-to-normal vision.
Materials and Procedures
All materials and procedures were identical to Experiment 1a, with the exception of the difficult true/false statements described below. Both no-question and active listening blocks presented 48 trials each; block order was counterbalanced across participants.
All eye-tracking task parameters and conditions were identical to Experiment 1. True/false statements were dichotomized as “easy” or “hard” based on previous research (Just et al. 2008). We used 53 easy statements, some of which were used in Experiment 1, and 54 hard statements. We blocked questions by difficulty, counterbalanced across participants. During active listening, the experimenter transitioned between content lists (easy versus hard or vice versa) after 12 trials where an eye movement was successfully executed. These transition points were unknown to participants. Participants were instructed that some items would be more difficult than others and that they should continue to listen to each statement and respond to the best of their knowledge. This design resulted in 24 trials per content difficulty condition.
Results and Discussion
Accuracy was 90.8% (SD = 7.47) for easy questions and 76.9% (SD = 12.34) for hard questions, both significantly above chance, t(23) = 27.12, P < .001 for easy questions, and t(23) = 10.68, P < .001, for hard questions. Accuracy was significantly greater for easy questions than for hard questions, t(23) = 13.88, P < .001, confirming the question difficulty manipulation.
Influence of Content Difficulty
Finally, there was no interaction between gap condition and question difficulty, F(1, 20) = .032, P = .861, ƞ2 = .002, suggesting that content difficulty of the statements did not differentially influence SRTs across gap and overlap trials.
No-Question versus Active Listening
To replicate the analyses from Experiment 1a, we collapsed across content difficulty to directly compare the influence of active listening on attentional disengagement.
We observed a significant main effect of trial type, F(1, 23) = 228.32, P < .0001, ƞ2 = .91, with faster SRTs in the gap condition (252.19 ms) than in the overlap condition (337.39 ms). We also observed a significant main effect of question condition, F(1, 23) = 15.49, P = .001, ƞ2 = .40, with faster SRTs in the no-question condition than in the active listening conditions. The interaction between trial type and question type was significant, F(1, 23) = 9.42, P = .005, ƞ2 = .29, indicating that active listening increases the gap effect compared to a no-question baseline.
Experiment 2 finds slowed attentional disengagement in the gap task under active listening but no effect of question difficulty. These findings suggest that the primary cost of active listening might be on encoding and maintenance of the statement into working memory, not the difficulty of information retrieval from semantic memory. However, working memory has many associated processing (e.g., encoding, maintenance, manipulation), and further work will be necessary to pinpoint the source of the disengage delay we have reported.
Our experiments indicated that active listening delayed overt attentional disengagement as measured by the gap task. Disproportionate slowing of SRTs occurred in the overlap condition under active listening—precisely the condition in which attention needed to be disengaged from central fixation prior to the ensuing saccade. Active listening is the necessary ingredient for delayed disengagement; passive listening did not significantly impact disengagement. Question difficulty did not have a measurable influence on SRTs.
The current results carry implications for the supervisory control of visual attention. Previous studies on cognitive load and attentional control typically use non-naturalistic secondary tasks such as digit maintenance or mental math (e.g., Han & Kim, 2004). In contrast, active listening is commonplace and draws on myriad processes, including working memory processes such as maintenance, updating, and manipulation. Our results demonstrate that even relatively low-level component operations of attentional orienting are impacted by daily activities—namely, active listening.
On the theoretical front, our findings extend the known influence of secondary tasks on eye movements and overt attention. A cognitive load impacts effortful behaviors, such as executing an antisaccade in the direction opposite a target (e.g., Roberts et al. 1994), as well as slowing other attentional phenomena, such as inhibition of return (Klein et al. 2006). However, our findings show that even more elementary behaviors such as attentional disengagement also are affected by a cognitive load.
On an applied front, our results suggest one possible source of the impairment cell-phone use produces on driving. A concurrent cellular-phone conversation impairs vehicle control, attention, and memory for on-road events (Horrey & Wickens, 2006). Studies that have included eye tracking are broadly consistent with our hypothesis that active listening, and, by extension, cell-phone conversation, delays attentional disengagement. If visual attention is slower to disengage during a cell-phone conversation, then the visual periphery should be sampled less often during a cell-phone conversation, producing an attentional tunnel vision. Atchley and Dressel (2004) reported a reduced field of view during a conversation, suggesting that attention might be ‘sticky’ and difficult to disengage while driving under a cognitive load. Slowed disengagement may partially explain increased crash risk during phone use, because drivers are delayed in attending to peripheral on-road events.
The influence of a secondary task on attentional disengagement suggests that attentional disengagement may show individual differences that covary with cognitive control measures. Individual differences in visual short-term memory predict attentional recovery from distraction (Fukuda & Vogel, 2011), which may be tied to the disengagement of attention from a visual distractor. Cognitive decline in older adults also delays attentional disengagement (Cosman et al. 2011, 2012). Such issues raise interesting avenues for further research.
We thank Cathleen Moore for use of the eye-tracking equipment and Nicole Jardine for technical support. This research was funded by a grant from the Toyota Collaborative Safety Research Center (CSRC), Detroit, Michigan, and in part by a grant from the National Science Foundation (BCS 11-51209).
- Fukuda, K., & Vogel, E. K. (2011). Individual differences in recovery time from attentional capture. Psychological Science, 22(3), 361–368Google Scholar
- Norman, D. A., & Shallice, T. (1986). Attention to action. In R. J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.), Consciousness and self-regulation (pp. 1–18). New York: Plenum Press.Google Scholar
- Saslow, M. G. (1967). Effects of components of displacement-step stimuli upon latency of saccadic eye movements. Journal of the Optical Society of America, 57, 1027–1029.Google Scholar