Abstract
In four experiments pupillary responses were used to examine attention allocation and encoding dynamics in free recall. In Experiment 1, pupillary responses increased (and then decreased) across serial position suggesting that attention was increasingly allocated to items during learning until working memory was overloaded. In Experiment 2, manipulating presentation duration resulted in larger and more sustained pupillary responses with increased presentation duration, suggesting that participants were likely engaging in more elaborative and attention-demanding processes. In Experiment 3a, manipulating list-length resulted in decreased pupillary responses across serial position suggesting that participants were prioritizing early list items and less attention was allocated to later items. In Experiment 3b, when list-length was known, pupillary responses in the long-list length condition tended to decrease across serial position whereas pupillary responses in the short list-length condition tended to increase and decrease across serial positon. These results suggest that participants flexibly allocate attention to items during encoding depending on the nature of the task and the types of processes that are engaged in. These results further suggest the potential of utilizing pupillary responses to track attention allocation during learning.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The ability to learn and remember important information is critical for a number of tasks and situations we encounter on an everyday basis. These range from relatively mundane tasks, such as encoding and remembering where you parked, to learning and remembering important information such as esoteric driving laws in Oregon needed for an upcoming driver’s test. In both cases it is critical that you pay attention to the information at learning to ensure it is properly encoded. In typical experiments it is not always possible to track how attention and effort are allocated across items during encoding. In the current study, we utilized pupillometry as a means of tracking variation in the allocation of attention to items during encoding in a delayed free-recall task.
Attention and encoding
Learning new information is thought to be an attention-demanding process. Information that we pay attention to tends to be remembered better than information that is unattended, or receives less attention. Thus, attention is needed not only to select relevant information for on-going processing, but also to allocate sufficient processing resources to relevant information (Chun & Turk-Browne, 2007). In agreement with these general ideas, much prior research has demonstrated that dividing attention during encoding results in poorer subsequent memory performance than when attention is fully allocated to encoding (e.g., Baddeley et al., 1984; Craik et al., 1996; Murdock, 1965). That is, when attention is fully allocated to encoding information, items are strongly encoded and subsequent recall tends to be high. When attention is divided between encoding and some secondary task, items are weakly encoded and recall is much lower. Thus, situations that allow for full attention at encoding should result in better remembering than situations where attention is divided at encoding.
Not only does attention influence remembering between conditions (full vs. divided), but attention also varies between items even in full attention conditions. That is, some items will receive more attention than others owing to differences in item characteristics (such as emotionally arousing items; McGaugh, 2006), but also due to factors such as when in the list items are presented. The notion that attention is important for encoding has also been used to explain primacy effects in recall whereby the first items presented are typically better recalled than subsequent items (e.g., Murdock, 1962; Tulving, 2007). Specifically, several models of immediate serial recall suggest that primacy effects arise, in part, because the amount of attention that is allocated to items decreases with each successive item (e.g., Brown et al., 2000; Farrell & Lewandowsky, 2002; Page & Norris, 1998). The first item receives the most attention and successive items receive less and less attention. Thus, in these accounts a primacy gradient of attention allocation during encoding accounts for primacy effects in immediate serial recall. A somewhat related idea has been proposed by Tulving (2007; see also Tulving & Rosenbaum, 2006) called the camatosis hypothesis. According to the camatosis hypothesis, primacy effects arise because encoding of the first item requires substantial effort, which leads to a fatiguing of neural assemblies, and thus less processing of successive items. That is, resources (potentially attentional) are depleted during the course of list presentation. Recent electrophysiological studies corroborate these ideas, suggesting that initial items get a bigger boost of attention than later items, resulting in primacy effects (Azizian & Polich, 2007; Healey & Kahana, 2020; Sederberg et al., 2006; Serruya et al., 2014). Collectively, prior work suggests that primacy effects arise, in part, from an inability to sustain attention to encoding activities during the presentation of a list (Healey & Kahana, 2016). Thus, in some cases, primacy effects can be seen as similar to a vigilance decrement that occurs during the presentation of the list. Of course this does not mean that all primacy effects arise due to differences in attention allocation to items, but rather that differential attention allocation can influence primacy effects in some situations.
There are additional ways in which differential attention to items could arise. For example, it is possible that participants are rehearsing the items during encoding (maintenance rehearsal) and rehearsal is an effortful process (e.g., Kahneman & Beatty, 1966). Specifically, prior research examining pupillary responses (see below) during an immediate memory-span task found that effort allocation tends to increase as the number of items increases during encoding and are presumably held in working memory (the loading function), and effort allocation then decreases as items are recalled in serial order from working memory (the unloading function; Gardner, Beltramo, & Krinksy, 1975; Kahneman & Beatty, 1966; Kahneman & Wright, 1971; Peavler, 1974). This suggests that attentional effort is allocated as more items are presented and rehearsed for recall. Thus, rather than attention decreasing across lists, this account suggests that the allocation of attention should increase across serial position.
The two prior accounts are primarily concerned with how the allocation of attention changes across items, but it is also important to examine how attention is allocated to each item during encoding. For example, prior research has suggested that maintenance rehearsal is composed of two processes: an early attention-demanding process and a later more automatic process (e.g., Naveh-Benjamin & Jonides, 1984; Phaf & Wolters, 1993). Naveh-Benjamin and Jonides (1984) suggested that the first component of rehearsal involves the retrieval and initiation of the proper rehearsal program, which is an attention-demanding effortful process. The second component is a more automatic continuous execution of the appropriate motor program. Similarly, Phaf and Wolters (1993) suggested that elaborative rehearsal requires a continuous allocation of attention to items, whereas maintenance rehearsal involves an early focusing of attention followed by habituation. Thus, there are likely differences in how much attention is allocated to individual items during encoding and whether the allocation of attention is continuous or transient depending on whether one is engaging in maintenance or elaborative rehearsal.
Collectively, prior research suggests a number of possible ways in which attention is allocated to items during encoding. Some accounts suggest that the allocation of attention should decrease across items, whereas other accounts suggest that the allocation of attention should increase across items. Furthermore, prior research has suggested that the allocation of attention is associated with the type of processing that occurs during encoding, with some processes (elaborative rehearsal) requiring more attention than other processes (maintenance rehearsal).
Pupil dilation as an index of attention allocation
Although prior research is consistent with the notion that attention allocation is important for encoding and that attention might be allocated differentially both within and across items during encoding, more evidence is needed. A potential indicator of the attention allocation to items during encoding are pupillary responses. A great deal of prior research suggests that the pupil dilates in response to the cognitive demands of a task (see Beatty & Lucero-Wagoner, 2000; Goldinger & Papesh, 2012; Laeng et al., 2012, for reviews). These effects reflect task-evoked pupillary responses where the pupil dilates relative to baseline levels due to increases in attention allocation across a number of tasks (Beatty & Lucero-Wagoner, 2000). Reviewing the literature up to that point, Kahneman (1973) suggested that pupil dilation is a reliable and valid psychophysiological marker of attentional allocation. That is, these task-evoked pupillary responses correspond to the intensive aspect of attention and provide an online indication of the intensity of attention (Just & Carpenter, 1993; Kahneman, 1973).
There is a long history of using pupillary responses to examine encoding processes (e.g., Beatty & Lucero-Wagoner, 2000; Engle, 1975; Goldinger & Papesh, 2012; Heaver & Hutton, 2011; Janisse, 1977; Kafkas & Montaldi, 2011; Naber et al., 2013; Papesh & Goldinger, 2015; Võ et al., 2008). For example, in a recognition memory task, Otero et al. (2011) found larger pupil dilations when participants recognized items that were encoded under deep-encoding conditions compared to shallow-encoding conditions, suggesting that deep processing was associated with greater attentional allocation than shallow processing (e.g., Craik & Byrd, 1982). Additionally, in another recognition memory task, Papesh et al. (2012) found that greater pupil diameter at encoding not only predicted subsequent recognition, but also confidence ratings during retrieval, with the most confident items being associated with the largest pupils at encoding. Similarly, in an immediate free-recall task, Ariel and Castel (2014) found that high-value items were associated with the largest pupil dilations during encoding, and these items tended to be the best recalled. Collectively, prior research suggests pupil dilation can be used to track how attention is allocated to items during encoding.
Research has also examined how attention allocation changes across items as more items are added. As noted previously, Kahneman and Beatty (1966) found increased pupillary dilation as working memory load increased in a memory-span task (see also Heitz et al., 2008; Peavler, 1974; Unsworth & Robison, 2015). With each item that was presented, the pupil tended to dilate, resulting in a loading function. Subsequent research has replicated and extended these findings by demonstrating that pupil dilation was larger for ungrouped than grouped items (Kahneman, Onuska, & Wolman, 1968), larger when participants are required to recall more items (Kahneman & Wright, 1971), and larger when participants had to transform the items and when the items were associated with reward (Kahneman, Peavler, & Onuska, 1968). Furthermore, if participants are informed that they no longer need to retain the items, the pupil constricts, suggesting that the items have been dropped from working memory (Johnson, 1971; Unsworth & Robison, 2018). Similarly, the pupil tends to constrict when more items are presented than can be rehearsed in working memory (overloading; Granholm et al., 1996; Granholm et al., 1997). Thus, rather than being a simple linear function between pupil dilation and number of items presented, the function seems to be more quadratic in nature demonstrating a loading function, an asymptote, and then overloading. Overall, these results are consistent with the notion that items are being held and rehearsed in working memory during encoding and that cumulative maintenance rehearsal is an active effortful process (Kahneman, 1973).
While a number of studies have examined pupillary responses in memory-span tasks, considerably less work has examined pupillary responses in free recall and how pupillary responses change across serial position. To our knowledge, only three prior studies have examined pupillary responses in free recall and how they change across serial position. For example, during a familiarization task in which participants were presented with pairs of items for immediate free recall, Kahneman and Peavler (1969) found that items presented early in a list were associated with larger pupils than items presented later in a list consistent with the idea that attention allocation was decreasing across the list (although they did not actually test whether the decline was significant). More recently, Miller, Gross, and Unsworth (2019) examined pupillary responses in a delayed free-recall task. In their first experiment they found increased pupil dilation from serial positions 1-6, but the pupil then constricted from serial positions 7-10 (a quadratic effect) consistent with loading and overloading of resources seen in the memory-span tasks (Granholm et al., 1996; Kahneman & Beatty, 1966). However, examining individual differences in working-memory capacity qualified these results by suggesting that high working-memory capacity individuals tended to demonstrate the quadratic pupillary function, whereas low working-memory capacity individuals demonstrated a decrease in pupil dilation across serial position. Thus, whereas the high working-memory capacity individuals seemed to rely on cumulative effortful rehearsal in which the allocation of attention increased (and then decreased) across items, the low working-memory capacity individuals allocated more attention to early items and subsequently less attention to later items. Examining pupillary responses for each item suggested that the pupil tended to dilate while the item was on screen. Similarly, Kucewicz et al. (2018) demonstrated that the pupil tended to dilate with the presentation of each word (dilation was larger for recalled vs. forgotten words), and there seemed to be some evidence that the pupil increased and decreased across serial position (although this was not explicitly tested).
In Miller et al.’s (2019) second experiment, items were associated with points (1–10) in value directed remembering paradigm (e.g., Ariel & Castel, 2014). In half of the lists the points were assigned in a descending order such that primacy items were worth the most points and recency items were worth the fewest points. In the other half of the lists the points were assigned in an ascending order such that primacy items were worth the fewest points and recency items were worth the most points. In the descending-points condition the quadratic function was again seen, such that the pupil initially dilated across serial position, but then decreased (and this did not change as a function of working-memory capacity). In the ascending condition, there was a more general increase in pupil dilation across the list, suggesting that more attention was allocated to recency items. Miller et al. interpreted these results as suggesting that participants can flexibly allocate attention across items depending on the properties of the task and the type of processing that participants are engaging in. In some situations, and for some participants, cumulative maintenance rehearsal processes are utilized where attention allocation increases across items up to a point and then decreases as resources are overloaded. In other situations, and for some participants, more attention is allocated to early items than later items, or more attention is allocated to later items than early items. Thus, it seems likely that the allocation of attention is partially dependent on the nature of the task, the type of processes that are engaged in during encoding, as well as ability differences.
Current study
The goal of the present study was to use pupillary responses as an online measure of attention allocation to better examine how attention is allocated to items during encoding in a delayed free-recall task. Specifically, if pupillary dilations provide an online measure of attentional allocation, we should be able to track how attention is allocated to items during encoding in order to better understand how attention is utilized during learning. As noted previously, prior research has suggested two possible ways in which attention is allocated across items during encoding. First, it is possible that participants cannot sustain their attention across items, and thus the allocation of attention decreases across items in the list such that early items receive the most attention consistent with primacy gradient models of immediate serial recall (e.g., Brown et al., 2000; Farrell & Lewandowsky, 2002; Healey & Kahana, 2016; Miller et al., 2019; Page & Norris, 1998; Tulving, 2007). We refer to this as the Sustained Attention account, which predicts (as shown in Fig. 1) pupil diameter should decrease across serial positions. Second, it is possible that the allocation of attention increases across items as participants attempt to cumulatively rehearse items leading to a loading function until working memory is overloaded (Granholm et al., 1996; Kahneman & Beatty, 1966; Miller et al., 2019). We refer to this as the Load-Overload account, which predicts (as shown in Fig. 1) pupil diameter should initially increase and then decrease across serial positions. Of course it is also possible that the allocation of attention does not change across the list, resulting in a null pupillary serial position effect. We refer to this as the Same account, which predicts (as shown in Fig. 1) that there should be no changes in pupil diameter across serial positions. Furthermore, it is possible that each of these accounts is correct in that participants can flexibly allocate their attention depending on the nature of the task and the type of processes that are engaged in.
Possible changes in pupil diameter as a function of serial position and theoretical account. See text for details
In addition to examining pupillary responses and attention allocation across items in a list, we were also interested in examining how attention is allocated to each item regardless of serial position. As noted previously, prior research suggests that when participants are engaging in maintenance rehearsal, attention is allocated early on during item presentation for word processing and setting up the rehearsal program, but then the amount of attention drops as the rehearsal program runs relatively automatically (e.g., Naveh-Benjamin & Jonides, 1984; Phaf & Wolters, 1993). However, when participants are engaging in elaborative rehearsal processes, attention is continuously allocated during item presentation as participants engage in overall more effortful processing. This suggests that when maintenance rehearsal processes are used, there should be a small brief pupil dilation early in encoding. When elaborative rehearsal processes are used there should be a large and sustained pupil dilation during the encoding period. It should be possible to distinguish between these possibilities by examining pupillary responses for each item.
Before continuing it is important to note that there are two general types of pupillary responses, which differ in their temporal properties: tonic and phasic (e.g., Beatty, 1982; Chiew & Braver, 2013; Kostandyan et al., in press; Unsworth & Robison, 2016). Tonic pupillary responses are changes in the pupil that occur relatively slowly across the duration of a task, list, or block of trials. Conversely, phasic responses are more short-lived pupillary responses tied to a specific event, item, or trial. Thus, the above possibilities examining changes in pupil dilation across items in a list represent changes in tonic pupillary responses, which we refer to as list-level changes. Possibilities examining changes in pupil dilation for each item regardless of serial position represent changes in phasic pupillary responses, which we refer to as item-level changes. Both of these responses will be examined in the current set of experiments.
To examine these issues, participants performed a delayed free-recall task while their pupils were continuously measured throughout the task. In Experiment 1 we examined pupillary responses in delayed free recall in an attempt to replicate Miller et al. (2019). In Experiments 2 and 3 we manipulated presentation duration and list-length, respectively, in order to further examine how these manipulations influenced pupillary responses. The use of pupil diameter should allow us to track how attention is being allocated to items during encoding and examine possible differences in both list-level and item-level pupillary responses.
Experiment 1
In Experiment 1 we examined pupillary responses during encoding of a delayed free-recall task. Prior research with this task demonstrated a quadratic list-level effect for pupillary responses consistent with increasing allocation of attention due to effortful cumulative rehearsal until working memory was overloaded (e.g., Miller et al., 2019). However, one issue with these results is that prior to performing the delayed free-recall tasks all participants completed three complex working-memory-span tasks to assess individual differences in working-memory capacity. These tasks, like other span tasks, require serial recall, and thus the results may have been due to participants using cumulative rehearsal in the span tasks and then continuing to use that same rehearsal strategy when performing the delayed free-recall task. That is, perhaps the pupillary results are due to serial order requirements on the span tasks carrying over to free recall. To examine this possibility and to see if we could replicate the same general pattern of results from Miller et al. (2019), participants completed the same delayed free-recall task as Miller et al., without performing any prior tasks.
Method
Participants
Participants were 45 undergraduate students recruited from the subject pool at the University of Oregon. Based on our prior pupillometry work (Miller et al., 2019), we determined that a minimum sample size of 25 participants would be sufficient to find a medium effect size, with a power of .80 and alpha set at .05 (two tailed). We aimed for a sample size of 35 participants. Participants (71.7% female) were between the ages of 18 and 22 years (M = 18.93, SD = 1.14) and received course credit for their participation. Data from five participants were excluded from analyses because of data-collection problems with the eye-tracker, and two participants were excluded for not following instructions, leaving a final sample of 38 participants.
Procedure
After calibration of the eye-tracker, participants were administered a delayed free-recall task consisting of five word lists containing ten words each. Word lists were initially composed of randomized nouns selected from the Toronto word pool (Friendly, Franklin, Hoffman, & Rubin, 1982), and all words were between three and five letters in length. Words (as well as the mask preceding/following each word) were presented in black text in Arial font (font size = 24) on a light gray background. Participants were tested individually in a dimly lit room. Properties such as ambient light, screen brightness, contrast, etc. were held constant across participants. All participants received the same lists of words and were instructed to recall as many words as possible from each list. The task began with a “Ready?” signal onscreen for 2 s, followed by a fixation period lasting 2.5 s (baseline pupil diameter). Each list began with the same “Ready?” signal and fixation period, which were followed by a series of words presented individually in the center of the screen for 3 s. Each word was preceded by a mask of five plus signs (e.g., “+++++”) for 500 ms and each word was followed by the same mask for 500 ms. After list presentation, participants then completed a 16-s distractor task that required participants to verbally report a series of eight three-digit numbers in descending order (adapted from Rohrer & Wixted, 1994). Each three-digit string was presented onscreen for 2 s. At recall, three question marks appeared in the center of the screen to prompt participants to recall as many words as possible within a 45-s window. Participants typed their responses in any order they wished and pressed “enter” after each word, thereby clearing the screen.
Eye-tracking
Pupil diameter was continuously recorded binocularly at 120 Hz using a Tobii T120 eye-tracker. Participants were seated 60 cm from the monitor with the use of a chinrest. Stimuli were presented on the Tobii T120 eye-tracker 17-in. monitor with a 1,024 × 768 screen resolution. Data from each participant’s left eye were used. Missing data points due to blinks, off-screen fixations, and/or eye-tracker malfunction were removed. We did not exclude whole words or lists for missing data. List-level pupillary responses were baseline corrected by subtracting out baseline pupil diameter from the last 400 ms of the fixation period on a list-by-list (i.e., each list had its own baseline) basis for each participant. Item-level pupillary responses were baseline corrected by subtracting out pupil diameter from the 500-ms mask screen preceding each item on an item-by-item basis (i.e., each item had its own baseline) for each participant. The pupil data for the 3-s encoding phase for each item were averaged into a series of 200-ms time windows and each 200-ms window was baseline corrected on an item-by-item basis.
Results and discussion
Behavioral effects
First we examined accuracy as a function of serial position. Overall, proportion correct was .62 (SE = .03). As shown in Fig. 2, examining proportion correct as a function of serial position suggested a main effect of serial position, F(9, 333) = 34.20, MSE = .04, p < .001, partial η2 = .48, such that there was a strong primacy effect but no recency effect, consistent with other work using delayed free recall (Glanzer & Cunitz, 1966).
Proportion correct as a function of serial position in Experiment 1. Error bars represent one standard error of the mean
Pupillary effects
Next we turn to our primary analyses of interest examining pupillary dilations. As noted previously, pupil diameter was measured continuously throughout the encoding period. Figure 3 shows the overall averaged pupillary responses during the presentation of the words. As can be seen, there seemed to be a general trend whereby the pupil increased and then decreased across serial positions. Additionally, there seemed to be an increase in pupil diameter that occurred for each word (Kahneman & Peavler, 1969; Kucewicz et al., 2018). Thus, in order to better examine these notions, we separately examined baseline corrected list-level and item-level pupillary responses.
Change in pupil diameter during list presentation. Numbers reflect serial position of each presented word
List-level pupillary responses
Our first set of analyses focuses on changes in pupillary responses as a function of serial position throughout the encoding phase. As shown in Fig. 4a, there was a main effect of serial position, F(9, 333) = 4.16, MSE = .007, p < .001, partial η2 = .10, such that pupil diameter increased up to position seven and then decreased back down to the same levels as position 1. Replicating Miller et al. (2019), there was a quadratic trend, F(1, 37) = 18.64, MSE = .012, p < .001, partial η2 = .34.
Item-level pupillary responses
Next we examined item-level pupillary responses to determine if the pupil dilated for the presentation of each word regardless of serial position.Footnote 1 As shown in Fig. 4b, there was a main effect of time bin, F(14, 518) = 2.46, MSE = .002, p = .002, partial η2 = .06, and, more specifically, a significant quadratic trend, F(1, 37) = 6.85, MSE = .004, p = .013, partial η2 = .16, indicating that the pupil dilated early in the encoding period and subsequently decreased. The peak was significantly different from baseline, t(37) = 3.63, p = .001.
Overall, the current list-level results replicated prior research in demonstrating a quadratic trend for pupillary responses across serial position (Miller et al., 2019). This occurred even though participants in the current study did not perform any tasks prior to the delayed free-recall task. Thus, it does not seem like the results from Miller et al. (2019) were due to participants performing the complex span tasks prior to the delayed free-recall task, which resulted in any carryover effects. Both Miller et al. and the current results suggested that the pupil tended to increase in dilation across serial position up to around position seven and then decrease in dilation. These results are consistent with the Load-Overload account in suggesting that attention is increasingly allocated across items as participants engage in a form of effortful cumulative rehearsal until working memory becomes overloaded, at which point attention allocation decreases (Granholm et al., 1996; Kahneman & Beatty, 1966; Miller et al., 2019). Examining the item-level pupillary responses suggested that the pupil dilated early in the encoding period and subsequently decreased. These results are consistent with the notion that when each word is presented there is a brief burst of attention in order to process the item and add the current word to the rehearsal list, but then attention allocation decreases (e.g., Naveh-Benjamin & Jonides, 1984; Phaf & Wolters, 1993). These results provide insights into how attention is allocated to items during encoding both across serial positions and within each item.
Experiment 2
The purpose of Experiment 2 was to replicate and extend Experiment 1. Specifically, in Experiment 1 examining the item-level pupillary responses suggested a brief pupillary dilation followed by constriction consistent with what would be expected if participants were rehearsing items. In Experiment 2 we wanted to see if we could get larger and more sustained item-level pupillary responses consistent with what you would expect if participants were engaging in more effortful elaborative processing. Prior research has demonstrated that as presentation duration increases, participants are more likely to use more elaborative strategies (such as organization and imagery), resulting in better recall performance (Stoff & Eagle, 1971; Unsworth, 2016). Thus, participants adaptively change their encoding strategies as a function of task demands and experience (Delaney & Knowles, 2005; Finley & Benjamin, 2012; Unsworth, 2016). Furthermore, in a paired-associates task, Miller and Unsworth (in press) recently demonstrated that when participants reported using effective learning strategies (such as imagery or sentence generation), they demonstrated larger pupillary responses and better recall than when they reported using more ineffective strategies (such as rote rehearsal). Therefore, in Experiment 2 participants performed the same delayed free-recall task as in Experiment 1, but on a third of the lists the words were presented for 2 s each, on another third of the lists the words were presented for 4 s each, and finally on the last third of lists the words were presented for 8 s each. For the 2-s presentation-duration condition we expected to replicate the results from Experiment 1 in terms of both list-level and item-level effects. For the 8-s presentation-duration condition, however, we expected to see a much larger item-level pupillary response, and this response should be sustained throughout the encoding period as participants continually allocate high levels of attention to the items. The 4-s presentation condition should likely fall somewhere in between the other two conditions. Thus, by manipulating presentation duration we should encourage participants to switch strategies and attention allocation policies, and this should be reflected in the item-level pupillary responses.
Method
Participants
Participants were 40 undergraduate students recruited from the subject pool at the University of Oregon. Similar to Experiment 1, we aimed for a sample size of 35 participants. Participants (56.1% female) were between the ages of 18 and 23 years (M = 19.60, SD = 1.08) and received course credit for their participation. Data from one participant were excluded from analyses because of data collection problems with the eye-tracker, leaving a final sample of 39 participants.
Procedure
Participants performed the same delayed free-recall task as in Experiment 1 with the following exceptions. There were nine total lists of ten words each from the same pool of words from Experiment 1. On a third of the lists the words were presented for 2 s each, on another third of the lists the words were presented for 4 s each, and finally on the last third of the lists the words were presented for 8 s each. Presentation duration was randomized so that participants did not know how long the words would be presented until the end of the first word. All other aspects of the task were the same as Experiment 1.
Eye-tracking
This was the same as in Experiment 1.
Results and discussion
Behavioral effects
Overall, proportion correct was .55 (SE = .03). There was a main effect of presentation duration, F(2, 76) = 40.51, MSE = .07, p < .001, partial η2 = .52, such that proportion correct increased with increasing presentation duration (2 s M = .47, SE = .03; 4 s M = .53, SE = .04; 8 s M = .64, SE = .03). There was a main effect of serial position, F(9, 342) = 33.26, MSE = .07, p < .001, partial η2 = .47, consistent with Experiment 1 and prior research. As shown in Fig. 5, there was a presentation duration by serial position interaction, F(18, 684) = 1.70, MSE = .07, p = .035, partial η2 = .04, such that the effect of presentation duration was mainly localized to the primacy items.
Proportion correct as a function of serial position and presentation-duration condition in Experiment 2. Error bars represent one standard error of the mean
Pupillary effects
Similar to Experiment 1, we examined baseline-corrected list-level and item-level pupillary responses. See the Appendix for the overall averaged pupillary responses during the presentation of the words.
List-level pupillary responses
Examining pupillary responses as a function of serial position throughout the encoding phase suggested that the main effect of presentation duration was not quite below the .05 criterion, F(2, 76) = 3.13, MSE = .20, p = .05, partial η2 = .08. Unlike Experiment 1, there was not a main effect of serial position, F(9, 342) = 1.35, MSE = .022, p = .21, partial η2 = .03. However, as shown in Fig. 6a, there was a presentation duration by serial position interaction, F(18, 684) = 2.49, MSE = .010, p = .001, partial η2 = .06. Examining each presentation duration separately suggested that there was an effect of serial position in the 2-s presentation-duration condition, F(9, 342) = 3.33, MSE = .014, p = .001, partial η2 = .08, and like Experiment 1, there was a significant quadratic trend, F(1, 38) = 13.78, MSE = .035, p = .001, partial η2 = .27. There was no effect of serial position in either the 4-s presentation condition, F(9, 342) = .95, MSE = .016, p = .48, partial η2 = .02, or the 8-s presentation-duration condition, F(9, 342) = 1.39, MSE = .011, p = .19, partial η2 = .04. Thus, only in the shortest presentation-duration condition was there a significant (quadratic) effect of serial position.
(a) Change in pupil diameter as a function of serial position and presentation-duration condition in Experiment 2. Error bars reflect one standard error of the mean. (b) Change in pupil diameter during the encoding period for each word as a function of presentation-duration condition in Experiment 2. Shaded areas reflect one standard error of the mean
Item-level pupillary responses
Next we examined the item-level pupillary responses as a function of presentation duration. Examining the mean pupillary responses suggested a main effect of presentation duration, F(2, 76) = 5.50, MSE = .002, p = .006, partial η2 = .13, such that pupil dilation tended to increase with increasing presentation duration (2 s M = .02, SE = .01; 4 s M = .01, SE = .01; 8 s M = .05, SE = .01). Examining each presentation duration separately suggested that there was no effect of time bin in the 2-s presentation-duration condition, F(9, 342) = 1.33, MSE = .002, p = .22, partial η2 = .03. However, there were significant effects of time bin in both the 4-s presentation condition, F(19, 722) = 1.74, MSE = .003, p = .026, partial η2 = .04, and in the 8-s presentation-duration condition, F(39, 1482) = 2.43, MSE = .003, p < .001, partial η2 = .06. As shown in Fig. 6b, all of the presentation-duration conditions demonstrated an initial brief pupil dilation (although not significant in the 2-s condition). Similar to Experiment 1, in the 4-s presentation-duration condition, following the initial dilation, the pupil then constricted. However, in the 8-s presentation-duration condition, the pupil dilated to its largest levels and maintained this level of dilation for most of the encoding period before decreasing back to the starting levels. The peak dilation was significantly different from baseline in both the 4-s and the 8-s conditions (all ps < .007).
Overall, the list-level results from Experiment 2 suggested that when presentation duration was short (2 s), there was a quadratic effect for pupillary responses across serial position consistent with the Load-Overload account and Experiment 1. However, for longer presentation durations, there was not a significant effect of serial position, consistent with the Same account suggesting overall similar allocation of attention to each item. One interesting finding from the list-level analyses was that the pupillary responses for the 8-s presentation-duration condition were generally smaller than those for the 2-s condition. This would seemingly suggest that this condition was associated with less effort than the 2-s condition. However, as seen in Fig. 11c in the Appendix, the pupillary response for the first word started off near baseline and demonstrated a small peak, but then demonstrated a large reduction in pupil diameter. Thus, the second word (and subsequent words) started off at a much lower level than the first word. This was likely due to the fact that participants did not know how long the presentation duration was going to be until the first word was presented. Thus, it is not necessarily the case that the 8-s condition is associated with lower effort (see below), but rather demonstrates the importance of examining both list-level and item-level effects.
Examining item-level pupillary responses suggested that at shorter presentation durations there was a general increase in pupil dilation and subsequent decrease consistent with Experiment 1. However, for the 8-s presentation-duration condition there was a large initial ramp up in pupil dilation that was maintained for most of the encoding period consistent with the notion that attention was continuously being allocated to encoding the items. Based on prior research that suggests that increased presentation duration is associated with a switch from maintenance rehearsal to elaborative processing (e.g., Stoff & Eagle, 1971; Unsworth, 2016), these results suggest that when presentation duration is short, attention is allocated briefly to process the item and potentially add the current item to the rehearsal list. However, when presentation duration is long and more elaborative processes can occur, attention is allocated in a more continuous fashion both within and between items. However, it should be noted that we did not directly examine participant’s encoding strategies, and thus we do not know if a switch in strategies occurred in the current study. Future research is needed to examine if switches in encoding strategies are accompanied by changes in pupillary responses as speculated here. Collectively, these results suggest that with increasing presentation duration, participants changed how they processed the items and changed their attention allocation policies.
Experiment 3a
The purpose of Experiment 3a was to replicate and extend the results from the prior experiments. In both Experiment 1 and the 2-s presentation-duration condition in Experiment 2 there was evidence for the Load-Overload account in terms of a quadratic pupillary response across serial position. In the 4-s and 8-s presentation-duration conditions in Experiment 2, the pupillary serial position functions were not significant, suggesting that similar amounts of attention were being allocated across serial positions consistent with the Same account. As such, there was evidence for the Load-Overload and Same accounts, but we have not seen evidence for the Sustained Attention account aside from what was seen for low working-memory capacity individuals in Miller et al. (2019). The purpose of Experiment 3a was to try and find evidence for the Sustained Attention account by manipulating list-length. We reasoned that relatively short list-lengths of ten items in the prior experiments could have encouraged participants to try and cumulatively rehearse items (especially in the short presentation-duration conditions), leading to quadratic pupillary serial position curves. However, with much longer list-lengths, participants might abandon this strategy and instead only focus on the early list items (or attempt to rehearse items in small chunks) rather than trying to rehearse the entire list in order. Evidence for such an idea comes from prior research examining the item-order hypothesis in free recall, which suggests that order information is particularly important for short lists, but not for long lists (Mulligan & Lozito, 2007). Thus, with long lists of items we expected to find a general decrease in pupillary responses across serial position consistent with the Sustained Attention account. For short lists of items we expected to replicate the prior experiments in finding a quadratic pupillary function. To test these notions, participants performed the same delayed free-recall task as in Experiment 1. On a third of the lists there were ten words per list, on another third of the lists there were 15 words per list, and finally on the last third of lists there were 20 words per list. By manipulating list-length, we should encourage participants to switch attention allocation policies, and this should be reflected in the list-level pupillary responses.
Method
Participants
Participants were 40 undergraduate students recruited from the subject pool at the University of Oregon. Similar to Experiment 1, we aimed for a sample size of 35 participants. Participants (57.5% female) were between the ages of 18 and 32 years (M = 21.18, SD = 3.18) and received course credit for their participation. Data from one participant were excluded from analyses because of data collection problems with the eye-tracker and data from one participant were excluded for failing to follow instructions leaving a final sample of 38 participants.
Procedure
Participants performed the same delayed free-recall task as in Experiment 1 with the following exceptions. There were nine total lists of words from the same pool of words from Experiment 1. On a third of the lists list-length was ten words, on another third of the lists list-length was 15 words, and finally on the last third of lists list-length was 20 words. List-length was randomized so that participants did not know how long the lists would be. All other aspects of the task were the same as Experiment 1.
Eye-tracking
This was the same as in Experiment 1.
Results and discussion
Behavioral effects
Overall, proportion correct was .48 (SE = .03). There was a main effect of list-length, F(2, 74) = 56.72, MSE = .006, p < .001, partial η2 = .61, such that proportion correct decreased with increasing list-length (10 M = .56, SE = .03; 15 M = .49, SE = .03; 20 M = .38, SE = .03). Examining each list-length separately suggested main effects of serial position in each, all Fs > 10, all ps < .001, all partial η2s > .21. As shown in Fig. 7, each list-length demonstrated characteristic serial position curves for delayed free recall with large primacy and no recency.
Proportion correct as a function of serial position and list-length in Experiment 3a. Error bars represent one standard error of the mean
Pupillary effects
Similar to the prior experiments, we examined baseline-corrected list-level and item-level pupillary responses. See the Appendix for the overall averaged pupillary responses during the presentation of the words.
List-level pupillary responses
Examining pupillary responses as a function of serial position for each list-length separately suggested no main effect of serial position in the list-length 10 condition, F(9, 333) = 1.61, MSE = .014, p = .11, partial η2 = .04. However, there were significant effects of serial position in both the list-length 15, F(14, 504) = 8.72, MSE = .017, p < .001, partial η2 = .20, and list-length 20, F(19, 684) = 5.49, MSE = .017, p < .001, partial η2 = .13, conditions. As shown in Fig. 8a, both the list-length 15 and 20 conditions demonstrated decreases in pupil dilation across serial position. Examining the first ten serial positions for each list-length together suggested no main effect of list-length, F(2, 70) = 1.74, MSE = .195, p = .18, partial η2 = .05, and no list-length by serial position interaction, F(18, 630) = 1.23, MSE = .010, p = .23, partial η2 = .03. There was, however, a main effect of serial position, F(9, 315) = 5.49, MSE = .018, p < .001, partial η2 = .14, suggesting that there was a general decrease in pupillary responses across serial position. Thus, unlike Experiment 1, which demonstrated that the pupil tended to increase and then decrease in dilation across serial position, in the current experiment the pupil tended to constrict across serial positions.
(a) Change in pupil diameter as a function of serial position and list-length in Experiment 3a. Error bars reflect one standard error of the mean. (b) Change in pupil diameter during the encoding period for each word and list-length in Experiment 3a. Shaded areas reflect one standard error of the mean
Item-level pupillary responses
Next we examined the item-level pupillary responses as a function of list-length. The main effect of list-length did not fall below the .05 criterion, F(2, 74) = 2.77, MSE = .014, p = .069, partial η2 = .07. Likewise there was no significant main effect of time bin, F(14, 518) = .96, MSE = .004, p = .490, partial η2 = .03, nor a list-length by time bin interaction, F(28, 1036) = 1.24, MSE = .001, p = .185, partial η2 = .03. As shown in Fig. 8b, there was a general trend of a brief dilation early in the encoding period; however, unlike the prior experiments, the peak was not significantly different from baseline in any condition (all ps > .06).
Overall, the list-level analyses suggested that, unlike the prior experiments, there was a consistent decrease in pupillary responses across serial position. These results are consistent with the notion that participants are allocating more attention to early list positions in accord with the Sustained Attention account. Furthermore, we expected to see a quadratic pupillary function for the list-length 10 condition consistent with the prior experiments and Miller et al. (2019), but this did not occur. Rather there were similar decreases in pupillary responses across list-length. One possible reason for this is that because participants did not know how long the list was they treated all lists the same. If participants had been explicitly told how long the list-length was going to be or if list-length was blocked, then perhaps we could have seen different functions for the different list-lengths. Although it should be noted that prior research using an overt rehearsal paradigm found that participants still cumulatively rehearse even when list-length is unknown (Ward, 2002). In terms of the item-level analyses there was not a significant pupillary response. Collectively, with long lists of items, we found evidence of a general decrease in pupillary responses across serial position and the pupillary responses for each item were small (and not significantly different from zero), suggesting that participants may have prioritized early list items and less attention was allocated to items later in the list.
Experiment 3b
The purpose of Experiment 3b was to replicate and extend the results from the prior experiment. In particular, across all list-lengths the list-level pupillary responses tended to decrease across serial positions. However, we expected to find evidence for the Load-Overload account when list-length was short (ten items), but evidence for the Sustained Attention account when list-length was long (20 items). As noted above, one possible reason for why we did not replicate the prior findings in terms of the Load-Overload account for the short list-length is that participants did not know how long the list-length was going to be and thus treated all list-lengths the same rather than switching allocation policies for the different list-lengths. If this is the case, then if participants know in advance how long the list is going to be, they can dynamically switch allocation policies in line with each list-length. To test this notion, participants performed the same delayed free-recall task as Experiment 1. On half of the lists there were ten words per list and on the other half of lists there were 20 words per list. List-length was blocked within participants (and counterbalanced between participants). Additionally, prior to each list participants were told how long the list-length would be. By explicitly telling participants how long the list will be and manipulating list-length we should encourage participants to switch attention allocation policies and this should be reflected in the list-level pupillary responses.
Method
Participants
Participants were 46 undergraduate students recruited from the subject pool at the University of Oregon. Similar to Experiment 1, we aimed for a sample size of 35 participants. Participants (66.7% female) were between the ages of 18 and 34 years (M = 19.6, SD = 2.73) and received course credit for their participation. Data from four participants were excluded from analyses because of data collection problems with the eye-tracker, leaving a final sample of 42 participants.
Procedure
Participants performed the same delayed free-recall task as in Experiment 1 with the following exceptions. There were six total lists of words from the same pool of words from Experiment 1. On half of the lists there were ten words per list and on the other half of lists there were 20 words per list. List-length was blocked within participants (and counterbalanced between participants). Additionally, prior to each list, participants were told how long the list-length would be. All other aspects of the task were the same as in Experiment 1.
Eye-tracking
This was the same as in Experiment 1.
Results and discussion
Behavioral effects
Overall, proportion correct was .49 (SE = .03). There was an effect of list-length, t(41) = 13.84, p < .001, such that proportion correct decreased with increasing list-length (10 M = .60, SE = .03; 20 M = .37, SE = .03). Examining each list-length separately suggested main effects of serial position in each, all Fs > 10, all ps < .001, all partial η2s > .21. As shown in Fig. 9, each list-length demonstrated characteristic serial position curves for delayed free recall with large primacy and no recency.
Proportion correct as a function of serial position and list-length in Experiment 3b. Error bars represent one standard error of the mean
Pupillary effects
Similar to the prior experiments, we examined baseline-corrected list-level and item-level pupillary responses. See the Appendix for the overall averaged pupillary responses during the presentation of the words.
List-level pupillary responses
Examining pupillary responses as a function of serial position for each list-length separately suggested a main effect of serial position in the list-length 10 condition, F(9, 369) = 2.54, MSE = .018, p = .008, partial η2 = .06. As seen in Fig. 10a, the pupillary response demonstrated a slight initial decrease, and then increased up to serial positons 7–9 before decreasing again. Thus, given the slight initial dip, the cubic trend was significant, F(1, 41) = 15.69, MSE = .022, p < .001, partial η2 = .28. There was also a significant effect of serial position in the list-length 20 condition, F(19, 779) = 3.02, MSE = .027, p < .001, partial η2 = .07. As shown in Fig. 10a, the list-length 20 condition demonstrated a decrease and then a plateau in pupil dilation across serial position. Examining the first ten serial positions for each list-length together suggested a main effect of list-length, F(1, 41) = 16.23, MSE = .180, p < .001, partial η2 = .28, with generally larger pupillary responses in the list-length 10 condition (M = -.02, SE = .03) compared to the list-length 20 condition (M = -.14, SE = .03). There was also a main effect of serial position, F(9, 369) = 3.13, MSE = .024, p = .001, partial η2 = .07. Importantly, there was also a list-length by serial position interaction, F(9, 369) = 4.18, MSE = .023, p < .001, partial η2 = .09. When list-length was short, the pupillary responses were more similar to what was seen in Experiments 1 and 2 (short presentation duration), consistent with the Load-Overload account. However, when list-length was long the results were similar to Experiment 3a, demonstrating a general decrease across early serial positions consistent with the Sustained Attention account. Thus, unlike Experiment 3a, there were clear list-level differences in pupillary responses between short and long list-lengths.
(a) Change in pupil diameter as a function of serial position and list-length in Experiment 3b. Error bars reflect one standard error of the mean. (b) Change in pupil diameter during the encoding period for each word and list-length in Experiment 3b. Shaded areas reflect one standard error of the mean
Item-level pupillary responses
Next we examined the item-level pupillary responses as a function of list-length. The main effect of list-length was not significant, F(1, 41) = .39, MSE = .026, p = .54, partial η2 = .01. There was, however, a significant main effect of time bin, F(14, 574) = 3.89, MSE = .004, p < .001, partial η2 = .09. The list-length by time bin interaction was not significant, F(14, 574) = .99, MSE = .002, p = .462, partial η2 = .02. As shown in Fig. 10b, both conditions demonstrated an initial brief pupil dilation followed by pupil constriction, similar to Experiment 1 and the short presentation condition in Experiment 2. The peak dilation was significantly different from baseline in both conditions (both ps < .001).
Overall, the list-level analyses suggested clear differences in the pupillary responses across serial position for the short and long list-lengths. In particular, in the long list-length condition the pupil demonstrated an initial decrease and then plateau consistent with Experiment 3a. These results are consistent with the notion that participants are allocating more attention to early serial positions in accord with the Sustained Attention account. In the short-list length condition, however, the pupillary response was more similar to what was seen in the prior experiments and Miller et al. (2019), in which the pupil tended to increase (after a brief initial dip) for middle serial positions and then decrease for the last serial position. These results are consistent with the Load-Overload account. Examining the item-level pupillary responses suggested that the pupil dilated early in the encoding period and subsequently decreased consistent with the prior experiments. Thus, when participants know the length of the list in advance, there are clear differences in how attention is being allocated to items in short and long list-lengths.
General discussion
In four experiments we conducted an initial examination of attention allocation during encoding in a delayed free-recall task utilizing pupillary responses. In particular, we were interested in how attention is allocated both across items in a list (list-level responses) and within each item (item-level responses), and how this changes as a function of different task manipulations. In terms of list-level responses we noted that there were three basic accounts of how attention might change across items in a list. First, the Sustained Attention account in accord with primacy gradient models suggests that attention should decrease across items in a list (e.g., Brown et al., 2000; Farrell & Lewandowsky, 2002; Healey & Kahana, 2016; Miller et al., 2019; Page & Norris, 1998; Tulving, 2007). Second, the Load-Overload account suggests that attention is increasingly allocated to items as participants engage in effortful cumulative rehearsal processes until working memory is overloaded (e.g., Granholm et al., 1996; Kahneman & Beatty, 1966; Miller et al., 2019). Finally, the Same account is essentially the null hypothesis suggesting that attention allocation does not change across items. The results from the experiments suggested evidence for all three accounts. In particular, in Experiment 1, the short presentation duration (2-s) condition in Experiment 2, and the short list-lengths in Experiment 3b we replicated prior research demonstrating a pupillary serial position curve where the pupil initially increased and then decreased across serial position in accord with the Load-Overload account. In the 4-s and 8-s presentation-duration conditions in Experiment 2, the pupil serial position curve was not significant, suggesting similar pupillary responses across serial position in accord with the Same account. However, in Experiment 3a (and long list-lengths in Experiment 3b) there was a general decrease in the pupillary responses across serial position. Thus, there was clear evidence for all three accounts depending on the nature of the task. When list-length was short (and the current list-length was known) and presentation duration was brief, the results were consistent with the Load-Overload account, suggesting the possibility that participants were attempting to cumulatively rehearse the entire list resulting in increased attention allocation across items until working memory was overloaded. When presentation duration was increased, participants likely engaged in more elaborative encoding processing resulting in a more continuous allocation of attention both within (see below) and across items. However, when list-length increased (and the current list-length was unknown) participants may have prioritized early list items, thereby allocating most of their attention to primacy items and less attention was allocated to subsequent items. As such, the current results suggest that there is likely no single pattern of attention allocation across items (as some models seem to suggest); rather, participants likely flexibly allocate their attention to items depending on the nature of the task and perhaps by the types of processes that are engaged in (i.e., cumulative rehearsal vs. elaborative processing). While these initial results are promising, future research is needed to further examine how participants flexibly allocate their attention to items across a list and whether list-level pupillary responses accurately track these changes in allocation policies. Furthermore, because all of the experiments here relied on delayed free recall with relatively short lists, future research is needed to examine these effects in other free-recall paradigms like immediate free recall and potentially with longer list-lengths than used in the current study.
In terms of item-level responses, prior research has suggested that when participants engage in maintenance rehearsal, attention is initially allocated to an item, followed by habituation, but when participants engage in elaborative rehearsal, attention is allocated in a more continuous fashion (e.g., Naveh-Benjamin & Jonides, 1984; Phaf & Wolters, 1993). The results from the experiments suggested evidence consistent with both ideas. When presentation duration was relatively short in each experiment, the pupil tended to dilate briefly early in the encoding period followed by constriction back to baseline levels consistent with the idea that attention was only briefly allocated to the item initially. However, when presentation duration was increased to 8 s in Experiment 2, there was a large initial ramp up in pupil dilation that was maintained for most of the encoding period before falling back down to baseline levels. This latter pattern is consistent with the notion that attention was being fairly continuously allocated to the item, as would be expected if participants were engaging in more elaborative processing of the item (e.g., Phaf & Wolters, 1993). Thus, there was evidence for both a brief initial allocation of attention to items and a more sustained allocation of attention to items depending on the presentation duration of the items. The one wrinkle to this pattern were the results from Experiment 3a where list-length was manipulated. In this experiment the item-level pupillary responses were not significantly different from zero. It is not clear what is causing these much smaller (and non-significant) item-level pupillary responses. It is possible that given the long (and unknown) list-lengths participants were devoting most of their attention to primacy items, as noted previously, and subsequent items were encoded in a less active fashion with little attention or rehearsal. It is also possible that these null results are simply a Type II error given that prior research (Kahneman & Peavler, 1969; Kucewicz et al., 2018; Miller et al., 2019) and the prior experiments all demonstrated item-level pupillary responses. In Experiment 3b when list-length was known, the item-level pupillary responses were significant and similar to the prior experiments. Future research is needed to better examine these effects and the extent to which these item-level pupillary responses are associated with different allocation policies associated with different types of encoding processes.
The current results broadly suggest that pupillary responses can be used to examine how attention is allocated to items during encoding in a delayed free-recall task. That is, pupillary responses can be used to examine how effortful processing is changing both within and between items during encoding. The current results suggest that sometimes greater pupil dilation and greater allocation of attention are associated with better recall. For example, in Experiment 2, longer presentation durations were associated with larger pupils and better recall. Similarly, in Experiment 3a (and long list-lengths in Experiment 3b), primacy items were associated with larger pupils and better recall. These results are consistent with prior research suggesting that greater dilation is associated with better memory both within and between subjects (e.g., Ariel & Castel, 2014; Kucewicz et al., 2018; Miller & Unsworth, in press; Miller et al., 2019). For example, Ariel and Castel (2014; see also Miller et al., 2019) found that words associated with higher point values were associated with larger pupil dilations during encoding and were more likely to be recalled than items associated with lower point values. Furthermore, in a paired-associates task, Miller and Unsworth (in press) found that when participants reported using effective learning strategies (such imagery or sentence generation), they demonstrated larger pupillary responses and better recall than when they reported using more ineffective strategies (such as rote rehearsal). Thus, some manipulations are associated with larger pupils and better recall. At the same time, other results suggest that there is not a clear mapping between greater pupil dilation and better memory. For example, in Experiments 1 and 2, quadratic effects (a cubic effect was seen in Experiment 3b) in the list-level analyses suggested the largest dilation occurred for mid-list items and these items were recalled worse than primacy items as is typically seen (see also Miller et al., 2019). Furthermore, in all four experiments examining subsequent memory effects at the item-level suggested no differences between subsequent remembered versus forgotten items in terms of pupil dilation1 (see Kucewicz et al., 2018, for contrasting results). Thus, across all analyses, the results suggest that the relation between pupil dilation and memory performance in free recall is more nuanced than simply assuming that greater dilation is associated with better memory. Future research is needed to better examine conditions in which there are positive, negative, and/or no relation between pupil dilation and memory performance both within and between subjects.
Collectively, the current results suggest that participants flexibly allocate their attention to items in a delayed free-recall task depending on the nature of the task and the types of processes that are engaged in. These results suggest the potential of using pupillary responses (both list- and item-level responses) to track attention allocation during encoding. Future research is needed to better examine flexible allocation of attention to items during learning and the extent to which pupillary responses can be used to track changes in attention allocation.
Notes
We conducted exploratory analyses examining differences in pupillary responses for subsequently remembered and subsequently forgotten items in each experiment. In Experiment 1 there was no difference between remembered (M = .011; SD = .05) and forgotten items (M = .012; SD = .06), t(37) = -.19, p = .85 (see Kucewicz et al., 2018 for contrasting results) . Similar results were obtained when examining only primacy items, t(7) = .96, p = .37, or when non-primacy items, t(37) = .36, p = .72. In Experiment 2 there was no difference between remembered (M = .023; SD = .05) and forgotten items (M = .027; SD = .06), t(38) = -.54, p = .59. Similar results were obtained when examining only primacy items, t(27) = -.26, p = .80, or when non-primacy items, t(38) = .85, p = .40. Similar results were obtained when examining each presentation-duration condition separately. In Experiment 3 there was no difference between remembered (M = .001; SD = .054) and forgotten items (M = .008; SD = .065), t(37) = -1.22, p = .23. Similar results were obtained when examining only primacy items, t(26) = .09., p = .93, or when non-primacy items, t(37) = .19, p = .85. Similar results were obtained when examining each list-length condition separately. We also did an analysis where the item-level pupillary responses predicted accuracy for each item and participant using logistic MLM. None of the analyses across experiments were significant (all ps >.37), suggesting that item-level pupillary responses did not predict subsequent memory. Thus, while Kucewicz et al. (2018) found evidence for pupillary subsequent memory effects in their study, our results did not demonstrate such effects. One potential reason for this discrepancy is that in our experiments we utilized larger samples of participants (N = 10 in Kucewicz et al., 2018), but each participants completed a smaller number of lists than in Kucewicz et al. in which participants completed seventeen lists of twelve words each. Thus, each participant had many more items from which to calculate pupillary responses for recalled vs. forgotten items. For example, in our Experiment 1 there were a total of 50 words per participant, whereas in Kucewicz et al. each participant was given 204 words. Given the within-subject nature of these analyses, having more items per subject likely resulted in greater power to find potentially small within-subject effects. Future research is needed to better examine the robustness of pupillary subsequent memory effects in free recall.
References
Ariel, R., & Castel, A. D. (2014). Eyes wide open: Enhanced pupil dilation when selectivity studying important information. Experimental Brain Research, 232, 337-344.
Azizian, A., Polich J. 2007. Evidence for attentional gradient in the serial position memory curve from event-related potentials. Journal of Cognitive Neuroscience, 19, 2071–2081.
Baddeley, A., Lewis, V., Eldridge, M., & Thomson, N. (1984). Attention and retrieval from long-term memory. Journal of Experimental Psychology: General, 113, 518–540.
Beatty, J. (1982). Phasic not tonic pupillary responses vary with auditory vigilance performance. Psychophysiology, 19, 167-172.
Beatty, J., Lucero-Wagoner, B., 2000. The pupillary system. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (Eds.), Handbook of Psychophysiology (pp. 142-162). Cambridge University Press, New York.
Brown, G.D.A., Preece, T., & Hulme, C. (2000). Oscillator-based memory for serial order. Psychological Review, 107, 127-181.
Chiew, K.S., & Braver, T.S. (2013). Temporal Dynamics of Motivation-cognitive Control Interactions Revealed by High-resolution Pupillometry. Frontiers in Psychology, 4, 15.
Chun, M.M., & Turk-Browne, N. B. (2007). Interactions between attention and memory. Current Opinion in Neurobiology, 17, 177-184.
Craik, F. I. M., & Byrd, M. (1982). Aging and cognitive deficits: The role of attentional resources. In F. I. M. Craik & S. E. Trehub (Eds.), Aging and cognitive processes (pp. 191-211). New York: Plenum.
Craik, F.I.M., Govoni, R., Naveh-Benjamin, M., & Anderson, N.D. (1996). The effects of divided attention on encoding and retrieval processes in human memory. Journal of Experimental Psychology: General, 125, 159–180.
Delaney, P.F., & Knowles, M.E. (2005). Encoding strategy changes in spacing effects in the free recall of unmixed lists. Journal of Memory and Language, 52, 120-130.
Engle, R.W. (1975). Pupillary measurement and release from proactive inhibition. Perceptual and Motor Skills, 41, 835-842.
Farrell, S., & Lewandowsky, S. (2002). An endogenous model of ordering in serial recall. Psychonomic Bulletin & Review, 9, 59-79.
Finley, J. R., & Benjamin, A. S. (2012). Adaptive changes in encoding strategy with experience: Evidence from the test expectancy paradigm. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 632-652.
Friendly, M., Franklin, P.E., Hoffman, D. & Rubin, D.C. (1982). The Toronto Word Pool: Norms for imagery, concreteness, orthographic variables, and grammatical usage for 1,080 words. Behavior Research Methods and Instruments, 14, 375-399.
Gardner, R.M., Beltramo, J.S., & Krinksy, R. (1975). Pupillary changes during encoding, storage, and retrieval of information. Perceptual and Motor Skills, 41, 951-955.
Glanzer, M., & Cunitz, A. R. (1966). Two storage mechanisms in free recall. Journal of Verbal Learning and Verbal Behavior, 5, 351-60.
Goldinger, S.D., & Papesh, M.H. (2012). Pupil dilation reflects the creation and retrieval of memories. Current Directions in Psychological Science, 21, 90-95.
Granholm, E., Asarnow, R. F., Sarkin, A. J., & Dykes, K. L. (1996). Pupillary responses index cognitive resource limitations. Psychophysiology, 33, 457–461.
Granholm, E., Morris, S. K., Sarkin, A. J., Asarnow, R. F., & Jeste, D. V. (1997). Pupillary responses index overload of working memory resources in schizophrenia. Journal of Abnormal Psychology, 106, 458–467.
Healey, M. K. and Kahana, M. J. (2016). A four–component model of age–related memory change. Psychological Review, 123, 23-69.
Healey, M. K. and Kahana, M. J. (2020). Age-related differences in the temporal dynamics of spectral power during memory encoding. PLoS ONE, 15, e0227274.
Heaver B., & Hutton S. B. (2011). Keeping an eye on the truth? Pupil size changes associated with recognition memory. Memory, 19, 398– 405.
Heitz, R. P., Schrock, J. C., Payne, T. W., & Engle, R. W. (2008). Effects of incentive on working-memory capacity: Behavioral and pupillometric data. Psychophysiology, 45, 119-129.
Janisse, M.-P. (1977). Pupillometry: The Psychology of the Pupillary Response. Washington, D.C.: Hemisphere Publishing Co.
Johnson, D.A. (1971). Pupillary responses during a short-term memory task: Cognitive processing, arousal, or both? Journal of Experimental Psychology, 90, 311-318.
Just, M.A., & Carpenter, P.A. (1993). The intensity dimension of thought: Pupillometric indices of sentence processing. Canadian Journal of Experimental Psychology, 47, 310–339.
Kafkas A., & Montaldi D. (2011). Recognition memory strength is predicted by pupillary responses at encoding while fixation patterns distinguish recollection from familiarity. Quarterly Journal of Experimental Psychology, 64, 1971– 1989.
Kahneman, D. (1973). Attention and effort. Upper Saddle River, NJ: Prentice Hall.
Kahneman, D., & Beatty, J. (1966). Pupil diameter and load on memory. Science, 154, 1583– 1585.
Kahneman D, Peavler WS (1969) Incentive effects and pupillary changes in association learning. Journal of Experimental Psychology, 79, 312–318.
Kahneman, D., Onuska, L., & Wolman, R.E. (1968). Effects of grouping on the pupillary response in a short-term memory task. Quarterly Journal of Experimental Psychology, 20, 309-311.
Kahneman, D., Peavler, W. S., & Onuska, L. (1968). Effects of verbalization and incentive on the pupil response to mental activity. Canadian Journal of Psychology, 22(3), 186–196
Kahneman, D., & Wright, P. (1971). Changes in pupil size and rehearsal strategies in a short-term memory task. Quarterly Journal of Experimental Psychology, 23, 187-196.
Kostandyan, M., Bombeke, K., Carsten, T., Krebs, R. M., Notebaert, W., & Boehler, C. N. (in press). Differential effects of sustained and transient effort triggered by reward: A combined EEG and pupillometry study. Neuropsychologia.
Kucewicz, M., Dolezal, J., Kremen, V., Berry, B., Miller, L., Magee, A., Fabian, V., & Worrell, G. (2018). Pupil size reflects successful encoding and recall of memory in humans. Scientific Reports, 8, 4949.
Laeng, B., Sirois, S., & Gredebäck, G. (2012). Pupillometry: A Window to the Preconscious?. Perspectives on Psychological Science, 7, 18-27.
McGaugh, J. L. (2006). Make mild moments memorable: Add a little arousal. Trends in Cognitive Sciences, 10, 245-247.
Miller, A.L., & Unsworth, N. (in press). Variation in attention at encoding: Insights from pupillometry and eye gaze fixations. Journal of Experimental Psychology: Learning, Memory, and Cognition.
Miller, A.L., Gross, M.P., & Unsworth, N. (2019). Individual differences in working memory capacity and long-term memory: The influence of intensity of attention to items at encoding. Journal of Memory & Language, 104, 25-42.
Mulligan, N. W., & Lozito, J. P. (2007). Order information and free recall: Evaluating the item-order hypothesis. Quarterly Journal of Experimental Psychology, 60, 732–751.
Murdock B.B. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64, 482–488.
Murdock, B. B. (1965). Effects of a subsidiary task on short term memory. British Journal of Psychology, 56, 413-419.
Naber, M., Frassle, S., Rutishauser, U., & Einhauser, W. (2013). Pupil size signals novelty and predicts later retrieval success for declarative memories of natural scenes. Journal of Vision, 13, 1-.20.
Naveh-Benjamin, M. & Jonides, J. (1984). Maintenance rehearsal: A two-component analysis. Journal of Experimental Psychology: Learning, Memory and Cognition, 10, 369-385.
Otero S. C., Weekes B. S., & Hutton S. B. (2011). Pupil size changes during recognition memory. Psychophysiology, 48, 1346–1353.
Page, M.P.A., & Norris, D. (1998). The primacy model: a new model of immediate serial recall. Psychological Review, 105, 761–781.
Papesh M.H., & Goldinger, S.D. (2015). Pupillometry and Memory: External Signals of Metacognitive Control. In Gendolla, G.H.E, Tops, M., & Koole, S. (Eds.), Handbook of Biobehavioral Approaches to Self-Regulation. NY: Springer. Pp. 125-139.
Papesh, M.H., Goldinger, S.D., & Hout, M.C. (2012). Memory strength and specificity revealed by pupillometry. International Journal of Psychophysiology, 83, 56-64.
Peavler, W.S. (1974). Pupil size, information overload, and performance differences. Psychophysiology, 11, 559-566.
Phaf, R.H., & Wolters, G. (1993). Attentional shifts in maintenance rehearsal. American Journal of Psychology, 106, 353-382..
Rohrer, D., & Wixted, J.T. (1994). An analysis of latency and interresponse time in free recall. Memory & Cognition, 22, 511 – 524.
Sederberg, P. B., Gauthier, L. V., Terushkin, V., Miller, J. F., Barnathan, J. A., and Kahana, M. J. (2006). Oscillatory correlates of the primacy effect in episodic memory. NeuroImage, 32, 1422–1431.
Serruya, M. D., Sederberg, P. B., and Kahana, M. J. (2014). Power shifts track serial position and modulate encoding in human episodic memory. Cerebral Cortex, 24, 403–413.
Stoff, D.M., & Eagle, M.E. (1971). The relationship among reported strategies, presentation rate, and verbal ability and their effects on free recall learning. Journal of Experimental Psychology, 87, 423-428.
Tulving, E. (2007). On the law of primacy. In M. A. Gluck, J. R. Anderson, & S. M. Kosslyn (Eds.), Memory and Mind: A Festschrift for Gordon H. Bower (pp. 31-48). New Jersey: Lawrence Erlbaum Associates.
Tulving, E., & Rosenbaum, R. S. (2006). What do explanations of the distinctiveness effect need to explain? In R. R. Hunt, & J. B. Worthen (Eds.), Distinctiveness and Memory (pp. 407-423). New York, NY: Oxford University Press.
Unsworth, N. (2016). Working memory capacity and recall from long-term memory: Examining the influences of encoding strategies, study time allocation, search efficiency, and monitoring abilities. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42, 50-61.
Unsworth, N. & Robison, M.K. (2015). Individual differences in the allocation of attention to items in working memory: Evidence from pupillometry. Psychonomic Bulletin & Review, 22, 757-765.
Unsworth, N. & Robison, M.K. (2016). Pupillary correlates of lapses of sustained attention. Cognitive, Affective, Behavioral Neuroscience, 16, 601-615
Unsworth, N. & Robison, M.K. (2018). Tracking working memory maintenance with pupillometry. Attention, Perception, & Psychophysics, 80, 461-484.
Ward, G. (2002). A recency-based account of the list length effect in free recall. Memory & Cognition, 30, 885-892.
Võ M. L., Jacobs A. M., Kuchinke L., Hofmann M., Conrad M., & Schacht A. (2008). The coupling of emotion and cognition in the eye: Introducing the pupil old/new effect. Psychophysiology, 45, 130– 140.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
(a) Change in pupil diameter during list presentation in the 2-s presentation-duration condition. (b) Change in pupil diameter during list presentation in the 4-s presentation-duration condition. (c) Change in pupil diameter during list presentation in the 8-s presentation-duration condition. Numbers reflect serial position of each presented word. Data from Experiment 2
(a) Change in pupil diameter during list presentation in the list-length 10 condition. (b) Change in pupil diameter during list presentation in the list-length 15 condition. (c) Change in pupil diameter during list presentation in the list-length 20 condition. Numbers reflect serial position of each presented word. Data from Experiment 3a
(a) Change in pupil diameter during list presentation in the list-length 10 condition. (b) Change in pupil diameter during list presentation in the list-length 20 condition. (c) Numbers reflect serial position of each presented word. Data from Experiment 3b
Rights and permissions
About this article
Cite this article
Unsworth, N., Miller, A.L. Encoding dynamics in free recall: Examining attention allocation with pupillometry. Mem Cogn 49, 90–111 (2021). https://doi.org/10.3758/s13421-020-01077-7
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13421-020-01077-7