Theories of metacognitive monitoring emphasize that the accuracy of judgments about future memory depends on the accessibility and the diagnosticity of the cues that are accessed for future memory experiences (e.g., Dunlosky & Matvey, 2001; Dunlosky & Metcalfe, 2009; Koriat, 1993, 1997; Koriat & Bjork, 2006). Different kinds of cues are likely to be accessed at different stages of the process of learning and remembering (e.g., Finn & Metcalfe, 2008). T. O. Nelson and Narens (1990) proposed a framework of metacognitive monitoring and control involving three basic phases of learning and remembering: acquisition (or encoding), retention, and retrieval. Different kinds of metacognitive monitoring, made during each of these three stages, can inform control processes during those stages, such as selection of encoding strategies during acquisition or termination of search during retrieval.

The present study evaluated the sensitivity of metacognitive judgments to implicit interference effects at retrieval. Implicit interference was manipulated by varying the number of words associated with either the cue or the target, or associative set size (D. L. Nelson, McEvoy, & Schreiber, 1990). The purpose of the experiments was to examine whether metamemory predictions made at different stages in the memory process are sensitive to this kind of implicit interference. Three types of metamemory predictions were examined: immediate judgments of learning (IJOLs) made during encoding, delayed JOLs (DJOLs) made after encoding but prior to recall during the retention interval (sometimes also called predictions of knowing, or POKs; Schreiber & Nelson, 1998), and feelings of knowing (FOKs) made after attempted cued recall, or at retrieval. The rationale of the study was that implicit interference will not impact judgments at all stages of the memory process. Specifically, because implicit interference effects occur at retrieval, we expected IJOLs made at encoding to be insensitive to the effects of implicit interference, whereas DJOLs and FOKs should access retrieval outcomes that are influenced by implicit interference. In the following sections, we outline the basis of our theoretical argument after explicitly defining and describing the metacognitive judgments used in our study.

Metacognitive judgments

Metacognitive judgments are often made for paired-associate items (e.g., bird–wings), in part because of their affordance for using the same cues for judgments and testing associative memory (e.g., cuing judgments and recall by presenting bird). IJOLs are made immediately after encoding of each item. For instance, immediately after studying bird–wings, the cue bird is presented, and people report their subjective confidence that they will correctly recall wings when cued to do so on the later test. The response format used in the present study was a probability scale ranging from 0 (certain not to recall the target) to 100 (certain to recall the target).

DJOLs are predictions that are made during the retention interval between encoding and retrieval. DJOLs can be made with a very short delay (e.g., T. O. Nelson & Dunlosky, 1991; Weaver & Kelemen 2003) or, as in this study, during a separate DJOL collection phase after all items have been studied. Otherwise, DJOLs have a similar format and response scale as IJOLs.

FOKs are predictions made after a cued-retrieval attempt, either immediately after the recall attempt or separately in a later block (Hart, 1965; MacLaverty & Hertzog, 2009; T. O. Nelson & Narens, 1990; Schacter, 1983). FOKs assess confidence about future recognition of the target that was paired with the studied cue. In the present study, after recall, item cues were presented in random order, and FOKs were made using the same probability scale as JOLs and DJOLs.

Impact of implicit interference on memory

The question posed by these experiments was whether the three types of metamemory judgments are impacted in the same way by implicit interference. In the present study, implicit interference refers to the number of associates of a given cue or target, or associative set (D. L. Nelson et al., 1990). Words vary in terms of the number of words that are associated to them; some words have a relatively small associative set (e.g., petals, 3), whereas some have a relatively large set (e.g., flower, 21). Implicit interference arises from competition among associates of either the cue or the target of a paired-associate item (e.g., flower–rose) during retrieval. Words that have a large set size evoke more implicit interference than do those with a small set size. When the target (e.g., rose) is studied alone and then cued with an associate via extralist cuing, recall is lower given a large-set-size cue (e.g., flower) than it would be given a small-set-size cue (e.g., petals). Implicit interference is greater given a large- than given a small-set-size cue or target (Eakin & Hertzog, 2006; D. L. Nelson & McEvoy, 1979; D. L. Nelson et al., 1990). Both cue- and target-set-size effects are eliminated when the cue–target pair is studied together via intralist cuing, regardless of whether the cue has a small or a large number of associates (Eakin & Hertzog, 2006; D. L. Nelson & McEvoy, 1979; D. L. Nelson et al., 1990).

According to the processing implicit and explicit representations model (PIER2; D. L. Nelson, McKinney, Gee, & Janczura, 1998), encoding a word generates both (1) an explicit episodic representation of the word and (2) implicit activation of related word concepts. Retrieval success during extralist cuing depends on sampling associated candidates that compete for retrieval; large-set-size cues and targets result in more interference relative to small-set-size cues and targets, resulting in lower probability of recall or set size effects. This interference is implicit because people are unable to identify which words have a small or large associative set. Under intralist cuing, both the cue and target provide meaning context, and only the intersecting set of associates of both words are activated when they are studied together. The result is an effectively equal sampling set for small- and large-set size cues and targets and, therefore, equivalent recall.

Implicit interference effects on metamemory

Prior research has demonstrated that metacognitive judgments can be affected by implicit interference effects. Schreiber (1998) demonstrated that both DJOLs and FOKs varied with target set size effects, mirroring implicit interference effects on recall. These effects were eliminated under intralist cuing. Likewise, Schreiber and Nelson (1998) found that both DJOLs and FOKs tracked cue set size effects in recall. Eakin and Hertzog (2006) replicated cue set size effects on DJOLs, and Eakin and Hertzog (2011) replicated the effects on FOKs. In both studies, people’s judgments accurately reflected the impact of implicit interference on recall. Taken together, these studies demonstrate that DJOLs and FOKs are apparently influenced in the same way—and in the same way as recall—by implicit interference.

Metamemory predictions made just prior to (i.e., DJOLs) or after (i.e., FOKs) retrieval are sensitive to the implicit interference from competing associates of both the cue and the target. The question we pose in the present study is whether this influence of implicit interference is theory-based prognostication (Koriat & Bjork, 2006) or a direct consequence of retrieval interference for accessibility to information about the target. We argue that this sensitivity to interference occurs because of the retrieval demands that precede both types of judgments.

Attempted retrieval has been demonstrated to have an effect on predictions at retrieval, such as DJOLs (e.g., Dunlosky & Nelson, 1994) and FOKs (e.g., MacLaverty & Hertzog, 2009). Therefore, we hypothesized that IJOLs, because they are made at encoding and prior to any retrieval attempt, would be immune from implicit interference effects. In the absence of experiencing the implicit interference effects at retrieval, it is possible that people could fail to predict the effects on memory. Although this specifically has not been tested, Schreiber (1998) showed that participants were not able to accurately identify which words had small or large associative sets. People are also not likely to have an implicit theory regarding the relationship between increased set size and increased interference. Koriat and Bjork (2005) demonstrated that IJOLs were influenced by whether the cue and target were associatively related but were also insensitive to the direction of association (forward vs. backward). Arguably, the direction of association is more readily deduced from the stimuli than is the associative set size of a particular stimulus.

Because we suspected that a word’s associative set size is neither directly manifest nor easily deduced, we expected that IJOLs would be insensitive to set size effects. Instead, the hypothesis was that metacognitive judgments would be sensitive to implicit interference only if the judgment was based on accessibility after target retrieval was attempted upon presentation of the cue. In other words, implicit interference influences the information that is accessible at the time of retrieval from long-term memory, so that metacognitive judgments that are influenced by retrieval access should, at least potentially, be sensitive to implicit interference. Conversely, IJOLs, which do not benefit from retrieval from secondary memory, should be insensitive to future implicit interference effects. Although the experiential quality of a large- versus small-set-size cue or target is available when IJOLs are made, without a retrieval attempt, the effect of many versus few associates on retrieval may not be anticipated. In contrast to IJOLs, DJOLs and FOKs are believed to be influenced by the outcomes of attempted target retrievals (Dunlosky & Nelson, 1994; MacLaverty & Hertzog, 2009; T. O. Nelson, Narens, & Dunlosky, 2004). If accessibility is the common factor for DJOLs and FOKs, then—because of the experience garnered via a retrieval attempt—both these judgments should manifest implicit interference effects.

Experiment 1 tested this hypothesis by having three groups of people study related paired associates for which the cue set size varied (small vs. large) under either extralist or intralist cuing procedures. We predicted that cue set size effects would be observed in memory when tested with extralist cuing and would be eliminated when tested with intralist cuing for all three judgment conditions. We anticipated that DJOLs and FOKs would be sensitive to cue set size effects, but that IJOLs would not.

General method

The procedure for the three experiments reported was identical; what changed were the materials used. The general procedure is described here, and unique details are provided when describing each experiment.

General procedure

The tasks were programmed using E-Prime version 1.1 (Psychology Software Tools, Inc.) and were executed on standard PCs. Figure 1 depicts the general procedure for the three experiments. All experiments manipulated prediction type (IJOL, DJOL, or FOK) and cue type (intralist or extralist cuing) as crossed between-subjects factors; participants were randomly assigned to the factors. In all conditions, participants were presented with an item to be memorized for 8 s and were instructed to encode the item using visual imagery. They pressed the space bar to indicate when the image was formed and then, after the full 8 s had passed, rated their images. The response options included vivid (clear with lots of detail), neutral (unclear and vague), or unable to form an image. A total of 44 items were presented in random order. For the intralist cuing procedure, the items were cue–target paired associates (e.g., brook–river) and were told to make predictions about and attempt retrieval of the paired target. For extralist cuing, participants viewed only the target from each paired associate during the encoding phase (e.g., river) and were told to make predictions about and attempt retrieval of the studied target that was related in meaning to the presented cue. An additional 6 items served as practice before each of the experimental phases.

Fig. 1
figure 1

The procedure for IJOLs, DJOLs, and FOKs. The bottom figure compares the procedure for the cued and no-cue IJOL extralist conditions used in Experiments 2 and 3

Full size image

IJOL procedure The IJOL procedure consisted of two phases: (1) the study-then-IJOL phase and (2) the recall phase. After the 8-s study time, the item was removed, and the cue was presented to prompt the IJOL judgment. Participants were instructed to judge the degree to which they could remember the target, given the cue. Judgments were made using a scale ranging from 0 (certain not to recall) to 100 (certain to recall); participants were encouraged to use the full range of the scale. Then participants rated the image they had formed during study, and the next item was presented.

During the recall phase, each of the cues was presented in a random order, and participants attempted recall of the associated target. Time to recall was unlimited. Participants were encouraged to try hard to recall and were permitted to guess if they were unable to do so. If unable to recall or guess, participants typed “NEXT” on the keyboard.

DJOL procedure The DJOL procedure consisted of three phases: (1) the study phase, (2) the DJOL phase, and (3) the recall phase. During the study phase, all items were studied without making any DJOL predictions. The DJOL prediction phase followed, during which each cue was presented and participants were instructed to judge the degree to which they predicted they could recall the target. DJOLs were made using the using the 0–100 scale. The recall phase was identical to that of the IJOL procedure.

FOK procedure The FOK procedure consisted of four phases: (1) the study phase, (2) the recall phase, (3) the FOK phase, and (4) the recognition phase. The study phase and recall phase procedures were identical to the DJOL procedure. The key difference between the DJOL and FOK procedures was that DJOLs were made prior to any recall attempt, and FOKs were made after the recall phase about future recognition. FOKs were made on all items and were examined separately for unrecalled items post hoc. During the FOK phase, each cue was presented, and participants were instructed to judge the degree to which they predicted they could recognize the target from among five alternatives, using the 0–100 confidence scale. A written recognition test was provided on a clipboard, along with a scantron form on which to mark responses. Each of the 44 cues was presented along with five alternatives, one of which was the associated target. Participants filled in the bubble corresponding to the alternative (a, b, c, d, or e) to indicate their response. The recognition test was self-paced. Each of the phases, except the recognition test, was preceded by a practice phase.

Experiment 1

Method

Design and participants

The design was a 2 × 2 × 3 mixed model factorial design. Cue set size (small, large) was manipulated within subjects. Cuing procedure (extralist, intralist) and prediction type (IJOL, DJOL, FOK) were manipulated between subjects. Dependent measures included probability of recall, prediction magnitude (sensitivity), and probability of recognition (FOK only). Participants were 150 undergraduates from Mississippi State University, who participated for course credit.

Materials

The key feature of the materials for Experiment 1 was that the cue varied in terms of the number of associates; 22 small-set-size cues and 22 large-set-size cues were used.

Stimulus materials A list of 44 related cue and target word pairs were created using the University of South Florida Word Association Norms (see D. L. Nelson et al., 1990, for details). Half of the word pairs had small-set-size cues (5 to 9 associates, M = 6.79, SD = 0.16), and half had large-set-size cues (16 to 24 associates, M = 19.75, SD = 0.10). We equated forward association strength (likelihood of providing target, given the cue; M = .12, SD = .03) and backward association strength (likelihood of providing cue, given the target; M = 0.03, SD = 0.01) across cue set size. The association strength between any given cue and target was relatively low; the target was never the most highly associated member of the cue’s associative set. We also equated target set size, printed word frequency (Kučera & Francis, 1967), concreteness, and connectivity (D. L. Nelson et al., 1998). In addition, we used the ListChecker Pro 1.2 program (Eakin, 2010) to ensure that each cue was related only to its intended target and not to any other target or cue on the list.

The five-alternative forced choice recognition test presented each of 44 cues (e.g., brook) along with its correct (previously studied) target (river) and four associated foils (babbling, creek, water, stream). The intended target had the strongest forward association strength with the cue from among the five alternatives only about 15% of the time. Four versions of the recognition test for each list were created, with the cue order and the order of the alternatives differing randomly across versions.

Results

Planned comparisons were conducted to examine the impact of the interaction between cue set size and cuing procedure on each judgment type. Specifically, a significant interaction between the two factors (and an examination of the means) would indicate that cue set size effects were obtained for extralist and eliminated for intralist cuing. All reported analyses were conducted using repeated measures, mixed design ANOVAs with prediction magnitude (sensitivity), probability of recall, and probability of recognition as dependent measures. The omnibus F for the full ANOVA for each planned comparison was significant. A criterion of p < .05 was required for significance in all comparisons. The mean prediction magnitudes for all judgment types for all three experiments are reported in Table 1. Probabilities of recall for all three experiments are reported in Table 2.

Table 1 Metamemory prediction magnitude for all judgment conditions for all three experiments
Full size table
Table 2 Recall performance for all judgment types for all three experiments
Full size table

DJOL results

DJOL magnitude The interaction between cue set size and cuing procedure was significant, F(1, 46) = 17.38, p < .001, η 2p = .27. Under extralist cuing, predictions were reliably higher for small-set-size (M = 62.19, SE = 2.42) than for large-set-size (M = 51.79, SE = 2.40) cues. Under intralist cuing, cue set size effects were eliminated. Mean DJOLs were similar for small-set-size (M = 80.75, SE = 2.52) and for large-set-size (M = 83.85, SE = 2.50) cues, F < 1.

DJOL recall The interaction between cue set size and cuing procedure was significant, F(1, 46) = 20.62, p < .001, η 2p = .31. Recall was higher for small-set-size (M = .52, SE = .04) than for large-set-size (M = .36, SE = .03) cues under extralist cuing. As was expected, set size effects were eliminated under intralist cuing. Recall levels were similar for small- and large-set-size cues (M = .73, SE = .04, and M = .75, SE = .03, respectively), F < 1.

FOK results

FOK magnitude The expected interaction between cue set size and cuing procedure for FOK magnitude of all items (recalled and unrecalled) was found, F(1, 49) = 18.69, p < .001, η 2p = .28, with reliable cue set size effects under extralist cuing. As had been the case for DJOLs, FOKs were higher for small-set-size (M = 73.35, SE = 2.87) than for large-set-size (M = 61.31, SE = 3.41) cues. Under intralist cuing, no set size effect was observed (small-set-size, M = 85.35, SE = 2.70; large-set-size, M = 84.53, SE = 3.22), F < 1.

FOK recall The interaction between cue set size and cuing procedure was significant, F(1, 49) = 33.43, p < .001, η 2p = .41, with cue set size effects obtained under extralist cuing and eliminated under intralist cuing. Under extralist cuing, the expected difference was found (small set size, M = .53, SE = .03; large set size, M = .41, SE = .03). Again, recall was roughly equivalent under intralist cuing for the two set size conditions (small-set-size, M = .79, SE = .03; large-set-size, M = .81, SE = .03), F < 1.

FOK recognition Recognition was better than recall (M = .85, SE = .03). However, the interaction between cue set size and cuing procedure for recognition memory was not significant, F < 1.

IJOL results

IJOL magnitude As was predicted, IJOL magnitude did not vary with cue set size for either cuing procedure , and the interaction between cue set size and cuing procedure was not significant, F < 1. JOLs were insensitive to implicit interference.

IJOL recall To our surprise, recall performance also did not vary as a function of cue set size or cuing procedure, F < 1. Cue set size effects were not obtained for either cuing procedure.

Cuing procedure results

For the DJOL and FOK conditions, significant main effects of cuing procedure on recall were also obtained, F(1, 46) = 39.25, p < .001, η 2p = .46, and F(1, 49) = 55.15, p < .001, η 2p = .53, respectively. Recall was better for intralist than for extralist cuing. This finding is typical in studies comparing the two (e.g., D. L. Nelson, McEvoy, Janczura & Xu 1993; D. L. Nelson et al., 1990, Experiment 2). However, for the IJOL condition, recall was similar for the two cuing procedures, p = .22.

Discussion

As was predicted, metamemory predictions made prior to test, after study, reflected the effects of implicit interference on memory. Cue set size effects were obtained in recall in both the DJOL and FOK conditions under extralist cuing and were eliminated under intralist cuing. In addition, both DJOLs and FOKs varied with cue set size in a manner similar to cued recall. Predictions were higher given small- than large-set-size cues under extralist cuing and equated under intralist cuing. Similar to prior research on FOKs under implicit interference effects (e.g., Eakin & Hertzog, 2011), FOKs tracked the effects on recall, rather than on recognition—the type of memory they were supposed to be predicting. Also similar to prior research, DJOLs were more accurate than IJOLs. The present results are consistent with arguments that DJOLs (but not IJOLs) are thought to be influenced in large part by target accessibility following either explicit or implicit retrieval attempts (T. O. Nelson et al., 2004). It appears that both DJOLs and FOKs are influenced by accessibility, given the influence of implicit interference upon them.

IJOLs did not vary with cue set size. Predictions were the same for small- and large-set-size cues, regardless of cuing procedure. Under the assumption that implicit interference effects arise only in the context of retrieval, this outcome was expected. However, in the IJOL condition, recall did not behave as expected; cue set size effects were not obtained for either cuing procedure. A closer examination of the procedure for IJOLs suggests a potential reason for the lack of cue set size effects in recall for that metamemory condition. According to PIER2, under extralist cuing, cue set size effects occur because the cue is not studied in conjunction with the target. Therefore, all of the associates of the cue, including the target, are part of the sampling set and compete for retrieval given the cue at recall. Cue set size effects are eliminated under intralist cuing because the cue and target are studied together, effectively reducing the sampling set to associates that are related to both the cue and the target. In the IJOL condition, the sampling set could be constrained in the extralist procedures because, although the target was presented alone under extralist cuing, its associated cue was presented immediately thereafter in order to elicit the IJOL. Therefore, the target and cue were presented in close temporal proximity, unintentionally resulting in a backward intralist cuing procedure (e.g., wings ➔ bird). This interpretation is supported by the finding that recall was better under intralist than under extralist cuing for the DJOL and FOK conditions but was the same for the two cuing procedures for the IJOL condition. Presumably, studying the target in conjunction with the cue that later prompts its recall, as occurs under intralist cuing, aids memory performance relative to studying the target in isolation, as occurs under extralist cuing. The benefit of the backward intralist cuing procedure in reducing implicit interference effects under extralist cuing also aided memory overall in that condition.

The lack of sensitivity of JOLs to implicit interference effects was rendered ambiguous given the lack of implicit interference effects on recall. In Experiment 2, a new method for collecting IJOLs was used to preserve implicit interference effects on memory, enabling a better evaluation of whether IJOLs are influenced in the same way by implicit interference as are predictions at retrieval.

Experiment 2

To avoid the backward intralist effect on memory, Experiment 2 implemented a no-cue IJOL condition, as would be obtained in free-recall JOLs (Castel, 2008). In this condition, participants made IJOLs immediately after the studied target when prompted, but without being provided with the cue word. Because the IJOL cue was not the related cue to be later used in extralist cuing, no winnowing of the target’s associative set should have occurred before extralist cuing. We predicted that set size effects on IJOLs would be observed in the no-cue IJOL condition, but not in the typical IJOL procedure. Our expectation was also that the two immediate IJOL conditions would produce equivalent IJOL magnitudes.

Method

Design and participants

The design was a 2 × 2 × 4 mixed model factorial design. Cue set size (small, large) was manipulated within subjects. Cuing procedure (extralist, intralist) and prediction type (DJOL, IJOL, FOK, no-cue IJOL) were manipulated between subjects. Dependent measures included probability of recall and prediction magnitude (sensitivity). Planned comparisons were conducted to examine the cue set size and cuing procedure interaction within each of the four prediction types for each of the relevant dependent measures. Participants were 249 undergraduate students at Mississippi State University, who participated for course credit.

Materials

The same materials as those used in Experiment 1 were used in Experiment 2.

Procedure

Experiment 2 followed the general procedure. For the no-cue IJOL condition, participants studied a word pair or target under the two cuing procedures as usual. However, under extralist cuing, they were instructed to predict their memory for the target they had seen just previously; a slide was presented after study that said “Enter your judgment from 0–100.” After making their judgment, they rated their image for vividness, and the next word pair or target was presented. For intralist cuing, after studying the cue–target pair, participants were instructed to judge their ability to remember the target they had just seen previously. Judgments were prompted in the same manner as in the extralist condition.

Results

Statistical procedures were identical to those in Experiment 1.

DJOL and FOK results

We replicated the basic pattern of results from Experiment 1 for DJOLs and FOKs. The magnitudes of both judgments were influenced by implicit interference in a manner that mirrored cued recall; implicit interference effects were obtained under extralist cuing but were eliminated under intralist cuing. In addition, recall was better under intralist than under extralist cuing.

IJOL results

The critical issue was the contrast between standard IJOLs and cue-only JOLs.

IJOL magnitude We had predicted no impact of implicit interference for IJOLs and expected no effects of cuing procedure on JOL magnitudes. The latter prediction was supported; there was not a significant relationship difference between IJOL and cue-only JOL magnitude under extralist cuing, p = .07. More critically, however, both types of IJOLs were insensitive to set size. For standard IJOLs, we found no reliable interaction between cue set size and cuing procedure, F(1, 52) = 2.25, p = .14, η 2p = .04. Likewise, no-cue IJOLs manifested no interaction, F < 1. Whether JOLs were elicited with presentation of the cue or not, IJOL magnitude did not vary with cue set size.

IJOL recall As in Experiment 1, cue set size and cuing procedure for standard IJOLs did not interact in influencing recall, F(1, 52) = 1.19, p = .66, η 2p = .004. Most critically, the interaction between cue set size and cuing procedure was significant in the new no-cue IJOL condition, F(1, 63) = 33.76, p < .001, η 2p = .35. A planned comparison of probability of recall for extralist cuing between the IJOL (M = .75, SE = .03) and no-cue IJOL (M = .53, SE = .02) conditions was significant, F(1, 55) = 38.93, p < .001, η 2p = .41. Cue set size effects on recall were obtained only under extralist cuing when a cue was not used to elicit the IJOL.

In the standard IJOL condition, memory was similar regardless of cuing procedure. However, for no-cue IJOLs, the typical advantage for intralist over extralist cuing was obtained. Thus, as hypothesized, removing the IJOL cue reinstated implicit interference effects on recall without producing a similar effect on IJOLs themselves.

Discussion

Experiment 2 demonstrated that the typical cue set size effects in recall could be obtained in the no-cue IJOL condition for which the to-be-presented cue was not used to prompt IJOLs. In addition, in the no-cue IJOL condition, memory overall was worse for extralist than for intralist cuing, a typical finding that was not obtained in the standard IJOL condition. Apparently, when the backward association between the target and cue was eliminated by removing the cue at prediction in the no-cue IJOL condition, typical implicit interference effects were obtained in recall. In contrast, when the cue was presented to prompt standard IJOLs, implicit interference effects were not obtained either in the IJOLs or in recall. Similar to Experiment 1, standard IJOLs in Experiment 2 did not vary with cue set size; nor were interference effects obtained in recall in the standard IJOL condition. The findings demonstrated that the lack of cue set size effects in memory in the standard IJOL condition was due to the backward association between the target and cue created when the cue prompted the IJOLs.

Although Experiment 2 succeeded in reinstating the implicit interference effect on recall, one can question whether the lack of cue set size effects on no-cue IJOLs is the strongest possible demonstration that IJOLs are insensitive to implicit interference effects. The cue whose set size generates implicit interference in this experiment was not present when the JOL was made in the no-cue IJOL condition. For the standard IJOL condition, the cue is present and can influence standard IJOLs, but its presence eliminates implicit interference effects at retrieval. The hypothesized pattern involves an effect on recall in one condition and no effect on judgments in another (in effect, the comparison is made between no-cue IJOL recall and standard IJOL judgments). A more compelling case for insensitivity of IJOLs to implicit interference would be to show that IJOLs are not influenced by implicit interference when the same generator of set size effects is available both at the time of the judgment and at recall. In Experiment 3, the materials were changed to create such a condition.

Experiment 3

Experiment 3 made the source of implicit interference present both at the time of the no-cue IJOL and at recall by varying the associative set size of the target, rather than the cue. In Experiments 1 and 2, the set size of the cue was manipulated while a constant target set size was maintained. In this experiment, the cue set size was maintained while the target set size was varied to create a list of small- and large-set-size targets. Target set size effects in memory are similar to cue set size effects; recall is better for small- than for large-set-size targets under extralist cuing and similar for the two under intralist cuing (e.g., D. L. Nelson, Schreiber & McEvoy, 1992). In addition, both DJOLs and FOKs have been shown to track target set size effects in memory (e.g., Schreiber, 1998). Manipulating the set size of the target within the no-cue IJOL condition allows for the potential influence of many versus few associates at prediction because the target is presented for prediction without an accompanied cue in order to avoid the backward effects of cue presentation on reducing the effective associative set (as in Experiment 2). At the same time, the stimulus with varying associative set size that generates implicit interference (i.e., the target) is still in immediate memory while making the IJOL.

Thus, the critical prediction was that the no-cue IJOL condition would generate implicit interference effects on recall without the IJOLs themselves manifesting the effect. The standard IJOL condition was also included in the experiment in order to test, as was expected, whether backward intralist cuing effects would also occur with target set size. For comparison purposes, the DJOL and FOK conditions that generated judgments sensitive to implicit interference in the previous experiments were also included.

Method

Design and participants

The design was a 2 × 2 × 4 mixed model factorial design. Target set size (small or large) was manipulated within subjects. Cuing procedure (extralist or intralist) and prediction type (IJOL, DJOL, FOK, or no-cue IJOL) were manipulated between subjects. Participants were 249 undergraduate students at Mississippi State University, who participated for course credit.

Materials

A new list of 44 related cue and target word pairs were created using the University of South Florida Word Association Norms (D. L. Nelson et al., 1990). Half of the word pairs had small-set-size targets (3 to 8 associates, M = 6.64, SD = 1.53), and half had large-set-size targets (15 to 25 associates, M = 19.82, SD = 3.14). All other factors equated on the cue set size list were also equated on the target set size list. ListChecker Pro 1.2 (Eakin, 2010) ensured that each cue was associatively related only to its intended target and not to any other target or cue on the list.

Procedure

The general procedures and those described in Experiment 2 for the no-cue IJOLs were also used in Experiment 3.

Results

Dependent measures and statistical procedures were identical to those in the previous experiments.

DJOL and FOK results

The pattern of results from Experiments 1 and 2 were obtained in Experiment 3 for DJOLs and FOKs. Target set size impacted both judgment types, and they tracked the effects in recall, whether implicit interference effects were obtained under extralist cuing or eliminated under intralist cuing. Recall was better under intralist than under extralist cuing.

IJOL results

The findings for the standard IJOLs replicated those from Experiments 1 and 2. There was no reliable interaction between target set size and cuing procedure for the judgments, F(1, 66) = 2.58, p > .10. However, the probability of recall also did not interact with target set size and cuing procedure, F < 1. There was no difference in recall overall between extralist and intralist cuing. Analogous to the first two experiments, a backward intralist effect was obtained by using the extralist cue to prompt the IJOL, eliminating target set size effects on recall that normally would be found under extralist cuing.

The critical data were generated by the no-cue IJOL condition. As was hypothesized, target set size and cuing procedure did not interact in affecting no-cue IJOLs, F(1, 53) = 1.91, p = .17. In contrast, target set size effects were obtained in recall; the interaction between target set size and cuing procedure was significant, F(1, 53) = 5.27, p = .03, η 2p = .09. The no-cue procedure avoided the backward intralist effect, so that target set size effects were obtained under extralist cuing and were eliminated under intralist cuing. In addition, the typical advantage for intralist versus extralist cuing was obtained in the no-cue IJOL condition.

Discussion

As was expected, DJOLs and FOKs tracked the impact of target set size on memory. The recall effects were smaller than previously found for cue set size in Experiments 1 and 2; target set size effects are frequently smaller than cue set size effects (e.g., D. L. Nelson et al., 1992). Regardless, they were reliable and systematic. Thus, metamemory predictions that are arguably influenced by target accessibility were impacted by implicit interference effects based on target set size.

In contrast, no-cue IJOLs were not influenced by target set size; predictions were similar for small- and large-set-size targets under both extralist and intralist cuing. Yet it was still the case that a pattern of implicit interference effects was obtained for extralist cued recall in the no-cue IJOL condition. Recall was better for small- than for large-set-size targets under extralist cuing, whereas target set size effects were eliminated under intralist cuing. Although implicit interference impacted memory, metamemory judgments made at the time of encoding did not reflect this impact.

General discussion

In the three experiments, metamemory predictions that were potentially influenced by retrieval—DJOLs and FOKs—were shown to track implicit interference effects in memory. In all three experiments, implicit interference effects were obtained in recall for both DJOLs and FOKs under extralist cuing and were eliminated for both under intralist cuing. In addition, memory was better for intralist than for extralist cuing overall. In all three experiments, standard IJOLs made at encoding failed to track implicit interference effects that are typically obtained at recall, and those effects were not obtained in recall for IJOLs. However, Experiments 2 and 3 established that the method used to measure IJOLs caused the lack of implicit interference effects in memory for that condition. Use of the extralist cue to prompt the IJOL resulted in a backward intralist cuing procedure, thereby eliminating the typical cue set size effects obtained under extralist cuing.

In Experiments 2 and 3, implicit interference did occur for extralist cued recall, but no-cue IJOLs did not predict this interference. Although the qualia that result in implicit interference were present when the cue (Experiment 2) or the target (Experiment 3) was presented in the standard IJOL condition, the judgments did not vary with associative set size, regardless of cuing procedure. These outcomes corroborated the hypothesis motivating this study—namely, that IJOLs are insensitive to implicit interference effects during retrieval, whereas DJOLs and FOKs are sensitive to those effects.

This study adds to our understanding of the qualitative differences between metacognitive judgments arising at study versus those that occur during retention or test. IJOLs at the initial study opportunity can be influenced by a number of different sources of information, such as stimulus characteristics (e.g., Koriat, 1997), encoding fluency (e.g., Hertzog, Dunlosky, Robinson, & Kidder, 2003), retrieval fluency of aspects of the cue, and so on (e.g., Benjamin, Bjork, & Schwartz, 1998; Castel, 2008; Finn & Metcalfe, 2008; Koriat & Bjork, 2006; Kornell & Bjork, 2009). However, when people make IJOLs, apparently they are not influenced by implicit retrieval interference or the variables that produce that interference. Even though, in principle, a variable like cue set size or target set size is computable from observed characteristics during study, observers apparently do not do so. Hence, we conclude that IJOLs are not influenced by variables that lead to implicit interference when a retrieval search for target information is generated by a request for a metacognitive judgment. Phenomena that arise only during retrieval access, including implicit interference effects, do not influence IJOLs. Only judgments that can be influenced by qualia generated by target feature accessibility are influenced by implicit interference. Admittedly, the present research demonstrates the dissociation of implicit interference effects on IJOLs and DJOLs without directly demonstrating the sources of information people actually used to make IJOLs in this experimental context.

In summary, when people make DJOLs and FOKs, their predictions are influenced by the implicit interference produced by a larger number of associates for large-set-size cues and targets than for those with a small associative set, unless the set is reduced by the presence of a related cue or target during intralist cuing. Conversely, IJOLs are not influenced by stimulus properties that lead to implicit interference at retrieval even under extralist cuing. The information people access when making IJOLs is not based on the same PIER2-type sampling of the target set. Nor, apparently, do people infer the effect of varying cue set size or target set size when making their IJOLs.