Intuition and metacognition: The effect of semantic coherence on judgments of learning

Abstract

The idea that two distinct modes of thought affect human cognition and behavior has received considerable attention in psychology. In the domain of metacognition, it is assumed that metacognitive judgments are based on both nonanalytic, experience-based processes and analytic, theory-based processes. This study examined whether the experience-based process of intuition underlies people’s predictions of their future memory performance (judgments of learning; JOLs). In four experiments, people made JOLs and took a test on compound remote associates, that is, groups of 3 words that were either remote associates of a single solution word (coherent triads) or had no common associate (incoherent triads). Previous research has shown that increased fluency of processing coherent triads produces brief positive affects that may underlie judgments. In all experiments, JOLs were higher for coherent than for incoherent triads. The same was true for recognition memory and free recall performance. Moreover, Experiments 2 and 3 demonstrated that coherent triads were processed more fluently (i.e., read more quickly) than incoherent triads. Finally, Experiments 3 and 4 showed that the effect of semantic coherence on JOLs occurred for participants who were aware and unaware of relations between all three triad words, but was more pronounced for aware participants. In sum, this study demonstrates that intuition impacts JOLs over and above theory-based processes.

The idea that two distinct modes of thought affect human cognition and behavior has received considerable attention in psychology (reviewed in Koriat, Bjork, Sheffer, & Bar, 2004). In the domain of metacognition, it is assumed that metacognitive judgments are based on both nonanalytic, experience-based processes and analytic, theory-based processes (see, e.g., Kelley & Jacoby, 1996; Koriat & Levy-Sadot, 1999). Consistent with this dual-basis view of metacognition, research has shown that people’s predictions of their future memory performance (judgments of learning; JOLs) rely on fluency of processing (e.g., Benjamin, Bjork, & Schwartz, 1998; Koriat & Ma’ayan, 2005; Undorf, Zimdahl, & Bernstein, 2017). However, it has also been proposed that JOLs mainly rely on theory-based processes (e.g., Dunlosky & Tauber, 2014; Mueller, Dunlosky, & Tauber, 2016; Mueller, Dunlosky, Tauber, & Rhodes, 2014).

According to the dual-basis view of metacognition, JOLs should be affected by intuition, which we define as feelings resulting from cognitive processes that operate outside awareness (cf. Topolinski & Strack, 2009a). To test this prediction, we investigated JOLs for compound remote associates, as they are often used in research on intuition (e.g., Bolte, Goschke, & Kuhl, 2003; Bowers, Regehr, Balthazard, & Parker, 1990). Compound remote associates are groups of three words that are remote associates of a single solution word (coherent triads; e.g., silk–cream–even, solution: smooth) or have no common associate (incoherent triads; e.g., deck–stool–pocket). People are able to intuitively judge whether triads are coherent without retrieving the solution word after approximately 1.5 s (Bolte & Goschke, 2005). Conversely, retrieving solution words usually takes more than 2 s (Bolte & Goschke, 2005). Presumably, increased fluency of processing coherent triads produces brief positive affects that may underlie judgments (e.g., Topolinski & Strack, 2009b).

Although no prior JOL study has used coherent and incoherent triads (but see Ackerman & Zalmanov, 2012, for a study of metacognition in problem solving), research on the effect of pair relatedness on JOLs is relevant. This work demonstrated that JOLs are higher for related than for unrelated paired associates (cf. Mueller, Tauber, & Dunlosky, 2013). Studies by Koriat and Bjork (2005) revealed that JOLs are also sensitive to subtle differences in relatedness: In asymmetrically associated pairs, JOLs were (at least numerically) higher when the dominant association was from the cue to the target rather than from the target to the cue. Both experience-based processes (i.e., processing fluency) and theory-based processes (i.e., the belief that memory is better for related pairs) drive the relatedness effect on JOLs (Undorf & Erdfelder, 2015). In all studies of the relatedness effect, materials differed in actual pair relatedness. Conversely, pairwise relatedness does not differ in coherent and incoherent triads (Bolte & Goschke, 2005).

In this study, people made JOLs and completed a memory test for coherent and incoherent triads. Participants were never informed about semantic coherence or common associates, and short presentation times ensured that they could not retrieve solution words. If, as predicted by the dual-basis view of metacognition, intuition affects metacognition, JOLs should be higher for coherent than for incoherent triads. If, however, intuition plays a minor role in metacognition, JOLs should be the same for coherent and incoherent triads. Based on prior research, we expect better memory performance for coherent than for incoherent triads (cf. Topolinski & Strack, 2008).

Experiment 1

In Experiment 1, participants saw coherent and incoherent triads for 1.8 s each. Immediately after the presentation of each triad, participants made a JOL.

Method

Participants and materials.

Participants were 45 University of Mannheim undergraduates. Stimuli were taken from the Horr, Braun, Zander, and Volz (2015) set of coherent and incoherent triads. From this set, we randomly selected 72 incoherent triads and 72 coherent triads that had been solved by less than 50 % of participants within 6 s (range: 1.80 %–49.60 %, M = 25.34 %, SD = 12.85). An effort was made to avoid including individual words in more than one triad. However, this was not always possible: 48 words were included in two triads and 10 words were included in three triads. The percentage of words presented in more than one triad did not differ across coherent and incoherent triads (coherent triads: 28.24 %, incoherent triads: 31.02 %, t < 1).

We constructed two study lists of 26 coherent and 26 incoherent triads and two practice lists of 10 coherent and 10 incoherent triads. Lists were comparable with respect to probabilities of solving coherent triads and percentages of words presented in more than one triad. In the study phase, each participant saw one practice and one study list, and half the participants saw each list. Practice lists were excluded from the analysis.

Procedure.

At the outset of the experiment, participants were told that the experiment concerned people’s perceptions of groups of words. They were instructed to read 72 groups of three words and to judge, for each group of words, how easy it would be to recognize at a later time. They were not told, however, that they were to remember the triads or that there would be a memory test. On every trial, a fixation stimulus (“!!!”) appeared on-screen for 500 ms. Immediately afterwards, a triad was presented for 1,800 ms, followed by a blank interval of 350 ms. Then the JOL prompt “The ease of recognizing?” appeared above a 10-point Likert scale ranging from 1 (very difficult) to 10 (very easy). Participants entered their JOL using labeled keys (the keys 1 to 0 were labeled 1 to 10). The next trial followed immediately. After the study phase, participants performed a filler task for 5 min (visual search). They then received a surprise recognition test that included the 72 studied and 72 new triads. Triads were presented sequentially, and participants made recognition judgments by clicking on buttons labeled “old” or “new.” In study and test phases, triads appeared in a random order with the restriction that practice lists preceded study lists.

Results and discussion

Table 1 shows JOLs, hit rates, false alarm rates, corrected hit rates P _r (hits minus false alarms; Snodgrass & Corwin, 1988), and bias index B _r (false alarms/[l - (hits + false alarms)]; Snodgrass & Corwin, 1988) for coherent and incoherent triads. JOLs were higher for coherent than for incoherent triads, t(44) = 10.02, p < .001, d = 1.51. The same was true for P _r and B _r, P _r: t(44) = 4.53, p < .001, d = 0.68; B _r: t(43) = 3.69, p = .001, d = 0.56 (B _r could not be calculated for one participant due to a false alarm rate of zero); results were unchanged when d’ and c were used to measure sensitivity and response bias. For completeness, we report correlations between JOLs and memory performance for this and the following experiments in Appendix A.

Table 1 Descriptive statistics for Experiments 1 and 2

Experiment 1 revealed that semantic coherence increased JOLs and memory performance. This finding shows that intuition affects JOLs and thus supports the dual-basis view of metacognition.

Experiment 2

Prior research indicated that processing fluency is higher for coherent than for incoherent triads. However, we did not measure processing fluency in Experiment 1. As this may be a possible concern (e.g., Mueller et al., 2016; Undorf & Erdfelder, 2011), we used reading time as a measure of fluency in Experiment 2 (cf. Topolinski & Strack, 2009a).

Method

Participants and materials.

Participants were 43 University of Mannheim undergraduates. Stimuli were the same as in Experiment 1.

Procedure.

The procedure was identical to that of Experiment 1 except that participants read the triads at their own pace. Prior to the study phase, participants were instructed to read each triad and then press the space bar. They were asked to read quickly and not to spend too much time thinking. If the response time on a trial exceeded 2.2 s, a 1,000-Hz warning tone was presented. The tone continued until participants pressed the space bar. Instructions asked participants to adjust their reading rate so as to avoid hearing the tone. The 2.2-s time limit was chosen to provide sufficient time for participants to experience fluency, but not enough time to retrieve solution words.

Results and discussion

Trials on which reading times were shorter than 300 ms or longer than 3.000 ms (1.83 %) were removed from the analysis. As can be seen in Table 1, coherent triads were read faster than incoherent triads, t(42) = 3.39, p = .002, d = 0.52. As before, JOLs were higher for coherent triads, t(42) = 7.17, p < .001, d = 1.11. The same was true for P _r and B _r, P _r: t(42) = 2.81, p = .007, d = 0.43; B _r: t(42) = 3.29, p = .002, d = 0.51.

Experiment 2 revealed that semantic coherence increased processing fluency as measured by reading time. Again, semantic coherence increased both JOLs and memory performance. Taken together, Experiments 1 and 2 demonstrate that intuition affects JOLs and thus support the dual-basis view of metacognition.

Experiment 3

Experiment 3 had three aims. First, we tested whether a belief about pair relatedness may account for the effect of semantic coherence on JOLs. Such a belief may have played a role in Experiments 1 and 2, because rated pairwise relatedness between triad words was higher for coherent than for incoherent triads (cf. Zander, Horr, Bolte, & Volz, 2015). In Experiment 3, we used coherent and incoherent triads that did not differ in pairwise relatedness. Second, we investigated whether theory-based processes contribute to the semantic coherence effect on JOLs. To this end, a postexperimental inquiry tested whether participants were consciously aware of relations between all three triad words. Notably, the dual-basis view is consistent with theory-based processes contributing to the semantic coherence effect. However, if the semantic coherence effect were solely due to theory-based processes, this effect would not provide support for the idea that intuition affects JOLs. Finally, awareness of relations reported in a postexperimental inquiry following a recognition test may either stem from the JOL phase or from the recognition phase. If, however, no triads were presented at test, awareness of relations should stem from the JOL phase. Therefore, in Experiment 3, one group of participants completed a free recall test, while another group completed a recognition test, as in the previous experiments. Consequently, Experiment 3 also explored whether semantic coherence increases recall performance.

Method

Participants and materials.

Participants were 110 University of Mannheim undergraduates. We omitted data from two participants whose reading times were longer than 3,000 ms for more than 71.15 % of the triads. The remaining participants were randomly assigned to either the recognition condition (n = 52) or to the recall condition (n = 56). Stimuli were 64 coherent and 64 incoherent triads taken from the Horr et al. (2015) set. As shown in Table 2, coherent and incoherent triads were equated on pairwise relatedness between triad words and several word characteristics. Fifty-six individual words were included in more than one triad (coherent triads: 27.60 %, incoherent triads: 29.69 %, t < 1).

Table 2 Means (and standard deviations) of pairwise relatedness and word characteristics for triad words in Experiments 3 and 4, separately for coherent and incoherent triads

We constructed two study lists of 26 coherent and 26 incoherent triads and two practice lists of six coherent and six incoherent triads. Lists were equated for probabilities of solving coherent triads, percentages of words included in two triads, pairwise relatedness, and the word characteristics listed in Table 2. Within each study list, no word was included in two triads.

Procedure.

For the recognition group, the procedure was identical to that of Experiment 2. The only difference between groups was that, at test, participants from the recall group were asked to write down as many triad words as they could remember, in any order. They were given 5 min for free recall.¹ Following the memory test, we asked a series of questions of graded directness to determine whether participants were aware of relations between triad words: “Did you notice systematic differences between word groups?” and “Were there relations between all three words of some word groups?” (If so: “On what proportion of trials?” and “Please describe these relations.”).

Results and discussion

Trials on which reading times were shorter than 300 ms or longer than 3,000 ms (0.94 %) were removed from the analysis. In the postexperimental inquiry, 34 of 52 participants from the recognition group and 25 of 56 participants from the recall group reported that they had noticed relations between all three triad words. Their descriptions of relations revealed that all of them had an understanding of coherence. For example, participants reported that words had one topic, belonged to each other, or described a scene. We refer to participants who noticed relations between all three triad words as aware of relations and to the remaining participants as unaware of relations. Participants from the recognition group were aware of relations more often than participants from the recall group, χ²(1) = 4.68, p = .031.

Table 3 shows descriptive statistics for all participants and separately for participants who were aware and unaware of relations. Reading time was submitted to a mixed three-way ANOVA with coherence (coherent, incoherent) as a within-subjects factor and test type (recognition, recall) and awareness (aware, unaware) as between-subjects factors. A significant effect of coherence revealed that coherent triads were read faster than incoherent triads, F(1, 104) = 9.58, p = .003, η_p ² = .08. No other effects were significant, all Fs < 1.81. A 2 (coherence) × 2 (test type) × 2 (awareness) ANOVA on JOLs revealed higher JOLs for coherent than for incoherent triads, F(1, 104) = 195.51, p < .001, η_p ² = .65, and higher JOLs in the recall than in the recognition group, F(1, 104) = 5.91, p = .017, η_p ² = .05. A significant interaction between awareness and coherence revealed that the effect of semantic coherence was particularly pronounced for participants who were aware of relations, F(1, 104) = 7.60, p = .007, η_p ² = .07. No other effects were significant, all Fs < 1. The main effect of test type on JOLs was unexpected. It is probably spurious, because the procedures were identical until the test phase.

Table 3 Descriptive Statistics for All Participants and Separately for Participants Who Were Aware and Unaware of Relations Between Triad Words in the Recognition and Recall Groups of Experiment 3

In the recognition group, two separate 2 (coherence) × 2 (awareness) ANOVAs on P _r and B _r revealed that both measures were higher for coherent than for incoherent triads, P _r: F(1, 50) = 5.70, p = .021, η_p ² = .10; B _r: F(1, 49) = 6.87, p = .012, η_p ² = .12 (B _r could not be calculated for one participant due to a false alarm rate of zero). No other effects were significant, all Fs < 1. In the recall group, a 2 (coherence) × 2 (awareness) ANOVA on recall performance revealed better recall for coherent than incoherent triads, F(1, 53) = 12.38, p < .001, η_p ² = .19. No other effects were significant, all Fs < 1. Relatively low levels of recall were probably due to the fact that participants were presented with many more words than in typical free recall experiments with JOLs (see, e.g., Mueller et al., 2014).

In Experiment 3, semantic coherence increased fluency as measured by reading time, JOLs, recognition memory, and recall performance. Beliefs about pair relatedness could not account for the semantic coherence effect on JOLs, because pairwise relatedness was the same for coherent and incoherent triads. The semantic coherence effect occurred for participants who were aware and unaware of relations between triad words, but it was more pronounced for aware participants. This pattern of results indicates that both intuition and theory-based processes underlie the semantic coherence effect on JOLs. It thus supports the dual-basis view of metacognition.

Experiment 4

Experiment 3 revealed that theory-based processes contributed to the semantic coherence effect on JOLs. In Experiment 4, we investigated whether this finding depended on participants having ample time to process triads when making JOLs. To this end, we asked people to read triads and make JOLs within the time limit for reading triads in Experiments 2 and 3. Moreover, we extended the postexperimental inquiry to test whether awareness of differences in ease of reading may contribute to the semantic coherence effect on JOLs.

Method

Participants and materials.

Participants were 52 University of Mannheim undergraduates. We omitted data from one participant whose response times were longer than 3,000 ms for all triads. All participants were presented with one study and one practice list from Experiment 3.

Procedure.

The procedure was similar to that of Experiment 3 with the following exceptions. The JOL prompt appeared on the screen simultaneously with the triad. Participants were instructed to read each triad and then make a JOL. The warning tone was presented if the response time on a trial exceeded 2.2 s. All participants completed a free recall test. We used the same postexperimental questionnaire as in Experiment 3 except that we added the question: “Did word groups differ in ease of reading?”

Results and discussion

Trials on which response times were shorter than 300 ms or longer than 3,000 ms (8.63 %) were removed from the analysis. In the postexperimental inquiry, 20 of 51 participants reported having noticed relations between all three triad words. On average, it took participants less than 1.70 s to read the triads and make a JOL (coherent triads: M = 1.68 s, SD = 0.40, incoherent triads: M = 1.68, SD = 0.37).

Descriptive statistics are shown in Table 4. A mixed two-way ANOVA on JOLs with coherence (coherent, incoherent) as a within-subjects factor and awareness (aware, unaware) as a between-subjects factor revealed higher JOLs for coherent than for incoherent triads, F(1, 49) = 12.20, p = .001, η_p ² = .20. A significant interaction revealed that the effect of semantic coherence was particularly pronounced for participants who were aware of relations between triad words, F(1, 49) = 4.20, p = .046, η_p ² = .08. There was no main effect of awareness, F(1, 49) = 4.03, p = .050. A 2 (awareness) × 2 (coherence) ANOVA on recall performance revealed better recall for coherent than for incoherent triads, F(1, 49) = 6.89, p = .012, η_p ² = .12, and better recall for participants who were aware of relations, F(1, 49) = 14.27, p < .001, η_p ² = .23. A significant interaction revealed that semantic coherence increased recall only in participants who were aware of relations, F(1, 49) = 11.93, p = .001, η_p ² = .20.

Table 4 Descriptive Statistics for All Participants and Separately for Participants Who Were Aware and Unaware of Relations Between Triad Words in Experiment 4

In the postexperimental inquiry, 35 of 51 participants reported that triads differed in ease of reading. The effect of semantic coherence on JOLs and recall performance was the same for participants who were aware and unaware of differences in ease of reading (see Appendix B for detailed analysis).

Experiment 4 demonstrated that semantic coherence increased JOLs and recall performance. As in Experiment 3, the semantic coherence effect on JOLs occurred for participants who were aware and unaware of relations between triad words, but was more pronounced for aware participants. Conversely, awareness of differences in ease of reading did not influence the semantic coherence effect on JOLs. Taken together, Experiments 3 and 4 indicate that both intuition and theory-based processes underlie the semantic coherence effect on JOLs. They thus support the dual-basis view of metacognition.

General discussion

These experiments examined whether metamemory is affected by intuition, which we define as feelings resulting from cognitive processes that operate outside awareness. We presented groups of three words that were either remote associates of a single solution word (coherent triads) or had no common associate (incoherent triads). Importantly, participants were not told about semantic coherence or common associates. Prior research showed that increased fluency of processing coherent triads produces brief positive affects that may underlie judgments (e.g., Topolinski & Strack, 2009b) and that coherent and incoherent triads do not differ in pairwise relatedness between triad words (Bolte & Goschke, 2005).

In all four experiments, JOLs were higher for coherent than for incoherent triads. The same was true for recognition memory and recall performance. Experiments 2 and 3 demonstrated that coherent triads were processed more fluently (i.e., read more quickly) than were incoherent triads. Finally, Experiments 3 and 4 showed that the semantic coherence effect on JOLs was particularly pronounced for participants who were aware of relations between triad words.

Taken together, this study revealed an effect of semantic coherence on JOLs. Furthermore, it suggested that intuition alone can produce this effect. At the same time, the semantic coherence effect was larger when theory-based processes also contributed. These results converge with findings on the effect of pair relatedness on JOLs (Undorf & Erdfelder, 2015). At a theoretical level, the current research supports the dual-basis view of metacognition according to which both experience-based and theory-based processes underlie JOLs (e.g., Kelley & Jacoby, 1996; Koriat, 1997; Koriat & Levy-Sadot, 1999).

On the basis of the present findings, it is unlikely that the semantic coherence effect on JOLs is entirely due to theory-based processes. To begin with, it was not due to deliberate applications of theories about how memory is affected by pairwise relatedness, word length, word frequency, concreteness, and ease of reading. Of course, other theories may play a role. Moreover, the semantic coherence effect occurred for participants who were aware and unaware of relations between triad words. Two observations suggest that our measure of theory-based processes (i.e., participants’ reports of noticing relations between all three triad words) was valid: First, aware participants’ descriptions of relations showed that they had an understanding of coherence. Second, taking a recall rather than a recognition test reduced awareness of relations, as did time pressure when making JOLs. Nevertheless, it may be worth using other measures of theory-based processes in future studies.

In sum, this work demonstrated that semantic coherence affected JOLs through intuition and theory-based processes. This semantic coherence effect may be well suited to investigate how nonanalytic, experience-based and analytic, theory-based processes combine in metacognition.

Appendices

Appendix A

Table 5 shows within-person gamma correlations between JOLs and memory for Experiments 1–4. For recognition groups (Experiments 1, 2, and 3), memory refers to old responses to studied words. For recall groups (Experiments 3 and 4), memory refers to correctly recalled words. For Experiments 2 and 3, the table also shows correlations between JOLs and reading time and between reading time and memory performance. For Experiments 3 and 4, correlations are presented for all participants and separately for participants who were aware and unaware of relations between triad words. Note that gamma correlations could not be calculated for some participants in Experiment 3 due to insufficient variability in JOLs or memory performance.

Table 5 Means (and standard deviations) of gamma correlations in Experiments 1–4

Across all conditions, correlations between JOLs and memory performance were significantly positive, correlations between JOLs and reading time were significantly negative, and correlations between reading time and memory performance did not differ from zero.

Appendix B

Table 6 shows JOLs and recall performance separately for participants who were aware and unaware of differences in ease of reading. To test whether awareness of differences in ease of reading impacted the semantic coherence effect, JOLs were submitted to a mixed two-way ANOVA with coherence (coherent, incoherent) as a within-subjects factor and awareness (aware, unaware) as a between-subjects factor. A significant effect of coherence revealed higher JOLs for coherent than for incoherent triads, F(1, 49) = 5.79, p = .020, η_p ² = .11. No other effects were significant, all Fs < 3.00. A 2 (coherence) × 2 (awareness) ANOVA on recall performance revealed better recall for coherent than for incoherent triads, F(1, 49) = 4.24, p = .045, η_p ² = .19. No other effects were significant, all Fs < 1.37. These results reveal that awareness of differences in ease of reading did not contribute to the semantic coherence effect on JOLs.

Table 6 Descriptive Statistics for Participants Who Were Aware and Unaware of Differences in Ease of Reading in Experiment 4