Look it up: Online search reduces the problematic effects of exposures to inaccuracies

Abstract

People often reproduce information they read, which is beneficial when that information is accurate. Unfortunately, people are also often exposed to inaccurate information, with subsequent reproductions allowing for problematic decisions and behaviors. One empirically validated consequence of exposures to inaccuracies is that after reading falsehoods, participants are more likely to make errors answering related questions than if they previously read accurate statements, particularly for unfamiliar information. Interventions designed to attenuate these reproductions are often ineffective, at least as studied in tasks that restrict participants to generating answers based on text content and relevant prior knowledge. In the real world, however, people have access to outside resources to evaluate information. In three experiments, we tested whether affording the option to search for relevant online information following exposure to inaccurate statements would reduce reproductions of those inaccuracies on a post-reading task. Participants given the opportunity to search for information were less likely to reproduce inaccurate information and more likely to produce correct responses, in comparison to the performance of participants who were not allowed to search. We also tested whether warnings about potentially inaccurate information would encourage searches and inform responses. While warnings increased searching, additional reductions in inaccurate reproductions were not observed. Given the contingencies of many lab tasks, reproductions of inaccurate information might be overestimated. Resources available in the real world can offer useful supports for reducing the influence of and uncertainty associated with inaccurate exposures, consistent with contemporary accounts of memory and comprehension.

Introduction

Reading accurate information can help us develop useful understandings of the world. Unfortunately, not everything we read is accurate, and taking up such information can have problematic consequences, instantiating uncertainty and affording undue consideration of false ideas. Consider that people refer to previously presented inaccuracies when answering questions about events even after those ideas have been discredited (Ecker, Lewandowsky, & Tang, 2010; Johnson & Seifert, 1994; Rich & Zaragoza, 2016). Exposures to inaccurate information also increases people’s errors when answering related questions, whether that information is presented in decontextualized declarative statements or embedded in fictional stories (e.g., Fazio, Brashier, Payne, & Marsh, 2015; Gerrig & Prentice, 1991; Marsh, Meade, & Roediger, 2003). A growing body of work has sought to reduce the deleterious consequences of exposures to inaccurate information, motivated by accounts of how memory and comprehension processes underlie the reported effects. We examined whether allowing people access to online resources after exposures to inaccurate content would reduce their reproductions of inaccurate ideas. This is of theoretical interest given that previous studies, despite being derived from processing accounts, have often failed to obtain even modest reductions (e.g., Eslick, Fazio, & Marsh, 2011; Fazio & Marsh, 2008; Hinze, Slaten, Horton, Jenkins, & Rapp, 2014; Marsh & Fazio, 2006). The current project also tests whether previous findings provide valid estimates of reproductions given the ease with which people can access online resources to support comprehension.

Reproductions of inaccurate information

Empirical projects consistently demonstrate that reading inaccurate information in fictional stories negatively impacts performance on subsequent tasks querying related information. These projects typically share two key methodological features (e.g., Fazio, Barber, Rajaram, Ornstein, & Marsh, 2013; Marsh et al., 2003; Rapp, Hinze, Kohlhepp, & Ryskin, 2014a). First, participants read texts containing statements that might confirm or contradict real-world facts and assertions. Participants are not told beforehand that the texts contain inaccuracies, being asked to read the texts as they would normally on their own. Fictional texts are often used as they can be modified (e.g., in character dialogue, in descriptions of scenes and settings, or in the provision of background information) to include different kinds of information. Fiction also often contains mixtures of truths, half-truths, and inaccuracies, allowing for such manipulations to avoid standing out as obviously aberrant so as to encourage careful processing (Rapp, Hinze, Slaten, & Horton, 2014b).

As a second shared feature, participants complete post-reading tasks, on the basis of memory alone, designed to assess the subsequent influence of previously presented falsehoods. For example, when tasked with judging the veracity of statements related to previously presented information, participants make more judgment errors after having read false than accurate claims (e.g., Appel & Richter, 2007; Prentice, Gerrig, & Bailis, 1997; Rapp, et al., 2014a). Other post-reading measures make the subsequent use of inaccuracies more explicit, with participants asked to answer general knowledge questions with critical questions related to statements manipulated in the previously read stories. For example, participants might be asked to identify the medical term for the inability to sleep (i.e., insomnia) after reading about story characters discussing this concept and correctly calling it insomnia, inaccurately identifying it as narcolepsy, or not labelling the term at all. Participants are never told that these questions relate to information from the stories, in efforts to reduce response strategies and task demands. In general, participants are more likely to answer critical questions with false information after reading related inaccurate statements than after reading related accurate statements or unspecified information (Fazio et al., 2013; Marsh et al., 2003). These inaccurate reproductions emerge for ideas that, based on norming (e.g., Nelson & Narens, 1980; Tauber, Dunlosky, Rawson, Rhodes, & Sitzman, 2013), people should know are wrong, with greater rates of inaccurate reproductions for information they are unlikely to already know. The latter cases are a particular challenge given people often do not possess the relevant knowledge with which to recognize, evaluate, or reject information they do not know is wrong.

Reducing inaccurate reproductions

People’s reproductions of inaccuracies can underlie misinformed conceptualizations and decisions (Rapp & Braasch, 2014; Rapp & Salovich, 2018). Many projects have attempted, via text and task manipulations, to reduce such reproductions. This has included requiring participants to re-read stories containing falsehoods (Marsh et al., 2003), presenting stories slowly (Fazio & Marsh, 2008), highlighting inaccurate text information (Eslick, et al., 2011), and offering warnings about potential falsehoods (Marsh & Fazio, 2006). Other projects have explicitly asked participants to retrieve relevant, accurate knowledge prior to reading falsehoods (Fazio et al., 2013; Rapp, 2008), or have interrupted reading to encourage evaluation or divert attention from inaccuracies (Donovan, Theodosis, & Rapp, 2018a). These manipulations are intended to increase the likelihood people detect inaccuracies and act on them, tagging encoded information as potentially wrong or activating accurate prior knowledge to refute it. To date, though, these interventions have yielded modest to negligible reductions, and sometimes even increases, in inaccurate reproductions.

One explanation for the durability of this phenomenon is predicated by the view that reading requires the simultaneous activation of text content and prior knowledge (Albrecht & O’Brien, 1993; Kendeou & O’Brien, 2014; Kintsch, 1994). As readers encode and comprehend text, relevant background knowledge becomes activated (Kintsch, 1988; McNamara & Kintsch, 1996). This can result in competition between newly encoded traces of text content and prior knowledge (van den Broek, Rapp, & Kendeou, 2005). In some cases, recently encoded traces are privileged over prior knowledge, owing to factors that include the fluency of the presented information (Oppenheimer, 2008), the plausibility of new text statements (Hinze et al., 2014), and the familiarity of text content to the reader (Storm, 2011). These traces are especially influential when people do not possess relevant prior knowledge to detect and reject potential discrepancies. As a result, recently read information may be readily available even when it contains falsehoods. That availability makes it a likely candidate for retrieval and post-reading influence when answering related test questions.

At the same time, people can also fail to engage in effective epistemic validation of what they have read, increasing the likelihood that inaccurate information will influence post-reading activities. Validation requires the application of prior knowledge in the consideration of recently presented information (Prentice & Gerrig, 1999). In the best of cases, people possess and utilize specific, relevant knowledge to evaluate the veracity of information. But in many situations, a lack of certainty about what one knows, an outright lack of knowledge necessary for evaluating a particular idea, or interruptions in any epistemic validation can compromise evaluative processes. Recent accounts have also suggested that people often engage in shallow processing of material, reflecting lazy engagement rather than deep reasoning and scrutinization (Pennycook & Rand, 2019). People’s prior inaccurate understandings, and overconfidence in their capacity to resist inaccuracies, can additionally imperil validation activities and evaluative mindsets (Mayo, 2019; Salovich & Rapp, 2020). Perhaps it is therefore unsurprising that people often reproduce inaccurate information they have recently read.

Evaluating information in the real world

One issue, applicable to the above kinds of projects conducted in lab settings, is that in the real world people often have access to comprehension supports. Besides relying on prior knowledge, they can ask others for assistance, consult other texts, and look up information on the internet. The frequency with which these resources are sought out and their credibility undoubtedly varies, but their exclusion in previous work calls into question whether identified patterns of reproductions are effective starting points for contemplating the effects of exposures to falsehoods. Resources like web search engines are ubiquitous tools (Lai, Lee, Chen, & Yu, 2017), with users identifying search as the most important activity associated with the internet (Ozkara, Ozmen, & Kim, 2016). Online search is a fundamental part of how people learn about and understand the world (Rieh, Collins-Thompson, Hansen, & Lee, 2016). The ability to search also influences people’s confidence about what they know (Ferguson, McLean, & Risko, 2015; Fisher, Goddu, & Keil, 2015; Hamilton, McIntyre, & Hertel, 2016). Beyond confidence judgments, engaging in relevant searches should increase the availability of accurate information, helping reduce the post-reading influence of inaccuracies.

The current project specifically examined the effects of allowing participants to search for additional information as they completed a post-reading task, following exposures to inaccurate information. There are several a priori reasons to predict that allowing people to search could reduce inaccurate reproductions. Recall that recently encoded inaccuracies remain available in memory to influence subsequent decisions. Engaging in relevant searches can make accurate information, if found, also readily available to participants, which is beneficial when participants are uncertain or possess limited knowledge. Searches could also allow participants to discredit information if they find counter evidence. These benefits have already been observed when participants are encouraged to apply prior knowledge to fact-check and edit texts without the use of external resources (Brashier, Eliseev, & Marsh, 2019; Rapp, et al., 2014a). That said, offering the option to utilize outside resources does not guarantee they will be used or used effectively. Thus, any benefits might be expected only if participants actually engage in searches.

In Experiments 1 and 2, participants read stories containing potentially accurate and inaccurate information, after which they answered trivia questions with a subset of items related to those contents. The expectation, based on previous work, was that participants would reproduce information presented in the stories to answer the questions, both when that information was likely to already be known, as well as and even more so when it was not. The focus in the current study was specifically on the reproductions that have consistently been reported following inaccurate presentations. In contrast to previous projects, half of the sample was allowed to conduct online searches while answering the questions and the other half was not. We hypothesized that participants allowed to search would be less likely to reproduce inaccurate information from previously read texts, and more likely to provide accurate responses, than would participants not given the option to search. This hypothesis relied on the notion that successful searches would make accurate information available and influential over previously read inaccuracies. A related possibility is that searches might encourage evaluative mindsets making it easier for participants to doubt the information they previously read. We note though that the critical questions referred to declarative ideas that may or may not have been of interest to participants, so searches for answers to even a subset of the questions might be considered burdensome and distracting for the task. In Experiment 3 we examined whether warnings about inaccurate story content might further encourage searches. Previous work has shown little benefit of warnings for reducing reproductions, but it is unclear whether warnings, when paired with resources for effectively acting on those warnings, are useful.

These experiments allowed for testing whether lab-based examinations overestimate people’s propensity to reproduce inaccuracies given that everyday knowledge supports have usually been unavailable to participants. Previous work has certainly documented the problematic consequences of exposures to inaccuracies, but has not afforded participants the opportunity to revisit previous claims or evaluate potential responses as they might do when interacting with information outside of the lab. The current project directly tested whether access to online search, and the degree to which searches were enacted, can overcome the influence of encoded falsehoods. Observed reductions in inaccurate reproductions would also help reinforce accounts appealing to competing traces and epistemic validations as informative for considering the effects of inaccurate exposures.

Experiment 1

Participants read stories containing accurate, inaccurate, and neutral (i.e., unspecified) statements. Afterwards they completed a questionnaire with critical items relating to the statements. Half of the participants were given the option to search online during questionnaire completion, and the other half were not. We predicted that participants allowed to search would reproduce inaccurate information at lower rates and produce accurate responses at higher rates than would participants who were not allowed to search. This was predicated on the idea that searches would encourage evaluative considerations and increase the availability of accurate ideas associated with the searches, informing accurate answers to the questions.

Methods

Participants

Two hundred and thirty-one participants recruited via Amazon’s Mechanical Turk completed the study for $6. Fifteen participants were eliminated due to failure to follow instructions (i.e., inconsistent demographic responses or failing to complete all portions of the experiment), leaving 216 in the final sample. All participants identified as native English speakers. Participants ranged in age from 23 to 69 years (M = 37.58, SD = 11.03). Due to a program error, gender information was collected only for 66% of the sample. Of those 144 participants, 73 identified as male and 71 as female. Screening procedures were employed to ensure that none of the participants had completed similar experiments conducted by the researchers.

Materials

The study was presented online via Qualtrics survey software, using four short narrative stories adapted from Marsh (2004). The stories presented fictional narratives of common, real-world events and topics (e.g., a science fair, a summer job). Each of the stories contained eight general knowledge statements for a total of 32 critical statements. Statements were embedded in story dialogue and descriptions of narrative events but were not critical to the unfolding plots. Each statement was presented in one of three forms: an inaccurate form (i.e., “She was from Wilmington, the capital of Delaware”), an accurate form (i.e., “She was from Dover, the capital of Delaware”), or a neutral form (i.e., “She was from the capital of Delaware” without a critical descriptor or term specified). Statement type was counterbalanced across three versions of each story, with each participant reading 8–12 statements of each type. Statements were also counterbalanced for difficulty: Half (16) were categorized as easy (i.e., likely to be known to participants), and half as hard (i.e., unlikely to be known to participants), based on general knowledge norms (Tauber et al., 2013). Statements pertained to a variety of topics including historical events and figures, popular culture, geography, science and medicine, and sports. Sample story passages and corresponding items are presented in Table 1. The instructions tasked participants with reading each story without mentioning the statement manipulations or the possibility of inaccuracies. Of the total 367 sentences included across the four stories, only 2.2– 3.3% across conditions included inaccuracies, helping to obfuscate the purpose of the experiment.

Table 1. Sample story passages and corresponding test items

A 74-item short-answer questionnaire assessed people’s reproductions of information from the stories. Thirty-two of the items queried information presented in the 32 critical story statements, with the remaining 42 items serving as fillers. Each item consisted of a single-sentence question (e.g., “What is the capital of Delaware?”), appearing identically across conditions. Participants were instructed to type their responses into blank spaces provided for each item. The questionnaire instructions did not reference the stories in any capacity so as not to predispose readers to using information from them. The instructions differed across conditions only with respect to how participants were told to respond if they were unsure of an answer. In the search condition, participants were told they could search online if they were unsure of answers to the questions, while participants in the no-search condition were told to write “no answer” if they were unsure, and not to search for answers online.

Procedure

Participants completed the experiment in a single, 1-h session, in an environment and with a device of their choosing. They were first tasked with reading the four stories in a self-paced manner. The instructions read, “In this survey you will be presented with a series of texts to read. Click the >> button to start reading.” Stories were presented as complete texts, one at a time, in counterbalanced order. Next, participants completed word problems as a distractor. Then they were presented with the questionnaire instructions for the search or no-search condition, depending on assignment. Participants in the search condition were instructed:

“Next you will complete a general knowledge questionnaire. Please type your response to each question in the space provided. If you are unsure of an answer, you may either write ‘no answer,’ or if you wish you can search online for the correct answer. Use whatever search engine or app you like to help you. After each question, you will be asked to indicate whether or not you searched online for the answer. There is no penalty for searching for answers or indicating no answer. We simply want to see how well people can do on this task with the option to use the internet to answer the questions.”

Participants in the no-search condition, in contrast, were instructed:

“Next you will complete a general knowledge questionnaire. Please type your response to each question in the space provided. If you are unsure of an answer, please write ‘no answer.’ We are interested in what you think is the right answer. Please do not use the internet to search for answers.”

The 74 short-answer questionnaire items were then presented one at a time. Participants in the search condition were also asked to answer, “Did you look up the answer to this item?” for each of their question responses by checking boxes marked “yes” or “no.” Finally, participants completed a brief demographics questionnaire and received a debriefing.

Design

The experiment was a 3 (statement type: inaccurate, accurate, or neutral) × 2 (item difficulty: easy or hard) × 2 (condition: search or no search) design. Statement type varied within subjects with participants reading inaccurate, accurate, and neutral statements in each of the four stories presented to them. Item difficulty also varied within subjects, with half of the experimental items each participant read classified as easy and half as hard. Condition varied between subjects with half of the participants (n = 108) assigned to the search condition and the other half (n = 108) assigned to the no-search condition.

Questionnaire coding

Questionnaire responses to the experimental items (N = 6,912) were coded using four categories. Responses reproducing inaccuracies from the stories were coded as inaccurate – story (e.g., incorrectly stating that the capital of Delaware is Wilmington, with Wilmington having appeared in a story statement). This fit 6.23% of responses in the data. Responses providing an accurate answer were coded as correct – story (e.g., stating that the capital of Delaware is Dover, with Dover being correct and appearing in the story), fitting 65.74% of responses. Responses that were incorrect but did not reproduce a story inaccuracy were coded as inaccurate – other (e.g., stating that the capital of Delaware is Delaware City, which is wrong and never appeared in a story), fitting 11.18% of responses. Unspecified responses were coded as no answer (e.g., stating “I don’t know” or “no answer” to the capital of Delaware question), which fit 16.13% of responses.Footnote 1 The experiment apparatus required a response to every item, such that blank responses could not be submitted. Accordingly, all response categories were mutually exclusive and fit the entirety of the data set. Two raters independently coded half of the responses in the data set, with the remaining half coded by one rater only. Inter-rater reliability for dual-coded responses was reliably high (κ = .98), with all disagreements resolved through discussion. Table 2 presents response rates across all coded categories as a function of search.

Table 2. Response rates as a function of search in Experiment 1

Results and discussion

Online search rates

Participants in the search condition were free to search online as they saw fit, so we began by examining the extent to which they reported engaging in searches. Participants self-reported engaging in searches for critical items (M = 18.20%, SD = 17.72) more than for filler items (M = 15.78%, SD = 17.50) [t (107) = 3.24, p = .002]. Focusing on critical items specifically, individual search rates ranged from 0.00% to 65.63% of the items. We assessed search rates for critical items using a repeated-measures ANOVA, with statement type (i.e., inaccurate, accurate, or neutral) and item difficulty (i.e., easy, hard) entered as within-subjects variables. Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 3.23, p = .227. A main effect of difficulty indicated that participants were more likely to report searching for hard (M = 32.56%, SD = 32.75) than for easy items (M = 4.67%, SD = 11.24), [F (1, 106) = 127.84, MS = 12.60, p < .001, ηp2 = .54]. This suggests participants were strategic in their searches, more often seeking out unfamiliar information than double-checking what they likely already knew. This was qualified by an interaction with statement type [F (1, 105) = 5.00, MS = 0.88, p = .008, ηp2 = .04]. Participants more often conducted searches for hard items when related information had earlier appeared in neutral statements (M = 35.24%, SD = 34.86) than in accurate statements (M = 29.44%, SD = 30.35), (p = .02). Search rates for information in inaccurate statements (M = 33.01%, SD = 32.94) were no different than from either neutral or accurate statements (all ps > .27). Search rates for easy items were equitably low across levels of statement type (all ps > .10). No main effect of statement obtained (F = 1.7).

Inaccurate questionnaire responses

To assess whether opportunities for online search reduced inaccurate reproductions, we examined responses for the category of inaccurate – story using a repeated-measures ANOVA with statement type (i.e., inaccurate, accurate, or neutral) and item difficulty (i.e., easy, hard) entered as within-subjects variables. Condition (i.e., search, no search) was entered as a between-subjects variable. Response rate was the dependent variable. Simple main effects tests with Bonferroni adjustments were used to assess all interactions. Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated for this analysis, χ2(2) = 27.44, p = .000, so reported findings reflect Greenhouse-Geisser corrections.

Replicating previous work using similar materials, a significant main effect of statement obtained. Participants were more likely to reproduce inaccuracies as answers after reading related inaccurate statements (M = 11.10%, SD = 17.65) than they were to spontaneously produce those inaccurate responses after reading related accurate (M = 3.12%, SD = 8.48) or neutral statements (M = 4.63%, SD = 10.36), [F (1.78, 213) = 60.88, MS = 4.01, p < .001, ηp2 = .22]. Also replicating previous work, a main effect of difficulty was observed, with participants more likely to inaccurately answer hard (M = 11.42%, SD = 16.37) than easy items (M = 1.14%, SD = 5.50) [F (1, 214) = 329.00, MS = 23.83, p < .001, ηp2 = .61]. These effects were qualified by an interaction between statement type and difficulty [F (1.77, 213) = 27.00, MS = 1.33, p < .001, ηp2 = .11]. For hard items, participants were more likely to reproduce inaccuracies after reading inaccurate statements (M = 13.35%, SD = 20.46) than to spontaneously produce them after reading neutral statements (M = 9.23%, SD = 13.09) (p < .001); inaccurate reproductions were also more likely following both inaccurate and neutral statements than following accurate statements (M = 5.38%, SD = 11.16), (ps < .03). For easy items, participants reproduced inaccuracies more often after reading inaccurate statements (M = 2.74%, SD = 8.90) than they spontaneously generated those inaccuracies after reading accurate (M = 0.00%, SD = 0.0) or neutral statements (M = 0.32%, SD = 2.38), (ps < .001). The frequency of spontaneous inaccurate responses did not differ between accurate and neutral statements (p > .99).

An overall main effect of condition was observed, with participants in the search condition (M = 5.17%, SD = 10.97) less likely to reproduce inaccurate information than were participants in the no-search condition (M = 7.40%, SD = 15.10) [F (1, 214) = 11.01, MS = 0.83, p = .001, ηp2 = .05]. The earlier reported main effects of statement and difficulty were qualified by interactions with search condition, highlighting search benefits. Participants in the search condition (M = 8.04%, SD = 13.74) were less likely than participants in the no-search condition (M = 14.15%, SD = 20.42) to reproduce inaccuracies after reading inaccurate statements [F (1.78, 213) = 9.10, MS = 0.60, p < .001, ηp2 = .04]. Spontaneous inaccurate productions following accurate or neutral statements did not differ across conditions (all ps > .33). Participants in the search condition (M = 9.32%, SD = 13.42) were also less likely than participants in the no-search condition (M = 13.52%, SD = 18.64) to reproduce inaccurate information as answers to hard items [F (1, 214) = 9.026, MS = 0.65, p = .003, ηp2 = .40]. In contrast, rates of inaccurate reproductions for easy items were similar across conditions (p = .55). A three-way interaction between statement, difficulty, and condition was observed. Participants in the search condition reproduced fewer inaccuracies on hard items after reading inaccurate statements (M = 13.35%, SD = 15.91) than did participants assigned to the no-search condition (M = 24.75%, SD = 22.76), [F (1.77, 213) = 8.45, MS = 0.42, p < .001, ηp2 = .05]. These results indicate that the opportunity to search reduced inaccurate reproductions, particularly for topics participants were unlikely to possess sufficient background knowledge to validate independently.

Next, looking at the search condition specifically, we compared rates of inaccurate – story responses for questionnaire items that participants self-reported searching for versus items they did not look up. Participants were less likely to use inaccurate information as answers on items for which they reported searching (M = 0.84%, SD = 3.05) as compared to items for which they did not report a search (M = 9.10%, SD = 8.69), [t (76) = -7.47, p < .001].Footnote 2 This indicates that the reductions in inaccurate reproductions were not due merely to the assigned condition or a “search-based” evaluative mindset, but likely to actual engagement in search for particular items.

Correct questionnaire responses

We next completed analogous analyses for responses fitting the correct – story coding category. Mauchly's Test of Sphericity indicated the assumption of sphericity was not violated, χ2(2) = 3.23, p = .199. A main effect of statement type indicated participants produced more correct responses after reading accurate statements (M = 70.28%, SD = 31.09) than after reading inaccurate (M = 62.18%, SD = 35.52) or neutral statements (M = 65.04%, SD = 34.67), [F (1, 213) = 23.95, MS = 5.44, p < .001, ηp2 = .17]. A main effect of difficulty obtained, with participants providing more correct responses for easy (M = 87.80%, SD = 17.40) than for hard items (M = 43.87%,SD = 32.23), [F (1, 214) = 659.27, MS = 287.69, p < .001, ηp2 = .75]. These main effects were qualified by an interaction between statement and difficulty [F (1, 213) = 3.37, MS = 1.33, p = .035, ηp2 = .02]: Participants were more likely to provide correct responses for hard items after reading accurate (M = 50.30%, SD = 29.53) than after reading neutral statements (M = 43.12%, SD = 33.37), with rates of correct responses following both accurate and neutral statements higher than following inaccurate statements (M = 38.19%, SD = 32.65), (all ps < .04). For easy items, correct response rates were higher following accurate statements (M = 90.27%, SD = 16.19) than following both neutral (M = 86.96%, SD = 18.19) and inaccurate statements (M = 86.16%, SD = 17.55) (ps < .002), while rates of correct responses did not differ between neutral and inaccurate statements (p > .99).

With respect to search benefits, a main effect of condition again obtained: Participants in the search condition produced more correct responses (M = 72.37%, SD = 30.74) than did participants in the no-search condition (M = 59.30%, SD = 35.74), [F (1, 214) = 31.82, MS = 22.25, p < .001, ηp2 = .13]. An interaction between condition and difficulty obtained, with participants in the search condition (M = 55.66%, SD = 32.43) more often producing correct responses for hard items than participants in the no-search condition (M = 32.01%, SD = 27.40), [F (1, 214) = 44.41, MS = 19.38, p < .001, ηp2 = .17]. In contrast, rates of inaccurate responses for easy items were equitable across conditions (p = .68). No other significant main effects or interactions were observed (all Fs < 3.05). We also compared performance for searched and unsearched items. Participants produced significantly more correct responses for searched (M = 88.60%, SD = 15.29) than unsearched items (M = 66.46%, SD = 17.58), [t (76) = 6.24, p < .001]. This again suggests actual search behaviors led to performance benefits.

Overall, participants reproduced inaccurate information more often after reading it than if they had not, replicating previous work. But providing the opportunity to engage in online search attenuated this pattern. Participants allowed to search reproduced less inaccurate information, and more accurate information, than did participants who were not allowed to search. Participants exhibited an average search rate of 18.20% of critical items, with follow-up analyses demonstrating that benefits accrued specifically for searched items. Given these promising results, we sought to replicate and extend the findings with the same materials and manipulations but with a different participant sample and experimental context (i.e., in person rather than through MTurk). Experiment 1 also relied on participants’ self-reported searches, so we next used screen recordings to capture participants’ actual search engagements. These recordings were intended to provide exploratory insight into potential reasons that searches were enacted.

Experiment 2

Methods

Participants

Ninety-six participants were recruited from Northwestern University’s subject pool, with 40 identifying as male, 53 as female, and three not reporting their gender identity. They ranged in age from 18 to 23 years (M = 18.81), receiving course credit for participating in a 1-h session. Screening procedures were employed to ensure none had completed similar experiments.

Materials

The materials were identical to those used in Experiment 1.

Procedure

The procedure was identical to Experiment 1 with three modifications. First, participants completed the study in person rather than remotely. This allowed for screen recording participants’ choices, which was the second modification. Screen recordings were initiated via QuickTime Player immediately after participants granted their consent. Third, because participants were now completing the task using a lab computer and dedicated web browser rather than their own computer, the search-condition instructions were slightly modified, replacing the sentence “Use whatever search engine or app you like to help you,” with “You may open a separate tab in this browser window to do so.”

Design

The design was identical to Experiment 1.

Questionnaire coding

The same coding scheme from Experiment 1 was applied to responses collected in Experiment 2 (N = 3,072). The inaccurate – story code fit 5.91% of the responses, correct – story fit 64.36% of the responses, inaccurate – other constituted 11.34% of the data, and no-answer responses fit 18.36% of the data.Footnote 3 Response categories were mutually exclusive and fit the entire data set. Fifty percent of the responses were independently coded by two raters with the remaining coded by one rater. Inter-rater reliability was high (κ = .98), with all disagreements resolved through discussion. Table 3 presents response-rate codings as a function of search.

Table 3. Response rates as a function of search in Experiment 2

Results and discussion

The same analytic procedures as in Experiment 1 were used where appropriate.

Online search rates

Participants self-reported searching for 36.22% (SD = 19.80) of critical items and 34.23% (SD = 20.97 ) of filler items, which patterned with searches in Experiment 1 but were not statistically different [t (47) = 0.97 , p > .05]. For critical items specifically, mean individual search rates ranged from 0.00% to 68.75%. We examined search rates for critical items using a repeated-measures ANOVA, with statement type (i.e., inaccurate, accurate, or neutral) and item difficulty (i.e., easy, hard) within-subjects variables. Mauchly's Test of Sphericity indicated the assumption of sphericity was not violated, χ2(2) = 3.38, p = .184. A main effect of difficulty indicated that participants were more likely to report searching for hard (M = 56.55%, SD = 31.96) than easy items (M = 15.89%, SD = 20.23), [F (1, 47) = 72.40, MS = 2.89, p < .001, ηp2 = .61]. Search rates for critical items in Experiment 2 were somewhat higher than in Experiment 1 but exhibited a similar pattern with participants seeking information about unfamiliar topics more so than double-checking or re-examining familiar topics.

A significant interaction between difficulty and statement again obtained [F (2, 46) = 90.14, MS = 4.70, p < .001, ηp2 = .66]. Participants were more likely to search for hard items after reading neutral (M = 62.90%, SD = 33.00) than after reading accurate statements (M = 51.60%, SD = 30.67), (p = .002), with search rates following inaccurate statements (M = 55.16%, SD = 31.77) no different than following neutral or accurate statements (all ps > .06). This pattern also emerged for easy items albeit at an overall lower rate, with searches more likely after reading neutral (M = 20.65%, SD = 23.08) than after reading accurate statements (M = 12.65%, SD = 18.99), (p = .046). Search rates following inaccurate statements (M = 14.36%, SD = 17.75) did not differ from the other statement types (all ps > .36). As in Experiment 1, decisions to search were influenced by likely familiarity with the topics and recently read text content.

Inaccurate questionnaire responses

We first assessed response rates for inaccurate – story codings. Mauchly's Test of Sphericity indicated the assumption of sphericity was violated for this analysis, χ2(2) = 51.26, p = .000, so Greenhouse-Geisser corrections were used. A main effect of statement type obtained, with participants significantly more likely to reproduce story inaccuracies after reading inaccurate statements (M = 13.07%, SD = 17.69) than they were to spontaneously produce those inaccuracies after reading accurate (M = 1.92%, SD = 5.60) or neutral statements (M = 2.75%, SD = 7.14), [F (1.40, 94) = 63.26, MS = 3.76, p < .001, ηp2 = .40]. A main effect of difficulty indicated that participants were more likely to reproduce inaccuracies when answering hard (M = 10.44%, SD = 15.32) than easy items (M = 1.39%, SD = 12.53), [F (1, 95) = 179.41, MS = 8.21, p < .001, ηp2 = .66]. And a main effect of search revealed that participants in the search condition (M = 4.30%, SD = 9.61) used inaccurate information to answer questions less often than did participants in the no-search condition (M = 7.52%, SD = 14.73), [F (1,95) = 14.94, MS = 0.72, p < .001, ηp2 =.14].

Three interactions qualified these main effects. First, a statement by condition interaction indicated participants in the search condition (M = 9.48%, SD = 13.84) were less likely than participants in the no-search condition (M = 16.66%, SD = 20.29) to reproduce inaccuracies after reading inaccurate statements [F (1.40, 94) = 4.02, MS = 0.24, p = .033, ηp2 = .04]. Spontaneous inaccurate productions following accurate or neutral statements did not differ across conditions (all ps > .14). Second, an interaction between condition and difficulty indicated participants in the search condition (M = 7.66%, SD = 12.12) utilized inaccuracies significantly less often for hard items than did participants in the no-search condition (M = 13.22%, SD = 17.57), [F (1, 95) = 8.60, MS = 0.39, p = .004, ηp2 =.08]. Inaccurate reproductions for easy items did not differ between search (M = 0.95%, SD = 4.00) and no-search conditions (M = 1.82%, SD = 7.85), (p = .25). Third, a statement by difficulty interaction obtained [F (1.40, 94) = 32.91, MS = 1.29, p < .001, ηp2 =.26 ]. For hard items, participants were more likely to reproduce inaccurate information after reading inaccurate statements (M = 21.98%, SD = 19.06) than they were to spontaneously produce inaccuracies after reading accurate (M = 3.83%, SD = 7.45) or neutral statements (M = 5.50%, SD = 9.33). At an overall lower rate, participants also reproduced inaccuracies to answer easy items more often after reading inaccurate statements (M = 2.74%, SD = 8.90) than they spontaneously generated those responses after reading accurate (M = 0.00%, SD = 0.0) or neutral statements (M = 0.32%, SD = 2.38), (ps < .001). Inaccurate response rates did not differ after participants read accurate and neutral statements for hard (p = .27) or easy items (p > .99). No other main effects or interactions obtained (all Fs < 2.47).

Finally, we again specifically compared performance on searched and unsearched items for participants in the search condition. Participants reproduced fewer inaccurate responses for searched (M = 1.12%, SD = 2.36) than for unsearched items (M = 12.93%, SD = 13.92), [t (42) = -5.74, p < .001]. This replicated the search benefits observed in Experiment 1.

Correct questionnaire responses

Mauchly's Test of Sphericity indicated the assumption of sphericity was violated, χ2(2) = 8.91, p = .012, so Greenhouse-Geisser corrections were used. A main effect of statement indicated that participants produced more correct responses after reading accurate statements in the stories (M = 71.09%, SD = 30.81) than after reading neutral (M = 64.43%, SD = 32.90) or inaccurate statements (M = 57.55%, SD = 34.05), [F (1.83, 94) = 31.47, MS = 4.54, p < .001, ηp2 =.25]. A main effect of difficulty showed that participants were more likely to produce correct responses for easy (M = 84.01%, SD = 19.13) than for hard items (M = 44.70%, SD = 32.30), [F (1,95) = 524.85, MS = 113.01, p < .001, ηp2 =.85]. An interaction between difficulty and condition also obtained [F (1, 95) = 56.02, MS = 12.06, p < .001, ηp2 =.37]. Participants in the search condition produced more correct responses for hard items (M = 64.74%, SD = 28.54) than did participants in the no-search condition (M = 24.66%, SD = 23.20). At an overall higher rate, participants in the search condition (M = 90.55%, SD = 16.07) also produced more correct responses for easy items than did participants in the no-search condition (M = 77.48%, SD = 19.76). Finally, a main effect of condition showed that participants in the search condition (M = 77.64%, SD = 26.49) produced more correct responses overall than did participants in the no-search condition (M = 51.07%, SD = 33.60), [F (1, 95) = 81.42, MS = 44.94, p < .001, ηp2 =.46]. Specifically in the search condition, participants produced more correct responses for searched items (M = 89.08%, SD = 9.23) than for unsearched items (M = 68.96%, SD = 16.44), [t (42) = -5.74, p < .001], further replicating Experiment 1. No other significant main effects or interactions were observed (all Fs < 3.03).

Screen recordings

All participants were screen recorded to maximize similarity across conditions, but only recordings of participants in the search condition (N = 48) included behaviors to be analyzed. Three participants’ recordings were lost due to QuickTime file processing errors. Neither the study procedures nor any data collected via Qualtrics were compromised by these errors as they occurred during the file saving process after participants completed their sessions. Thus, data from these three participants were applicable for inclusion in the previous analyses but excluded from the screen recording analysis (N = 45).

To analyze the recordings, we developed a coding scheme based on a priori understandings of online search processes and grounded categorizations derived from a preliminary review of a subset of recordings. The coding scheme contained four general categories, each with possible sub-codes. (Table 4 provides a summary of the coding categories.) The full set of recordings included 1,440 observations, based on behavioral activity for the 32 items for each of the 45 participants. Twenty-five percent of the data set was dual-coded by two independent raters trained in the coding scheme, with the remaining coded by one rater only. Inter-rater reliability for the dual-coded recordings was reliably high (κ = .93).

Table 4. Search codes applied to screen recordings, Experiment 2

The first category, validity of search report, identified whether or not behaviors evident in the screen recordings matched the self-reports participants provided for questionnaire items. Recorded responses to a questionnaire item were coded as either a match (i.e., a participant self-reported “Yes” to a search prompt with a search evident in the screen recording, or self-reported “No” with no search evident) or a mismatch (i.e., a participant self-reported “Yes” to the prompt without any search evident in the screen recording or vice versa). 495 searches were coded as matches, and seven as mismatches, indicating self-reports were for the most part confirmed. Of the seven mismatches, six were false negatives in which participants self-reported “No” but did search. Those searches occurred after an answer had already been provided and did not change following the search. Only one mismatch reflected a participant reporting “Yes” but without evidence of a search. We added the six false negatives and removed the one false positive to the 495 matching cases for a total of 500 searches, to be categorized using the remaining codes.

The three remaining categories and their sub-codes attempted to identify specific features of search behaviors and were not mutually exclusive. The code search type identified whether participants formatted their search terms as the exact question for a given item (e.g., “What is the name of the man who invented the steamboat ‘Clermont’?”), as a rephrased question (e.g., “Who invented the Clermont steamboat?”), or as a keyword search (e.g., “Clermont steamboat”). If a participant completed multiple, unique searches for a single item, it received as many codes as relevant for categorization. Keyword searches (N = 415 or 83.00% of searches) were far more common than rephrased questions (N = 90 or 18.00%) or exact questions (N = 20 or 4.00%).

The search destination coding specified where participants “landed” after conducting a search, with three relevant sub-codes: search suggestions (i.e., relevant information automatically populating under the search bar while a search query was being entered), Google search results page (i.e., actual results pages appearing post-search entry), and webpage (i.e., visits to specific websites). If multiple outcomes were possible, multiple sub-codes were applied. Each applicable sub-code was quantified; for example, if a participant viewed a page of Google search results, and navigated to two unique webpages from those results, multiple sub-codes would be applied. We only applied the search suggestions sub-code if it was clear that a participant ceased their search efforts after suggestions had auto populated under the search bar. This was applicable to 39 searches (7.80%). Google search results page codings were applied when participants “landed” on a page of results and entered their response. (Participants never used any other search engine.) This was the most common sub-code, fitting 457 of the observed searches (91.40%). Finally, the webpage sub-code involved actual visits to webpages (e.g., Wikipedia), applicable to 44 of the observed searches (8.80%).

Finally, the search role code attempted to capture whether a search appeared to factor into a participants’ response. These codes were mutually exclusive and fit the entirety of the data set (N = 500 searches). An augmentation sub-code was applicable to any instance of a search preceding a response, with search terms not already including any potential answer. This code fit 423 of the 500 searches (84.60%). An incompletion sub-code was applied when participants searched and ultimately responded “no answer” or “I don’t know.” This fit ten of the 500 searches (2.00%). A verification sub-code applied to instances in which participants appeared to possess relevant knowledge prior to searching, evidenced in two ways. First, participants could enter an answer, then conduct a search, and then actually submit their initial response without any changes beyond minor edits (defined as a superficial change that did not alter an original meaning or reading, e.g., capitalizing or changing the spelling of an initial answer). Second, participants’ searches could have included a correct response (e.g., using “Dover” as a search term for the item “What is the capital of Delaware?” and ultimately responding “Dover”). That their original response remained unchanged suggests these searches served the purpose of verification. This sub-code fit 32 of the 500 searches observed (6.40%). The emendation sub-code was applied when participants formulated responses before their search, but ultimately changed that response after their search. This fit 35 of the 500 searches observed (7.00%).Footnote 4

The screen recordings provide a type of manipulation check, indicating that participants were almost always honest with respect to identifying when they actually searched. As exploratory data, the recordings also highlighted that participants preferred to utilize keyword searches, relying on information obtained from Google search result lists. Most importantly, their searches were often conducted to augment their performance, with searches usually generated before responses were provided, and information from those searches informing participants’ ultimate responses. Participants certainly engaged in other types of search behaviors, but to a lesser degree likely due to the kinds of materials and topics being tested, the number of potential searches to be conducted, and the contingencies of the lab task. Nevertheless, the screen recordings highlight that searches were directly influential for reducing participants’ inaccurate reproductions, and increasing their correct responses, because they afforded easy access to useful information that could be readily applied to answer the test questions.

Overall, the results from Experiments 1 and 2 indicate that rates of search can be beneficial, attenuating inaccurate reproductions. These search rates are no doubt reflective of participants’ interest and engagement in the task, and the degree to which they considered searches important and worth their time to enact in the experiment. Note that participants in Experiment 2 searched more than twice as often as did participants in Experiment 1, both for the critical questionnaire items (36.22% and 18.20%, respectively), and across the entire set of questionnaire items (35.84% and 17.30%, respectively). Recall that participants in Experiment 1 completed the task over MTurk, while participants in Experiment 2 completed the task in person, with these contexts potentially incentivizing rapid performance (e.g., in Experiment 1) or setting task demands (e.g., in Experiment 2) that could differentially affect search enactments. But across both experiments, searches reduced inaccurate and increased accurate responses.

Given the utility of searches for participants’ performance, we wondered whether explicitly encouraging searches could support additional benefits. In Experiment 3 we incorporated a warning about the possible inclusion of inaccuracies in the stories as a means of motivating participants to consult online resources. Warnings on their own have not yielded substantial reductions in reproductions of inaccurate content (Marsh & Fazio, 2006), but also have not been paired with access to resources that could help people act on those warnings. In tandem, warnings and searches might encourage evaluative mindsets (Mayo, 2019), supporting performance on the questionnaire task. We predicted that coupling warnings with the opportunity to search would reduce inaccurate reproductions and increase correct responses, as compared to the opportunity to search alone. The warnings were included immediately prior to the questionnaire task to avoid encouraging story reading strategies that may have differed from those in Experiments 1 and 2, and to specifically target participants’ search decisions.

Experiment 3

Methods

Participants

Two hundred and thirty-four participants recruited via Amazon’s Mechanical Turk completed the experiment for $6, none of whom had participated in Experiments 1 or 2. Eighteen participants were eliminated due to failure to follow experiment instructions, leaving 216 participants in the final sample. All participants identified as native English speakers, ranging in age from 20 to 61 years (M = 34.86, SD = 9.04). Ninety identified as female, 125 as male, and one did not identify with a gender binary.

Materials

The materials were identical to Experiment 1 with two changes. First, all participants were instructed that they could search for questionnaire answers if they wished, receiving the search condition instructions from Experiment 1. Second, the instructions included additional information associated with each condition. Following the original instructions, participants in the no-warning condition also read:

“There is no penalty for searching for answers or indicating no answer. We simply want to see how well people can do on this task with the option to use the internet to answer the questions.”

Participants in the warning condition also received these instructions, preceded by the following:

“Please keep in mind that some of the information you previously read was inaccurate. As such it may be especially useful to you to search online for answers as you complete the questionnaire.”

As before, the instructions neither specified inaccuracies nor mentioned the story contents.

Procedures

The procedure was identical to Experiment 1, now with all participants asked to self-report whether they completed an online search for each questionnaire item.

Design

The experiment utilized a 3 (statement type: inaccurate, accurate, or neutral) × 2 (item difficulty: easy or hard) × 2 (condition: warning or no warning) design. The design emulated Experiment 1, this time with half of the sample (n = 108) given a warning about inaccuracies potentially appearing in the stories, and the remaining half (n = 108) receiving no such warning.

Questionnaire coding

The same coding scheme from Experiments 1 and 2 was applied to responses in Experiment 3 (N = 6,912). Codes for inaccurate – story accounted for 4.07% of responses in the data set; correct – story accounted for 73.87% of the data; inaccurate – other represented 12.06% of the data; and 10.00% of responses fit no answer.Footnote 5 Half of the responses were coded by two independent raters, with the remaining responses coded by one rater only. Inter-rater reliability for dual-coded responses was reliably high (κ = .94), with all disagreements resolved through discussion. See Table 5 for response rates across all coded categories as a function of search.

Table 5. Response rates as a function of search in Experiment 3

Results and discussion

Online search rates

Participants self-reported searching for critical items (M = 24.70%, SD = 21.41) more than for filler items (M = 22.34%, SD = 22.50) [t (215) = 4.19 , p < .001]. For critical items specifically, individual search rates ranged from 0.00% to 100.00%. For the analysis, Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 3.03, p = .220. A main effect of difficulty indicated participants were more likely to search for hard (M = 41.62%, SD = 35.35) than for easy items (M = 8.25%, SD = 17.90), [F (1, 214) = 331.65, MS = 36.07, p < .001, ηp2 = .61]. Participants who received warnings (M = 28.90%, SD = 33.89) reported searching more often than did participants who did not receive warnings (M = 20.97%, SD = 30.78) [F (1, 214) = 7.53, MS = 2.03, p = .007, ηp2 = .03]. There was an interaction between difficulty and condition: For hard items, participants conducted more searches when given warnings (M = 47.96%, SD = 35.02) than when not (M = 35.27%, SD = 34.58), [F (1, 214) = 6.78, MS = 0.74, p = .01, ηp2 = .03]. For easy items, search rates did not differ across conditions (p = .11). No other effects or interactions were observed (all Fs < 1.28).

Inaccurate questionnaire responses

Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated, χ2(2) = 48.46, p = .000, so Greenhouse-Geisser corrections were applied. A main effect of statement indicated participants were more likely to reproduce inaccurate information after reading inaccurate statements (M = 7.71%, SD = 13.74) than to spontaneously produce those responses after reading accurate (M = 1.80%, SD = 5.50) or neutral statements (M = 2.75%, SD = 7.80), [F (1.66, 213) = 51.76, MS = 2.35, p < .001, ηp2 = .19]. A main effect of difficulty indicated participants were more likely to reproduce inaccurate information for hard (M = 7.30%, SD = 12.72) than easy items (M = 0.88%, SD = 4.18), [F (1, 214) = 253.71, MS = 10.58, p < .001, ηp2 = .54]. There was also an interaction between statement and item difficulty [F (1.75, 213) = 24.04, MS = 0.76, p < .001, ηp2 = .10]. Participants were more likely to reproduce inaccurate information as answers to hard items after reading inaccurate statements (M = 13.10%, SD = 16.66) than they were to spontaneously produce inaccurate responses after reading accurate (M = 3.60%, SD = 7.35) or neutral statements (M = 5.21%, SD = 10.16). Participants reproduced inaccurate information at commensurate rates whether in the warning (M = 3.90%, SD = 9.79) or no-warning condition (M = 4.28%, SD = 10.19), [F (1, 214) = 0.68]. Across conditions, participants reproduced fewer inaccuracies for items they specifically searched for (M = 0.46%, SD = 1.80) than for items they did not (M = 8.35%, SD = 9.53), [t (165) = -9.44, p < .001]. No other effects or interactions obtained (all Fs < .48).

Correct questionnaire responses

Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 1.10, p = .995. A main effect of statement indicated that participants provided more correct responses after reading accurate (M = 77.62%, SD = 26.42) than inaccurate (M = 70.13%, SD = 30.10) or neutral statements (M = 73.88%, SD = 29.96), [F (1, 213) = 18.55, MS = 3.16, p < .001, ηp2 = .08]. A main effect of difficulty indicated participants provided correct responses more often for easy (M = 86.97%, SD = 17.36) than hard items (M = 60.78%, SD = 32.26), [F (1, 214) = 289.74, MS = 120.10, p < .001, ηp2 = .58]. A main effect of condition indicated that participants in the warning condition produced more correct responses (M = 76.34%, SD = 71.41) than did participants in the no-warning condition (M = 71.41%, SD = 18.00), [F (1, 214) = 4.73, MS = 4.14, p = .03, ηp2 = .02]. No other effects or interactions were significant (all Fs < 2.10). Across warning conditions, participants produced more correct responses for items they searched for (M = 81.79%, SD = 18.84) than for items they did not (M = 67.26 %, SD = 19.06), [t (165) = 4.27, p < .001].

The results of Experiment 3 replicated the previous findings, with participants reproducing inaccurate information following exposure, and with reproductions reduced when searches were applied. The warning was beneficial, with participants engaging in more searches in the warning than in the no-warning condition, and with warned participants overall producing more correct responses. However, the warning did not reduce reproductions of inaccurate content below rates observed in the search only condition, suggesting boundaries to those benefits.

General discussion

The information that people read, from both fiction and non-fiction sources, influences thought and behavior. For example, previous work consistently shows that participants reproduce information from stories, when relevant to the real world, to answer related questions. Story contents need not be accurate, and thus a variety of conditions intended to reduce inaccurate reproductions have been tested, albeit with limited success. We examined whether allowing people to consult online sources while answering post-reading questions would help attenuate inaccurate reproductions. Because everyday reading usually allows for consulting outside resources, this examination provides a crucial assessment of the external generalizability of previous empirical concerns and claims. It also puts accounts appealing to information availability and accessibility as integral to people’s reproductions to direct empirical test. That searches would be beneficial was not a foregone conclusion, given that some projects have failed to substantially reduce the influence of inaccurate information even when people are asked to think about or are reminded of accurate ideas (e.g., Rapp, 2008).

In Experiment 1, MTurk participants read stories containing accurate and inaccurate statements about the real world. After reading, they completed a questionnaire including items relevant to story statements. Participants were either allowed to search online for additional information while answering the questions or asked not to do so. We predicted that participants in the search condition would reproduce fewer story inaccuracies as answers to questions, and produce more correct responses overall, than would participants in the no-search condition. Participants given the option to search did so modestly, self-reporting searches on average for 18.20% of the 32 critical items. They were more likely to search when contemplating answers to hard than easy items, suggesting their searches were intended to supplement their existing knowledge base more so than to double-check familiar ideas. Searches for hard items also occurred most often when neither accurate nor inaccurate information had appeared in the stories, again suggesting searches were intended to fill in knowledge gaps.

As in previous studies, participants were more likely to reproduce inaccurate information after reading it than to spontaneously produce those inaccurate responses. But searches were beneficial, as participants in the search condition reproduced fewer inaccurate ideas after reading them (8.04%) and produced more overall correct responses (72.37%) than did participants in the no-search condition (14.15% and 59.30%, respectively). Search benefits were most clearly observed for hard items, again highlighting the utility of searches for supplementing knowledge. Performance on easy items, which participants rarely looked up, was associated with few inaccurate reproductions and high rates of correct responses. Participants thus benefited from consulting online resources for items they were unlikely to be able to answer correctly on their own. Their self-reported searches indicated benefits were obtained for questions they looked up, linking decreases in inaccurate reproductions directly to their efforts.

Experiment 2 was intended to replicate these patterns, and to examine search behaviors beyond a self-report query. Participants completed the study in a lab setting with their search behaviors screen recorded. Searches occurred more often in this experiment than in Experiment 1, with participants searching for 36.22% of critical items. The observed patterns nevertheless resembled those in Experiment 1. Participants were more likely to reproduce inaccurate information if they read it than if they did not, with reproductions reduced (9.48%) and correct responses more likely (77.64%) when participants were allowed to search as compared to when they were not permitted to do so (16.66% and 51.07%, respectively). Participants were more likely to search for, and obtain benefits from, looking up information about hard than easy items.

Screen recordings provided useful insight as to when and how searches likely benefited post-reading performance. Participants typically employed keyword searches and consulted Google result pages rather than individual webpages when searching for relevant information. Searches most often reflected participants attempting to augment their understandings with new information, consistent with searches targeting information unlikely to already be known. Their searches less often reflected double-checking or corrections of previously provided answers. In addition, participants very rarely indicated having looked up information when they had not, providing confidence in the search self-reports collected in Experiment 1.

Given the observed search benefits, in Experiment 3 we tested whether participants might be encouraged to engage in even more searches, with concomitant benefits for questionnaire performance. All MTurk participants were allowed to search, with half also warned about the potential inclusion of inaccuracies in the stories. Participants who were warned self-reported more searches for critical items than did participants who were not warned, with searches again conducted more often for hard than easy items. Warned participants also produced more correct responses (76.34%) than did participants who received no warning (71.41%). This again was observed most clearly for hard items, providing further evidence that participants searched for information they were unlikely to have already known. Warnings, however, did not further reduce inaccurate reproductions, with similar reproduction rates observed across conditions. Warned participants were more likely to search, but those searches did not appear to be necessarily or optimally directed towards information that refuted previously read inaccuracies. This may be at least partially attributable to the general warnings that participants received. Specific warnings calling out the particular falsehoods in a text, with useful examples, may be more effective than ambiguous alerts about the potential for inaccuracies (Crozier & Strange, 2019; Higham, Blank, & Luna, 2017). These warnings can operate in a manner consistent with so-called inoculation approaches, which involve pre-exposing people to misleading ideas to foster subsequent recognition of their flaws and rejection of their claims (e.g., Cook, Lewandowsky, & Ecker, 2017; Roozenbeek & Van Der Linden, 2019). While general warnings can be useful with respect to some kinds of contents (e.g., discredited causes as in Peshkam, Mensink, Putnam, & Rapp, 2011), concise declarative statements that are not integral to story plots may require more explicit directives, as advocated for in inoculation approaches, to make readers aware of inaccurate inclusions and of their potential for influencing post-reading responses.

We foregrounded the effects of exposures to inaccurate information as a motivating framework for the project. But an important consideration is that, across the experiments, participants more often reproduced inaccurate information they were unlikely to know much about than information they were likely to know was wrong. Previous projects have reported similar patterns. Additionally in the current project, participants’ searches often involved looking up information related to unfamiliar as compared to familiar claims. Some of these search behaviors might reflect not just responses specifically intended to validate previous inaccurate claims, but also general attempts to deal with uncertainty while completing the task. Relevant for this consideration though, participants self-reported more searches when answering questions related to information appearing in the stories than for questions unrelated to text content (significant in Experiments 1 and 3, and in the same numeric direction in Experiment 2). This suggests that participants’ searches were often intended to verify text statements. But whether searches involved attempts to validate the accuracy of previously read information, or general uncertainty about how to respond to questions, the observed reductions in participants’ reproductions of inaccuracies indicate that searches supported post-reading performance.

Additionally, the observed search benefits have implications for contemporary discussions of the influence of inaccurate information during everyday reading, for explanations as to why reliance on inaccurate information occurs, and for accounts of epistemic validation. Research has consistently shown that people use falsehoods in texts to complete subsequent tasks, derived largely from lab studies in which participants can only rely on their knowledge as they are not permitted access to outside sources. Demonstrating that readers reproduce text information might therefore be somewhat unsurprising, as is especially observed for unfamiliar content, but also informative and disappointing when prior knowledge should be useful and relevant (Fazio et al., 2013). These studies have helped identify the cognitive processes and behavioral products underlying experiences with inaccurate information (Rapp & Braasch, 2014; Rapp, 2016). However, the experimental contexts associated with those studies may not be ideal for establishing whether inaccurate reproductions obtain during naturalistic reading, given that people can use other resources to supplement or even overrule text content or prior knowledge. As the current findings demonstrate, readers may consult outside resources if given the opportunity, which reduces inaccurate reproductions relative to when that access is restricted. Claims about the influence of inaccurate information in everyday situations may be overstated if people’s everyday resources and practices are left out of any analyses.Footnote 6

Explanations as to why people are influenced by exposures to inaccuracies often appeal to the availability of recently processed information. This information, whether accurate or inaccurate, may be more readily accessible than the knowledge people possess, affording its use. Recently presented information will exert an even stronger influence when people do not possess relevant knowledge to detect potential inaccuracies and effectively interrogate ideas. Allowing participants to engage in online searches permits examination as to whether increasing the availability of valid information, alongside or outside of prior knowledge, helps attenuate inaccurate reproductions. The current findings are consistent with the view that making valid information readily available to participants can improve performance. Recent exposures to accurate information helped overcome preceding exposures to inaccuracies. Previous work has assessed similar effects at encoding rather than at test: When people consider and apply relevant prior knowledge they possess while reading inaccurate story content, they are less likely to show an influence of that inaccurate information on post-reading tasks (Rapp et al., 2014a). Providing access to accurate information at encoding or at retrieval, obtained from useful prior knowledge or valid external sources, helps reduce the influence of inaccuracies, which further confirms availability as a critical mechanism in emerging accounts.

Other not necessarily opposing accounts have focused on epistemic validation, examining whether and how people evaluate unfolding discourse during or after comprehension (Braasch & Bråten, 2017; Richter & Schmid, 2010; Singer, 2013). These accounts focus on the degree to which people possess and apply knowledge to evaluate the veracity of information they read. There are, however, many instances in which prior knowledge may be partially or wholly lacking, making successful independent validation impossible. People can and do seek out information to supplement their knowledge during and, as observed here, after reading. While epistemic validation was not central to the current project, the results establish the utility of attending to the practices readers engage in to support evaluation and comprehension. These practices might, for example, underlie mindsets that influence epistemic processes and products (Mayo, 2019). They may also be useful for further evaluating the circumstances under which people scrutinize information or adopt shallow processing approaches that omit careful reasoning (e.g., Hennes, Kim, & Remache, 2020; Pennycook & Rand, 2019).

The benefits observed in these experiments are promising, with important issues to be addressed in future work. Our questionnaire materials had well-defined answers that were readily available via search. Research should evaluate the utility of search for other types of inaccurate and even ambiguous or inconsistent information (e.g., Flemming, Kimmerle, Cress, & Sinatra, 2020). This has proven useful for examining students’ understandings of historical topics with real world materials from varied sources (e.g., Britt & Aglinskas, 2002; Donovan, Zhan, & Rapp, 2018b; Seixas, 1994; Wineburg, 2000). Topics with greater ambiguity and uncertainty could yield different search patterns and outcomes. People might be less likely, for example, to search if they believe doing so will be difficult, time-consuming, or less than satisfactory. The topics we tested pertained to mundane issues lacking in controversy and unrelated to values and attitudes. While much of the information we contend with every day is indeed mundane, we are also confronted with controversial topics that should involve consultations with reputable sources and data. Unfortunately, when inaccuracies align with people’s beliefs, they may forgo doing so or, worse, select sources that support their beliefs (Fischer, Jonas, Frey, & Schulz-Hardt, 2005; Hart et al., 2009). These represent cases in which searches may actually prove ineffective or detrimental.

While item difficulty played an important role in identifying when participants searched, other factors influence when and how often people consult sources. This could include state and trait dependent curiosity, needs for accuracy and closure, topic preferences, confidence in prior knowledge or text sources, and information seeking habits. Examining these sources of variability should prove useful for understanding search practices and outcomes. The searches that participants enacted here were not required, may have distracted from completing the task efficiently, and need not have been a motivated priority for participants. That they still proved beneficial indicates the need for further examination of when they are likely to occur. To date, searches have been usefully examined in consumer decision making (De Los Santos, 2018; Johnson, Moe, Fader, Bellman, & Lohse, 2004; Xiang & Gretzel, 2010) and educational experiences (see Rieh, et al., 2016, for a review), among other topic areas. Here they were applied as a means of examining both descriptive and explanatory accounts of the consequences of exposures to inaccurate information.

Notes

  1. 1.

    We did not generate hypotheses for the categories of inaccurate – other or no answer, but include response rate analyses for these codes here for completeness. For the inaccurate – other category, Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 2.68, p = .262. Participants generated more inaccurate – other responses after reading neutral (M = 12.54%, SD = 17.93) than after reading inaccurate statements (M = 9.98%, SD = 15.23), with response rates after reading accurate statements (M = 11.32%, SD = 17.04) no different from the other statement types [F (1, 214) = 6.38, MS = 0.53, p = .002, ηp2 = .03]. Participants also generated more inaccurate - other responses for hard (M = 16.61%, SD = 19.06) than for easy items (M = 5.95%, SD = 12.00), [F (1, 214) = 22.30, MS = 167.95, p < .001, ηp2 = .44]. No other effects or interactions were observed (all Fs < 2.19).

    For the no-answer category, Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 2.48, p = .289. Participants produced more no-answer responses after reading neutral (M =17.31%, SD = 25.78) than after reading accurate statements (M = 14.59%, SD = 23.82), with no-answer responses after reading inaccurate statements (M = 16.26%, SD = 25.54) no different from either of the other statement types [F (1, 214) = 5.31, MS = 0.44, p = .005, ηp2 = .02]. Participants generated more no-answer responses for hard (M = 27.60%, SD = 29.39) than for easy items (M = 4.51%, SD = 11.28), [F (1, 214) = 284.33, MS = 91.29, p < .001, ηp2 = .57]. Participants in the search condition (M = 11.69%, SD = 22.51) generated fewer no-answer responses as compared to participants in the no-search condition (M = 20.42%, SD = 26.70), [F (1, 214) = 17.49, MS = 12.33, p < .001, ηp2 = .08]. A significant interaction between difficulty and condition indicated that participants in the no-search condition (M = 36.10%, SD = 28.40) produced more no-answer responses for hard items than did participants in the search condition (M = 19.10%, SD = 27.90), [F (1, 214) = 33.82, MS = 10.86, p < .001, ηp2 = .14]. Finally, a three-way interaction between statement, difficulty, and condition obtained: Participants in the no-search condition (M = 39.78%, SD = 28.47) generated more no-answer responses to hard items than did participants in the search condition (M = 18.42%, SD = 27.57), specifically after reading neutral statements [F (1, 214) = 4.27, MS = 0.30, p = .015, ηp2 = .02. No other interactions were observed (all Fs < 1.41).

  2. 2.

    Given that the majority of participants did not report searching for information related to easy items at all, response rates for easy, searched items could not be calculated for majorities of participants. Accordingly, we collapsed across levels of difficulty for all comparisons of response rates for searched versus unsearched items. Mean response rates for each level of difficulty are nevertheless presented in Tables 1, 2, and 5, although again, the average response rates for easy, searched items reflect only a small number of applicable cases and are thus subject to skew.

  3. 3.

    We again provide analyses for the coding categories of inaccurate – other and no answer for completeness. For inaccurate – other responses, Mauchly's Test of Sphericity indicated the assumption of sphericity was violated, χ2(2) = 6.16, p = .046, so Greenhouse-Geisser corrections were applied. A main effect of difficulty showed participants were more likely to provide incorrect responses unrelated to the potentially included inaccuracies for hard (M = 15.93%, SD = 17.82) than easy items (M = 6.74%, SD = 16.27), [F (1, 95) = 92.01, MS = 8.71, p < .001, ηp2 =.49]. This was qualified by a crossover interaction between difficulty and condition [F (1, 95) = 5.57, MS = 0.53, p = .02, ηp2 =.06]. Participants in the search condition generated more inaccurate – other responses for hard items (M = 17.23%, SD = 19.14) than did participants in the no-search condition (M = 14.63%, SD = 16.36). The opposite pattern was obtained for easy items, with participants in the search condition generating fewer inaccurate – other responses (M = 5.54%, SD = 12.67) than did participants in the no-search condition (M = 7.94%, SD = 13.36). This may be due to the wide variety of information participants in the search condition may have had access to, with commensurate difficulty involved in selecting correct responses for hard as compared to easy items. No other main effects or interactions obtained (all Fs < 1.23). As in Experiment 1, no significant difference in rates of inaccurate – other responses were observed across search (M = 11.38%, SD = 17.23) and no-search conditions (M = 11.28%, SD = 15.28).

    For the coded category of no answer, Mauchly's Test of Sphericity indicated the assumption of sphericity was not violated, χ2(2) = 3.74, p = .154. A main effect of difficulty [F (1, 95) = 168.92, MS = 29.53, p < .001, ηp2 =.64] indicated that participants were more likely to provide no answers for hard (M = 28.93%, SD = 29.44) than easy items (M = 7.79%, SD = 14.51). This main was qualified by two interactions. First, a statement by difficulty interaction was observed [F (2, 94) = 3.38, MS = 0.25, p = .04, ηp2 =.08]. For hard items, participants were more likely to provide no response after reading neutral (M = 33.00%, SD = 31.61) than after reading inaccurate (M = 26.89%, SD = 27.43) or accurate statements (M = 26.88%, SD = 28.99), (ps < .026). In contrast, no answer responses for easy items did not differ as a function of statement type (all ps > .11). Additionally, there was an interaction between condition and difficulty [F (1, 95) = 58.65, MS = 10.25, p < .001, ηp2 =.38]. Participants in the search condition were less likely to provide no response to hard items (M = 10.37%, SD = 20.28) than were participants in the no-search condition (M = 47.48%, SD = 26.33). Participants in the search condition were also less likely to provide no response to easy items (M = 2.83%, SD = 8.45) than were participants in the no-search condition (M = 12.76%, SD = 17.36). In addition, a main effect of condition showed that search condition participants (M = 6.60%, SD = 15.97) were less likely to provide no response than were participants in the no-search condition (M = 30.12%, SD = 27.73), [F (1, 95) = 61.34, MS = 36.67, p < .001, ηp2 =.39]. No other significant main effects or interactions were obtained (all Fs < 2.53).

  4. 4.

    Participants occasionally looked up answers after submitting a response. Because the experiment did not allow backtracking, these searches occurred after the question had already been answered and could not inform participants’ responses. These were classified as retroaction, and only occurred eight times in total. We report them here for completeness.

  5. 5.

    We again provide response rates for the categories of inaccurate – other or no answer in Experiment 3 for completeness. For inaccurate – other responses, Mauchly's Test of Sphericity indicated that the assumption of sphericity had not been violated, χ2(2) = 5.18, p = .075. There was a main effect of difficulty, with participants more often providing these responses for hard (M = 17.42%, SD = 17.29) than easy items (M = 6.83%, SD = 12.53), [F (1, 214) = 155.89, MS = 20.17, p < .001, ηp2 = .42]. No other effects or interactions obtained (all Fs < 2.57).

    For no-answer responses, Mauchly's Test of Sphericity indicated that the assumption of sphericity was violated, χ2(2) = 7.77, p = .020, so Greenhouse-Geisser corrections were applied. A main effect of statement indicated that participants produced no-answer responses more often after reading neutral (M = 10.82%, SD = 22.59) than after reading accurate statements (M = 8.92%, SD = 20.25), with rates following inaccurate statements (M = 10.25%%, SD = 20.37) no different from the other statement types [F (1.94, 213) = 3.39, MS = 0.17, p = .036, ηp2 = .02]. A main effect of difficulty also obtained, with participants producing more no-answer responses for hard (M = 15.18%, SD = 26.43) than for easy items (M = 4.82%, SD = 11.76), [F (1, 214) = 76.03, MS = 19.01, p < .001, ηp2 = .26]. Finally, a main effect of condition indicated participants in the warning condition (M = 7.52%, SD = 17.36) produced fewer no-answer responses than did participants in the no-warning condition (M = 12.48%, SD = 24.02), [F (1, 214) = 4.75, MS = 3.65, p = .03, ηp2 = .02]. No significant interactions were observed (all Fs < 3.33).

  6. 6.

    We note that rates of inaccurate reproductions were, as compared to in other studies, relatively low, even for participants who were not permitted to search. We have no ready explanation as to why the overall pattern was attenuated as compared to in other projects, but importantly, when participants were allowed to search, reductions were nevertheless observed and also more substantial than has been reported in previous projects. Given the problems associated with the use of inaccurate information, investigating methods of reducing reproductions to the lowest possible levels remains an important set of theoretical and applied concerns.

References

  1. Albrecht, J. E., & O’Brien, E. J. (1993). Updating a mental model: Maintaining both local and global coherence. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 1061-1070.

    Google Scholar 

  2. Appel, M., & Richter, T. (2007). Persuasive effects of fictional narratives increase over time. Media Psychology, 10, 113–134.

    Google Scholar 

  3. Braasch, J. L., & Bråten, I. (2017). The discrepancy-induced source comprehension (D-ISC) model: Basic assumptions and preliminary evidence. Educational Psychologist, 52, 167-181.

    Article  Google Scholar 

  4. Brashier, N. M., Eliseev, E. D., & Marsh, E. J. (2019). An initial accuracy focus prevents illusory truth. Cognition.

  5. Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20, 485-522.

    Article  Google Scholar 

  6. Cook, J., Lewandowsky, S., & Ecker, U.K.H. (2017). Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their influence. PLOS ONE, 12, e0175799.

    PubMed Central  PubMed  Article  Google Scholar 

  7. Crozier, W. E., & Strange, D. (2019). Correcting the misinformation effect. Applied Cognitive Psychology, 33, 585-595.

    Google Scholar 

  8. De Los Santos, B. (2018). Consumer search on the internet. International Journal of Industrial Organization, 58, 66-205.

    Article  Google Scholar 

  9. Donovan, A. M., Theodosis, E., & Rapp, D. N. (2018a). Reader, interrupted: Do disruptions during encoding attenuate misinformation effects? Applied Cognitive Psychology, 32, 775-786. DOI: https://doi.org/10.1002/acp.3464

    Article  Google Scholar 

  10. Donovan, A. M., Zhan, J., & Rapp, D. N. (2018b). Supporting historical understandings with refutation texts. Contemporary Educational Psychology, 54, 1-11.

    Article  Google Scholar 

  11. Ecker, U. K. H., Lewandowsky, S., & Tang, D. T. W. (2010). Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & Cognition, 38, 1087-1100. DOI:https://doi.org/10.3758/MC.38.8.1087

    Article  Google Scholar 

  12. Eslick, A. N., Fazio, L. K., & Marsh, E. J. (2011). Ironic effects of drawing attention to story errors. Memory, 19, 184-191. DOI: https://doi.org/10.1080/09658211.2010.543908

    Article  PubMed Central  PubMed  Google Scholar 

  13. Fazio, L. K., & Marsh, E. J. (2008). Slowing presentation speed increases illusions of knowledge. Psychonomic Bulletin & Review, 15, 180–185. DOI:https://doi.org/10.3758/PBR.15.1.180

    Article  Google Scholar 

  14. Fazio, L. K., Barber, S. J., Rajaram, S., Ornstein, P. A., & Marsh, E. J. (2013). Creating illusions of knowledge: Learning errors that contradict prior knowledge. Journal of Experimental Psychology: General, 142, 1–5.

    Article  Google Scholar 

  15. Fazio, L. K., Brashier, N. M., Payne, B. K., & Marsh, E. J. (2015). Knowledge does not protect against illusory truth. Journal of Experimental Psychology: General, 144, 993-1002.

    Article  Google Scholar 

  16. Ferguson, A. M., McLean, D., Risko, E. F. (2015). Answers at your fingertips: Access to the internet influences willingness to answer questions. Consciousness and Cognition, 37, 91-102.

    PubMed  Article  PubMed Central  Google Scholar 

  17. Fischer P., Jonas, E., Frey, D., & Schulz-Hardt, S. (2005). Selective exposure to information: The impact of information limits. European Journal of Social Psychology, 35, 469–492.

    Article  Google Scholar 

  18. Fisher, M., Goddu, M. K., & Keil, F. C. (2015). Searching for explanations: How the Internet inflates estimates of internal knowledge. Journal of Experimental Psychology: General, 144, 674-687.

    Article  Google Scholar 

  19. Flemming, D., Kimmerle, J., Cress, U., & Sinatra, G.M. (2020) Research is tentative, but that’s okay: Overcoming misconceptions about scientific tentativeness through refutation texts. Discourse Processes, 57, 17–35.

  20. Gerrig, R. J., & Prentice, D. A. (1991). The representation of fictional information. Psychological Science, 2, 336–340.

    Article  Google Scholar 

  21. Hamilton, K. A., McIntyre, K. P., & Hertel, P. T. (2016). Judging knowledge in the digital age: The role of external-memory organization. Applied Cognitive Psychology, 30, 1080-1087.

    Article  Google Scholar 

  22. Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psychological Bulletin, 135, 555–588.

    PubMed Central  PubMed  Article  Google Scholar 

  23. Hennes, E.P., Kim, T., & Remache, L.J. (2020). A goldilocks critique of the hot cognition perspective on climate change skepticism. Current Opinion In Behavioral Sciences, 34, 142-147.

    Article  Google Scholar 

  24. Higham, P. A., Blank, H., & Luna, K. (2017). Effects of postwarning specificity on memory performance and confidence in the eyewitness misinformation paradigm. Journal of Experimental Psychology: Applied, 23, 417– 432.

    PubMed  PubMed Central  Google Scholar 

  25. Hinze, S. R., Slaten, D. G., Horton, W. S., Jenkins, R., & Rapp, D. N. (2014). Pilgrims sailing the Titanic: Plausibility effects on memory for misinformation. Memory & Cognition, 42, 1-20.

    Article  Google Scholar 

  26. Johnson, E. J., Moe, W. W., Fader, P. S., Bellman, S., & Lohse, G. L. (2004). On the depth and dynamics of online search behavior. Management Science, 50, 299-308.

    Article  Google Scholar 

  27. Johnson, H. M., & Seifert, C. M. (1994). Sources of the continued influence effect: When discredited information in memory affects later inferences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1420-1436.

    Google Scholar 

  28. Kendeou, P., & O’Brien, E. J. (2014). The knowledge revision components (KReC) framework: Processes and mechanisms. In Rapp, D. N., and Braasch, J. L. G. (Eds.), Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 353–379). Cambridge, MA: MIT Press.

    Google Scholar 

  29. Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction–integration model. Psychological Review, 95, 163-182.

    PubMed  Article  PubMed Central  Google Scholar 

  30. Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist, 49, 294-303.

    PubMed  Article  PubMed Central  Google Scholar 

  31. Lai, K., Lee, Y. X., Chen, H., & Yu, R. (2017). Research on web search behavior: How online query data inform social psychology. Cyberpsychology, Behavior, and Social Networking, 20, 596-602. DOI: https://doi.org/10.1089/cyber.2017.0261

    Article  Google Scholar 

  32. Marsh, E. J. (2004). Story stimuli for creating false beliefs about the world. Behavior Research Methods, Instruments, & Computers, 36, 650–655. DOI:https://doi.org/10.3758/BF03206546

    Article  Google Scholar 

  33. Marsh, E. J., & Fazio, L. K. (2006). Learning errors from fiction: Difficulties in reducing reliance on fictional stories. Memory & Cognition, 34, 1140–1149. DOI:https://doi.org/10.3758/BF03193260

    Article  Google Scholar 

  34. Marsh, E. J., Meade, M. L., & Roediger, H. L. (2003). Learning facts from fiction. Journal of Memory and Language, 49, 519–536.

    Article  Google Scholar 

  35. Mayo, R. (2019). Knowledge and distrust may go a long way in the battle with disinformation: Mental processes of spontaneous disbelief. Current Directions in Psychological Science, 28, 1 -6.

    Article  Google Scholar 

  36. McNamara, D. S., & Kintsch, W. (1996). Learning from text: Effects of prior knowledge and text coherence. Discourse Processes, 22, 247-287.

    Article  Google Scholar 

  37. Nelson, T. O., & Narens, L. (1980). Norms of 300 general-information questions: Accuracy of recall, latency of recall, and feeling-of-knowing ratings. Journal of Verbal Learning and Verbal Behavior, 19, 338–368. DOI:https://doi.org/10.1016/S0022-5371(80)90266-2

    Article  Google Scholar 

  38. Oppenheimer, D. M. (2008). The secret life of fluency. Trends in Cognitive Science, 12, 237-241.

    Article  Google Scholar 

  39. Ozkara, B. Y., Ozmen, M., & Kim, J. Y. (2016). Exploring the relationship between information satisfaction and flow in the context of consumers’ online search. Computers in Human Behavior, 63, 844-859.

    Article  Google Scholar 

  40. Pennycook, G., & Rand, D.G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39-50.

    PubMed  Article  PubMed Central  Google Scholar 

  41. Peshkam, A., Mensink, M. C., Putnam, A. L., & Rapp, D. N. (2011). Warning readers to avoid irrelevant information: When being vague might be valuable. Contemporary Educational Psychology, 36, 219-231.

    Article  Google Scholar 

  42. Prentice, D. A., & Gerrig, R. J. (1999). Exploring the boundary between fiction and reality. In S. Chaiken & Y. Trope (Eds.), Dual-process theories in social psychology (pp. 529-546). New York, NY, US: Guilford Press.

    Google Scholar 

  43. Prentice, D. A., Gerrig, R. J., & Bailis, D. S. (1997). What readers bring to the processing of fictional texts. Psychonomic Bulletin & Review, 4, 416–420.

    Article  Google Scholar 

  44. Rapp, D. N. (2008). How do readers handle incorrect information during reading? Memory & Cognition, 36, 688–701.

    Article  Google Scholar 

  45. Rapp, D. N. (2016). The consequences of reading inaccurate information. Current Directions in Psychological Science, 25, 281–285.

  46. Rapp, D. N., & Braasch, J. L. G. (Eds.). (2014). Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences. Cambridge, MA: MIT Press.

    Google Scholar 

  47. Rapp, D. N., & Salovich, N. A. (2018). Can’t we just disregard fake news? The consequences of exposure to inaccurate information. Policy Insights from the Behavioral and Brain Sciences, 5, 232-239.

    Article  Google Scholar 

  48. Rapp, D. N., Hinze, S. R., Kohlhepp, K., & Ryskin, R. A. (2014a). Reducing reliance on inaccurate information. Memory & Cognition, 42, 11-26.

    Article  Google Scholar 

  49. Rapp, D. N., Hinze, S. R., Slaten, D. G., & Horton, W. S. (2014b). Amazing stories: Acquiring and avoiding inaccurate information from fiction. Discourse Processes, 38, 50-74.

    Article  Google Scholar 

  50. Rich, P. R., & Zaragoza, M. S. (2016). The continued influence of implied and explicitly stated misinformation in news reports. Journal of Experimental Psychology: Learning, Memory, and Cognition, 42, 62-74.

    PubMed  PubMed Central  Google Scholar 

  51. Richter, T., & Schmid, S. (2010). Epistemological beliefs and epistemic strategies in self-regulated learning. Metacognition and Learning, 5, 47-65.

    Article  Google Scholar 

  52. Rieh, S. Y., Collins-Thompson, K., Hansen, P., & Lee, H-J. (2016). Toward searching as a learning process: A review of current perspectives and future directions. Journal of Information Science, 42, 19–34.

  53. Roozenbeek, J., & Van Der Linden, S. (2019). The fake news game: actively inoculating against the risk of misinformation. Journal of Risk Research, 22, 570-580.

    Article  Google Scholar 

  54. Salovich, N. A. & Rapp, D. N. (2020). Metacognition and reliance on inaccurate information. Manuscript submitted for publication.

  55. Seixas, P. (1994). Students’ understanding of historical significance. Theory and Research in Social Education, 22, 281-304.

    Article  Google Scholar 

  56. Singer, M. (2013). Validation in reading comprehension. Current Directions in Psychological Science, 22, 361-366.

    Article  Google Scholar 

  57. Storm, B. C. (2011). Retrieval-induced forgetting and the resolution of competition. In A. S. Benjamin (Ed.), Successful remembering and successful forgetting: A festschrift in honor of Robert A. Bjork (pp. 89–105). New York, NY: Psychology Press.

    Google Scholar 

  58. Tauber, S. K., Dunlosky, J., Rawson, K. A., Rhodes, M. G., & Sitzman, D. M. (2013). General knowledge norms: Updated and expanded from Nelson and Narens (1980) norms. Behavioral Research Methods, 45, 1115-1143.

    Article  Google Scholar 

  59. van den Broek, P., Rapp, D. N., & Kendeou, P. (2005). Integrating memory-based and constructionist processes in accounts of reading comprehension. Discourse Processes, 39, 299–316.

    Article  Google Scholar 

  60. Wineburg, S. (2000). Making historical sense. In P. N. Stearns, P. Seixas, & S. Wineburg (Eds.), Knowing, teaching, and learning history: National and international perspectives (pp. 306-325). New York, NY: New York University Press.

    Google Scholar 

  61. Xiang, Z., & Gretzel, U. (2010). Role of social media online travel information search. Tourism Management, 31, 179-188. DOI:https://doi.org/10.1016/j.tourman.2009.02.016

    Article  Google Scholar 

Download references

Acknowledgements

We thank Sid Horton, Rachel Mersey, Nikita Salovich, Michael Spikes, and three anonymous reviewers for their helpful comments on the manuscript. We also thank Rayna Weiser and Yulan Chen for their assistance with data collection and coding.

Open Practices Statement

None of the data or materials for the experiments reported here are available, and none of the experiments were preregistered.

Author Note

The authors declare that there were no conflicts of interest with respect to the authorship or publication of this article.

Author information

Affiliations

Authors

Corresponding author

Correspondence to David N. Rapp.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Donovan, A.M., Rapp, D.N. Look it up: Online search reduces the problematic effects of exposures to inaccuracies. Mem Cogn 48, 1128–1145 (2020). https://doi.org/10.3758/s13421-020-01047-z

Download citation

Keywords

  • Inaccurate information
  • Online search
  • Reading comprehension
  • Memory
  • Text processing