The present study assessed the combined effects of context and textual discrepancies on readers’ standards for a documents model representation and standards for presentation. We contrasted a university context that emphasized evaluation with regard to academic norms with a personal context that lacked this emphasis. In addition, we researched effects of beliefs about science. The context manipulation impacted readers’ standard for a documents model representation and standard of presentation. With regard to discrepancy, the experiment replicated and extended prior findings.
Effects of the context on standards of a documents model representation and standards of presentation
The first main goal of the study was to examine context effects on participants’ standards for constructing document models, as opposed to simpler representations. In the university context, participants more frequently switched between texts, used more adversative connectors when writing about the documents, used more corroborating expressions in the essays, and were more likely to explicitly address potential discrepancies in their essays. These findings strongly suggest that participants in the university condition attached more importance to conveying a balanced presentation of the content of the documents, which required the construction of a documents model (e.g., Britt & Rouet, 2012).
With regard to standards for presentation, we found that participants in the university context spent more time on the task/essay page than participants in the personal context. This is consistent with the finding that participants in a university context almost always wrote a continuous essay, as opposed to bullet points in a personal context. Writing continuous text requires more time than just writing some notes. Thus, the participants in the university context condition invested time in communicating their findings, probably because they had a different standard for how the product had to look like, that is for the presentation. Differences in readers’ standards for presentation are probably due to the presence of an explicit audience in this context (see Cho & Choi, 2018). However, further studies are needed to clarify whether it is the audience alone that produced the effects observed in the present study.
All in all, the findings can be interpreted on a general level as support for the RESOLV model and on a more specific level with regard to differences between the university and the personal context. First, the findings on context effects support on a more general level the notion that students interpret a given task (e.g., “write an overview”) against the background of the specific context as it was suggested in RESOLV. In the present study, the actual task was the same in both conditions. We provided the students with background information about the context, but we did not instruct them to, for example, write nicely because they write for their professor. This is an inference that the participants themselves drew, only because we placed the current task in one or the other context. Thus, the present findings support the notion that task instructions are interpreted differently depending on the context of the task. As suggested in RESOLV, readers seem to use context features to infer what exactly they are supposed to do and how to understand the task. Our data support the view that contextual cues shape students' interpretation of an explicit task request (e.g., “write an overview”), and their generation of different reading goals and strategies, depending on the context.
With regard to university versus personal context, we suggest that a university context prompt the activation of academic norms and standards which are not (or not as strongly) activated in a personal context. These academic norms probably encompass not only norms about sources, source citations, and a balanced presentation of the literature, but also norms about an academic style of writing, which is connected to the use of sophisticated words, including connectors, and that mere notes are not sufficient. In contrast, in the personal context these norms are not (or not as strongly) activated. Idiosyncratic norms can be applied instead. Moreover, the personal context did not involve any external audience for the essay. To put it in RESOLV terms, the only audience was the self (Britt et al., 2018). Thus, differences in audience (and not just the academic norms) may have contributed to the observed context effect.
Yet, in the present study we could observe most of this only indirectly. That is, we did not directly measure the goals and strategies that students represented in their task model nor their norms and standards that were supposedly activated by the different contexts. Further studies should explicitly test both goals/strategies and norms/standards to confirm our interpretation of results. More work is also needed on how people interpret tasks. For example, there may be one aspect in task understanding (e.g., the standard to write continuous text in the university condition) that drives other results (e.g., the effect on the length of essay) or the context influences several aspects independently (which the low correlations in Table 5 suggest for the present study). Future studies, which may also use other methods such as think-aloud, may give further insight into this.
We also did not assess the mental representation of the texts, but only the students communication thereof. Thus, we cannot conclude that the mental representation of the texts is different across contexts. A different communication may be due to a different mental representation, but it could also be caused by a different understanding of what needs to be communicated. Since we found no context effect on the time spent reading, but an effect on the time spent on the task/essay page, this may be a first hint that the mental representation of the texts may not necessarily be different across contexts, but how a representation is presented. Yet, this needs further exploration.
Effects of discrepancy: replication and extension of DIS-C effects
As in prior studies (e.g., Braasch et al., 2012; Kammerer & Gerjets, 2014; Kammerer et al., 2016; Rouet et al., 2016), we found that sources providing discrepant information were more often cited explicitly in an essay. Although participants did not spend more time reading discrepant texts (e.g., Hakala & O'Brien, 1995; Kammerer & Gerjets, 2014), they more often switched texts when the contents were discrepant, which we interpret as comparison processes across texts. Readers facing discrepant documents might act more strategically by comparing information across documents (see, e.g., List & Alexander, 2020). This corroborates and extends prior findings that discrepancy strongly promotes readers’ attention to sources and the integration of source and content information into a documents model (Braasch et al., 2012; Rouet et al., 2016).
Additionally, communicating information about discrepant topics needs more work—as evidenced both in longer essays and more time spent on writing the essay. To our knowledge, this extends prior findings on the DIS-C effect. The finding is all the more noteworthy because for consistent topics, four different reasons (across the two documents) covering four different aspects supporting the claim were provided, whereas for discrepant topics, the four reasons covered only two different aspects, namely two from a pro and two from a con perspective. Consequently, readers actually had more unique pieces of information to report when reading consistent versus discrepant topics. However, the participants did not seem to experience a need to write as much about consistent topics as about discrepant topics. Although an analysis of the essays with regard to the number of reasons reported would be interesting, such an analysis is also confronted with a problem that is difficult, if not impossible, to solve: How exactly to count the number of reasons participants provided, in a way that is reliable across raters. Especially in the case of discrepant topics, there are many ways of reporting reasons that are interrelated but different (i.e., complementary, contradictory), as is the case in the present study. This makes an objective counting of reasons almost impossible. However, even without this analysis, the present results suggest that reading from discrepant sources increases participants’ standard for completeness in reporting about what they read. That is, discrepant information may trigger them to report all sides of the discrepancy, whereas consistent information is not reported completely but only to an extent that readers feel that they sufficiently supported the point.
Finally, we observed an interaction of context and discrepancy on the use of adversative connectors in participants’ essays. Participants used more connectors when writing in a university context, especially when the sources provided discrepant information. This finding indicates that the participants were more likely to have represented the content as a documents model. In a situation with consistent texts, building an overall situation model is usually sufficient, and no documents model is necessary (Britt et al., 1999). In a situation with discrepant texts, however, a documents model becomes necessary, and readers might even write why they believe in one or another claim (see Rouet et al., 2016). In order to report a documents model, adversative connectors and more words overall are necessary. The results overall do not support our hypothesis that context would increase or attenuate the D-ISC effect. Rather the results suggest that both discrepancy and a university context that emphasized evaluation additively increased readers’ attention to source information and its representation.
Influence of beliefs about science
There was a main effect of utility of science on the number of text switches: Participants with more positive beliefs more often compared information across texts. We also found interaction effects: Participants with more trust in science in the university context more often explicitly mentioned in their essay that a topic was discrepant. Also in a university context, participants with a higher perceived utility of science spent more time on the texts.
Thus, interaction effects of beliefs about science and context were in the direction that more positive beliefs boosted behavior and use of information in the university context, and not in the personal context. Positive beliefs about science may motivate students to apply academic norms when the context demands it, or else, positive beliefs about science may raise the importance of already salient academic norms in the university context. Yet, since there are only a few effects, these considerations have to be confirmed in further studies.
The challenges of experimentally researching the context
Experimentally researching context effects poses several challenges. Ideally, participants would work in different contexts in reality. However, whereas it is quite easy to place students into a university context, it is practically difficult to collect detailed data about information processing in authentic personal contexts. We did get evidence that our manipulation was effective (for instance, through the content analysis of participants’ essays and by means of manipulation control questions), and the procedure is in line with previous research (e.g., reading from a burglar's vs. a homebuyer's perspective: Schraw et al., 1993). However, the study says little about the magnitude of the context effect as it occurs naturally, since the participants were still in the same real context: sitting at home in front of their computer, participating in a scientific study. This might have affected the results.
Second, contexts differ with regard to several aspects (see also Table 1). On the one hand this means that there is a wide variety of possible scenarios within a university context and a personal context. In addition to the two scenarios implemented in the present study, there might also be university scenarios with no audience or where academic norms are not salient and not important, whereas it is also possible to come up with personal context scenarios in which there is an audience and/or in which academic norms are salient and important. We do not assume that these two represent the defining elements of university or personal context, respectively. On the other hand, contexts are not easily manipulable such that only one context aspect is varied, since often aspects covary with other aspects. For example, in many contexts the requester is also the audience or at least part of the audience. Finding meaningful scenarios with a requester but without audience (which could then be compared to a scenario with requester and audience), is difficult. Moreover, often stakes covary with a requester and/or an audience. Thus, there is still a lot of theoretical and empirical work to do to better describe the influence of context on reading.
Related to that is the problem that it is difficult to identify causal mechanisms of context effects when several aspects of context vary at the same time. Moreover, it could also be that not one single context aspect, but a combination of different aspects (e.g., the presence of high stakes and an audience) causes an effect, but neither single variation of these aspects. Probably several methodological approaches are necessary to advance the field: While experimental research can help confirm causal explanations, ethnographic research or content analysis of real-world contributions to the social media, for example, may help identify aspects of the context or combinations of aspects of the context that are relevant and trigger different processing of multiple documents.
Likewise, future research may try to identify context differences in the actual processing of multiple documents. Such research could disentangle, for example, whether the context effects observed in the present study are caused by differences in the (construction of) the mental representation of the documents, or by differences in the presentation of this representation. With regard to discrepancies, for example, such research could also follow up whether discrepancies in the personal context are a) less likely to be noted, b) less likely to be mentally represented, or c) only less likely to be explicitly addressed in the essay (see Stadtler & Bromme, 2014). In the present study, we used only a few measures that directly addressed the process (i.e., time on texts, time on task/essay page, text switches). So, we started this kind of research, but more research is needed.
Apart from the limitations that arise from the challenges of context manipulation, the present study entails a number of other limitations. First, the present sample was not a representative but an ad-hoc sample. Therefore, there might be a (self-)selection of students who value science and scientific norms, which might bias results with regard to the personal context such that this sample also applies scientific norms in their personal lives. Given the high mean rating of perceived utility of science (about 4 out of 5), this might be true. However, the latter finding could also be due to our sample being a university student sample.
Second, the present experiment used a rather long procedure. As can also be deduced from the position effects we found, there might be fatigue or motivation loss effects. Yet, this should have resulted in a less careful way of working on the study, thus disguising existing context effects. Therefore, the effects we report here can be assumed to be a conservative estimation of really existing effects. On the pro side, our approach allows us to research context and discrepancy effects in a way that is less affected by topic effects, for example due to topic interest or characteristics of the specific set of texts that were used. Thus, the effects reported in the present study represent a general pattern that is not necessarily replicable in every single set of documents. Future research on multiple document comprehension should consider using several topics.
Third, future work could benefit from including also direct measures of the assumed mechanism, for example measures of activated norms, or think-aloud or eye tracking procedures to support the current interpretation of results. Also, more fine-grained log-data may be of use, for example with regard to source information. An additional limitation of our study is that the task request and the editor for writing the essay were on the same page. Therefore, we cannot disentangle time data of re-reading the task and writing the essay.
Last but not least, the procedure did not include a comprehension measure, but only behavioral and productive measures as well as a memory post-test. Further studies could also measure comprehension immediately after reading the texts. Moreover, we did not measure reading and writing skills or prior knowledge. Whereas prior knowledge effects should be reduced by our approach of using several topics, reading and writing skills may account for some variance, especially with regard to the measures that are directly derived from the essay (e.g., length, number of adversative connectors). However, given the considerable sample size and the within-participant design for the factor discrepancy, we can assume that these skills follow similar distributions across condition, such that neither context nor discrepancy effects can be explained by individual differences.
Consequences for theory and practice
The results of the present study support the general provision of the RESOLV model (Britt et al., 2018) that the physical and social context influences reading processes. Yet, the RESOLV model only enables coarse predictions regarding which aspects of the context can make a difference and how exactly they influence reading processes and outcomes. The analysis of prior manipulations of context as well as our own manipulation show that there are several aspects intertwined inseparably across naturally occurring contexts. Further studies on the RESOLV model might want to focus on which aspects exactly contribute to a difference in the reading process. The present results suggest that the presence or absence of an audience plays a role for the presentation of information. It might also be that it is not a single aspect of the context which changes reading processes, but an additive or interactive combination of several aspects. Thus, with regard to assessing theoretical considerations about the role the context for reading, the present study took a first step, but many others are to follow.
With regard to educational practice, the present results suggest that writing in an academic context increases students’ standards for comprehending and reporting information. In addition, writing from discrepant sources increased students’ attention to sources. These findings support the use of multiple discrepant documents as a means to stimulate critical thinking and the principled integration of sources and contents in students’ essays. Thus, multiple-document assignments can support university students’ learning by promoting heuristics like corroboration or sourcing, or more generally: by promoting standards for a documents model. This is especially true if there are discrepancies across documents. While in the present research, this discrepancy was on the level of provided claims and reasons, differences at other levels, such as discrepant wording across documents, can support university students’ learning (Schoor et al., 2019).