Abstract
Students with high reading proficiency typically achieve better results in science assessments, indicating the importance of reading proficiency. Since the process of reading is a complex interaction between properties of a text and a reader, the linguistic demands of a text might affect text comprehension. Certain linguistic features, such as complex syntactic structures and low word frequency, have been found to create higher cognitive load. However, studies investigating the influence of linguistic features on test item difficulty and students’ text comprehension in science have hitherto let to inconclusive results. The present study investigates whether the linguistic demands of expository text affect German students’ text comprehension in the domain of physics. Within an experimental study, we presented three introductory texts on different subtopics of thermodynamics and 27 single-select, multiple-choice items to 812 secondary school students (grades 7–9). Items measured students’ text comprehension (dependent variable); the linguistic demands of each text were systematically varied across three levels (independent variables) while other features of text quality and content were held constant. The results of the item response theory analysis indicated no consistent differences in item difficulty across levels of linguistic demands. Moreover, differential analyses of subgroups presented no consistent differences in solution frequencies of items related to different linguistic demands. Furthermore, while the highest linguistic demand of the texts led the students to perceive a lower comprehensibility, their text comprehension was not affected. Hence, this study provides evidence that the influence of linguistic features on text comprehension is at most low and might be overestimated in present discussions.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
There is wide agreement among scholars that academic and scientific language skills are highly important to communicate, process, comprehend, and apply scientific knowledge (e.g. Schleppegrell, 2004; Snow, 2010; Stoffelsma & Spooren, 2019). On the one hand, academic and scientific language supports teaching and learning because of its precise and condensed nature. On the other hand, the special features of academic and science language differ from students’ everyday linguistic repertoire, which results in challenges for learning the necessary language alongside the more content-focused competency in science education. Thus, academic language can be assumed to play a crucial role by either supporting or hindering science learning and understanding, which might lead to unequal learning opportunities. In fact, research currently indicates influences of reading proficiency and academic language skills on performance in mathematics and science assessments (Cooper et al., 2014; Härtig et al., 2015; Prediger et al., 2015), suggesting that high linguistic demands of items in science assessments hinder students from demonstrating their full potential. Moderate-to-strong correlations between text comprehension and science performance in large-scale assessments corroborated this finding (Cromley, 2009; Korpershoek et al., 2015; O’Reilly & McNamara, 2007; Yore et al., 2004). A reason for this might be that items used in science assessments evaluate students’ competencies and knowledge (e.g. Organization for Economic Cooperation and Development [OECD] 2017), which require students to read and comprehend text. However, studies that vary the linguistic demands of science and mathematics tasks systematically—and predominantly on the level of lexicon and syntax—to measure the influence of linguistic features on item difficulty have found small, inconsistent, or even expertise-reversal effects of increased text cohesion for students with high, prior, subject-specific knowledge (e.g. Haag et al., 2015; Höttecke et al., 2018; Kieffer et al., 2009). These results indicate that the linguistic presentation of test items only marginally affects an average student’s ability to comprehend and answer test items. Furthermore, these results cannot be attributed to text comprehension alone since the items used in these studies targeted subject-specific procedural skills and text comprehension at the same time without systematically separating these factors. Thus, variance in test performance is mostly due to subject-specific procedural knowledge and skills. The high correlation of reading proficiency and science performance in large-scale assessments might indicate an influence of students’ language skills on their learning and understanding in science.
We assume that the effect of linguistic demand is task-specific and depends on the subject-specific context. We argue that expository text for knowledge-building—typical in physics textbooks—places greater demand on students’ text comprehension, which might result in stronger effects of the text’s linguistic presentation on students’ comprehension compared to items in an assessment. Research on the effect of linguistic features on comprehension of expository text is far from coherent (e.g. Härtig et al., 2019; McNamara & Kintsch, 1996; Schmitz, 2015). Additionally, analysis of German science schoolbooks shows that the linguistic demands are generally high and do not systematically increase across grades (e.g. Berendes et al., 2017; Greasser et al., 2011; Kohnen et al., 2017a). Thus, the linguistic demands of schoolbooks are not sufficiently adapted to the language proficiencies of students. As a result, students’ perceived text complexity of textbooks is high (Härtig et al., 2012a, 2012b; Starauschek, 2003). Therefore, our research aims at a better understanding of the effect of linguistic demands on science learning and understanding by drawing on students’ text comprehension of expository text in physics. In view of inconsistent research results, we ask the following general research question: Does a systematic variation of linguistic demands in expository text in physics affect students’ text comprehension? To answer this research question, a model of text comprehensibility (Göpferich, 2009) used to draft the texts and isolate the effect of linguistic demand on text comprehension is outlined below. Furthermore, a theoretically and empirically derived heuristic model of linguistic demands in German to systematically vary linguistic features is presented. After the theoretical framework, development of our science text comprehension instrument is described, followed by results and discussion.
Theoretical Framework
Text Comprehension
Research on text comprehension has shown that text comprehension is a complex interplay between properties of a text and the reader (Schnotz, 1994). Text comprehension is a result of successful reading; it requires bottom-up processes in which information must be decoded from the text’s linguistic structure. In addition, readers relate, compare, and integrate textual information top-down into their prior knowledge to construct a mental representation of the textual information. Consequently, text comprehension means gaining and applying knowledge at the same time. While the general aim of school should be to support students in the development of skills to understand domain-specific academic text, scholars have suggested that expository text be adapted or even simplified to the level of students’ reading proficiency to facilitate their learning (Göpferich, 2009; Hsu & Yang, 2007; Schleppegrell, 2004).
Model of Text Comprehensibility
Several measures have been suggested to operationalize text difficulty based solely on textual features and without acknowledging the role the reader plays. The so-called readability scores based on the length of words and number of words per sentence are limited since they take only the text’s surface features into account. Comprehensibility models go beyond readability scores by acknowledging the role of lexis, syntactics, and semantics for text comprehension (Göpferich, 2009). Such models have been criticized as being imprecise, incomplete, or of limited practicability for researchers outside linguistics (Lutz, 2015). The comprehensibility model by Göpferich (2009) overcomes these problems and, therefore, was chosen to draft the text used in our study. It covers six dimensions:
-
• Structure is concerned with the internal semiotic order, consistency, and cohesion of information presented across a text
-
• Brevity/conciseness refers to the ratio of information represented to the length of a text; hence, a rather concise text refrains from any additional information
-
• Stimulating additives might stimulate a reader; yet, too many additions might be overwhelming and deviate their attention
-
• Simplicity is concerned with the linguistic demands (e.g. choice of words and sentence structure)
-
• Accuracy specifies the level to which a text is free of contradictions, technically correct, and follows the conventions of a genre-appropriate textual function
-
• Perceptibility generally means the ease with which a text is perceived depending on its visual appearance including the font, numerations, and any further typographic elements
The highest level of comprehensibility is not simply located at the maximum of each dimension; instead, the optimum of each dimension depends on characteristics such as a given type of text, its purpose, and target group. The linguistic demands of any text and the peculiarities of scientific and academic language belong to the simplicity dimension. Since all dimensions are interconnected, a single dimension cannot be changed without affecting the others.
Perception of Text Complexity
An additional feature that may influence the impact of textual demands on text comprehension is how complex a text is perceived by readers; a reader's perception of text complexity and comprehensibility affect their reading strategies. A perceived high complexity of a text might challenge a reader to read with increased care and attention, which in turn results in higher text comprehension. In fact, this might cause and explain the expertise-reversal effect mentioned above since readers with high prior knowledge might feel underchallenged by a text’s high cohesion, leading to superficial text scanning and lack of comprehension. Yet studies focusing on students’ perceived text complexity usually lack data about their actual text comprehension. Härtig et al., (2012a, 2012b) have shown that students perceive expository text in science textbooks as rather complex and incomprehensible. A comparative survey on two science textbooks showed that higher text cohesion decreases the level of perceived text complexity (Starauschek, 2003). Yet in an experimental study science text were varied on two levels of local substantival textual cohesion (i.e. neighboring sentences connected by the same noun), but no effect was indicated on perceived complexity (Starauschek, 2006). Tolochko et al. (2019) have shown that perceived complexity is only affected by a text’s semantics, not by its syntax. A reason might be that students struggle with rating a text’s syntax without prior training (Funke et al., 2013). In brief, research has not clearly indicated any effect of perceived complexity on text comprehension. Therefore, the present study investigates the effects of linguistic demand on both text comprehension and perceived complexity.
Model of Linguistic Demand
In this section, we briefly outline that research hitherto has not led to clear results which linguistic features either affect students’ science test performance or their science text comprehension. Based on findings of the two fields of research, a model will be outlined that allows for a systematic variation of linguistic demand along three major aspects.
Influence of Linguistic Features on Item Difficulty
Several studies from educational as well as cognitive psychology vary linguistic features of items to isolate the effect of linguistic demand on test performance. A high density of technical terms has been found to decrease test performance (e.g. Cassels, 1980; Schiemann, 2011; Snow, 2010; Stiller et al., 2016). The use of the passive voice has been shown to generate difficulty (e.g. Berndt et al., 2004; Bransford & Johnson, 2014; Cornelis, 1996), but it has also been found to have no effect (Cassels, 1980). The simultaneous variation of numerous linguistic item features has either shown small (Kettler et al., 2012; Plath & Leiss, 2017; Prophet & Badede, 2009), non-consistent (Höttecke et al., 2018), or even contradictory effects (Rivera & Stansfield, 2004). Meta-studies have corroborated the finding that the effect of linguistic demands of items is at most small (Kieffer et al., 2012; Pennock-Roman & Rivera, 2011). Pöhler et al. (2017) even revealed that a variation of how an item was presented (i.e. lacking any context, visually or textually contextualized) has no effect on its difficulty, indicating that cognitive complexity has a higher impact on science performance and text comprehension compared to linguistic demand.
Another approach to identifying complex language features is the linguistic post hoc analyses of data from large-scale assessments. Heppt et al. (2015) found academic terms, words with three or more syllables, and word compounds to be difficult while word count, complex sentence structures, and noun phrases were nonsignificant. Cruz Neri et al. (2021), on the other hand, identified word count as having a positive effect on students with high reading proficiency. Dempster and Reddy (2007) found that complex sentence structure and unfamiliar words lead to increased item difficulty. Prediger et al. (2015) have shown that linguistic demands have an effect and that, while complex or unfamiliar words were hardly regarded as challenging, complex sentence structures were.
Effects of Linguistic Features on Comprehension of Expository Text
Scholars have identified specific (i.e. mostly syntactical) linguistic features of science text as being difficult (Kareva & Echevarria, 2013), yet studies in this area are rare. Deppner (1989) simplified science text by reducing redundancy and complexity of sentence structure and used more frequently occurring words. She could not find any general effects on the text comprehension of grade 8 students except for those with Turkish as their heritage language. Studies on the semantic structure of science text have shown that text cohesion has either a positive (McNamara & Kintsch, 1996; Schmitz, 2015) or no effect on text comprehension (Kohnen et al., 2017b; Rabe & Mikelskis, 2007). In addition, Hsu and Yang (2007) have shown that a higher level of numerous types of text cohesion (i.e. lexical, semantical, syntactical, text-picture-relation) results in higher text comprehension. Studies taking a deeper look into the effect of text cohesion have taken readers’ language proficiency and prior knowledge as controls. They show that readers with high prior knowledge comprehend a text better if cohesion is low (Kamalski et al., 2008; Linderholm et al., 2008; McNamara & Kintsch, 1996; McNamara et al., 1996). This expertise-reversal effect has been challenged by evidence presented by others (Schmitz, 2015). If both general reading proficiency and prior knowledge are considered, different interactive effects of text cohesion on text comprehension were found (O’Reilly & McNamara, 2007; Ozuru et al., 2009) and yet refuted (Schmitz, 2015).
Previous research did not lead to coherent results and generally points toward small effects of linguistic features on item difficulty as well as text comprehension. Thus, further research is needed to show whether and which specific linguistic features of academic and scientific language influence text comprehension. Only density of technical terms is consistently found to decrease students’ performance in assessments. Linguistic text features such as cohesion, passive voice, unfamiliar (low frequent) words, words with numerous syllables, word count, noun and participial phrases, and complex sentence structure usually create small effects on students’ comprehension; these features are considered in the model used in the systematic variation of linguistic demands in this study. It is based upon a model developed by an interdisciplinary group of scholars in linguistics, science education, and empirical educational research presented in detail elsewhere (e.g. Heine et al., 2018; Plath & Leiss, 2017). It considers the specifics of academic language used for teaching and learning that are frequently assumed to generate textual difficulty (Schleppegrell, 2004) by applying three major aspects of linguistic demands.
A Model of Linguistic Demands Based on Three Major Aspects
Our model systematically varies numerous linguistic features that have been shown to create higher cognitive load in eye-tracking, reading-time, and EEG studies or to require a broader linguistic knowledge base of peripheral and less frequent linguistic elements along three major aspects (Heine et al., 2018):
-
1. Structural complexity represents the cognitive load determined by the quality and quantity of cognitive processes while reading. That is, the longer a particular information needs to actively be held in working memory until a unit of information can finally be processed (e.g. a sentence), the higher the cognitive load. This is the case when a noun and an associated pronoun or relative clause are rather distant from each other (Gabler, 2013; Konieczny, 2000) or when particle verbs are used in long constructions (Levy & Keller, 2013; Levy et al., 2012). In addition, complexity is found to be increased by nominalizations and nominal phrases (Fang et al., 2006; Kliegl et al., 2004; Solomyak & Marantz, 2010)
-
2. Transparency describes how unambiguously information is presented. Processing semantically complex information (Fusté-Herrmann, 2008; Štekauer, 2005)—such as words with double meanings (Köhne et al., 2015), idiomatic phrases (Irujo, 1993), the passive voice (Berndt et al., 2004), and specific academic vocabulary (Cassels, 1980; Schiemann, 2011; Snow, 2010; Stiller et al., 2016)—requires knowledge of less obvious meanings. As a result, cognitive processing of information is less automatic
-
3. Frequency, which often correlates with structural complexity and transparency, indicates how frequently a word (e.g. home = frequent, electron = less frequent), clause-type (e.g. canonical word order = more frequent than noncanonical word order, active voice = more frequent than passive voice, etc.), or grammatical structure are used in a particular language. Highly frequent structures create less cognitive load because they allow for automatic processing (Hulstijn, 2015)
Considering these three aspects, Heine et al. (2018) presented specific principles to establish three levels of linguistic demand; the cutoff points of which are meant to be interpretative and dependent on purpose and the target group of a text. The use of technical terms essential to an understanding of content is held constant across all three levels of linguistic demand, due to the expected strong effects of these terms on text comprehensibility and because technical terms are not translatable into linguistic structures of lower demand without loss of specific meaning. The three-level operationalization of the model (i.e. low–medium–high demand of text with the same content) has previously been shown to allow for only small effects on student performance in physics and mathematics test items and were not coherent across all items (Höttecke et al., 2018; Plath & Leiss, 2017). However, the model can be regarded as theoretically solid; it uses linguistic features that have been shown to create processing difficulty and assumes that the simultaneous variation of several of these features creates a cumulative higher cognitive load, which in turn results in measurably less comprehension (e.g. Paas et al., 2003).
Hypotheses of Present Study
As has been demonstrated in the previous sections, research has not shown coherent results regarding if, and if so how, linguistic features of a text affect students’ text comprehension. The present study tested two hypotheses: (H1) A higher linguistic demand of an expository science text results in lower text comprehension. We expected small effects of linguistic demand (independent variable) on students’ text comprehension in science (dependent variable). Given the above considerations about the possible mediating role of perceived linguistic complexity and comprehensibility, we furthermore examined if (H2) perceived comprehensibility confounds with students’ text comprehension in science.
Method
In two qualitative pre-studies and one quantitative pre-study, an instrument to measure German secondary students’ science text comprehension was developed and validated. As part of the validation, correlations of students’ science text comprehension with general reading proficiency (LGVT instrument; Schneider et al., 2007) and prior knowledge concerning the physics content (Einhaus, 2007) were used as control variables. The basic idea of the LGVT is to present participants with a narrative text in which numerous words are blanked and then the right word must be chosen from three options. Because prior domain-specific knowledge and general reading proficiency are among the strongest predictors of learning in general and text comprehension in particular (e.g. DIME Model: Cromley & Azevedo, 2007; Schmitz, 2015; Shapiro, 2004), we expected moderate-to-high correlations of students’ science text comprehension with their general reading proficiency and prior knowledge.
We presented the validated instrument consisting of three introductory texts on three subtopics of thermodynamics (i.e. particle model, thermal expansion, perception of heat) and a total of 27 single-select, multiple-choice items to 812 secondary school students (age M = 13.1, SD = 0.87; 51.7% female; 86.9% academic-school track; 39.6% grade 7; 43.0% grade 8; 17.3% grade 9). All items were targeting students’ text comprehension. The sample size exceeded the size of 690 that was requested by an a priori power analyses for a one-way analysis of variance (ANOVA) with three groups conducted with G*Power (\(\alpha\)= 0.05, \(1-\beta\)= 0.95, expected small effect size f = 0.15). Most students in our sample spoke German at home (75.1%); a minority (24.8%) used at least one additional language at home.
As demonstrated in the schematic schedule of the main study (Fig. 1), the order of the three subtopics (i.e. particle model, thermal expansion, perception of heat) and the corresponding text comprehension items were the same for each student. The linguistic demand (i.e. A = low, B = medium, C = high) was systematically rotated to reduce order effects on linguistic demand and to control learning effects on physical content.
Materials
Development of Text Variants
First, we applied stable comprehensibility measures according to Göpferich (2009) to maintain a similarly high degree of well-formedness/comprehensibility for all texts and to control any additional variance that might otherwise overshadow any effect of linguistic demand on text comprehension (Fig. 2, left box). To this aim, the application of Göpferich’s dimensions on the four text versions about issues in thermodynamics were checked, rechecked, and verified by a group of trained linguistic experts (Hackemann, 2022; Fig. 2, left cycle). Figure 2 presents all steps followed in the development of text variants in our study.
All texts begin with a description and explanation of an everyday physical phenomenon, followed by three sections that address frequent pre-instructional ideas and elaborate further on the phenomenon (e.g. structure, stimulative additives). All information presented is concerned with the phenomenon at stake only while any narrative elements are avoided (e.g. brevity). Each section begins with a subheading to clarify content structure (e.g. perceptibility). Through this structure, the texts partly fit the definition of refutation text, which has been empirically proven to foster learning (e.g. Ariasi & Mason, 2014; Mason et al., 2019). Classic refutation texts make students’ pre-instructional ideas about a particular topic explicit, argue directly against them, and finally introduce a new scientific concept as a viable way of thinking (Ariasi & Mason, 2014). Text structures in our study differ from refutation text as students’ pre-instructional ideas are merely addressed as a guiding principle for a physics explanation instead of being central.
Although textbooks are usually multicoded and encompass text as well as images, tables, charts and the like, this study is based upon the use of plain text only to isolate the single effect of linguistic demand (e.g. Brown & Hudson, 1998; Hung, 2014; Jian, 2019). The decision was made against an extraction of expository text from German physics textbooks because they hardly meet the high baseline of comprehensibility according to Göpferich (2009) and because they had been found difficult to comprehend (e.g. Härtig et al., 2012a, 2012b).
Qualitative Validation of Texts’ Comprehensibility
To ensure the overall comprehensibility and perceptibility of all texts in our study, a pre-study was conducted based upon focused interviews (Merton & Kendall, 1979). In total, 16 grade 8 students first read one of the draft texts and then answered a series of questions concerning text comprehensibility and text perception. The results of the qualitative content analysis following the guidelines of Boyatzis (1998) and Mayring (2015) (Cohens K=0.82) showed that one text version was less comprehensible compared to the others; therefore, it was deleted from the text pool. The remaining three texts were rated by the students as more comprehensible compared to typical textbooks (Hackemann, 2022); students’ estimations of text comprehensibility fit well with the comprehensibility predicted by the model of Göpferich (2009). Additionally, minor inconsistencies and unanticipated comprehension problems (e.g. with the technical term matter) led to further comprehension improvements to the remaining three texts.
Validation of Implementation of the Model of Linguistic Demands
The three texts developed up to this point had to be varied across levels of linguistic demand. This variation was collaboratively and iteratively discussed by experts of physics education and linguistics to ensure that further linguistic dimension (e.g. structure, brevity, etc.) as well as the quality of how physics content (e.g. accuracy) was held constant (Fig. 2, right cycle). Table 1 contains a short, translated section of text that presents how linguistic demands were varied.
Table 2 shows detailed information about the variants of linguistic demand of the texts used in our study and reports mean frequencies of the linguistic features varied across the three levels. Word frequency was estimated through one of the standard language corpora for German (https://wortschatz.uni-leipzig.de/de) with a scale of word frequency from 1 (very frequent) to 25 (very rare) with all domain-specific technical terms excluded from the analysis. The three levels—low (1–9), medium (10–13), and high (> 13)—were used to match the frequency levels to the overall categories of linguistic demands: A (low), B (medium), and C (high). All presented values matched our model.
As part of the validation, a score for readability and cohesion (Kulgemeyer & Starauschek, 2014) was calculated and are shown in Table 3. With few exceptions, the values indicate that the implementation of the model applied in this study (Heine et al., 2018) indeed does result in the expected variation of linguistic demand. Values for word count, 3 + -syllables words, and the Vienna expository text formula (Bamberger & Vanacek, 1984) describe the structural complexity of a text and, therefore, increase as expected with level of linguistic demand from A across B to C. However, local text cohesion of nouns per sentence ran contrary to the model. This can be explained by the drastically reduced number of sentences in B-variants and particularly in C-variants (see number of word count per sentence in Table 2). To take this into consideration, we established the local text cohesion of nouns per clause additionally to the score provided by Kulgemeyer and Starauschek (2014). We assumed that the ratio of neighboring clauses connected by the same noun is a stronger indicator for a text’s cohesion in comparison to the ratio of neighboring sentences connected by the same noun, especially when complexity of sentence structures increases. Table 3 reports values for the local text cohesion of nouns per clause, corroborating the intention of the model. Additionally, values of the mean ratio of factual terms corroborated the model by being almost constant across levels of linguistic demand.
To summarize, all texts developed for our study meet a high baseline of comprehensibility (Fig. 2: left cycle). They address three different topics in thermodynamics. All texts were systematically varied on three levels of linguistic demand (Fig. 2: right cycle). This was done by a systematic variation of linguistic features frequently assumed to create higher structural complexity, cognitive load, and lower transparency (Fig. 2: Model of linguistic demand). Our major expectation was that this variation should result in different levels of student comprehension (H1).
Measures
Development of Science Text Comprehension Items
Mullis et al. (2015) presented four comprehension processes used in the PIRLS Framework, which differed only marginally from processes applied in other studies: “Focus on and Retrieve Explicitly Stated Information; Make Straightforward Inferences; Interpret and Integrate Ideas and Information; Evaluate and Critique Content and Textual Elements” (p. 13). To assess the construct of science text comprehension (dependent variable), we applied the first three cognitive processes but not the fourth since it would require readers to make a justified judgment based upon their wider understanding of the world, which is not a focus in our study. Each item was presented with one attractor and four distractors. For illustration, a translated item is presented below; it addresses the lowest cognitive process (i.e. focusing and retrieving explicitly stated information) and refers to the section presented in Table 1:
How long has the particle model existed?
-
• More than 20 years.
-
• More than 100 years.
-
• More than 200 years.
-
• More than 500 years.
-
• More than 2000 years.
To validate our instrument, we conducted two pre-studies that are described below.
Qualitative Validation of Science Text Comprehension Items. First, the quality, general comprehension, and perception of 51 items were evaluated by applying the method of stimulated recall (Gass & Mackey, 2017). For this purpose, 32 secondary school students read one text variant and then answered 10 items each. After completing an item, students were asked to justify their choice and which knowledge they thought they had applied to solve the item (e.g. which sections of the text appeared to matter to them). If an item was answered incorrectly, students were immediately provided with the correct answer and were asked if—and if so, why—they changed their mind. By applying a qualitative content analysis (Cohens K=0.85; Mayring, 2015), we examined students’ verbal explanations and answers of the multiple-choice items to determine if the explanation and answer chosen confounded. Overall, 37 items functioned as intended and appeared to be comprehensible, while 7 items were revised by either improving the prompt or any of the multiple-choice options. Another 7 items were eliminated from the pool mostly because students were able to choose the right answer without any reference to the text.
Quantitative Confirmatory Analyses of Science Text Comprehension Test. Second, we evaluated item quality of our science text comprehension test in a quantitative pilot study of 147 secondary school students who answered 11 prior knowledge items, read three texts, and then answered 45 text comprehension items each. The results were used to either develop or delete items by checking item and test quality values (e.g. MNSQ fit, one-dimensionality, differential item functioning of the IRT analysis) of both tests separately.
As expected, most items of the science text comprehension test met the requirements of IRT modeling (Robitzsch et al., 2020; Rost, 2004). Nine items were eliminated because either (a) their MNSQ infit was out of the range of 0.8 and 1.2, (b) their MNSQ outfit was out of the range of 0.7 and 1.3, or (c) they showed differential item function with split-criteria gender, school track, or chance in the Mantel–Haenszel test with p \(\ge\) 0.05. The final item pool reached the required values in the DETECT test for one-dimensionality (weighted DETECT = 0.32, ASSI = 0.15, RATIO = 0.22 in which values of DETECT > 0.40, ASSI > 0.25, RATIO > 0.26 are interpreted as more dimensionality; Zhang & Stout, 1999). These results all supported construct validity since only a small number of misfitting items occurred and the test proved to be one-dimensional.
Multiple-choice options were either rephrased or their order changed if students with a high weighted likelihood estimation (WLE) of person ability chose a distractor rather than the correct attractor. This was the case when the point biserial correlation between person WLE and an attractor of an item was \({r}_{pb.WLE}\le\) 0.3 or for distractors \({r}_{pb.WLE}>\) 0. The same procedure was required if the relative frequency of a distractor was below 5%. After deletion and reformulation, 35 items remained in the item pool for the main study. Items that addressed more similar physics content were excluded in the next step because test time constraints in the main study led to a total of 27 items (9 per text). The aims of this procedure were to address as broad a range of cognitive abilities as possible and to distribute item difficulties equally across each text.
Results
The prior knowledge test met all requirements of IRT modeling stated above; therefore, all items were used to estimate item and person parameters. The WLE reliability of the prior knowledge test turned out to be low (R = 0.37) due to the small number of items, with a mean person parameter of M = 0.001 (SD = 0.95) and mean item difficulty of M = 0.49 (SD = 1.27). Due to the restricted test time, administration of an increased number of items in the prior knowledge test would have resulted in fewer items in the science text comprehension test. This was not an option for us since text comprehension was our dependent variable.
Two items of the science text comprehension test presented values outside the range of \(0.8\le \mathrm{MNSQ}-\mathrm{Infit}\le 1.2\), thereby violating the IRT modeling requirements; they were excluded from further analysis. The remaining 25 items met all requirements of the IRT model and were used to estimate item and person parameters. For each item, three difficulty values were estimated in accordance with the corresponding linguistic demand A, B, or C of the text read before answering the item. The WLE reliability of the science text comprehension test was sufficient (R = 0.78) with a mean person parameter of M = 0.01 (SD = 1.08) and a mean item difficulty of M = -0.41 (SD = 0.80).
Correlations of Science Text Comprehension with Further Constructs
We first checked if the results of our science text comprehension test correlated with traits of a test person as control variables according to spearman. There were positive correlations between science text comprehension and reading proficiency (r = 0.39), prior knowledge (r = 0.50), cultural capital (r = 0.22), class grade (r = 0.24), and school track (non-academic/academic) (r = 0.33), all at p < 0.001. Cultural capital was operationalized by a books-at-home index. The number of books at home has been indicated as a strong predictor for students’ reading proficiency (e.g. Mullis et al., 2015). As expected, reading proficiency and prior knowledge were the strongest predictors of science text comprehension. In accordance with Härtig et al. (2012a, 2012b) and Höttecke et al. (2017), low correlations of reading proficiency either in general or science showed that they are separate but related constructs. Unsurprisingly, prior knowledge had the highest correlation since we only selected items from Einhaus’ (2007) thermodynamic competence test that corresponded to the subtopics addressed by our instrument. All correlations were moderate and corroborated the validity of our science text comprehension test in addition to the results of the pre-studies. Additionally, multilingual students who speak more than one language at home and students who only speak German at home accomplished similar results, whereas multilingual students who speak a single language other than German at home accomplished significantly lower results (F(739, 2) = 6.86, p < 0.001, with post hoc Tukey test at p < 0.05). Nevertheless, the latter result should not be overstated since research has shown how speaking a single language other than German at home confounds with lower reading proficiency, prior knowledge, cultural capital, and chosen school track. While at first sight multilingualism is seen to correlate with science text comprehension, on second sight regression modeling indicates a minor role of multilingualism under control of reading proficiency and prior knowledge according to research (e.g. OECD, 2017; Prediger et al., 2015) and our results (Hackemann, 2022).
Effect of Linguistic Demands on Science Text Comprehension (H1)
Our expectation that a text drafted with a high linguistic demand would lead to lower students’ text comprehension (H1) must be rejected. Table 4 shows there is a slight increase in mean item difficulty from linguistic demand A across B to C as expected, but this effect is not significant.
Additionally, no significant differences in item solution frequency in subgroups of poor to strong readers and students with low to high prior knowledge (divided by split-criteria median of prior knowledge person ability and reading proficiency) were found. Even after computing a one-way ANOVA for comparisons in top/bottom quantiles of students’ reading proficiency, no differences in item solution frequency across linguistic demands were found (top quantile F(2, 72) = 0.15; p = 0.86; bottom quantile F(2, 72) = 0.30; p = 0.74) or for students’ top (F(2, 72) = 0.02; p = 0.98) and bottom (F(2, 72) = 0.02; p = 0.98) quantiles regarding prior knowledge or any other computed subgroups. Additionally, no significant differences of item difficulty across linguistic demand for any isolated text could be identified (Hackemann, 2022).
Figure 3 shows the Wright map of all computed item difficulty values of our science text comprehension test ordered by subtopics in thermodynamics. Item difficulty values corresponding to the same item and different linguistic demands are connected by black lines to accentuate differences across the three levels of linguistic demand. Some of the black lines indicate the expected influence of linguistic complexity on item difficulty across levels of linguistic demand (e.g. Item A7) while others show the opposite (e.g. T7) or almost no difference (e.g. A15). Difficulty values of items referring to the text presented the latest appeared to be generally higher than others (F(2, 72) = 3.80; p = 0.03). This result is due to an ordering effect since students’ attention might have decreased across test time, resulting in higher item difficulty values.
Note. Item difficulty values corresponding to the same item but to different levels of linguistic demand (A-B-C) are connected by black lines. Items starting with A refer to the text particle model, items starting with T refer to the text thermal expansion, and items starting with W refer to the text perception of heat.
Perceived Comprehensibility of Our Instrument (H2)
As expected, our results indicate that the students’ perceived degree of comprehensibility was affected by the linguistic demands. Text comprehensibility was assessed by asking all students to rate the texts on levels from 1 (very comprehensible) to 6 (not comprehensible). Students’ perceived comprehensibility was 2.50 (SD = 1.12) for the low linguistic demand A, 2.52 (SD = 1.11) for the medium linguistic demand B, and 2.88 (SD = 1.2) for the high linguistic demand C. A one-way within-subject ANOVA showed that the effect of linguistic demands on students’ comprehensibility rating as an indicator of their perceived complexity was significant (F(2, 1512) = 40.24; p < 0.001). Two paired-samples t tests were applied as post hoc comparison tests to investigate if levels of linguistic demand might have resulted in significant differences of perceived comprehensibility. Significant differences of perceived comprehensibility were found between variants A and C (t(754) = − 7.82; p < 0.001) as well as B and C (t(754) = − 7.26; p < 0.001); differences of perceived comprehensibility of linguistic demand between A and B failed to be significant (t(754) = − 0.53; p = 1.00). This is seen as proof of students perceiving variant C as harder to comprehend than the other two. Nevertheless, (H2) must be rejected as well; the perceived comprehensibility does not confound with students’ actual text comprehension because the latter does not vary significantly across levels of linguistic demand (Table 4).
Discussion
The present experiment assessed the influence of linguistic features as well as perceived comprehensibility on students’ ability to comprehend expository science text. However, linguistic demand could not explain the variance in science text comprehension we found in our sample of secondary school students.
Challenging to Read, Easy to Comprehend?
Even though a significant effect of linguistic demands on text comprehension was not detected, our results point to the fact that students perceive a text as less comprehensible if it is linguistically demanding. More succinctly, linguistic demands were perceived as a challenge, but, in terms of text comprehension, there were none. This result appears to be contradictory but can be explained in plausible ways. It may be the case that if a text is perceived as linguistically demanding students will feel challenged. Consequently, they might read the text more carefully, more slowly, and with an increased level of attention and focus. Hence, an effect of linguistic demands on text comprehension might be compensated by an increased awareness of these demands, resulting in an adaption of reading strategies. Moreover, the results presented in this paper corroborate findings of Tolochko et al. (2019) on the interrelation of perceived complexity and actual text comprehension, thereby putting into question studies that were based upon students’ perceived complexity and comprehensibility as an indicator of actual text comprehension.
Based on the research and findings presented here, we assume that the perception of high linguistic demand may lead to an adaptation of reading strategies resulting in a compensatory effect. Whether reading strategies will be adapted in such a case may be mediated by students’ motivation and interest in the content presented to them. This would mean that students’ lower performance in assessments might not be related to their reading proficiency alone but to affective traits.
Is the Influence of Linguistic Complexity on Text Comprehension Currently Overstated?
As stated by Pohl (2016), recent research is doing the second step before the first if concepts we do not yet fully understand are used to inform pedagogical programs for academic language acquisition. Instead, we should first deepen our knowledge about the effects of academic language features on learning in general and on text comprehension in particular. We contribute to the body of research by unveiling that the systematic variation of linguistic features found to create higher cognitive load—including academic language features frequently assumed to generate difficulty in text comprehension—does not significantly affect text comprehension after all. Nevertheless, even on a rather demanding level of linguistic complexity, the level of text comprehension has been found not to decrease. In accordance with prior research (Härtig et al., 2019), the results of our study provide further evidence that the correlation of students’ reading proficiency and science performance found in large-scale assessments might be less related to the linguistic demand posed in assessment items or expository texts than expected. All this implies that a reduction of linguistic demands in science classes will most likely not support science learning for regular secondary school students in general.
Limitations
Due to limited test time as well as regulations of the corona-pandemic, a limited number of factors influencing text comprehension and item difficulty were controlled. Thus, our results are based upon a convenience sample that underrepresented the non-academic school track. The design itself has limitations as well; the generally high text comprehensibility of our instrument could partly explain the results. Maybe drafting an expository physics text by means of a comprehensibility model like Göpferich’s (2009) results in a strong reduction of cognitive load or even the absence of any overload. If this is true, a generally high comprehensibility of an expository science text might lead to a compensation of its high linguistic demand. As a result, this limitation allows for the apprehension that authors of school textbooks should applicate comprehensibility models that support science text comprehension and learning.
In corroboration with Höttecke et al. (2018), our results query the assumption of a linear correlation of linguistic demand and level of text comprehension suggested by Heine et al. (2018). Contrasting ample psychological research varying singular surface features, the more realistic variation of linguistic demands by multiple surface features does not explain different levels of text comprehension in our study. We assume that science text comprehension is a rather complex activity influenced by various factors usually not controlled in experiments. Further research and development of such models need to consider additional influencing factors of text comprehension with a focus on mediating effects of linguistic demand such as the above-mentioned affective and motivational traits.
Future Research
All this implies that further research is required. Since it is unclear if the text of our instrument is more comprehensible in comparison to those in textbooks, studies comparing our text with expository physics textbooks focusing on the same content are needed. Most likely certain specifics of our text—such as its structure, the consideration of students’ pre-instructional ideas, and a focus on a physical phenomenon—foster students’ comprehension independently of their linguistic demands. Additionally, it is still unclear how a reader’s perceived complexity affects their attention or motivation to deal with a text, leading to an adaption of reading strategies and varying levels of text comprehension. In future studies, the adaption of reading strategies as well as motivational and affective factors should be considered with a closer focus on interaction effects.
References
Ariasi, N., & Mason, L. (2014). From covert processes to overt outcomes of refutation text reading: The interplay of science text structure and working memory capacity through eye fixations. International Journal of Science and Mathematics Education, 12(3), 493–523.
Bamberger, R., & Vanacek, E. (1984). Lesen-Verstehen-Lernen-Schreiben [Reading-comprehending-learning-writing]. Jugend und Volk.
Berendes, K., Vajjala, S., Meurers, D., Bryant, D., Wagner, W., Chinkina, M., & Trautwein, U. (2017). Reading demands in secondary school: Does the linguistic complexity of textbooks increase with grade level and the academic orientation of the school track? Journal of Education Psychology, 110(4), 1–26.
Berndt, R. S., Mitchum, C. C., Burton, M. W., & Heandiges, A. N. (2004). Comprehension of reversible sentences in aphasia: The effects of verb meaning. Cognitive Neuropsychology, 21(2), 229–244.
Boyatzis, R. E. (1998). Transforming qualitative information. SAGE.
Bransford, J. D., & Johnson, M. K. (2014). Good-enough language processing: Evidence from sentence-video matching. Journal of Psycholinguistics Research, 43(1), 27–43.
Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. Teachers of English to Speakers of Other Languages, 32(4), 653–675.
Cassels, J. R. T. (1980). Language and thinking in science: Some investigations with multiple choice questions. University of Glasgow.
Cooper, B. R., Moore, J. E., Powers, C. J., Cleveland, M., & Greenberg, M. T. (2014). Patterns of early reading and social skills associated with academic success in elementary school. Early Education and Development, 25(8), 1248–1264.
Cornelis, L. (1996). English and Dutch: The passive difference. Language Sciences, 18(1–2), 247–264.
Cromley, J. G. (2009). Reading achievement and science proficiency: International comparisons from the Program on International Student Assessment. Reading Psychology, 30(2), 89–118.
Cromley, J. G., & Azevedo, R. (2007). Testing and refining the direct and inferential mediation model of reading comprehension. Journal of Educational Psychology, 99(2), 311–325.
Cruz Neri, N., Guill, K., & Retelsdorf, J. (2021). Language in science performance: Do good readers perform better? European Journal of Psychology of Education, 36, 45–61.
Dempster, E. R., & Reddy, V. (2007). Item readability and science achievement in TIMSS 2003 in South Africa. Science Education, 91(6), 906–925.
Deppner, J. (1989). Fachsprache der Chemie in der Schule: Empirische Untersuchung zum Textverständnis und Ansätze zur sprachlichen Förderung türkischer und deutscher Schülerinnen und Schüler [Terminology of chemistry in school: Empirical investigation of text comprehension and approaches to language acquisition of Turkish and German students] (Doctoral dissertation), University of Heidelberg.
Einhaus, E. (2007). Schülerkompetenzen im Bereich Wärmelehre – Entwicklung eines Testinstruments zur Überprüfung und Weiterentwicklung eines normativen Modells fachbezogener Kompetenzen [Concepts of thermodynamics – Development of an instrument to assess and develop a normative model of science competence]. Logos.
Fang, Z., Schleppegrell, M. J., & Cox, B. E. (2006). Understanding the language demands of schooling: Nouns in academic registers. Journal of Literacy Research, 38(3), 247–273.
Funke, R., Wieland, R., Schünenberg, S., & Melzer, F. (2013). Exploring syntactic structures in first-language education: Effects on literacy-related achievements. L1 Educational Studies in Language and Literature, 13(1), 1–24.
Fusté-Herrmann, B. (2008). Idiom comprehension in bilingual and monolingual adolescents (Doctoral dissertation), University of South Florida. USF Tampa Graduate Theses and Dissertations. https://digitalcommons.usf.edu/etd/256
Gabler, K. (2013). Wortstellungsvariation im Deutschen: Psycholinguistische Untersuchungen zur Relativsatzposition [Variation of wording in German: Psycholinguistic investigation of positioning relative clauses] (Doctoral dissertation), Philipps-Universität Marburg.
Gass, S., & Mackey, A. (2017). Stimulated recall methodology in applied linguistics and L2 research (2nd ed.). Routledge.
Göpferich, S. (2009). Comprehensibility assessment using the Karlsruher comprehensibility concept. Journal of Specialized Translation, 11, 31–52.
Greasser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234.
Haag, N., Heppt, B., Roppelt, A., & Stanat, P. (2015). Linguistic simplification of mathematics items: Effects of language minority students in Germany. European Journal of Psychology and Education, 30(2), 145–167.
Hackemann, T. (2022). Textverständlichkeit sprachlich variierter physikbezogener Sachtexte [Text comprehensibility of factual physics tests in diverging language demands] (Unpublished doctoral dissertation), University of Hamburg.
Härtig, H., Fraser, N., Bernholt, S., & Retelsdorf, J. (2019). Kann man Sachtexte vereinfachen? – Ergebnisse einer Generalisierungsstudie zum Textverständnis [Can you simplify expository texts? – Results of a generalization study on text comprehension]. Zeitschrift Für Didaktik Der Naturwissenschaften, 25, 273–287.
Härtig, H., Heitmann, P., & Retelsdorf, J. (2015). Analyse der Aufgaben zur Evaluation der Bildungsstandards in Physik – Differenzierung von schriftsprachlichen Fähigkeiten und Fachlichkeit [Analyses of items to evaluate the educational standards in physics – Differentiation between written language proficiency and content knowledge]. Zeitschrift Für Erziehungswissenschaft, 18, 763–779.
Härtig, H., Kauertz, A., & Fischer, H. E. (2012a). Nutzung von Schulbüchern zur Unterrichtsvorbereitung in Physik [Use of textbooks for class preparation in physics]. Der Mathematische Und Naturwissenschaftliche Unterricht, 65(4), 197–200.
Härtig, H., Pehlke, C., Fischer, H. E., & Schmeck, A. (2012b). Sind Fachsprache und Fachwissen bezogen auf Physik unterscheidbar? [Can technical language and specialist knowledge be distinguished in relation to physics?]. Zeitschrift Für Didaktik Der Naturwissenschaften, 18, 381–390.
Heine, L., Domenech, M., Otto, L., Neumann, A., Krelle, M., Leiss, D., Höttecke, D., Ehmke, T., & Schwippert, K. (2018). Modellierung sprachlicher Anforderungen in Testaufgaben verschiedener Unterrichtsfächer: Theoretische und empirische Grundlagen [Modeling language requirements in test tasks of various subjects: Theoretical and empirical foundations]. Zeitschrift Für Angewandte Linguistik, 69, 69–96. https://doi.org/10.1515/zfal-2018-0017
Heppt, B., Haag, N., Böhme, K., & Stanat, P. (2015). The role of academic-language features for reading comprehension of language-minority students and students from low-SES families. Reading Research Quarterly, 50(1), 61–82.
Höttecke, D., Ehmke, T., Krieger, C., & Kulik, M. (2017). Vergleichende Messung fachsprachlicher Fähigkeiten in den Domänen Physik und Sport [Assessment of language competencies in the domains of physics and sport]. Zeitschrift Für Didaktik Der Naturwissenschaften, 23(1), 53–69.
Höttecke, D., Feser, M. S., Heine, L., & Ehmke, T. (2018). Do linguistic features influence item difficulty in physics assessments? Science Education Review Letters, 6(1), 1–6.
Hsu, P.-L., & Yang, W.-G. (2007). Print and image integration of science texts and reading comprehension: A systematic functional linguistic perspective. International Journal of Science and Mathematics Education, 5(4), 639–659.
Hulstijn, J. H. (2015). Language proficiency in native and non-native speakers: Theory and research. John Benjamins.
Hung, Y.-N. (2014). “What are you looking at?” An eye movement exploration in science text reading. International Journal of Science and Mathematics Education, 12(2), 241–260.
Irujo, S. (1993). Steering clear: Avoidance in the production of idioms. International Review of Applied Linguistics in Language Teaching, 31(3), 205–220.
Jian, Y.-C. (2019). Reading instructions facilitate signaling effect on science text for young readers: An eye-movement study. International Journal of Science and Mathematics Education, 17(2), 503–522.
Kamalski, J., Sanders, T., & Lentz, L. (2008). Coherence marking, prior knowledge and comprehension of informative and persuasive texts: Sorting things out. Discourse Processes, 45(4), 323–345.
Kareva, V., & Echevarria, J. (2013). Using the SIOP model for effective content teaching with second and foreign language learners. Journal of Education and Training Studies, 1(2), 239–248.
Kettler, R. J., Dickenson, T. S., Bennett, H. L., Morgan, G. B., Gilmore, J. A., Beddow, P. A., Swaffield, S., Turner, L., Herrera, B., Turner, C., & Palmer, P. W. (2012). Enhancing the accessibility of high school science tests: A multistate experiment. Exceptional Children, 79(1), 91–106.
Kieffer, M. J., Lesaux, N. K., Rivera, M., & Francis, D. J. (2009). Accommodations for English language learners taking large-scale assessments: A meta-analysis on effectiveness and validity. Review of Educational Research, 79(3), 1168–1201.
Kieffer, M. J., Rivera, C., & Francis, D. J. (2012). Practical guidelines for the education of English language learners: Research-based recommendations for the use of accommodations in large-scale assessments, 2012 update. RMC Research Corporation, Center on Instruction. https://files.eric.ed.gov/fulltext/ED537635.pdf
Kliegl, R., Grabner, E., Rolfs, M., & Engbert, R. (2004). Length, frequency, and predictability effects of words on eye movements in reading. European Journal of Cognitive Psychology, 16(1–2), 262–284.
Köhne, J., Kronenwerth, S., Redder, A., Schuth, E., & Weinert, S. (2015). Bildungssprachlicher Wortschatz – linguistische und psychologische Fundierung und Itementwicklung [Educational vocabulary – Linguistic and psychological foundation and item development]. In A. Redder, J. Naumann, & R. Tracy (Eds.), Forschungsinitiative Sprachdiagnostik und Sprachförderung – Ergebnisse (pp. 67–92). Waxmann.
Kohnen, N., Bernholt, S., Retelsdorf, J., & Härtig, H. (2017a). Textverständnis im Physikunterricht [Text comprehension in physics classes]. In C. Meurer (Ed.), Implementation fachdidaktischer Innovation im Spiegel von Forschung und Praxis. Gesellschaft für Didaktik der Chemie und Physik Jahrestagung in Zürich 2016 (pp. 436–439). Universität Regensburg.
Kohnen, N., Härtig, H., Bernholt, S., & Retelsdorf, J. (2017b). Leichte Sprache im Physikunterricht [Easy language in physics classes]. In B. Bock, U. Fix, & D. Lange (Eds.), Leichte Sprache im Spiegel theoretischer und angewandter Forschung (pp. 337–342). Frank & Timme.
Konieczny, L. (2000). Locality and parsing complexity. Journal of Psycholinguistic Research, 29(6), 627–645.
Korpershoek, H., Kuyper, H., & van der Werf, G. (2015). The relation between students’ math and reading ability and their mathematics, physics, and chemistry examination grades in secondary education. International Journal of Science and Mathematics Education, 13(5), 1013–1037.
Kulgemeyer, C., & Starauschek, E. (2014). Analyse der Verständlichkeit naturwissenschaftlicher Fachtexte [Comprehensibility analyses of expository science texts]. In D. Krüger, H. Schecker, & I. Parchmann (Eds.), Methoden in der naturwissenschaftsdidaktischen Forschung (pp. 241–253). Springer Spektrum.
Levy, R., Fedorenko, E., Breen, M., & Gibson, T. (2012). The processing of extraposed structures in English. Cognition, 122(1), 12–36.
Levy, R., & Keller, F. (2013). Expectation and locality effects in German verb-final structures. Journal of Memory and Language, 68(2), 199–222.
Linderholm, T., Everson, M. G., van den Broek, P., Mischinski, M., Crittenden, A., & Samuels, J. (2008). Effects of causal text revisions on more and less skilled readers? Comprehension of easy and difficult text. Cognition and Instruction, 18(4), 525–556.
Lutz, B. (2015). Verständlichkeitsforschung transdisziplinär: Plädoyer für eine anwenderfreundliche Wissensgesellschaft [Transdisciplinary comprehensibility research: A plea for a user-friendly knowledge society]. V&R Unipress.
Mason, L., Zaccoletti, S., Carretti, B., Scrimin, S., & Diakidoy, I.-A.N. (2019). The role of inhibition in conceptual learning from refutation and standard expository texts. International Journal of Science and Mathematics Education, 17(3), 483–501.
Mayring, P. (2015). Qualitative Inhaltsanalyse – Grundlagen und Techniken [Qualitative content analysis: Basics and techniques]. Beltz.
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interaction of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14(1), 1–43.
McNamara, D. S., & Kintsch, W. (1996). Learning from text: Effects of prior knowledge and text coherence. Discourse Processes, 22(3), 247–288.
Merton, R. K., & Kendall, P. L. (1979). Das fokussierte Interview [The focused interview]. In C. Hopf & E. Weingarten (Eds.), Qualitative Sozialforschung (pp. 171–204). Klett-Cotta.
Mullis, I. V. S., Martin, M. O., & Sainsbury, M. (2015). PIRLS 2016 reading framework. In I. V. S. Mullis & M. O. Martin (Eds.), PIRLS 2016 assessment framework (2nd ed., pp. 11–29). Boston College TIMSS & PIRLS International Study Center.
Organization for Economic Cooperation and Development. (2017). PISA 2015 science framework. In PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (rev. ed., pp. 19–48). Author. https://doi.org/10.1787/9789264281820-3-en
O’Reilly, T., & McNamara, D. (2007). The impact of science knowledge, reading skill, and reading strategy knowledge on more traditional “high-stakes” measures of high school students’ science achievement. American Educational Research Journal, 44(1), 161–196.
Ozuru, Y., Dempsey, K., & McNamara, D. (2009). Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction, 19(3), 228–242.
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4.
Pennock-Roman, M., & Rivera, C. (2011). Mean effects of test accommodations for ELLs and non-ELLs: A meta-analysis of experimental studies. Educational Measurement: Issues and Practice, 30(3), 10–28.
Plath, J., & Leiss, D. (2017). The impact of linguistic complexity on the solution of mathematical modelling tasks. ZDM, 48(7), 1–13.
Pohl, T. (2016). Die Epistemisierung des Unterrichts diskurses – ein Forschungsrahmen [The epistimation of instructional discourse: A research framework]. In E. Tschirner, O. Bärenfänger, & J. Möhring (Eds.), Deutsch als fremde Bildungssprache. Das Spannungsfeld von Fachwissen, sprachlicher Kompetenz, Diagnostik und Didaktik (pp. 55–80). Stauffenberg.
Pöhler, B., George, A. C., Prediger, S., & Weinert, H. (2017). Are word problems really more difficult for students with low language proficiency? Investigating percent items in different formats and types. IEJME – Mathematics Education, 12(7), 667–687.
Prediger, S., Wilhelm, N., Büchter, A., Gürsoy, E., & Benholz, C. (2015). Sprachkompetenz und Mathematikleistung – Empirische Untersuchung sprachlich bedingter Hürden in den Zentralen Prüfungen 10 [Language competence and mathematics performance – Empirical investigation of language-related obstacles in the central examinations 10]. Journal Für Mathematik-Didaktik, 36(1), 77–104.
Prophet, R. B., & Badede, N. B. (2009). Language and student performance in junior secondary science examinations: The case of second language learners in Botswana. International Journal of Science and Mathematics Education, 7(2), 235–251.
Rabe, T., & Mikelskis, H. F. (2007). Kohärenzbildungshilfen und Selbsterklärungen: Fördern sie das Physiklernen [Fostering coherence and self-explanations: Are they fostering physics learning?]. Zeitschrift Für Didaktik Der Naturwissenschaften, 13, 33–51.
Rivera, C., & Stansfield, C. W. (2004). The effect of linguistic simplification of science test items on score comparability. Educational Assessment, 9(3), 79–105.
Robitzsch, A., Kiefer, T., & Wu, M. (2020). TAM: Test analysis modules (R package version 3.4–26) [Computer software]. https://CRAN.R-project.org/package=TAM
Rost, J. (2004). Lehrbuch Testtheorie – Testkonstruktion [Textbook test theory – Test construction]. Hans Huber Verlag.
Schiemann, P. (2011). Fachsprache in biologischen Testaufgaben [Terminology in biological assessment items]. Zeitschrift Für Didaktik Der Naturwissenschaften, 17, 115–136.
Schleppegrell, M. J. (2004). The language of schooling – A functional linguistics perspective. Lawrence Erlbaum Associates.
Schmitz, A. (2015). Verständlichkeit von Sachtexten – Wirkung der globalen Textkohäsion auf das Textverständnis von Schülern [Comprehensibility of expository texts – Effect of a text’s global cohesion on students’ text comprehension]. Springer. https://doi.org/10.1007/978-3-658-12016-0
Schnotz, W. (1994). Aufbau von Wissensstrukturen. Untersuchung zur Kohärenzbildung beim Wissenserwerb mit Texten (Fortschritte der psychologischen Forschung) [Construction of knowledge: Investigation of coherence building with expository texts]. Beltz.
Schneider, W., Schlagmüller, M., & Ennemoser, M. (2007). LGVT – Lesegeschwindigkeits- und -verständnistest für die Klassen 6–12 [Reading speed and comprehension test for grades 6–12]. Hogrefe.
Shapiro, A. M. (2004). How including prior knowledge as a subject variable may change outcomes of learning research. American Educational Research Journal, 41(1), 159–189.
Snow, C. E. (2010). Academic language and the challenge of reading for learning about science. Science, 328(5977), 448–450.
Solomyak, O., & Marantz, A. (2010). Evidence for early morphological decomposition in visual word recognition: A single-trial correlational MEG study. Journal of Cognitive Neuroscience, 22(9), 2042–2057.
Starauschek, E. (2003). Ergebnisse einer Schülerbefragung über Physikbücher [Results of a student survey about physics textbooks]. Zeitschrift Für Didaktik Der Naturwissenschaften, 9, 135–146.
Starauschek, E. (2006). Der Einfluss von Textkohäsion und gegenständlichen externen piktoralen Repräsentationen auf die Verständlichkeit von Texten zum Physiklernen [The influence of textual cohesion and physical external pictorial representations on the comprehensibility of texts for physics learning]. Zeitschrift Für Didaktik Der Naturwissenschaften, 12, 127–157.
Štekauer, P. (2005). Meaning predictability in word formation. John Benjamins.
Stiller, J., Hartmann, S., Mathesius, S., Straube, P., Tiemann, R., Nordmeier, V., Krüger, D., Upmeier, Z., & Belzen, A. (2016). Assessing scientific reasoning: A comprehensive evaluation of item features that affect item difficulty. Assessment & Evaluation in Higher Education, 41(5), 721–732.
Stoffelsma, L., & Spooren, W. (2019). The relationship between English reading proficiency and academic achievement of first-year science and mathematics students in a multilingual context. International Journal of Science and Mathematics Education, 17(2), 905–922.
Tolochko, P., Song, H., & Boomgaarden, H. (2019). “That looks hard!”: Effects of objective and perceived textual complexity on factual and structural political knowledge. Political Communication, 36(4), 609–628.
Yore, L. D., Hand, B., Goldman, S. R., Hildebrand, G. M., Osborne, J. F., Treagust, D. F., & Wallace, C. S. (2004). New directions in language and science education research. Reading Research Quarterly, 39(3), 347–352. https://doi.org/10.1598/RRQ.39.3.4
Zhang, J., & Stout, W. (1999). The theoretical detect index of dimensionality and its application to approximate simple structure. Psychometrika, 64(2), 213–249.
Acknowledgements
The authors would like to thank all staff and students participating in the study as well as Prof. Dr. Knut Schwippert and all members of the working groups Language Education and Multilingualism (Ruhr University Bochum) and of Physics Education (University of Hamburg) for support and feedback.
Funding
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical Approval and Consent to Participate
The study reported in this paper had the Institute of education monitoring and quality development of Hamburg (Institut für Bildungsmonitoring und Qualitätsentwicklung - lfBQ) aproval (181-24.03/218,62). Informed consent was obatined from every schools headmasters and all participants as well as their legal guardians. Data handling and analysis followed the rights of the General Data Protection Regulation (Datenschutz-Grundverordnung - DSGVO).
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Hackemann, T., Heine, L. & Höttecke, D. Challenging to Read, Easy to Comprehend? Effects of Linguistic Demands on Secondary Students’ Text Comprehension in Physics. Int J of Sci and Math Educ 20 (Suppl 1), 43–68 (2022). https://doi.org/10.1007/s10763-022-10306-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10763-022-10306-1