Introduction

While the reading research community has a long history of competing viewpoints about numerous aspects of reading, the field has reached consensus around the importance of preventing reading difficulties in the early elementary grades (K-3) rather than relying on remediation of reading difficulties in the upper-elementary grades (4–6; Juel, 1988; Snow, Burns, & Griffin, 1998). Unfortunately, despite major national initiatives such as Reading First and Title I that were designed to prevent reading difficulties before fourth-grade (Gamse, Bloom, Kemple, & Jacob, 2008), remediation of reading difficulties remains abundantly necessary in fourth-grade and beyond, particularly for low-income and minority children (Chall, Jacobs, & Baldwin, 1990; Daane, Campbell, Grigg, Goodman, & Oranje, 2005; Snow, 2002). For example, Biancarosa and Snow (2006) noted that over half of all Black and Latino children in fourth-grade scored “below basic” in reading—the lowest performance level—on recent administrations of the National Assessment of Educational Progress (NAEP). Using longitudinal data, Loveless (2007) found that recent cohorts of fourth- to eighth-grade (2003–2007) minority children have made similarly low reading gains on NAEP relative to earlier cohorts of U.S. school children (1998–2002). Both cross-sectional and longitudinal analyses of NAEP highlight low levels of literacy attainment among minority children and a general stagnation in reading growth in the upper elementary and middle grades.

To improve the reading skills of underperforming children, educators must identify and implement interventions that address the heterogeneity of reading difficulties among older children. Guidance from the National Reading Panel (2000) report suggests that phonemic awareness, phonics, fluency, vocabulary, and comprehension are five key components of scientifically-based reading instruction. Since many struggling readers require remediation in more than one area, Deshler and Hock (2007) have proposed an Adolescent Reading Model (ARM) for improving both word recognition (i.e., phonological awareness, decoding, sight word reading, fluency) and language comprehension abilities (background knowledge, syntax, vocabulary, text structures). Therefore, adopting a mixed-method intervention makes sense particularly if it addresses weaknesses that are typical of struggling readers in grades 4–6. The present study describes how a mixed-methods literacy intervention like READ 180 may address the reading difficulties of older children.

Addressing reading difficulties among children in the upper elementary grades

Children who fail to acquire basic word reading skills in the primary elementary grades (K-3) typically fall behind in reading during the late elementary grades (4–6) and undergo a marked deceleration in comprehension after fourth-grade (Chall, Jacobs, & Baldwin, 1990). Over time, poor readers tend to avoid reading and engage in less self-initiated reading habits than good readers who master the alphabetic principle in the early elementary grades. As good readers engage in additional reading practice both inside and outside, they enjoy greater improvements in reading ability than poor readers. This phenomenon contributes to Matthew effects in reading, whereby poor readers have less exposure to print and fall further behind good readers (Juel, 1988; Stanovich, 1986), and variability in print exposure in the elementary grades contributes to individual differences in reading achievement (Adams, 2006; Anderson, Wilson, & Fielding, 1988; Stanovich, 1986, 2000; Torgesen, 2005). Given the deficits in print exposure that have accumulated across the elementary grades, it is often difficult for educators to address the large disparities in reading practice and skill between good and poor readers in the upper elementary and middle grades (Ceci & Papierno, 2005; Juel, 1988; Torgesen et al., 2006). Torgesen (2005), for example, has argued that it becomes increasingly difficult to close the gap in fluency, vocabulary, and comprehension for older children who have struggled to read for several years and have major deficits in reading practice. Consequently, many poor readers in the upper elementary and middle grades have small sight word vocabularies and limited opportunities to acquire the background knowledge needed to comprehend grade-level texts.

Struggling readers are especially apt to fall behind when making the transition from “learning to read” to “reading to learn the new.” Chall, Jacobs, and Baldwin (1990) coined the term “fourth-grade slump” to describe how deficits in vocabulary underlie the deceleration in reading comprehension among low-income children in the upper elementary grades. Indeed, many struggling readers fall behind most rapidly on measures of vocabulary in the late elementary grades and struggle to understand the more complex demands of reading abstract content (Chall et al., 1990; Kieffer & Lesaux, 2007). As a result, teachers are called upon to employ a mix of instructional methods to improve the reading skills of poor readers in the upper-elementary and middle grades (Buly & Valencia, 2002).

READ 180: a mixed-method approach to reading instruction

READ 180 is a mixed-method approach (Slavin, Cheung, Groff, & Lake, 2008) to literacy instruction that is designed to help struggling readers in grades 4–12 improve their word reading efficiency, reading comprehension and vocabulary, and oral reading fluency. In the full 90-min version of READ 180, teachers begin with a 20–30 min whole group lesson and then create small groups of children who participate in three 20-min activities, in which reading practice is scaffolded by computer activities, leveled books, and teacher lessons tailored to the reading level of each small group (Hasselbring & Goin, 2004).

During the whole group lesson, teachers use non-fiction text to develop children’s background knowledge in history, geography, science, and math. Teachers then provide direct instruction in high-utility vocabulary words that appear across these content areas. For example, in a unit on immigration, teachers ask questions to activate prior knowledge, model fluent reading from a passage on the topic, invite students to read the text together, and introduce target words that appear in subsequent READ 180 reading activities. In the vocabulary lessons, teachers usually begin by pronouncing the target word (e.g., citizenship), defining the word, using the word in sentences, asking questions that require students to use the target word, and providing children with multiple exposures of each target word (Beck, McKeown, & Kucan, 2002).

After the whole group lesson, teachers create three small groups of ~4–6 children. Each small group rotates through three 20-min activities, including (1) individualized computer-assisted instruction, (2) independent and modeled reading of leveled books, and (3) teacher-directed lessons tailored to the needs of a small group. Depending on the specific reading level of the children in each of the three small groups, teachers may review word reading strategies, model fluent reading, or review reading comprehension strategies (Deshler & Hock, 2007; Slavin, Cheung, Groff, & Lake, 2008).

Although READ 180 is a widely used intervention for struggling readers in U.S. public schools, there is a dearth of rigorous studies by independent evaluators (Slavin, Cheung, Groff, & Lake, 2008). There are four main limitations with the existing evaluations of READ 180. First, literacy scholars have underscored the need for more independent experimental evidence on the efficacy of the READ 180 program (Deshler & Hock, 2007). A best evidence synthesis (Slavin, Cheung, Groff, & Lake, 2008) suggested that READ 180 had “moderate evidence of effectiveness,” yielding a mean weighted effect size of +.24 on reading comprehension tests from eight studies using quasi-experimental designs. Second, none of these quasi-experimental studies measured improvements in word reading efficiency and oral reading fluency which are skills that READ 180 is designed to improve. Third, evaluations that compare the performance of children in READ 180 classrooms and non-READ 180 classrooms during the regular school day fail to isolate the instructional effects of the intervention on student outcomes. In their best evidence synthesis of READ 180, Slavin, Cheung, Groff, and Lake (2008) noted that “many students in READ 180 classes received considerably more instructional time in reading than did their counterparts in control classes…[and] instructional time was confounded with the effects of the program itself” (p. 295). Fourth, READ 180 includes a mix of instructional activities that are designed to increase print exposure and reading practice. Specifically, in two out of the three 20-min rotations, the goal is to increase the time children spend reading leveled books and the number of words children read in the computer activities. To date, no studies have examined whether independent reading of leveled books and reading words on the computer activities predict posttest reading scores. Past research suggests that individual differences in print exposure may explain residual variance in posttest reading scores after measures of prior reading ability are partialed out (Stanovich, 2000).

Study goals

To address the four limitations in previous evaluations of READ 180, we used an experimental design to estimate treatment effects, measured improvements on multiple measures of reading achievement, and eliminated the confound between time and the instructional effects of READ 180 by controlling the amount of time children spent in their after-school program. Finally, we examined whether two measures of print exposure which were part of the READ 180 rotations—book reading and the number of words read on the computer activities—explained variance in posttest reading scores.

Method

Participants and design

The present study was part of a larger implementation study (Hartry, Fitzgerald, & Porter, 2008) that was conducted in a high-poverty school district located in southeastern Massachusetts. Children were recruited from three elementary schools with a large percentage of struggling readers. Struggling readers were identified as children in grades four to six who scored below proficiency on the Massachusetts Comprehensive Assessment System (MCAS), a standards based assessment of the state English language arts curriculum. Children were eligible for this study if they scored below proficiency on their most recent MCAS English language arts assessment (Grades 3–5).

A power analysis was undertaken to identify the number of children needed to detect an effect size of ~.20 standard deviations on standardized tests of reading. Assuming a .70 to .80 correlation between pre and posttest reading measures, the power analysis suggested that ~250–350 children were needed to detect an effect size of +.20 standard deviation (two-tailed test with α = .05), which is in line with the mean effect size of +.24 in Slavin et al.’s (2008) review of READ 180. In this study, 294 children received active consent to participate in the study. As shown in Table 1, Black and Latino children comprised over 70% of the sample and 81% of the children received free- or reduced-price lunch.

Table 1 Characteristics of the sample at the beginning of the study (N = 294)
Full size table

Procedures

The READ 180 program and the district after-school program were administered 4 days per week for ~23 weeks from October 2005 to April 2006. There were a total of 20 certified teachers in this study; 10 teachers used the READ 180 materials and 10 teachers used the district after-school program. For both groups of children, the after-school program began with a 1-h session for snack and homework assistance. In the second 1-h session, teachers followed the curriculum in either the READ 180 program or the district after-school program. In the present study, the 90 min READ 180 model was adapted to fit a 60 min timeframe in order to accommodate the district’s after-school program schedule (Hartry et al., 2008). As a result, READ 180 teachers did not implement the whole group lesson. Instead, READ 180 teachers implemented three 20-min rotations.

Table 2 provides information on the activities that were part of the three READ 180 rotations and the district after-school program. READ 180 differed from the district after-school activities because of its exclusive focus on improving children’s reading skills. During the 20-min individualized computer-assisted READ 180 instruction, children participated in scaffolded reading practice with videos, leveled text, and word reading and fluency activities. To begin the computer activities, children chose from a selection of content areas topics (e.g., people and culture, science and math, and history and geography). Next, children watched a video which was designed to create mental models that support comprehension of text that appeared in subsequent reading activities (McNamara, Miller, & Bransford, 1991). After the video introduction, children moved on to the Reading Zone and listened to related texts that were narrated and then completed ten comprehension questions about the text. Children were required to correctly answer ten questions in order to receive credit for having read all the words in the text, yielding a useful measure of print exposure measure in this study. The next two computer activities (Spelling Zone and Word Zone) provided opportunities for children to practice spelling and reading words that were embedded in the Reading Zone. Throughout the activities, vocabulary building tasks such as reading words in context and options to have words read out loud or defined are available. In the final computer activity, children were directed to the Success Zone where they completed comprehension and fluency activities. Children concluded by making a final recording of the passage and listening to their oral reading of the text. In the second READ 180 rotation, children read books that were matched to their Lexile level and were given the option to have an audio recording to scaffold their independent reading. During independent reading time, students had access to all the books and were directed toward independent reading books that matched their Lexile. The third rotation involved teacher-directed lessons that were tailored to the reading level of children in each small group. The small group lesson was typically composed of a mini-lesson in which teachers helped children read phonetically difficult words, modeled fluent reading, and reviewed reading comprehension strategies (e.g., using context clues, identifying key words).

Table 2 Curriculum and instructional activities in 60 minute after-school program
Full size table

The district after-school program included both literacy and non-literacy related activities and the amount of time devoted a specific activity varied each day. As shown in the final column of Table 2, the district after-school program did not include daily computer-assisted reading activities and independent and modeled reading of leveled texts. In short, it included no activities that were comparable to rotations 1 and 2 in READ 180. Instead of following a structured 20-min rotation, teachers had the flexibility to implement lessons that were designed to encourage attendance in after-school programs by making activities interesting and diverse for children with poor academic performance. Each teacher was provided with a selection of 16 activities to choose from or they could develop their own activities. The activities included informal art based projects, games, and commercially developed materials for after-school programs, including InstaCamp themed kits (i.e., astronomy, history, geography, space exploration) and a math/literacy curriculum (KidzLit/KidzMath). Thus, the district after-school program was not specifically focused on improving reading and there was no structured time in the daily schedule to practice reading leveled text. The after-school program did, however, include some teacher-directed lessons that gave children opportunities to deepen content knowledge in history, geography, science, and math.

Measures

To address our research questions, we conducted pre and posttest observations of READ 180 classrooms to assess fidelity of intervention; administered pre and posttest measures of word reading efficiency, reading comprehension and vocabulary, and oral reading fluency; collected data on student attendance; administered a posttest survey of children’s after-school experiences; and, obtained a measure of the number of words children in the experimental condition read during the computer-assisted reading activities using the READ 180 software.

Test of Word Reading Efficiency (TOWRE). The Test of Word Reading Efficiency (TOWRE, Torgesen, Wagner, & Rashotte, 1999) is a nationally normed measure of word reading accuracy and fluency. It is individually administered and provides an efficient means of monitoring the growth of two kinds of word reading skills that are critical in the development of overall reading ability: the ability to recognize familiar words as whole units or “sight words” (max = 104) and the ability to “sound out” pseudo-words (max = 63). Reported alternate forms reliability coefficients exceed .90, and test/retest reliability coefficients range from .83 to .96.

Group Reading Assessment and Diagnostic Evaluation (GRADE). The Group Reading Assessment and Diagnostic Evaluation (GRADE) is group administered and includes the following subtests: vocabulary, sentence comprehension, and passage comprehension (Williams, 2001). Scores can be computed separately for vocabulary and comprehension (sentence and passage comprehension). Each child was given a test booklet that corresponded to his grade level (Level 4–6). Reported alternate form reliabilities were above .87 for levels 4, 5, and 6; test-retest reliabilities were .98, .77, .94, respectively, for each corresponding level.

DIBELS Oral Reading Fluency (DORF). The Dynamic Indicators of Basic Early Literacy Skills Oral Reading Fluency (DORF) is a standardized, individually administered test of children’s ability to read connected text with accuracy and speed (Good & Kaminski, 2002). Children read a passage aloud for 1 min and words omitted, substituted, and hesitations of more than three-seconds are scored as errors. Words self-corrected within 3 s are scored as accurate. The number of words correctly read per minute (WCPM) is the oral reading fluency rate. Good and Kaminski (2002) reported test-retest reliabilities for elementary grades ranging from .92 to .97. Children were administered three grade appropriate passages at pretest and posttest and the middle score from the three readings was used as the measure of oral reading fluency.

Massachusetts Comprehensive Assessment System (MCAS). The MCAS is a performance-based assessment that evaluates student knowledge and mastery of the Massachusetts curriculum frameworks in English language arts and three other content areas (mathematics, science and technology/engineering, and history and social science). Scaled scores of 240 on the MCAS are used as the cut point for determining whether students have met the proficient performance level determined by state policymakers. Scaled scores on the MCAS English language arts assessments in grades 4, 5, and 6 were used as the performance-based outcome measure.

Attendance. After-school teachers and supervisors kept attendance records on the children from the beginning of the study in October 2005 to the end of the study in May 2006.

Posttest Survey of After-school Programs. Children completed a 32-item survey about their reading motivation and after-school experiences. Items 1–19 included measures adopted from existing surveys of intrinsic motivation to read (Wigfield & Guthrie, 1997). Items 20–27 specifically asked about children’s experiences in their after-school programs. We used items 20–27 to assess the students’ attitudes and level of engagement with respect to their after-school program. Four items asked children to respond to the following items: I learn interesting things at my after-school program, I learn a lot at my after-school program, I like to go to my after-school program, and there are too many rules to follow at my after-school program. Response options were scored as (1) not true at all, (2) not too true, (3) somewhat true, and (4) very true. The last four items probed children’s after-school activities, including what participants thought about their after-school program (1 = I do not like my program, 2 = my program is okay, 3 = I like my program), how often participants read books each week (1 = none, 2 = 1 day a week, 3 = 2–3 days a week, 4 = everyday), and how much time adults helped with homework (1 = never, 2 = less than 15 min, 3 = 15–30 min, 4 = more than 30 min).

Measures of Print Exposure in READ 180. We measured children’s print exposure in READ 180 in two ways. First, we used the self-reported data on the frequency with which children read books (M = 2.79, SD = 1.19, Min = 1, Max = 4) to determine the number of days per week that children read books during the READ 180 intervention. Second, we obtained a measure of the number of words children read through the computer-assisted reading activities, which provided an estimate of each child’s level of print exposure during the implementation of the READ 180 computer activities. After correctly answering 10 comprehension and vocabulary questions on the first READ 180 computer activity (i.e., Reading Zone), each student received credit for reading the text and the computer generated data on the total number of words each child read during the intervention (M = 14,301, SD = 9,829).

Fidelity of READ 180 Implementation. Each READ 180 classroom was observed in November 2005 and again in April 2006 and rated using a 3-point scale (low fidelity = 1 to high fidelity = 3) to assess the fidelity of implementation for each of the three 20-min rotations (for details see Hartry, Fitzgerald, & Porter, 2008). In computer rotations implemented with high fidelity, the session lasted 18–20 min, children had access to working computers, software, microphones, and headphones, and were engaged in the READ 180 computer activities. Independent and modeled reading rotations were rated with high fidelity if children had access to READ 180 books at their Lexile level and were engaged in reading books for 18–20 min. Raters evaluated small group lessons as being implemented with high fidelity if children were grouped according to their reading level and received teacher-directed lessons on word study, fluency, or comprehension strategies that were related to READ 180. In general, raters observed teachers implementing all rotations with high fidelity. Averaged across all classrooms, the mean fidelity scored in November and April was 3.0 and 2.8 for the computer rotations, 3.0 and 2.8 for the independent reading rotations, and 2.9 and 2.3 for the small-group rotations.

Results

Treatment effects

Prior to estimating the treatment effects, we undertook analyses to check for baseline equivalence on pretests following random assignment. There was no statistically significant difference between the children in READ 180 and the district after-school program on pretest measures of word reading efficiency, reading comprehension and vocabulary, and oral reading fluency (all p’s > .20). From pretest to posttest, nearly 10% of the original sample was lost to attrition, reducing the sample from 294 children at pretest to 264 children at posttest. However, there was no significant relationship between attrition rates and condition, χ2 (294, 1) = .094, p > .10. These findings suggest that non-equivalence in reading skills at baseline and differential attrition did not threaten the internal validity of the treatment effects.

Moreover, there was no statistically significant difference on the GRADE total reading test for the final sample of 264 children who took all pretests and posttests (M = 90.66, SD = 11.39) and the 22 children who took only the pretest (M = 88.55, SD = 12.55), t(284) = −.83, p > .20. These findings indicate that the external validity of the results was not compromised by the attrition of particularly low- or high-performing children.

Does READ 180 improve word reading efficiency, reading comprehension and vocabulary, oral reading fluency, and attendance in after school programs?

Table 3 displays the pretest and posttest means and standard deviations on all reading measures for the final sample of children in READ 180 and the district after-school program. Visual inspection of the descriptive statistics suggests that the children in READ 180 and the district after-school program experienced gains on all measures of word reading efficiency and the measure of reading comprehension. Paired t-tests revealed significant improvements for both groups of children on the TOWRE total score, sight word reading, and phonetic decoding and the GRADE total score and comprehension score. However, there were no significant pre-posttest gains on the GRADE vocabulary tests for children in READ 180, t(132) = −.55, NS, and for children in the district after-school program, t(130) = .01, NS.

Table 3 Comparison of pretest and posttest means standard deviations on all reading measures, by experimental condition (N = 264)
Full size table

In order to estimate the impact of READ 180 on reading outcomes, we conducted ANCOVA on each posttest score using the relevant pretest score as the covariate. Children in the READ 180 condition performed no better than children in the district after-school program on measures of total word reading efficiency, F(1, 261) = .09, NS, phonetic decoding, F(1, 261) = .45, NS, and sight word reading, F(1, 261) = 1.93, p = .17. Similarly, there were no significant differences between the two groups on the GRADE total test, F(1, 261) = .32, NS, the comprehension subtest, F(1, 261) = .42, NS, and the vocabulary subtest F(1, 261) = .08, NS. However, children in READ 180 scored significantly higher on the measure of oral reading fluency than the control group, F(1, 261) = 4.41, p = .037.

After the conclusion of the study in April, all participating children took the MCAS in English language arts. At each grade level, an MCAS scaled score of 240 represented the minimum score for determining whether children were proficient in reading. The descriptive statistics in Table 3 suggest that the mean MCAS scores for students who participated in both the READ 180 and the district after-school program were below the proficiency cut score of 240. ANCOVA, with total reading scores from the GRADE as the covariate, also revealed no significant difference between the two groups on MCAS, F(1, 261) = 1.11, p = .29.

The final analysis compared attendance rates between the two groups. Attendance rates were significantly higher for children in the READ 180 group (M = 68.07, SD = 19.63) than for children in the district after-school program (M = 61.44, SD = 23.16), t(262) = 2.509, p = .0127.

The last column of Table 3 displays effect sizes (Hedges, 1981) based on the difference between the covariate-adjusted posttest means divided by the pooled within group standard deviation. The magnitude of the effect size was positive and significant for attendance (ES = .31) and the curriculum-based measure of oral reading fluency (ES = .12). The magnitude of the effect size for the GRADE measures in comprehension (ES = −.09) and vocabulary (ES = .03) was smaller than the effect sizes for oral reading fluency and word reading efficiency.

Does the impact of READ 180 differ by ethnicity, free or reduced-price lunch status, gender, reading level, and grade level?

Because READ 180 was designed to help subgroups of lower-performing children, we also conducted analyses to test interactions between the treatment and background characteristics. We found no evidence that effects on the measure of word reading efficiency and reading comprehension and vocabulary differed by ethnicity, free or reduced-price lunch status, or gender. Using the 25th percentile of the TOWRE pretest score to split the sample into good and readers, we found no significant treatment by reading level interaction effect on reading comprehension and vocabulary.

There was some evidence, however, that the magnitude of the treatment effect differed by grade level on measures of attendance and oral reading fluency. In Grade 4, children in READ 180 (M = 68.50, SD = 19.47) had significantly higher attendance rates than children in the district after-school program (M = 55.55, SD = 25.79), t(91) = 2.708, p = .008. In grades 5 and 6, however, there was no significant difference in attendance rates between the two groups. Similarly, ANCOVA on the oral reading fluency scores suggested that only grade 4 children in the READ 180 group read more fluently than the children in the district after-school program, F(1, 90) = 9.93, p = .002. There were no significant differences in oral reading fluency for children in grade 5 and 6. In other words, the magnitude of the effect size for oral reading fluency in grade 4 (ES = .25) was larger than the effect sizes in grade 5 (ES = .00) and grade 6 (ES = .07). These findings indicate that the significant main effects on attendance and fluency were driven primarily by gains among grade 4 children in the READ 180 condition.

Does READ 180 improve attitudes toward after-school literacy instruction, academic engagement, and time spent reading books?

We used posttest survey data to address questions about children’s after-school experiences. READ 180 children were more likely than control students to agree with the following questions: “I learned interesting things,” t(261) = 2.18, p < .05, and “I learned a lot,” t(260) = 2.79, p < .01. READ 180 children reported reading books more frequently (M = 3.34, SD = 1.03) than control children (M = 2.23, SD = 1.08), t(261) = 8.81, p < .001. More specifically, READ 180 children reported reading books ~3–4 days a week compared to ~1 day a week for control group children. We did not observe significant differences between the two groups on the other survey items.Footnote 1

Do measures of print exposure explain significant variance in word reading efficiency, reading vocabulary and comprehension, and oral reading fluency after all pretest reading scores are partialed out?

We undertook descriptive, correlational, and regression analyses to understand whether measures of print exposure explained significant variance on measures of word reading efficiency, reading comprehension and vocabulary, and oral reading fluency. These analyses were based on data for 119 out of 133 children in READ 180 with pre and posttest reading scores, survey data on book reading, and computer data on the number of words read. Fourteen children had missing computer data and were not included in subsequent analyses. There was no statistically significant difference on the TOWRE total pretest score between the 119 children (M = 89.16, SD = 12.39) who were included in the analyses and the 14 children (M = 90.71, SD = 13.29) who were excluded from the analyses due to missing data, t(131) = .44, NS. Similarly, there was no significant difference between these two groups of children on GRADE total pretest scores.

To begin, Table 4 displays descriptive statistics of the total number of words a child read in the individualized computer-assisted activities and attendance rates. Because the number of words read on READ 180 displayed a non-linear relationship with posttest reading scores, we included a logarithm transformation of the measure in the analyses involving parametric tests. The descriptive statistics highlight clear differences in children’s attendance rates and the number of words that children read on the READ 180 computer activities. Children whose attendance rate was at 90th percentile read 26,017 words, on average, whereas children whose attendance rate was at the 50th and 10th percentile read 12,109 words and 4,766 words, respectively. These findings suggest that children with high attendance rates typically read more words on the READ 180 computer lessons than children with low attendance rates.

Table 4 Descriptive characteristics of print exposure measures for children in READ 180 (n = 119)
Full size table

Table 5 displays zero-order correlations among the dependent and independent variables. All correlations among the pretest and posttest reading measures were significant (p’s < .01). Moreover, the total number of words read on the READ 180 computer activities was positively correlated with pretests and posttests. However, the mean number of days per week that children reported reading books was unrelated to all pretest and posttest reading measures. These correlations suggest that only the print exposure measure from the computer activities was positively associated with reading scores.

Table 5 Intercorrelations among posttest reading measures and print exposure for READ 180 children (n = 119)
Full size table

Table 6 displays the results from a series of multiple regression analyses involving the GRADE comprehension and vocabulary scores. Table 7 displays results from multiple regression analyses involving oral reading fluency, sight word reading, and phonemic decoding. Pretest reading scores were entered in model 1 followed by the variables representing the two print exposure measures in model 2. Therefore, the regressions analysis in model 2 removed the effects of pretest scores in reading comprehension and vocabulary, oral reading fluency, phonemic decoding, and sight word reading and then examined whether reading books and reading words on the READ 180 computer activities explained variance on each posttest. Each table presents the unstandardized regression coefficient (B), standard error of B (SE B), the standardized regression coefficients (β), and R 2.

Table 6 Multiple regression analyses with reading comprehension and vocabulary scores for children in READ 180 (n = 119)
Full size table
Table 7 Multiple regression analyses with oral reading fluency, sight word reading, and phonemic decoding scores for children in READ 180 (n = 119)
Full size table

Results from two regression analyses revealed significant relations between the total words read on the READ 180 activities and posttest scores. In Table 6, the total number of words read was positively associated with scores on the GRADE vocabulary posttests. After partialing out all pretest scores, the number of words children read in the READ 180 computer activities explained an additional 4% of the variance on the vocabulary posttest scores. In Table 7, the total number of words reads was positively associated with scores on the TOWRE sight word reading posttests. After partialing out all pretest scores, the number of words children read in the READ 180 computer activities explained an additional 3% of the variance on the sight word reading posttest scores. These two findings suggest that reading words on the READ 180 computer activities was positively associated with posttest scores on vocabulary and sight word reading after scores on all pretests were partialed out. Using a multi-level model to account for the clustering of children within classrooms, we replicated the regression results reported in Tables 6 and 7. The number of words children read in READ 180 explained significant variance reading vocabulary and sight word reading after pretests were partialed out. In addition, there was no significant variability in posttest reading scores across classrooms.1

Discussion

The purpose of this investigation was to examine the efficacy of a mixed-method approach to literacy intervention and to examine whether two print exposure measures in READ 180 explained variance in reading outcomes. Designed as a comprehensive literacy intervention for struggling readers in grades 4–12, the 90-min version of READ 180 employs a mix of teacher-directed whole group lessons and three 20-min instructional activities designed to improve word reading efficiency, reading comprehension and vocabulary, and oral reading fluency. In this study, we evaluated a modified 60-min version of READ 180, including individualized computer-assisted reading activities, independent and modeled reading of leveled books, and teacher-directed lessons for small groups of children.

Although Slavin et al. (2008) reported an effect size of +.24 in a review of eight quasi-experimental evaluations of READ 180, our experimental study revealed no significant impact on norm-referenced measures of word reading efficiency and reading comprehension and vocabulary. In addition, there was no significant effect on the MCAS English language arts tests. Both children in READ 180 and the district after-school program enjoyed significant pretest to posttest gains on measures of word reading efficiency and reading comprehension, suggesting that the curriculum and instruction in each after-school program was equally effective in producing reading gains. Moreover, there was no evidence that poor fidelity of implementation or differential attrition rates undercut the efficacy of READ 180. What, then, explains the discrepancy between the positive results reported in previous quasi-experimental evaluations of READ 180 and the results in the current experiment?

First, our modified 60-min version of READ 180 did not include teacher-directed whole-group lessons for building vocabulary which is critical to enhancing the efficacy of the intervention for struggling readers (Beck, McKeown, & Kucan, 2002; Kieffer & Lesaux, 2007). The full 90-min version of READ 180 includes ~30-min of whole-class teacher-directed instruction of high utility words that appear frequently across content areas. Direct instruction of words that appear across content areas helps struggling readers acquire the vocabulary and background knowledge to improve their comprehension of grade level texts (Beck, McKeown, & Kucan, 2002). Chall et al. (1990) noted that the limited vocabulary of many low-income children underlies the fourth-grade slump in reading comprehension that continues during the middle and high school grades. In our study, reading vocabulary was the one measure on which READ 180 children did not enjoy significant pre to posttest gains, suggesting that the absence of 30-min whole group instruction may have limited vocabulary gains and the overall efficacy of the READ 180 intervention.

Second, the district after-school curriculum included some literacy activities that may have promoted gains in word reading efficiency and comprehension. Based on our review of the curriculum materials, teachers in the district after-school program were able to select from a variety of activities that were designed to deepen children’s background knowledge and to provide opportunities to read leveled text. For example, teachers in the district after-school program were able to use the KidzLit program which includes a five-part process in which children hear books read aloud by the teacher, make connections to their own lives, and express their feelings about the book through discussions, art, and writing activities. The teacher-directed lessons in the district after-school program were most similar to the READ 180 rotation in which teachers tailored lessons for small groups of children. Given the similarities in some of the instructional strategies used by teachers in both programs, it is possible that the contrast between the two approaches was not strong enough for the READ 180 activities to produce significant gains in word reading efficiency and reading comprehension and vocabulary.

Third, correlational and experimental studies underscore the challenge of remediating reading difficulties among older children. Some longitudinal studies suggest that the gap in skills between good and poor readers grows larger over time (Juel, 1988; Bast & Reitsma, 1998). Good readers who master the alphabetic principle in the early grades are more likely than poor readers to become proficient at reading texts independently, to enjoy reading outside school, and to improve their word reading efficiency, reading comprehension and vocabulary (Stanovich, 1986). Over time, the early deficits in reading practice and skills accumulate and struggling readers are unable to close the gap between themselves and proficient readers through the elementary grades and into the middle grades. There is some evidence that remedial interventions for struggling readers may be less effective with older children than younger children. For example, a large-scale randomized controlled trial of four remedial interventions produced significant effects among children in grade 3 but not in grade 5 on measures of reading accuracy, fluency, and comprehension (Torgesen et al., 2006). More specifically, the magnitude of the treatment effect for the oral reading fluency measure (i.e., words correctly read in 1 min) was larger in grade 3 (ES = .27) than grade 5 (ES = .00). In our study, there was a significant impact on oral reading fluency, which was driven by the larger effect sizes in Grade 4 (ES = .25) than in Grade 5 (ES = .00) and Grade 6 (ES = .07). Consistent with findings from a remedial reading intervention involving a multi-grade sample, findings from our study also suggest that it may be more difficult to remediate the reading difficulties of older children than younger children.

Because only fourth-grade children in READ 180 enjoyed significant gains in attendance rates, it is unclear whether the efficacy of specific components of the READ 180 intervention or the general increase in instructional time promoted improvements in oral reading fluency among fourth-grade children. Moreover, since the significant effects on fluency in fourth-grade did not generalize to the norm-referenced measures of word reading efficiency and reading comprehension and vocabulary, there is a clear need to replicate these potential grade-level differences in the efficacy of READ 180 in a future experiment.

The regression analyses also provide some tentative hypotheses about the mechanisms through which specific READ 180 activities may improve reading ability. After partialing our pretest scores in word reading efficiency, reading comprehension and vocabulary, and oral reading fluency, the number of words children read on the READ 180 computer activities was positively associated with posttest scores in reading vocabulary and sight word reading efficiency. However, the number of days per week that children reported reading books did not predict posttest scores. This finding suggests that computer aided reading instruction with videos and leveled text predicts improvements on measures of sight word reading efficiency and reading vocabulary. The READ 180 computer based activities that incorporate leveled text followed by comprehension questions may help children read sight words efficiently and acquire knowledge of word meanings (Deshler & Hock, 2007).

The number of words children read in the READ 180 computer activities served as a useful proxy for measuring print exposure because students received credit for reading the words in the computer activities only if they correctly answered 10 comprehension questions about the text. In other words, children had to demonstrate comprehension of words and text to receive credit for having read the words in the passage. This method of assessing print exposure is potentially more accurate than self-reported measures of book reading. Relying on self-report measures can result in social desirability biases and may not provide reliable estimates of print exposure (Stanovich, 2000). Nonetheless, the major limitation of the regression analysis is the correlational nature of the findings, which do not permit clear causal inferences about the impact of reading words in READ 180 on reading outcomes. To address this limitation, a follow-up experimental study might examine the impact of giving children varying amounts of time on the computer activities to isolate the causal effects of additional print exposure on reading outcomes.

Ultimately, findings from this study suggest that it may be unreasonable to expect one reading intervention—even a comprehensive, mixed-method approach to literacy instruction—to address all areas of reading weakness simultaneously. Indeed, a supplemental literacy intervention that occurs for a single year is unlikely to compensate for multiple years of reading failure. Furthermore, leaving out 30-min of teacher-directed lessons on vocabulary may further weaken the efficacy of the READ 180 intervention. In this respect, researchers and practitioners should view READ 180 as a mixed-method approach to literacy instruction where teacher-directed instruction and computer-assisted learning are both essential to improving the literacy skills of struggling readers.