What Happens to Efficiency and Equity? The Cost Implications of Developmental Education Reform

Abstract

Florida passed legislation in 2013 that exempted many students from developmental education, and required colleges to implement new instructional modalities for optional developmental education courses. This paper uses an economic model of remediation to examine how changes in college course taking before and after the reform altered costs to completing introductory college-level courses in math and English for students and institutions, and whether cost savings differ by racial/ethnic subgroup. We also take into account changes in short-term costs relative to course completion rates to examine whether the reform is cost effective overall. We find that the total cost per completer (including costs to both students and institutions) decreased after the reform by $894 in English and $1851 in math. This resulted in efficiency gains of 34% in English and 30% in math. The results from Florida provide promising evidence that statewide developmental education reforms may be an effective mechanism for reducing costs to students and institutions. Reform efforts may also help to reduce racial/ethnic gaps in the costs to gateway completion in both subject areas, which has important implications for equity.

Developmental education has significant cost implications for both students and institutions, with up to $7 billion spent annually on instructional expenditures for developmental education programs nationwide (Scott-Clayton et al. 2014). Researchers and policymakers have also expressed concerns that despite these high costs, traditional developmental education programs are ineffective, particularly for student assigned to multiple remedial courses who have a low likelihood of ever passing a college-level class in the same subject area (Bailey and Jaggars 2016). Students incur a substantial portion of developmental education costs, as they must pay tuition to enroll in developmental education courses even though they do not receive any college credit for completing them. Postsecondary institutions also devote substantial resources to providing developmental education courses and associated support services such as tutoring for underprepared students. Public institutions have faced increased scrutiny over the past decade due to calls at both the state and national levels to hold institutions more accountable for the high costs and low success rates of traditional developmental education programs (Education Commission of the States 2011).

In this study, we examine the cost effectiveness of a large-scale reform of developmental education enacted at all Florida College System (FCS) institutions since fall 2014. The FCS includes all 28 state colleges (formerly known as community colleges) in the state. The cost of developmental education in Florida prior to the reform during the 2009/2010 academic year was estimated at $154 million, which included $81 million paid by institutions (primarily via state appropriations) and $73 million paid by students through tuition (Underhill 2013). The majority of these costs (up to 70%) result from students who never complete their developmental education requirements. To address concerns about the cost and effectiveness of developmental education, Florida lawmakers passed Senate Bill 1720 (SB 1720) in 2013. This legislation greatly reduced the number of students required to take developmental education, and also required colleges to offer different instructional strategies for students remaining in developmental education courses. We use student-level records from the population of first-time-in-college (FTIC) students at all Florida state colleges to examine how costs to both students and institutions changed due to differences in college coursetaking patterns before and after the reform. We also take into account the change in short-term costs relative to gateway completion rates (for the initial college-level courses counting toward the degree requirements in math and English) to examine whether the reform is cost effective overall.

While previous studies have examined the impacts of various developmental education reform initiatives, relatively little research has explored the costs of these programs and whether the benefits exceed the costs. A primary rationale of the reform was to lower the costs of developmental education by reducing the total number of credits attempted, and we have a unique opportunity to examine whether this actually happened. These comparisons of costs and benefits have important considerations for policymakers as they evaluate different options for programs that they may consider adopting to improve student success.

Additionally, our study examines costs to both students and institutions to explore the distribution of the financial burden of developmental education. It also examines differences by students’ race/ethnicity, which has important equity implications. The costs of traditional developmental education programs may be unevenly allocated among racial/ethnic groups, as Black and Hispanic students have an increased probability of assignment to developmental education relative to their White counterparts (Attewell et al. 2006; Bettinger and Long 2005), and are more likely to need multiple levels of remediation (Bailey et al. 2010). Given that these subgroups experience different educational trajectories, the reform may alter coursetaking patterns and their associated costs in a way that benefits (or harms) some groups more than others. Therefore, we consider whether the reform addresses the broader system of inequalities that have historically shaped developmental education in terms of who enters developmental education, who gets stuck in unsuccessful pathways, and who may face unintended consequences from reform efforts.

Specifically, we address the following research questions:

• 1. How do changes in the pathways that students take to gateway courses before and after SB 1720 alter costs to students and to institutions?

  • a. Are there differences in the costs to students by racial/ethnic subgroup?

• 2. Is the developmental education reform more or less cost effective than traditional developmental education programs prior to SB 1720 in terms of average cost per gateway completer?

We proceed as follows: first, we review the literature on the costs of developmental education and the cost effectiveness of developmental education reform initiatives. Next, we provide a summary of Florida’s policy context and the developmental education reform efforts under SB 1720. We then describe the data and methods we used to estimate a simple economic model of remediation examining changes in costs of college coursetaking. In the findings section, we show that the reform was cost effective, resulting in substantial savings to both students and institutions. Additionally, Black and Hispanic students experienced greater cost savings under the reform relative to White students, thus narrowing racial/ethnic gaps. We conclude with limitations of our analysis and implications for policymakers interested in developmental education reform.

Literature Review

Aggregate estimates of the national costs of developmental education vary widely depending on the data used and the type of measurement approach. Researchers have estimated the annual cost of developmental education at $1.1 billion (Pretlow and Wathington 2012), $3.6 billion (Alliance for Excellent Education 2011), and $7 billion (Scott-Clayton, Crosta, and Belfield 2014). All of these annual estimates are based on the cost of developmental education for FTIC students at public 2-year and 4-year colleges. They take into account institutional expenditures on developmental education courses incurred by institutions and tuition paid by students for enrollment in developmental education courses, but do not factor in the cost of student time or the impact of on student outcomes. Notably, none of these prior studies has examined differences in costs incurred to students from different subgroups such as racial/ethnic backgrounds.

The low-range estimate of $1.1 billion is based on 2004–2005 state appropriations to higher education in Texas that were generalized nationwide (Pretlow and Wathington 2012). The authors estimated the costs by multiplying the percentage of freshmen credit hours in developmental education by the general state-level expenditures at state 2- and 4-year colleges. Based on these estimates, the authors project that developmental education accounts for 0.48% of total higher education revenues. However, these estimates do not account for the wide variation in spending across states.

The middle-range estimate of $3.6 billion from the Alliance for Excellent Education (2011) used data from the Delta Cost Project and the National Postsecondary Student Aid Study (NPSAS) administered by the U.S. Department of Education. The cost was calculated by multiplying the average institutional funding and tuition costs of a course in 2- and 4-year public institutions in each state by the number of students under the age of 25 who took at least one developmental education course in each state, according to NPSAS data. The study likely underestimates the total costs to colleges because it only factors in the cost of one developmental education course per student even though many students are required to take more than one developmental education course.

The largest cost estimate of $7 billion (Scott-Clayton, Crosta, and Belfield 2014) attempts to account for students who took more than one developmental education course. The authors based this estimate on the total institutional expenditures per full-time equivalent (FTE) student at 2-year public institutions, which includes cost incurred by both the colleges and students. The estimate assumes that the 3.1 million first-time students who entered any type of 2- or 4-year institution each took an average of 1.3 remedial courses. The authors calculated the cost of a course by dividing the total expenditure per FTE by eight, assuming students take eight courses per year. They estimate a cost of approximately $1620 per student for each remedial course taken. One limitation is that this estimate includes the cost of developmental education for beginning students in all types of institutions at the average rate for public two-year institutions.

Estimates are also available for statewide costs of developmental education in Florida prior to the implementation of SB 1720. A legislative analysis of 2009–2010 data estimated annual costs of developmental education to Florida colleges at $81 million (Underhill 2013). Students contributed $73 million in tuition and fees for developmental education courses, for a total investment of $154 million. The costs for students in the legislative analysis are somewhat higher than a national analysis examining state-by-state developmental education costs for tuition paid by students and their families (Jimenez et al. 2016). The national analysis used data from Complete College America and the Integrated Postsecondary Education Data System (IPEDS) to multiply total developmental education enrollment rates for first-time college students by the average net price per student. It estimated the out-of-pocket tuitions costs of developmental education in Florida in 2009–2010 at $61 million. Florida had the third highest out-of-pocket costs for developmental education among the 50 states. Nationwide, costs of developmental education ranged from slightly more than $1 million in Alaska to more than $205 million in California.

Examining changes in student costs to developmental education has important implications for equity. A study by Melguizo et al. (2008) examined variation in costs to students in the Los Angeles Community College district based on the levels of English and math courses taken in the first semester (remedial, basic, intermediate, or transfer-level). Among students who subsequently transferred to a 4-year university, the total cost of community college attendance ranged from $4000 for students who enrolled directly in transfer-level courses to $7000 for students who enrolled in at least one remedial course. Additionally, nearly 60% of remedial students were African American or Latino/a, indicating that community colleges inadequately serve their missions of promoting access and equity. While community colleges provide a basic choice of college attendance with relatively low tuition, underrepresented students disproportionately bear the additional costs and time required to take remedial courses that are nontransferable.

Developmental Education Reforms

A first wave of reforms to developmental education began to spread across the United States in the mid- to late 2000s (Jaggars and Bickerstaff 2018). These early reform efforts consisted of largely isolated practices, such as providing additional advising, tutoring services, or learning communities. These reforms tended to be small in scope and only focused on short-term outcomes. Around 2010, there was a second wave of developmental education reforms that focused more on the wide scale adoption of practices including revised procedures for course placement, implementation of accelerated strategies that may allow students to complete developmental courses more quickly, and changes to the pedagogy of developmental education courses. These current reforms have typically resulted in positive impacts on short-term outcomes like the completion of gateway courses, but there have been challenges to scaling up these reform efforts to serve all underprepared students.

Florida’s reform removed the requirement for placement testing for exempt students, essentially lowering the placement cutoff score to 0. Prior research has found that the use of placement tests to assign students into developmental education courses results in frequent misplacement, particularly in English (e.g. Leeds and Mokher 2019; Scott-Clayton et al. 2014). Underplaced students could likely succeed in a higher course level, but instead are required to take an additional developmental education course. These students have to deal with issues of discouragement and demotivation, and have their time to degree unnecessarily extended. Removing placement test cutoffs could allow more students to progress to college-level coursework more quickly.

Florida’s reform also required all colleges to replace traditional semester-long developmental education courses with one or more accelerated instructional strategies. Colleges schedule compressed courses for a greater number of hours over fewer weeks so that students may potentially complete two courses in the same semester. Contextualized courses integrate math or English remedial content in an applied context related to students’ meta-majors, or major course pathways. Modularized courses assess students’ skills and allow them to complete computerized modules for only those content areas that they have not yet mastered. Co-requisite courses allow students to enroll concurrently in developmental education and gateway courses in the same subject area. There is a growing body of research indicating that these types of accelerated instructional strategies are associated with improvements in student outcomes such as gateway course enrollment rates and credit accumulation (Hodara and Jaggars 2014; Jaggars et al. 2014; Kalamkarian et al. 2015; Ran and Lin 2019; Okimoto and Heck 2015). Yet it remains unknown how much of these improvements in student success may be attributed to the effectiveness of these instructional strategies relative to the weakening of the negative effects of underplacement in traditional semester-long developmental courses.

Recently, studies have begun to examine the costs of traditional developmental education courses relative to one of the reform models specified under SB 1720, co-requisite remediation. Under the co-requisite model, students take developmental education and college-level courses simultaneously. The Community College of Baltimore County implemented a co-requisite remediation initiative, the Accelerated Learning Program (ALP) in 2007. This was first time that the co-requisite model was developed, piloted, and implemented at scale. Students who placed into upper-level developmental writing enrolled in a gateway English class while simultaneously enrolling in a companion ALP course taught by the same instructor. An examination of the costs of the program relative to traditional developmental education revealed that ALP is more cost-effective (Jenkins et al. 2010). The study followed completers through the developmental course sequence, as well as the credit-bearing English 101 and 102 course sequence. Although ALP cost more per enrollee than traditional developmental education, a larger percentage of ALP students passed both for-credit courses, so the reform had a lower cost per completer. The cost per ALP student to pass the developmental education course and both credit-bearing courses was $2680, compared to $3122 for the traditional developmental education sequence.

A cost study of co-requisite developmental education in Tennessee found similarly positive results (Belfield et al. 2016). In 2015, Tennessee implemented co-requisite remediation in math, reading, and writing in community colleges statewide. Researchers estimated the net effect of both traditional and co-requisite remediation on passing initial credit-bearing math and writing courses, finding that co-requisite remediation was the most cost-effective approach. The co-requisite model was 50% more efficient for math courses and 11% more efficient for writing courses. The total cost for co-requisite math was $3840 per successful student versus $7220 for the traditional model, while the total cost for co-requisite writing was $3350 per successful student versus $3750 for the traditional approach.

Policy Context

While prior studies provide some promising evidence that developmental education reforms may have the potential to reduce costs to students and institutions, the scope of Florida’s reform is much broader than other contexts. SB 1720 mandated three major changes to the state’s developmental education system. First, the bill made placement testing and developmental education optional for exempt students meeting certain criteria. Exempt students include those who entered ninth grade in a Florida public high school during or after the 2003–2004 academic year and graduated with a standard high school diploma, as well as students who serve as an active member of any branch of the U.S. Armed Services. While these students still have the option to take a placement test and/or developmental education coursework, it is no longer required, and exempt students may now enroll directly in introductory college-level courses. First year enrollment rates in developmental math at FCS institutions declined following the reform from 38% in fall 2013 to 22% in fall 2014 (Hu et al. 2016). There were similar declines in enrollment following the reform in developmental reading (from 21 to 10%) and developmental writing (16 to 11%).

Second, FCS institutions became required to offer developmental courses using at least one of four instructional strategies designed to improve developmental education completion rates and help students progress through courses more efficiently. These approaches include compressed, contextualized, modularized, and co-requisite courses. An accountability report from the Florida College System (2017) indicates that compressed courses were the most common instructional strategy for developmental courses in all three subject areas (reading, writing, and math), followed by modularized courses. Finally, colleges became required to provide enhanced advising and student support services (such as tutoring) to facilitate student enrollment in the new courses and provide support for student success. All public state colleges in the FCS were required to implement these comprehensive reforms to their developmental education programs by the 2014 fall semester. Institutions were required to make these changes without additional state appropriations.

English and Math Pathways at FCS Institutions

English includes up to two levels (lower-level and upper-level) of developmental education courses in both reading and writing prior to the first gateway English course. Math includes two levels of developmental education (lower-level and upper-level). Additionally, math includes a third course (prerequisite) that counts for elective credit but does not fulfill the general education requirements for an associate’s degree. We consider all developmental education and prerequisite courses as “pre-gateway” courses, whereas we refer to the initial courses in math and English that fulfill the degree requirement as “gateway” courses. Prior to SB 1720, advisors placed students into pre-gateway or gateway courses based on their scores on the Postsecondary Education Readiness Test (PERT) or another approved common placement test such as Accuplacer, ACT, or SAT. After the reform, exempt students could choose their own course placement and enroll directly into gateway courses. This means that in in English, exempt students could enroll directly into English Composition 1 (ENC1101), the gateway course fulfilling the state English requirement for the associate’s degree. In math, exempt students most often enrolled in one of the prerequisite courses (mostly commonly MAT 1033: Intermediate Algebra) prior to enrollment into one of the four gateway math courses approved by the state (MAC1105: College Algebra, MGF1106: Mathematics for Liberal Arts I, MGF1107: Mathematics for Liberal Arts II, or STA2023: Statistical Methods I). The legislation made only developmental education courses optional for exempt students, and most students were still required to take a pre-requisite course in math like Intermediate Algebra.

Understanding the Cost Implications of Developmental Education Reform

One of the primary rationales of SB 1720 was to reduce the costs of developmental education. The assumption was that exempt students would take fewer developmental courses and progress directly to college-level courses, thus reducing the total number of credits attempted. We consider three hypothetical pathways to completing the first gateway course in English, and how the costs to both students and institutions would differ under each scenario. These hypothetical pathways provide a rationale for how we might expect costs to change under the developmental education reform.

Costs to Students

Prior to SB 1720, a successful student placed into developmental education could have been required to pass two pre-gateway courses (a lower level developmental course and an upper level development course) and one gateway course for a total of three courses, which is equivalent to 9 credit hours. Given an average in-state tuition rate of $81 per credit hour, the total cost to the student would have been $725. If the student was exempt from developmental education after SB 1720, and was able to pass the gateway course on the first attempt (taking only 3 credit hours), the total cost to the student would decrease to $242. However, if the student struggled to pass the gateway course then the costs could be higher after the reform. Students in Florida are required to pay the full cost of instruction if they fail the same course twice and attempt it again. The full cost, which is equivalent to the out-of-state tuition rate, is approximately four times greater than the subsidized in-state tuition rate. This means that if a student failed the gateway course twice before passing it on the third attempt, the total cost to the student would increase to $1445. Thus, there are very large differences in the student costs of these differential pathways, ranging from $242 to $1,445.¹

Costs to Institutions

Changes in the pathways that students take to gateway courses also have cost implications for institutions. In Florida, developmental education courses are more expensive to provide, costing institutions $218 per credit hour compared to $181 per hour for lower-level courses including gateway courses (Florida Department of Education 2016).² Variation in class size by course type is a contributing factor to these differential costs. For example, Broward College capped enrollment in developmental math courses at 30 students, while the gateway College Algebra course had a cap of 40 students prior to the reform (Iossi 2013). Thus the instructional costs to institutions under SB 1720 will decline, regardless of whether students have to retake gateway courses, as long as the total number of credit hours attempted does not exceed what students would have otherwise taken under the traditional developmental sequence prior to the reform. Under our hypothetical scenarios, the cost to institutions ranges from $1851 for the traditional sequence of two pre-gateway courses and one gateway course, to $543 if the student is able to bypass developmental education and pass a gateway course on the first attempt.

Potential Improvements to Cost Efficiency

Our earlier work has shown some initial promising results from Florida’s developmental education reform, particularly in regards to cohort-based passing rates, or the percent of students in the cohort who have successfully completed a gateway course. In the first year after SB 1720, the predicted probability of both taking and passing gateway courses in the first semester increased by 5.3% points in English and 3.7% points in mathematics relative to the year prior to the reform (Hu et al. 2016). Further, first semester enrollment rates declined in developmental education courses at a more rapid rate for Black and Hispanic students, compared to White students (Park-Gaghan et al. in press). Racial/ethnic achievement gaps in cohort-based passing rates of gateway courses narrowed in English Composition, and were eliminated in Intermediate Algebra within the first year of the reform. This suggests that the reform may improve cost efficiency by increasing the number of successful completions in gateway courses, particularly among certain racial/ethnic groups. If the impacts of the reform disproportionately favor minority groups, there should also be disproportionate cost savings.

Additionally, SB 1720 may alter the costs of developmental education for students who continue to take these courses through the availability of new instructional strategies. These courses are designed to be more cost effective than traditional courses by potentially allowing students to complete developmental education more quickly and with fewer credits. For example, instead of taking a traditional 3-credit hour developmental education course, some students may be able to take a 1- or 2-credit hour modularized course that covers only the material that the student has not yet mastered. In addition, co-requisite courses, which allow students to complete developmental education and college-level courses simultaneously, have been demonstrated as cost effective in other locales like Baltimore (Jenkins et al. 2010) and Tennessee (Belfield et al. 2016).

Methods

We follow a simple economic model of remediation examining changes in costs of college coursetaking by building on the work of Belfield et al. (2016). While their model accounts for gateway course enrollment and passing rates during the first two semesters of college enrollment, we extend our model to include eight semesters from the fall of year one to the spring of year three. Many students, particularly those who attend part-time, do not complete gateway courses in their first year of college so tracking students for a longer duration will provide a more complete estimate of total costs over time. Our model also accounts for differential costs to students based on the number of attempts, and examines heterogeneity in student costs by racial/ethnic subgroup.

We address the first research question by calculating the average total cost to students and institutions of the pathways taken to complete the first gateway course, separately for math and English. We estimate results pre- and post-SB 1720, and disaggregate the findings based on whether the students ultimately completed a gateway course by the end of the third year of college enrollment. This timeline tracks students’ coursetaking pathways for eight semesters, including summers. We disaggregate the results by completion status because non-completers are unlikely to experience any significant earnings gains in the labor market from an unsuccessful attempt at a gateway course. Under an effective reform, we would expect to see both an increase in the number of completers and a lower cost per completer. Additionally, in order to consider the reform as equitable, we would expect to see reductions in racial/ethnic gaps in terms of completion rates and cost effectiveness.

We begin by estimating the costs (C) to the first gateway course for student s as:

$$C_{s} = \left( {\sum Credits_{initial} *Tuition_{instate} } \right) + \left( {\sum Credits_{retake} *Tuition_{outstate} } \right)$$

(1)

where the total number of credits attempted in both pre-gateway and gateway courses is multiplied by the in-state tuition rate for the initial two attempts, and by the out-of-state tuition rate for additional retakes. We then calculate the average student costs as:

$$\overline{{C_{s} }} = \frac{{\sum C_{s} }}{n},$$

(2)

where the sum of the individual student costs is divided by the total number of students in the sample. We also disaggregate the results for average student costs by racial/ethnic subgroups.

The costs to institutions is calculated similarly, but takes into account the differential cost per credit hour for institutions to provide pre-gateway and gateway courses. The costs for each institution i is calculated as:

$$C_{i} = \left( {\sum Credits_{pregateway} *Cost_{pregateway} } \right) + \left( {\sum Credits_{gateway} *Cost_{gateway} } \right).$$

(3)

We then calculate the average institution costs as:

$$\overline{{C_{i} }} = \frac{{\sum C_{i} }}{{n_{i} }},$$

(4)

where the sum of the individual institution costs is divided by the total number of students in the sample enrolled at the corresponding institution.

To address the second research question about whether student pathways to the first gateway course are more cost effective under SB 1720, we calculate a cost-effectiveness ratio (CER) as follows:

$$CER = \frac{{N_{cohort} \left( {\overline{{C_{s} }} + \overline{{C_{i} }} } \right)}}{{N_{completers} }},$$

(5)

where the sum of student and institution costs is multiplied by the total number of students in the cohort, and then divided by the total number of gateway course completers. We compare the CER ratios before and after SB 1720 to evaluate each according to both their costs and their effects in regards to the desired short-term outcome of students successfully completing gateway courses.

We also examine the extent to which any gains in cost effectiveness may be attributed to students opting out of developmental education versus participating in accelerated developmental education instructional strategies in the post-reform period. For these analyses, we limit the sample to students in the post-reform cohorts with placement tests scores that would have placed them into developmental education, and compare the costs for those who ever took a developmental education course versus those enrolled directly into a college-level course.³ Even though students were no longer required to take the placement test upon college enrollment following SB 1720, over 70% of students in our post-reform cohort have placement test scores because most Florida public school students were required to take the placement test in grade 11 as part of another statewide initiative for high schools. We calculate the same CER ratio as described above in Eq. 5 for students in the post-reform cohort scoring below college-ready, and compare the results based on whether students ever took a developmental education course.

Data

Developmental education in Florida is provided almost exclusively by the 28 public community colleges, referred to as “state colleges.” The sample includes all first-time-in-college (FTIC) students at Florida’s 28 state colleges who enrolled three years prior to SB 1720 (2011/12 cohort) and during the first year after SB 1720 (2014/15 cohort). This means the analysis includes the whole cohorts of incoming students in both years. Approximately two-thirds of students (65%) met the exemption criteria. Students in the pre-policy cohort only received exposure to the traditional developmental education program during the three-year timeframe for this analysis, while students in the post-policy cohort only received exposure to the developmental education reform. We exclude students in the 2012/2013 and 2013/2014 cohorts because they were initially subject to the traditional developmental education program, but became exempt in subsequent years after the passage of SB 1720. We tracked student coursetaking outcomes for eight semesters from the first fall semester of college enrollment until the spring semester of the third year.

We use data from Florida’s K-20 Education Data Warehouse (EDW), which tracks all Florida public school students remaining in-state from Kindergarten to postsecondary education. For these analyses, we use data from college enrollment records, course transcript records, and student background characteristics. We use the course transcript data to track student enrollment and passing outcomes for each math and English course to create a coursetaking pathway for each student. The pathway includes the total number of credits attempted in each course, the number of retake credits due to three or more attempts in the same course, and the passing outcome (pass/fail) for each course. Students are coded as successfully completing a course if they passed by earning a grade of C- or higher.

Overall, we found that student characteristics were very similar in the pre- and post-reform cohorts (Table 1). The greatest difference was that students in the pre-reform period were less likely to receive free- or reduced-price lunch in high school (31%) relative to the post-reform reform cohort (40%), a standardized mean difference of -0.18. Across both cohorts, just over 20% of students were black and about one-third were Hispanic, and 53% were female. The average age is approximately 21 years for both cohorts. We also examined whether there were any differences in academic preparation based on math and English coursetaking in high school, and found small standardized mean differences of less than 0.10 for all tracks.⁴ Just over half of the students in both cohorts completed an “at-risk” or “basic” track in English, and 57% completed an “at-risk” or “basic” track in math.

Table 1 Descriptive statistics on student characteristics for the pre- and post-reform cohorts

The source of data on costs per credit hour for institutions and tuition costs for students is the Fact Book for the Florida College System (Florida Department of Education 2016). For the cost to students, the average in-state tuition rate is $80.52 per credit hour. If a student fails the same course twice, they are required to pay an out-of-state tuition rate of $360.65 per credit hour for any subsequent retakes attempted (Florida Administrative Code 6A-14.0301 2016). For the cost to institutions, the cost per credit hour is $181 for gateway courses and $218 for pre-gateway courses. This includes direct instructional costs and support costs associated with each credit hour. General revenues from state appropriations and tuition comprise the main source of these operating funds, although lottery proceeds also provide some state funds. The state allocates appropriations to each institution annually as a lump sum, which institutions utilize to pay for operational expenses and to provide various programs and services. All dollar values are from the 2014–2015 academic year, which represents the most recent year available. This ensures consistency in the costs per credit across both cohorts, so that any cost differences are only attributable to changes in coursetaking patterns after the reform.

Limitations

Our analysis is limited in that it only accounts for the short-term benefits from the pathway to the first gateway course. Benefits may be even greater if the reform sets students on a more successful trajectory that increases the likelihood of receiving a degree and reduces the time to degree completion. A study by the Georgetown University Center on Education and the Workforce found that obtaining an Associate’s degree is associated with a $200,000 increase in lifetime earnings over completing some college without a degree (Carnevale et al. 2013). Therefore, the long-term benefits in the workforce may greatly exceed the short-term benefits of coursetaking pathways if the reform continues to improve student outcomes beyond the first gateway course.

Further, our analysis does not account for other costs, such as the additional advising and student support services provided to students. Institutional leaders did not receive any additional funding to implement reform efforts; however, they may have redistributed existing resources. Many of the reform’s costs involved reallocating personnel, meaning that staff time was unavailable for other activities, even if those costs did not show up in a budget. This means that our analysis may underestimate total costs of the reform because it only captures the costs of changes in course pathways taken by students.

Results

We begin by looking at the change in the number of pre-gateway courses taken before and after SB 1720 (Table 2). We find that the percent of students not taking any pre-gateway courses in English increases 16.1 percentage points from 65.6 to 81.6%. There is a corresponding decline in pre-gateway course participation rates, particularly among students taking more than one pre-gateway courses. In math, there were smaller changes in the percentage of students taking no pre-gateway courses with values of 30.4% before the reform and 33.8% after (a difference of 3.4% points). These results are as expected, as students were only exempt from developmental education courses under the reform and may still have been required to take a pre-gateway course prerequisite such as MAT 1033. However, we find that after the reform, students are more likely to be taking only one or two pre-gateway courses in math, and less likely to be taking three or more pre-gateway courses. The results for both subjects suggests that the costs to both students and institutions should decline if pass rates for the gateway courses remain consistent or increase, as students are taking fewer pre-gateway courses.

Table 2 Number of pre-gateway courses attempted in math and English, pre- and post-SB 1720

Next, we examine changes over time in cohort-based passing rates for gateway courses, based on the number of students who both take and pass a gateway course divided by the total number of students in each cohort. After SB 1720, there is an 11-percentage point increase in the percent of students who both take and pass a gateway English course in the first semester, from 29% in the 2011 cohort to 40% in the 2014 cohort (Fig. 1). This difference diminishes slightly over time, although students in the post-reform period are still six percentage points more likely to complete a gateway English course by the end of the third year compared to students in the pre-reform cohort. The results in math indicate that only 8% of students passed a gateway course in the first fall semester both before and after the reform (Fig. 2). However, pass rates for the post-reform cohort surpass those of students in the pre-reform cohort by 4% to 6% in each of the subsequent semesters. By the end of the third year, cohort-based passing rates for gateway math courses are 34% for the pre-policy cohort and 39% for the post-policy cohort. These results also suggest that the costs to both students and institutions in both subjects should decline after the reform since more students are successfully completing a gateway course and doing so in fewer semesters.

Fig. 1

Cohort-based passing rates for gateway English courses, pre- and post-SB 1720

Fig. 2

Cohort-based passing rates for gateway math courses, pre- and post-SB 1720

Costs to Students and Institutions of English Pathways

Table 3 demonstrates how the costs for the pathways taken to complete the first gateway course in English changed after the reform for both institutions and students. The total costs to institutions decreased after the reform, from $1040 to $721, a difference of $289 per student. The average number of credits for pre-gateway courses, which cost more for institutions to provide, decreased by 1.57 credits from 2.69 to 1.11 credits per student. There was also a slight increase in the average number of gateway credits from 2.47 to 2.81 (a difference of 0.34 credits), but the total number of credits attempted (in both pre-gateway and gateway courses) was 1.23 credits lower after the reform. Institutions reduced expenditures by approximately $363 for each non-completer, compared to $235 per completer.

Table 3 Cost to students and institutions for English gateway completers and non-completers, pre-and post-SB 1720

The average costs to students decreased from $443 to $338, a difference of $106. Students completed less than a quarter of one credit at the out-of-state tuition rate for attempting the same course more than twice, so the majority of the difference in cost is attributed to changes in the number of credits attempted at the in-state tuition rate. The total number of credits for the pathway to the first gateway course declined by just over one credit, from 5.16 to 3.92 credits. The changes in costs over time differed depending on whether students completed a gateway course. Those who ultimately did not complete a gateway course spent less money under the SB1720 context than non-completers in the pre-reform context, a difference of $143.This suggests that there were fewer wasted resources among students unlikely to experience future benefits in the labor market from unsuccessfully attempts toward the first gateway course. In comparison, the average cost savings for completers was $85 per student.

Costs to Students and Institutions of Math Pathways

The results in math also indicate cost savings to both students and institutions (Table 4). The average total institutional costs declined from $1450 to $1185, representing a cost savings of $264 per student. The number of credits attempted in pre-gateway courses, which are more expensive for institutions to provide, decreased from 5.20 credits to 3.76 credits. Conversely, the number of credits attempted in gateway courses increased slightly from 1.72 to 2.02 credits. As with English, there were large reductions in costs among non-completers under SB 1720 relative to the pre-reform context, with an average difference of $357 per non-completer. Additionally, institutional costs declined by $142 per student for completers.

Table 4 Cost to students and institutions for math gateway completers and non-completers, pre-and post-SB 1720

The costs to students of completing the first gateway math course declined by $123 from an average total cost of $640 per student to $517. The number of credits completed at both the in-state and out-of-state tuition rates decreased, and the total number of credits attempted declined from 6.92 to 5.78 credits. There were greater differences in costs after the reform for non-completers than completers of gateway courses, at $176 versus $51, respectively.

Changes in Student Costs by Racial/Ethnic Subgroups

The magnitude of the cost savings gained by students varied by racial/ethnic subgroups (Table 5). Prior to the reform there were substantial racial/ethnic gaps in the cost to gateway completion in both subject areas, with the lowest gateway completion costs for White students, followed by Hispanic students, and then Black students. Many of these gaps narrowed after the reform, particularly for Black students. The average cost savings in English was nearly four times greater for Black students ($210) relative to White students ($58). In math, the cost savings for Black students ($192) was about twice the cost savings for White students ($96). Hispanic students also experienced slightly greater cost savings compared to White students in English ($97 versus $58) and in math ($112 versus $96).

Table 5 Cost savings to students by race/ethnicity and subject area for gateway completers and non-completers, pre-and post-SB 1720

Overall Changes in Cost Effectiveness

Table 6 shows the cost effectiveness of English gateway pathways before and after SB 1720. We consider how the total costs to both students and institutions have changed since the reform, relative to the percent of students who successfully completed a gateway course. The cost effectiveness ratio represents the change in the amount of resources expended to produce one successful gateway completer. The average cost per completer in English decreased from $2645 to $1751, a savings of $894. This translates into a 34% efficiency gain under the reform. After disaggregating the results by race, the greatest efficiency gains are for Black students (42%), followed by Hispanic students (28%), and then White students (25%). Math had a slightly lower efficiency gain of 30%, with the average cost per completer declining from $6212 to $4361. Results for math also demonstrated greater efficiency gains for underrepresented racial/ethnic groups, with rates of 42% for Black students, followed by 33% for Hispanic students, and then 25% for White students. These results provide evidence of an equity-generating reform.

Table 6 Cost effectiveness of English and math gateway pathways, pre- and post-SB 1720

Lastly, we wanted to provide insight into the extent to which improvements in cost efficiency may be due to students opting out of developmental education courses versus students participating in accelerated developmental education instructional strategies in the post-reform period. We limited the sample to students in the post-reform cohort who had placement test scores that indicated they were below college-ready. The results indicate that the average costs per completer were three to four times greater for students who took developmental education courses relative to those who enrolled directly into college-level courses (Table 7).This suggests that the greatest gains to cost efficiency were due to students who opted out of developmental education. However, these results must be interpreted with caution, as students with higher levels of motivation or preparation may be more likely to opt out of developmental education.

Table 7 Cost effectiveness of English and math gateway pathways for students in the post-reform cohort who scored below college-ready on the placement test, by enrollment status in developmental education

Discussion

Given the high costs and limited evidence of the effectiveness of traditional developmental education programs (e.g. Jaggars and Bickerstaff 2018; Scott-Clayton, Crosta, and Belfield 2014), it is important to understand whether improvements can be made through large-scale reform efforts. Florida has implemented one of the most comprehensive developmental education reforms to-date, making changes to placement policies, curriculum and instruction, and advising and student support services. We used statewide data from the population of FTIC students at FCS institutions to compare changes before and after the reform to the cost to the successful completion of the first college-level math and English courses. Overall, our results indicate that Florida’s reform has contributed to short-term cost savings to both students and institutions. The average cost of the pathways taken to complete the first gateway course in English decreased by $106 for students and $289 for institutions. The greatest reduction in costs were among non-completers, suggesting that fewer resources were wasted among those unlikely to experience future benefits in the labor market for their unsuccessful credit attempts. In math, the average cost savings were $123 for students and $264 for institutions. The total number of credit hours attempted by students declined after the reform, particularly among pre-gateway courses, which were more expensive for institutions to provide. These gains are especially impressive given that the reform was implemented statewide in a relatively short time with no additional funding. Prior studies on developmental education reforms in other settings have found positive impacts on short-term outcomes like gateway course completion for participants in pilot programs, but it is often difficult to sustain these improvements when scaling reform efforts to serve the majority of underprepared students (Jaggars and Bickerstaff 2018).

We also found that very few students before or after SB 1720 enrolled in the same math or English course more than twice, so there were minimal changes in the amount of out-of-state tuition paid by students for repeated attempts. In both subject areas, gateway course completion rates by the end of the third year were 5 to 6 percentage points higher after the reform. The total cost per gateway completer decreased after the reform by $894 in English and $1851 in math, resulting in efficiency gains of 34% and 30%, respectively. In comparison, Tennessee’s co-requisite developmental education was associated with efficiency gains of 50% in math 11% in writing among first year coursetaking patterns (Belfield et al. 2016). Thus, our findings suggest that while Florida’s reform is not as cost effective as Tennessee’s in math, it is more consistent in improving efficiency across multiple subject areas.

Reform efforts may also help to reduce racial/ethnic gaps in the costs to gateway completion in both subject areas, which has important implications for equity. Black students experienced the greatest benefits, with cost savings two to four times greater than for White students. We also found that Black and Hispanic students had greater efficiency gains than White students in terms of cost per completer in both subject areas, providing compelling evidence of an equity-generating reform. While previous research has documented how Black and Hispanic students are more likely to be placed into developmental education courses (Attewell et al. 2006; Bailey et al. 2010; Bettinger and Long 2005; Melguizo et al. 2008), the financial implications of these outcomes tends to be understudied. By implementing reform initiatives to accelerate student success into gateway courses, not only do students improve their educational outcomes, but they also save money in the process. This may help to reduce racial/ethnic economic disparities in postsecondary education, as Black students tend to incur substantially more student debt than White students (Addo et al. 2016).

We posit that there may be two mechanisms through which underrepresented students experience greater efficiency gains under the reform. First, removing requirements for placement testing and developmental education may weaken the negative effects of underplacement that occur under traditional college course placement policies. Standardized tests tend to have less predictive strength for students of color who are more likely to underperform relative to their true abilities (e.g. Bowen et al. 2018; Hoffman and Lowitzki 2005). This is not necessarily due to biased tests, but may be attributed to other factors such as stereotype threat (Steele 2003) and less access to test preparation activities among underrepresented students (Buchmann et al. 2010; Park and Becks 2015). This issue is beginning to gain greater attention in policy arenas, as the University of California system is currently facing a lawsuit that challenges the use of ACT or SAT in admissions as it may lead to discrimination against applicants on the basis of income and race (Nadworny, 2019). Yet the issue tends to remain under-addressed in community colleges where underrepresented students disproportionately test into developmental education. By removing barriers to college-level courses, there may be differential improvements to the costs to gateway completion, particularly for racial/ethnic minority students who historically have faced the greatest barriers.

Second, the enhanced advising and support services offered through the reform may have contributed to efficiency gains, particularly for students of color. The types of inequities faced by underrepresented students in developmental education are not unique to postsecondary education. Academic preparation is developed in K-12 schools, and Black and Hispanic students are more likely to come from underperforming and underfunded schools. The challenges that these students face from underprepration may be further exacerbated in community colleges that provide insufficient support such as advising and tutoring, as well as misalignment between developmental and college-level courses (Melguizo et al. 2008). Florida’s SB 1720 is unique from developmental education reforms in other states in that it required colleges to prepare implementation plans for providing enhanced support services for students. Results from a survey of FCS leaders indicate that colleges significantly increased the emphasis on academic advising and academic supports, and that leaders perceived these enhanced services to be among the most effective changes that occurred under the reform (Hu et al. 2018). Mandating colleges to provide additional student support can be an important policy lever for ensuring that students have an increased likelihood of succeeding in college-level coursework, particularly among those who come from disadvantaged backgrounds.

Implications for Policy and Future Research

Policymakers in other states should consider whether their personnel can take on additional responsibilities and examine the value of the activities staff would need to give up to focus on reform efforts similar to those under SB 1720. While Florida implemented the developmental education reform without providing any additional appropriations to institutions, it still required institutions to reallocate substantial resources from other areas. Future research should consider start-up costs, such as developing curriculum for developmental education courses under new instructional strategies and providing professional development for instructors, as well as recurring costs such as the resources needed to provide additional support services.

In 2017, Florida policymakers cut approximately $25 million to the FCS, most of which was intended for investment in developmental education (Smith 2017). The rationale for the cuts was that FCS institutions would be spending less on developmental education under SB 1720. However, this assumption does not take into account additional resources that colleges have spent to reform their curricular approaches to developmental education or offer students additional advising and academic support services. A recent survey of Florida college administrators focused on the implementation of the reform found that colleges made numerous changes including adding new orientation and advising resources, spending more time with students identified as at-risk of academic failure through early alert systems, increasing the duration of advising sessions, and changing the format or content of orientation sessions (Hu et al. 2017). FCS institutions have estimated that approximately $31 million per year is spent on supporting advising, tutoring, and early alert systems (Florida College System 2017). This suggests that much of the cost savings to institutions from reducing developmental education enrollments may have been used to provide enhanced support services. While many of these services were made available to the general student population, some resources were prioritized in serving incoming students. For example, College of Central Florida added First Year Success Advisors to help students make a more successful transition to college.

While the increasing support for first-year students is promising, institutions should also consider whether adequate resources are available to continue to support students as they progress further into their degree programs. This is particularly important if reform initiatives are helping students to persist longer. Belfield et al. (2014) developed an efficiency model to demonstrate that reform initiatives which improve college persistence rates among entering students may incur greater costs in later years. Colleges tend to cross-subsidize the costs of more expensive upper-level courses with savings from less expensive entry-level courses. Therefore, if more students are persisting in college for longer periods of time, institutions will face increased financial pressures due to greater enrollments in upper-level courses. Colleges may also need to continue to provide additional tutoring and student support services if students who previously would have placed into developmental education, are now progressing to higher level courses. Belfield et al. (2014) note, “We suspect that this is one reason why many reforms are either short-lived—Colleges simply cannot afford to implement them for long at scale, or they are or ineffectual, such that their expense is offset by deteriorations in the quality of provision elsewhere in the college” (p. 330). Ensuring student success throughout the educational pipeline remains a key challenge for higher education practitioners and policymakers alike in Florida and beyond as they seek to improve outcomes for underprepared students.