Textbooks are traditional and useful learning resources for college students, but commercial texts books have been widely criticized for their high costs, restricted access, limited flexibility, and uninspiring learning experiences (Colvard et al., 2018; Hilton, 2019; Smith, 2009). These shortcomings may impact students’ academic outcomes, as many students cannot access required instructional materials or experience delays in accessing these materials, potentially leading to high withdrawal and failure rates (e.g., Colvard et al., 2018; Florida Virtual Campus, 2016). Open Education Resources (OER) have emerged as an alternative to commercial textbooks that proponents argue can lead to increased affordability, access, and quality of instruction (Colvard et al., 2018; Hilton, 2019). OER are digital materials that are freely available and openly licensed, allowing instructors and students to adapt, reuse, and share them. OER can be any type of content including entire textbooks, assessments, articles, lesson plans, videos, or individual images. Instructors can use OER in place of commercial resources to ensure that students have access to required instructional materials from the beginning of a course. Students have the opportunity to redirect these cost savings towards other educational expenses, for example, enrolling in more credits, or towards personal expenses such as food and rent. OER also has been theorized to enable improved instructional quality by prompting faculty to change the way they think about content and learning opportunities, adopting new approaches that center and empower students and nourishing a participatory culture of learning and knowledge creation (Kaatrakoski et al. 2017; Bali et al., 2020).
There has been a dramatic increase in the awareness and use of OER in recent years. A national survey of university faculty and department chairs assessing OER awareness and perceptions showed that the percentage of respondents familiar with OER increased from 34 to 46% over 3 years (Seaman & Seaman, 2018). Twenty-six percent of faculty who teach required introductory courses reported using OER for required course material in a national survey conducted in 2017/18 (Spilovoy et al., 2020). The Covid-19 pandemic may further increase the appeal of OER. For instance, the recent shift to online teaching and learning from traditional instructional settings calls for improved access to high quality digital learning resources (Means & Neisler, 2020). Additionally, the economic shock to GDP and employment has been accompanied by a sharp decline in college enrollment, particularly at community colleges, which serve a share of low-income students (Sedmak, 2020). Cost savings associated with OER may be more critical to students than ever before.
Evidence suggests that use of OER can improve students’ course outcomes (Colvard et al., 2018; Hilton, 2016, 2019). Initially, OER adoption efforts were often undertaken by individual faculty on individual courses or sections and not meaningfully connected to the broader educational goals of the institution (e.g., Hilton, 2016). A number of studies have examined course-level impacts of OER use on student outcomes, and one study examined both course completion and enrollment intensity over two terms for students who took OER courses (Fischer et al., 2015). However, previous research has not examined the effects on longer term academic outcomes of institution-level efforts that enable students to take multiple OER courses as part of a degree pathway. The impact of OER degrees may be different from unconnected course conversions, as converting entire degree programs to OER requires colleges to intentionally integrate OER work into their organizational strategies, policies, and practices, requiring coordinated efforts.
The current study examined how an OER degree pathway initiative impacted key student outcomes from studying 11 community colleges in the USA. A primary goal of the research is to determine, with as much confidence as possible, whether the availability of OER degree pathway options enables students to attempt and complete more college credits and thus progress more quickly towards attaining degrees.
OER degree pathway
Achieving the Dream’s OER Degree Initiative aimed to increase college affordability and student success by catalyzing an institutional commitment to OER. This initiative was motivated by the expectation that an “OER degree,” or pathway of OER courses that together meet the requirements for a degree program, could set an ambitious goal for scaling up OER course offerings. Converting courses to OER could reinvigorate curriculum and pedagogy, leading to improved quality of instruction. At the same time, redesigning courses with OER is often time-consuming and demands new skills and supports for instructors. Scaling within institutions thus entailed an organizational effort that bridged departments and units, potentially enhancing institutional culture.
Launched in 2016, the OER Degree Initiative sought to promote affordability and innovation at community colleges by supporting large-scale OER adoption. Over two and a half years, the initiative supported 38 community colleges across 13 states in the USA in building degree pathways using only OER instructional materials. Eighteen of the grantees participated in multi-college consortia that received funding from the initiative. The scale of the OER Degree Initiative created an opportunity to explore how OER adoption at scale affects students, instructors, and institutions.
This initiative took place in a broader context of growing interest and support for OER programs at the state and national level, as OER have emerged as a promising and potentially transformative contribution to solving the problem of college affordability. The OER degree model was first pioneered at Tidewater Community College as a “Z degree” (indicating zero cost) and then spread across the state of Virginia and elsewhere. Subsequently, in 2018, the New York legislature allocated $16 million to the State University of New York’s (SUNY) and City University of New York (CUNY) systems to support OER work, in addition to funding the Open SUNY Textbooks (OST) program, which offers a curated selection of OER textbooks and courses at no cost through an online platform (Government of New York State, 2018). The U.S. Department of Education and many states have also launched #GoOpen initiatives to support broader adoption. In 2016, the California legislature awarded over $100 million in grants to support development of OER degrees through a “zero-textbook-costs” (ZTC) program at two dozen community colleges, and participants reported tentative though early signs of encouraging results (Burke, 2019).
Achieving the Dream’s OER Degree Initiative grantee colleges were required to convert at least one section of each of 20 courses required for specific associate degrees, though some colleges converted entire courses, and some developed more than one OER degree pathway. A requirement of the grant was that all OER degree courses be certified by Lumen Learning as meeting open license standards. The Achieving the Dream’s OER Degree Initiative required courses to use all openly licensed instructional content except in specific circumstances. For example, instructors could require students to purchase essential tangible goods, such as laboratory equipment for lab courses or art supplies for a studio course. Grantees were permitted to use copyrighted primary sources such as novels in required courses, but elective courses that used copyrighted material did not count towards the OER degree pathway. Grantees were permitted to charge fees for OER courses, but students must have unrestricted access to course materials from day 1. In many cases, colleges needed to revise courses that had been considered OER but that used proprietary content.
Efficacy of OER
Although there was no prior study examining the effects of OER degrees on student outcomes systematically, the existing evidence about the impacts, potential benefits, and overall opinion of individual OER courses or sections is mostly encouraging. For instance, prior research suggests that OER courses have the potential to save students’ money with the same or modestly improved course outcomes (Fischer et al., 2015; Hilton & Laman, 2012). A synthesis study of 9 studies (Hilton, 2016) and a meta-analysis of 36 studies (Hilton, 2019) that focused on OER and its impacts in postsecondary environments found that students achieved the same or better learning outcomes when using OER compared to when they used commercial textbooks; students also saved a significant amount of money from using OER instead of commercial textbooks. The aforementioned “throughput” study authors attributed the positive impacts they detected on student course completion and enrollment intensity on access and affordability, as opposed to differences in instructional design (Fischer et al., 2015).
Additionally, in a recent meta-analysis, Clinton and Khan (2019) called for the need to provide a nuanced approach to examine academic outcomes, and they further divided student outcomes into student learning performance (e.g., GPA) and student progress towards the course completion (e.g., withdrawal rate). In the meta-analysis, they showed that there were no differences in learning performance between OER and commercial textbooks, but students using OER had a significantly lower withdrawal rate compared to students using commercial textbooks. Other research has found that broader indicators of students’ progress such as overall course completion, early credit accumulation, and cumulative college GPA are important predictors of degree attainment (Moore & Shulock, 2009).
Importantly, given that random controlled trials (RCT) testing course or program-level interventions such as OER are rarely feasible in postsecondary settings, it is critical to consider and control for student background characteristics in the study designs and analyses. For instance, Griggs and Jackson (2017) argued that researchers will not be able to generate meaningful results with external validity for OER efficacy studies without controlling for relevant variables. As an example, one confounder frequently discussed in prior studies was student prior performance (Cassidy, 2015). Clinton et al. (2019) found that students in the OER course produced higher GPA compared to students using commercial textbooks while both groups were taught and graded by the same instructor. However, students in the OER group had higher prior achievement (e.g., high-school GPA). Without controlling for student prior achievement or other confounding variables in the analyses, it is difficult to attribute the performance difference observed between the two courses to the use of OER. In addition, it is also helpful to understand whether or not OER has differential effects on subgroups of students (e.g., students with lower prior achievement) (Clinton & Khan, 2019). Thus, for the current study, the research team both controlled for background characteristics in the main impact analysis to generate accurate estimates and conducted moderation analysis to examine the differential effects of OER for subgroups.
Furthermore, Hilton (2019) indicated that, quite often, data metrics used in different studies or even within the same study can be very different, rendering it difficult for scholars to come up with synthesized results in meta-analyses. As an example, Croteau (2017) examined 3,847 college students in Georgia who used OER, but the study was based on inconsistent data (e.g., completion rates, and grade distributions) reported by different faculty members. Similarly, Ozdemir and Hendricks (2017) studied 28 faculties who provided evaluations regarding the impact of OER on student learning. Unfortunately, the outcome variables varied widely, from improved scores on exams, to anecdotal evidence, to no data or explanation at all. The lack of control of the variables used in the analysis limited the value of the overall study. Thus, the current study used the same data collection process, analysis approaches, and reporting methods across multiple settings, generating consistent results that can be synthesized.
In this paper, we aimed to answer two research questions:
Did students who took OER degree classes make greater progress towards degrees compared with similar students who take traditional classes, when we control for student background characteristics?
Were OER degrees more or less beneficial to particular subgroups of students (e.g., Pell grant eligible)?