The William and Flora Hewlett Foundation has defined open educational resources as:
teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge (Hewlett 2013).
OER materials eschew traditional copyright in lieu of licenses that allow others to retain, reuse, revise, remix, and redistribute the materials (Hilton et al. 2010; Wiley et al. 2014). The vast majority of the OER utilized in this study were available for free online. Thus, digital versions could be accessed on a wide variety of devices.
Open textbooks, which are a collection of OER aggregated in a manner that resembles a traditional textbook, take many shapes and forms. Typically, free digital versions of the textbook are made available to students. In addition, students who wish to purchase print versions of the textbooks can do so, at prices as low as $5 per textbook. While the quality of open textbooks varies, many go through rigorous editorial and design processes. Perhaps not surprisingly, students are favorably disposed towards replacing costly commercial textbooks with free open textbooks. Bliss et al. (2013) studied open textbook adoption at eight different institutions of higher education. Fifty-eight teachers and 490 students across the eight colleges completed surveys regarding their experiences in utilizing the open texts. Bliss and colleagues found that approximately 50 % of students said that the OER textbooks were of the same quality as traditional textbooks and nearly 40 % said that they were better. In their free-response comments, students focused on several benefits of the open textbooks, including cost-savings. For example, one student said, “I have no expendable income. Without this free text I would not be able to take this course.” In the same study, researchers found that 55 % of teachers adopting OER reported that the open materials were of the same quality as the materials they had previously used, and 35 % felt that they were better. One teacher in the study pointed out that “The materials were free to my students, which reduced a barrier to their chances for academic success.”
While Stratton et al. (2007) noted that results have been mixed in studies examining the relationship between student finances and their success in continuing through completion, several studies have indicated that greater financial resources correlate positively with student persistence. For example, Paulsen and St. John (2002) demonstrated that “the responsiveness of poor and working-class students to tuition increases is alarmingly high–reducing their probability of persisting by 16 and 19 %, respectively, per $1000 increment in tuition” (p. 229). While Paulsen and St. John did not discuss the cost of textbooks, it is interesting to note that the figure they used for an increase in tuition ($1000), is approximately the same amount of money full-time college students typically spend on textbooks per year. Thus one could argue that reducing textbook costs to zero could potentially increase persistence rates.
While not usually measured directly, it is possible that the use of no-cost or low-cost OER might free students’ resources to support increased credit loads which then enhance progress toward graduation. Wiley et al. (2015) analyzed the cost savings in courses with sections that used OER and sections that did not. The average cost of commercial textbooks across the courses was $140.85 which represented a potential total cost of $1,324,017.68 for that sample. In that instance, OER could have saved over one million dollars in textbook costs, which could have been applied directly to tuition for additional courses.
While financial reasons might be particularly persuasive to students and other educational stakeholders, the core purpose of education is to support learning. If the adoption of open textbooks decreases costs but also negatively influences student learning, educators should well view them with skepticism. While encouraging this skepticism, the authors fully acknowledge that institutions and educators everywhere trade improved affordability for lower outcomes on a regular basis. For example, colleges universally forego providing a full-time tutor for each student. Even though Bloom’s two-sigma work suggests this would greatly increase student learning, colleges instead choose to place students in educationally sub-optimal but significantly more affordable classes with many other students and a single instructor. Because this particular trade of sub-optimization for affordability is well established and broadly accepted, it is essentially invisible to many faculty. By contrast, a decrease in student learning associated with the adoption of open textbooks would be novel and likely to draw the negative attention of faculty, students, and other stakeholders. However, if learning outcomes actually improved in settings where open textbooks are utilized, there may be significant policy implications.
Perhaps because OER is relatively new, little research has been performed on how its utilization influences student learning. To date, six studies have compared student performance with and without implementing OER. These studies vary in rigor and all state that there are limitations to their findings. Nevertheless, they constitute the research done to the present time.
Lovett et al. (2008) measured the efficacy of an OER statistics module in comparison with the traditional educational model at Carnegie Mellon University. In two separate semesters, they invited students who had registered for an introductory statistics class at Carnegie Mellon to participate in an experimental online version of the course. Of those who volunteered, approximately one-third were randomly selected to take the online course, while the remaining two-thirds who had volunteered became the control group. The control group took the traditional, face-to-face statistics class at Carnegie Mellon. Researchers compared the results of these two groups in fall 2005 by examining their test scores (three midterms and one final exam), and found that there was no significant difference between the two groups. This experiment was replicated in spring 2006 with the same, non-significant, result. Thus, utilizing OER resulted in cost-savings without improving—or sacrificing—learning outcomes.
In another study focused on Carnegie Mellon’s open statistics modules, Bowen et al. (2014) compared the use of a traditional textbook in a face-to-face lecture class with that of a blended approach utilizing OER. Six hundred and five students took the OER version of the course, while 2439 took the traditional version. Bowen and colleagues found that, while students who utilized OER scored slightly higher than their peers on standardized exams, the difference was not statistically significant. A potential confound was that those utilizing OER received blended learning instead of traditional face-to-face instruction. Thus it is possible that the pedagogy masked the influence of OER. Nevertheless, it is relevant to note that in this study that the use of OER did not lead to lower student outcomes.
In a non-experimental case study, Hilton and Laman (2012) compared the performance of 690 students using an open textbook in an introductory psychology class to the performance of 370 students who used a traditional textbook in a previous semester. They concluded that students who used the open textbook achieved better grades in the course, had a lower withdrawal rate, and scored better on the final examination.
Feldstein et al. (2012) found that students in courses using open textbooks typically had higher grades and lower failure and withdrawal rates than those in courses with traditional textbooks. However, they did note significant limitations to their study suggesting that they provided only interesting data to be more rigorously pursued in the future.
Similarly, a case study presented by Hilton et al. (2013) focused on four math classes at Scottsdale Community College. These classes used the same departmental exam for each course for several years, which allowed faculty members to compare how students did on department exams when OER were used as compared with previous semesters. OER replaced traditional learning materials in fall 2012, and student results at the end of this semester were approximately the same as those obtained by students in fall 2011 and fall 2010.
Pawlyshyn et al. (2013) found that when OER material was integrated into the math courses at Mercy College, student learning significantly increased. The pass rates of math courses increased from 63.6 % in fall 2011 (when traditional learning materials were employed) to 68.9 % in fall 2012 when all courses were taught with OER. Similarly, students who were enrolled in OER versions of a reading course performed better than their peers who enrolled in the same course using non-OER materials.
Recent research indicates that a majority of faculty members perceive OER to be of approximately the same quality as traditional textbooks. Allen and Seaman (2014) surveyed 2144 college professors regarding OER. Of the 34 % (729) who were aware of OER, 61.5 % indicated OER had about the same “trusted quality” as traditional resources, 26.3 % said that traditional resources were superior, and 12.1 % said that OER were superior. Similarly, 68.2 % said that the “proven efficacy” were about the same, 16.5 % said that OER had superior efficacy, and 15.3 % said that traditional resources had superior efficacy.
Allen et al. (2015) studied an experimental class of 478 students that used OER known as ChemWiki for its primary textbook, while a control class of 448 utilized a commercial textbook. These two sections were taught the same semester at consecutive hours using the same faculty member and teaching assistants in order to control for potential confounds. Students in these classes received the same exams. No significant differences were found between the two groups. Beginning of the semester pre-tests combined with final exams showed no significant differences in individual learning gains between the two groups, thus indicating that OER could be substituted without any negative impact on learning.
While the aforementioned research provided interesting contextual case studies and varying degrees of statistical rigor, much more work needs to be done to ascertain the relationships between the use of OER and student academic performance. The purpose of this study was, therefore, to explore whether the use of open textbooks at 10 colleges significantly predicted learning outcomes in a group of 16,727 post-secondary students.
In the present study we sought to address the following questions:
-
1.
Comparing students who utilize OER and those who do not, is there a difference in the number of students who complete a course?
-
2.
Comparing students who utilize OER and those who do not, is there a difference in the number of students who pass a course with a C- or better grade?
-
3.
Comparing students who utilize OER with those who do not, is there a difference in the course grade?
-
4.
Comparing students who utilize OER and those who do not, is there a difference in the number of credits they take in the semester they used OER (fall)?
-
5.
Comparing students who utilize OER and those who do not, is there a difference in the number of credits they take the semester after the one in which they utilized OER (winter)?