Abstract
The purpose of this mixed-methods study was to investigate whether blended learning increased student achievement in middle schools at a public charter school system. Qualitative data was gathered through classroom observations (N=8), teacher interviews (N=8), and student focus groups (N=6). Quantitative analysis involved a difference-in-difference (DID) regression analysis. Data was gathered from seven schools and 44 classrooms. Level of blended learning implementation was recorded based on school principal rating on a 5-point scale. Data showed that a 1-point increase in blended learning was estimated to lead to a.05 standard deviation increase on the Measures of Academic Progress (MAP) math assessment on average, by .10 standard deviations for boys, .20 standard deviations for African American students, and .23 standard deviations for students with 504 plans (p < .05). Qualitative data suggests that blended learning was most impactful when teachers used data from adaptive digital content for differentiation. Limitations to this study as well as recommendations for future research are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Briggs, K. C. (2014). Blended learning vs. face-to-face instruction: A quantitative evaluation of student achievement in algebra I. Available from ProQuest Dissertations & Theses Full Text: The Humanities and Social Sciences Collection. (1640913684)
Brodersen, R. M., & Melluso, D. (2017). Summary of research on online and blended learning programs that offer differentiated learning options (REL 2017–228). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Central. http://ies.ed.gov/ncee/edlabs. Accessed 2 May 2018.
Columbia University (n.d.). Difference-in-Difference estimation. https://www.mailman.columbia.edu/research/population-health-methods/difference-difference-estimation. Accessed 20 Jan 2019.
Education Elements (2015, September 29). The positive power of personalized learning: Education Elements impact 2014–2015. https://cdn2.hubspot.net/hubfs/428371/Impact_report_2014_-15/EducationElements_ImpactReport_2014-2015.pdf. Accessed 30 Mar 2019.
Fazal, M., & Bryant, M. (2019). Blended learning in middle school math: The question of effectiveness. Journal of Online Learning Research, 5(1), 49–64.
Halverson, L. R., Spring, K. J., Huyett, S., Henrie, C. R., & Graham, C. R. (2017) Blended learning research in higher education and K-12 settings. In Spector M., Lockee B., & Childress M. (Eds) Learning, design, and technology. Springer, Cham. Retrieved May 10, 2019, from https://doi.org/10.1007/978-3-319-17727-4_31-1.
Herold, B. (2016, February 20). Personalized learning: What does the research say? https://www.edweek.org/ew/articles/2016/10/19/personalized-learning-what-does-the-research-say.html. Accessed 10 May 2019.
Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008, November 24). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172–177. https://onlinelibrary.wiley.com/doi/full/10.1111/j.1750-8606.2008.00061.x. Accessed 10 May 2019.
Hillman, N. (2018, July 10). Using difference-in-differences in higher education research. https://web.education.wisc.edu/nwhillman/index.php/2018/07/10/using-difference-in-differences-in-higher-education-research/. Accessed 25 May 2019.
Johns, S., & Wolking, M. (n.d.). The core four of personalized learning: The elements you need to succeed. https://www.edelements.com/hubfs/Core_Four/Education_Elements_Core_Four_White_Paper.pdf. Accessed 2 May 2018.
Kazakoff, E., Macaruso, P., & Hook, P. (2018). Efficacy of a blended learning approach to elementary school reading instruction for students who are English Learners. Educational Technology Research and Development, 66(2), 429–449. Retrieved August 28, 2018, from. https://doi.org/10.1007/s11423-017-9565-7.
Lin, L. (2018). Student learning and engagement in a blended environment: A mixed methods study [Electronic version]. In I. Bouchrika, N. Harrati, & P. Vu (Eds.), Learner experience and usability in online education (pp. 256-269). https://doi.org/10.4018/978-1-5225-4206-3.
Mackey, K. (2015, September 19). Proof points: Blended learning success in school districts. Christensen Institute. https://www.christenseninstitute.org/publications/proof-points/. Accessed 21 June 2018.
Mathews, A. (2017). Blended learning: A mixed-methods study on successful schools and effective practices. Available from ProQuest Dissertations & Theses Full Text (10634258).
Means, B., Toyama, Y., Murphy, R., Baki, M., & Jones, K. (2010, September). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf. Accessed 20 January 2019.
Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: a meta-analysis of the empirical literature. Teachers College Record, 115(3). https://www.sri.com/sites/default/files/publications/effectiveness_of_online_and_blended_learning.pdf. Accessed May 18 2018.
NWEA (2019). MAP growth: Precisely measure growth and performance. https://www.nwea.org/map-growth/. Accessed 20 Jan 2019.
Pane, J. F., Steiner, E. D., Baird, M. D., & Hamilton, L. S. (2015). Continued progress: Promising evidence on personalized learning. Santa Monica, CA: RAND Corporation, 2015. https://www.rand.org/pubs/research_reports/RR1365.html. Accessed 1 May 2019.
Ready, D. (2014, December 4). Student mathematics performance in the first two years of Teach to One: Math. http://www.newclassrooms.org/wp-content/uploads/2016/09/Teach-to-One_Report_2013-14.pdf. Accessed 1 May 2019.
Staker, H. C. (2011, May). The rise of K–12 blended learning profiles of emerging models(Rep.). https://www.christenseninstitute.org/wp-content/uploads/2013/04/The-rise-of-K-12-blended-learning.emerging-models.pdf. Accessed 21 June 2018.
Urban Institute (n.d.). Difference-in-difference and panel methods. https://www.urban.org/research/data-methods/data-analysis/quantitative-data-analysis/impact-analysis/quasi-experimental-methods/difference-difference-and-panel-methods. Accessed 10 Mar 2019.
Wang, Y., Han, X., & Yang, J. (2015). Revisiting the blended learning literature: using a complex adaptive systems framework [Electronic version]. Journal of Educational Technology & Society, 18 (2), 380–393. http://arktos.nyit.edu/login?url=http://search.ebscohost.com.arktos.nyit.edu/login.aspx?direct=true&db=ehh&AN=102557882&site=eds-live&scope=site. Accessed 2 May 2019.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Research involving Human Participants
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional review board and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Fazal, M., Panzano, B. & Luk, K. Evaluating the Impact of Blended Learning: a Mixed-Methods Study with Difference-in-Difference Analysis. TechTrends 64, 70–78 (2020). https://doi.org/10.1007/s11528-019-00429-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11528-019-00429-8