Advertisement

Technology, Knowledge and Learning

, Volume 22, Issue 3, pp 335–352 | Cite as

Balancing Student Success: Assessing Supplemental Instruction Through Coarsened Exact Matching

  • Maureen A. Guarcello
  • Richard A. Levine
  • Joshua Beemer
  • James P. Frazee
  • Mark A. Laumakis
  • Stephen A. Schellenberg
Original Research
First Online:

Abstract

Supplemental Instruction (SI) is a voluntary, non-remedial, peer-facilitated, course-specific intervention that has been widely demonstrated to increase student success, yet concerns persist regarding the biasing effects of disproportionate participation by already higher-performing students. With a focus on maintaining access for all students, a large, public university in the Western United States used student demographic, performance, and SI participation data to evaluate the intervention’s efficacy while reducing selection bias. This analysis was conducted in the first year of SI implementation within a traditionally high-challenge introductory psychology course. Findings indicate a statistically significant relationship between student participation in SI and increased odds of successful course completion. Furthermore, the application of Coarsened Exact Matching reduced concerns that increased course performance was attributed to an over-representation of higher performing students who elected to attend SI Sessions.

Keywords

Learning analytics High impact practice Program assessment Propensity score matching 
This is a preview of subscription content, log in to check access.

References

  1. Arendale, D. (1997). Supplemental Instruction (SI): Review of research concerning the effectiveness of SI from the University of Missouri-Kansas City and other institutions from across the United States. In S. Mioduski & G. Enright (Eds.), Proceedings of the 17th and 18th annual institutes for learning assistance professionals: 1996 and 1997. Tucson: University Learning Center, University of Arizona.Google Scholar
  2. Blackwell, M., Iacus, S., King, G., & Porro, G. (2009). CEM: coarsened exact matching in Stata. The Stata Journal, 9, 524–546.Google Scholar
  3. Creswell, J. W., Plano Clark, V. L., Gutmann, M., & Hanson, W. (2003). Advanced mixed methods research designs. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.Google Scholar
  4. Dawson, P., van der Meer, J., Skalicky, J., & Cowley, K. (2014). On the effectiveness of supplemental instruction: A systemic review of supplemental instruction and peer-assisted study sessions literature between 2001 and 2010. Review of Educational Research., 84(4), 609–639.CrossRefGoogle Scholar
  5. Fayowski, V., & MacMillan, P. D. (2008). An evaluation of the supplemental instruction programme in a first year calculus course. International Journal of Mathematical Education in Science and Technology, 39, 843–855.CrossRefGoogle Scholar
  6. Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42, 8.CrossRefGoogle Scholar
  7. Iacus, S. M., King, G., & Porro, G. (2009). CEM: Software for coarsened exact matching. Journal of Statistical Software, 30, 9.CrossRefGoogle Scholar
  8. Iacus, S. M., King, G., & Porro, G. (2011). Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association, 106, 345–361.CrossRefGoogle Scholar
  9. Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact matching. Political Analysis, 20, 1–24.CrossRefGoogle Scholar
  10. International Center for Supplemental Instruction. (2014). Supplemental Instruction supervisor manual. Kansas City, MO.Google Scholar
  11. Keller, B., & Tipton, E. (2016). Propensity score analysis in R: A software review. Journal of Educational and Behavioral Statistics, 41(3), 326–348.CrossRefGoogle Scholar
  12. King, G., & Nielsen, R. (2016). Why propensity scores should not be used for matching. Working paper.Google Scholar
  13. Laumakis, M., Graham, C., & Dziuban, C. (2009). The Sloan-C pillars and boundary objects as a framework for evaluating blended learning. Journal of Asynchronous Learning Networks, 13(1), 75–87.Google Scholar
  14. Martin, D., & Arendale, D. (1993). Supplemental instruction: Improving first-year student success in high-risk courses (2nd ed.). Columbia: National Resource Center for the First Year Experience and Students in Transition, University of South Carolina.Google Scholar
  15. McCarthy, A., Smuts, B., & Cosser, M. (1997). Assessing the effectiveness of supplemental instruction: A critique and a case study. Studies in Higher Education, 22, 221–231.CrossRefGoogle Scholar
  16. R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. https://www.R-project.org.
  17. Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Society, 79, 516–524.CrossRefGoogle Scholar
  18. Stevens, G., King, G., & Shibuya, K. (2010). Deaths from heart failure: using coarsened exact matching to correct cause-of-death statistics. Population Health Metrics, 8, 6.CrossRefGoogle Scholar
  19. Stock, W. A., Ward, K., Folsom, J., Borrenpohl, T., Mumford, S., Pershin, Z., et al. (2013). Cheap and effective: The impact of student-led recitation classes on learning outcomes in introductory economics. The Journal of Economic Education, 44(1), 1–16.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • Maureen A. Guarcello
    • 1
  • Richard A. Levine
    • 2
  • Joshua Beemer
    • 3
  • James P. Frazee
    • 1
  • Mark A. Laumakis
    • 4
  • Stephen A. Schellenberg
    • 5
  1. 1.Instructional Technology ServicesSan Diego State UniversitySan DiegoUSA
  2. 2.Department of Mathematics and StatisticsSan Diego State UniversitySan DiegoUSA
  3. 3.Computational Science Research CenterSan Diego State UniversitySan DiegoUSA
  4. 4.Department of PsychologySan Diego State UniversitySan DiegoUSA
  5. 5.Division of Undergraduate StudiesSan Diego State UniversitySan DiegoUSA

Personalised recommendations

Cite article

Buy options