Research in Higher Education

, Volume 57, Issue 2, pp 123–151 | Cite as

Using a Regression Discontinuity Design to Estimate the Impact of Placement Decisions in Developmental Math

  • Tatiana Melguizo
  • Johannes M. Bos
  • Federick Ngo
  • Nicholas Mills
  • George Prather
Article

Abstract

This study evaluates the effectiveness of math placement policies for entering community college students on these students’ academic success in math. We estimate the impact of placement decisions by using a discrete-time survival model within a regression discontinuity framework. The primary conclusion that emerges is that initial placement in a lower-level course increases the time until a student at the margin completes the higher-level course they were not assigned to by about a year on average but in most cases, after this time period, the penalty was small and not statistically significant. We found minor differences in terms of degree applicable and degree transferable credit accumulation between students placed initially in the lowerlevel course.

Keywords

Developmental math Community colleges Evaluation Discrete time hazard model Regression discontinuity design 

Supplementary material

11162_2015_9382_MOESM1_ESM.docx (3 mb)
Supplementary material 1 (DOCX 3100 kb)

References

  1. Allison, P. D. (2005). Fixed effects regression methods for longitudinal data using SAS. Cary, NC: SAS Institute.Google Scholar
  2. Angrist, J. D., & Lavy, V. (1999). Using Maimonides’ rule to estimate the effect of class size on scholastic achievement. Quarterly Journal of Economics, 114(2), 533–575.CrossRefGoogle Scholar
  3. Bailey, T. (2009). Challenge and opportunity: Rethinking developmental education in community college. New Directions for Community Colleges, 145, 11–30.CrossRefGoogle Scholar
  4. Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review, 29, 255–270.CrossRefGoogle Scholar
  5. Bettinger, E. P., Boatman, A., & Long, B. T. (2013). Student supports: Developmental education and other academic programs. The Future of Children, 23(1), 93–115.CrossRefGoogle Scholar
  6. Bettinger, E., & Long, B. T. (2009). Addressing the needs of under-prepared students in higher education: Does college remediation work? Journal of Human Resources, 44, 736–771.CrossRefGoogle Scholar
  7. Boatman, A. & Long, B. T. (2010). Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation. National Center for Postsecondary Research Working Paper.Google Scholar
  8. Boylan, H., Bliss, L., & Bonham, D. (1994). Characteristics components of developmental programs. Review of Research in Developmental Education, 11(1), 1–4.Google Scholar
  9. Boylan, H., Bliss, L., & Bonham, D. (1997). Program components and their relationship to students’ performance. Journal of Developmental Education, 20(3), 1–8.Google Scholar
  10. Calcagno, J. C., & Long, B. T. (2008). The impact of remediation using a regression discontinuity approach: Addressing endogenous sorting and noncompliance. Working Paper 14194. Cambridge, MA: National Bureau of Economic Research.Google Scholar
  11. Caliendo, M., Tatsiramos, T., & Uhlendorff, A. (2013). Benefit duration, unemployment duration, and job match quality: A regression-discontinuity approach. Journal of Applied Econometrics, 28, 604–627.CrossRefGoogle Scholar
  12. California Community College Chancellor’s Office. (2011). Matriculation Handbook. Retrieved from http://www.cccco.edu/Portals/4/SS/Matric/Matriculation%20Handbook%20(REV.%2009-2011).pdf.
  13. Clotfelter, C. T., Ladd, H. F., Muschkin, C. G., & Vigdor, J. L. (2014). Developmental Education in North Carolina Community Colleges. Educational Evaluation and Policy Analysis,. doi:10.3102/0162373714547267.Google Scholar
  14. Fan, J., & Gijbels, I. (1996). Local polynomial modeling and its applications. London: Chapman & Hall.Google Scholar
  15. Fong, K., Melguizo, T., Bos, H., & Prather, G. (2013). A different view on how we understand progression through the developmental math trajectory. Policy Brief 3. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.
  16. Fong, K., Melguizo, T., & Prather, G. (2015). Increasing success rates in developmental math: The complementary role of individual and institutional characteristics. Research in Higher Education. doi:10.1007/s11162-015-9368-9.Google Scholar
  17. Gamse, B. C., Jacob, R. T., Horst, M., Boulay, B., Unlu, F. (2008). Reading First impact study: Final report. NCEE 2009-4038. Washington DC: National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.Google Scholar
  18. Greenwood, M. (1926). The natural duration of cancer (Vol. 33, pp. 1–26)., Reports on Public Health and Medical Subjects London: Her Majesty’s Stationery Office.Google Scholar
  19. Ham, J., & LaLonde, R. (1996). The effect of sample selection and initial conditions in duration models: Evidence from experimental data on training. Econometrica, 64(1), 175–205.CrossRefGoogle Scholar
  20. Imbens, G., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635.CrossRefGoogle Scholar
  21. Lazarick, L. (1997). Back to the basics: Remedial education. Community College Journal, 68, 11–15.Google Scholar
  22. Lesik, S. A. (2007). Do developmental mathematics programs have a causal impact on student retention? An application of discrete-time survival and regression-discontinuity analysis. Research in Higher Education, 48(5), 583–608.CrossRefGoogle Scholar
  23. Martorell, P., & McFarlin, I, Jr. (2011). Help or hindrance? The effects of college remediation on academic and labor market outcomes. The Review of Economics and Statistics, 93(2), 436–454.CrossRefGoogle Scholar
  24. McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: a density test. Journal of Econometrics, 142(2), 698–714.CrossRefGoogle Scholar
  25. Melguizo, T., Bos, H., & Prather, G. (2011). Is developmental education helping community college students persist? A critical review of the literature. American Behavioral Scientist, 55(2), 173–184.CrossRefGoogle Scholar
  26. Melguizo, T., Bos, H., & Prather, G. (2013). Are community colleges making good placement decisions in their math trajectories? Policy Brief 1. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.
  27. Melguizo, T., Hagedorn, L. S., & Cypers, S. (2008). The need for remedial/developmental education and the cost of community college transfer: Calculations from a sample of California community college transfers. The Review of Higher Education, 31(4), 401–431.CrossRefGoogle Scholar
  28. Melguizo, T., Kosiewicz, H., Prather, G., & Bos, J. (2014). How are community college students assessed and placed in developmental math? Grounding our understanding in reality. Journal of Higher Education, 85(5), 691–722.CrossRefGoogle Scholar
  29. Moss, B., & Yeaton, W. (2006). Shaping policies related to developmental education: An evaluation using the regression-discontinuity design. Educational Evaluation & Policy Analysis, 28(3), 215–229.CrossRefGoogle Scholar
  30. Moss, B., Yeaton, W., & Lloyd, J. E. (2014). Evaluating the effectiveness of developmental math by embedding a randomized experiment within a regression discontinuity design. Educational Evaluation & Policy Analysis, 36(2), 170–185.CrossRefGoogle Scholar
  31. Murnane, R. J., & Willett, J. B. (2011). Methods matter: Improving causal inference in education and social science research. Oxford: Oxford University Press.Google Scholar
  32. National Center for Public Policy and Higher Education & Southern Regional Education Board. (2010). Beyond the Rhetoric: Improving College Readiness through Coherent State Policy. Washington, D.C.Google Scholar
  33. Ngo, F., Kwon, F., Melguizo, T., Bos, H., & Prather, G. (2013). Course placement in developmental mathematics: Do multiple measures work? Policy Brief 4. California Community College Collaborative. Rossier School of Education, University of Southern California. http://www.uscrossier.org/pullias/research/projects/sc-community-college/.
  34. Ngo, F., & Kwon, W. (2015). Using multiple measures to make math placement decisions: Implications for access and success in community colleges. Research in Higher Education, 56(5), 442–470.CrossRefGoogle Scholar
  35. Robinson, J. P. (2011). Evaluating criteria for English learner reclassification: A causal-effects approach using a binding-score regression discontinuity design with instrumental variables. Educational Evaluation and Policy Analysis, 33(3), 267–292.CrossRefGoogle Scholar
  36. Schneider, M., & Yin, L. (2011). The hidden costs of community colleges. Washington, DC: American Institutes for Research. Retrieved from http://www.air.org/files/AIR_Hidden_Costs_of_Community_Colleges_Oct2011.pdf.
  37. Schochet, P. A. (2006). Regression discontinuity design case study: National evaluation of early reading first. Princeton, NJ: Mathematica Policy Research.Google Scholar
  38. Scott-Clayton, J., & Rodriguez, O. (2015). Development, discouragement, or diversion? New evidence on the effects of college remediation. Education Finance and Policy, 10(1), 4–45.CrossRefGoogle Scholar
  39. Singer, J., & Willett, J. (1993). It’s about time: Using discrete-time survival analysis to study duration and the timing of events. Journal of Educational Statistics, 18(2), 155–195.CrossRefGoogle Scholar
  40. Singer, J., & Willett, J. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press.CrossRefGoogle Scholar
  41. Strong American Schools. (2008). Diploma to nowhere. Retrieved from http://www.edin08.com.
  42. Urquiola, M., & Verhoogen, E. (2009). Class-size caps, sorting, and the regression-discontinuity design. American Economic Review, 99(1), 179–215.CrossRefGoogle Scholar
  43. Zuckerman, I. H., Lee, E., Wutoh, A. K., Xue, Z., & Stuart, B. (2006). Application of regression-discontinuity analysis in pharmaceutical health services research. Health Services Research, 41, 550–563.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Tatiana Melguizo
    • 1
  • Johannes M. Bos
    • 2
  • Federick Ngo
    • 1
  • Nicholas Mills
    • 2
  • George Prather
    • 3
  1. 1.Rossier School of EducationUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.American Institutes for ResearchSan MateoUSA
  3. 3.Los Angeles Community College DistrictLos AngelesUSA

Personalised recommendations