Effects of Single-Sex Schooling in the Final Years of High School: A Comparison of Analysis of Covariance and Propensity Score Matching
- 2k Downloads
Typically, the effects of single-sex schooling are small at best, and tend to be statistically non-significant once pre-existing differences are taken into account. However, researchers often have had to rely on observational studies based on small non-representative samples and have not used more advanced propensity score methods to control the potentially confounding effects of covariates. Here, we apply optimal full matching to the large historical longitudinal dataset best suited to evaluating this issue in US high schools: the nationally representative High School and Beyond study. We compare the effects of single-sex education in the final 2 years of high school on Grade 12 and post-secondary outcomes using the subsample of students attending Catholic schools (N = 2379 students, 29 girls’ schools, 22 boys’ schools, 33 coeducational schools) focusing on achievement-related, motivational and social outcomes. We contrast conventional Analysis of Covariance (ANCOVA) with optimal full matching based on the propensity score that provides a principled way of controlling for selection bias. Results from the two approaches converged: When background and Year 10 covariates were controlled, uncorrected apparent differences between the school types disappeared and the pattern of effects was very similar across the two methods. Overall, there was little evidence for positive effects of single-sex schooling for a broad set of outcomes in the final 2 years of high school and 2 years after graduation. We conclude with a discussion of the advantages of propensity score methods compared to ANCOVA.
KeywordsSingle-sex schooling Propensity score Causal inference High School and Beyond study
This research was supported in part by grants to the second author from the UK Economic and Social Research Council and the King Saud University in Saudi Arabia. Requests for further information about this investigation should be sent to Benjamin Nagengast, Department of Education, Center for Educational Science and Psychology, University of Tübingen, Europastr. 6, 72072 Tübingen, Germany; E-mail: firstname.lastname@example.org
- Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton: Princeton University Press.Google Scholar
- Austin, P. C., Grootendorst, P., & Anderson, G. M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine, 26, 734–753. doi: 10.1002/sim.2580.PubMedCrossRefGoogle Scholar
- Burgess, S., Greaves, E., Vignoles, A., & Wilson, D. (2009). Parental choice of primary school in England: what ‘type’ of school do parents choose? (Working Paper No. 09/224). Bristol, UK: The centre for market and public organisation. Retrieved from www.bristol.ac.uk/cmpo/publications/papers/2009/wp224.pdf.
- Caspi, A. (1995). Puberty and the gender organization of schools: How biology and social context shape the adolescent experience. In L. J. Crockett & A. C. Crouter (Eds.), Pathways through adolescence: Individual development in relation to social contexts (pp. 57–74). Mahwah: Erlbaum.Google Scholar
- Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya-A, 35, 417–446.Google Scholar
- Daly, P., & Shuttleworth, I. (1997). Determinants of public examination entry and attainment in mathematics: Evidence on gender and gender-type of school from the 1980s and 1990s in Northern Ireland. Evaluation and Research in Education, 11, 91–101. doi: 10.1080/09500799708666919.CrossRefGoogle Scholar
- R Development Core Team (2010). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org.
- Diamond, A., & Sekhon, J. S. (2006). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. (Working paper). University of California, Berkeley. Retrieved from http://sekhon.berkeley.edu/papers/GenMatch.pdf.
- Eliot, L. (2011). Single-sex education and the brain. Sex Roles, this issue. doi: 10.1007/s11199-011-0037-y.
- Hattie, J. (2008). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.Google Scholar
- Heckman, J. J., & Vytlacil, E. J. (2007a). Econometric evaluation of social programs, part I: Causal models, structural models and econometric policy evaluation. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of econometrics (Vol. 6, pt. 2, pp. 4779–4874). Amsterdam, the Netherlands: North-Holland.Google Scholar
- Ho, D. E., Imai, K., King, G. & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42, (8). Retrieved from http://www.jstatsoft.org/v42/i08/paper.
- Lee, V. E. (1998). Is single-sex secondary schooling a solution to the problem of gender inequity? In American Association for University Women Educational Foundation (Ed.). Separated by sex. A critical look at single-sex education for girls (pp. 41–52). Washington, D.C.: American Association for University Women Educational Foundation.Google Scholar
- Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: John Wiley.Google Scholar
- Lumley, T. (2010). Survey: analysis of complex survey samples [Computer software]. R package version 3.22–4.Google Scholar
- Mael, F., Alonso, A., Gibson, D., Rogers, K., & Smith, M. (2005). Single-sex versus coeducational schooling: A systematic review. Doc# 2005-01. Washington: Department of Education. Office of Planning, Evaluation and Policy Development.Google Scholar
- Manski, C. F. (2010). Identification of treatment response with social interactions.Working paper. Evanston: Northwestern University, Department of Economics and Institute for Policy Research.Google Scholar
- Marsh, H. W., Smith, I. D., Marsh, M. R., & Owens, L. (1988). The transition from single-sex to coeducational high schools: Effects on multiple dimensions of self-concept and on academic achievement. American Educational Research Journal, 25, 237–269. doi: 10.3102/00028312025002237.CrossRefGoogle Scholar
- Marsh, H. W., Owens, L., Marsh, M. R., & Smith, I. D. (1989). The transition from single-sex to coeducational high schools: Teacher perceptions, academic achievement, and self-concept. British Journal of Educational Psychology, 59, 155–173. doi: 10.1111/j.2044-8279.1989.tb03088.x.CrossRefGoogle Scholar
- Muthén, L. K. & Muthén, B. O. (1998–2010). Mplus User’s Guide. Sixth Edition. Los Angeles, CA: Muthén & MuthénGoogle Scholar
- Nagengast, B. (2009). Causal inference in multilevel designs. Unpublished doctoral dissertation. School of Social and Behavioural Sciences. Germany: Friedrich-Schiller-Universität Jena.Google Scholar
- National Center for Educational Statistics. (1986). High school and beyond, 1980: sophomore cohort second follow-up (1984). Data file user’s manual. Ann Arbor: Inter-university Consortium for Political and Social Research.Google Scholar
- Neyman, J. (1923/1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5, 465–480.Google Scholar
- Park, H., Behrman, J. R., & Choi, J. (2010). Causal effects of single-sex schools on college attendance: Random assignment in Korean high schools. PSC Working Paper Series, 15.Google Scholar
- Riordan, C. (1990). Girls and boys in school: Together or separate? New York: Teachers College Press.Google Scholar
- Riordan, C. (1994). Single-gender schools: Outcomes for African and Hispanic Americans. Research in Sociology of Education and Socialization, 10, 177–205.Google Scholar
- Riordan, C. (1998). The future of single-sex schools. In AAUW Educational Foundation (Ed.), Separated by sex (pp. 53–62). Washington, DC: American Association for University Women Educational Foundation. Retrieved from http://www.aauw.org/research/upload/SeparatedBySex.pdf.
- Rosenbaum, P. R. (1991). A characterization of optimal designs for observational studies. Journal of the Royal Statistical Society—Series B, 53, 597–610.Google Scholar
- Rubin, D. B. (1990b). Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472–480.Google Scholar
- Smithers, A., & Robinson, P. (2006). The paradox of single-sex and coeducational schooling. Buckingham: Carmichael Press.Google Scholar
- Steyer, R., von Davier, A. A., Nachtigall, C., & Buhl, T. (2000). Causal regression models I: Individual and average causal effects. Methods of Psychological Research Online, 5, 39–71.Google Scholar
- U.S. Department of Education (2006). Nondiscrimination on the basis of sex in education programs or activities receiving financial assistance: final rule, Federal Register, 34 CFR Part 106, 25 October.Google Scholar