Abstract
Typically, the effects of single-sex schooling are small at best, and tend to be statistically non-significant once pre-existing differences are taken into account. However, researchers often have had to rely on observational studies based on small non-representative samples and have not used more advanced propensity score methods to control the potentially confounding effects of covariates. Here, we apply optimal full matching to the large historical longitudinal dataset best suited to evaluating this issue in US high schools: the nationally representative High School and Beyond study. We compare the effects of single-sex education in the final 2 years of high school on Grade 12 and post-secondary outcomes using the subsample of students attending Catholic schools (N = 2379 students, 29 girls’ schools, 22 boys’ schools, 33 coeducational schools) focusing on achievement-related, motivational and social outcomes. We contrast conventional Analysis of Covariance (ANCOVA) with optimal full matching based on the propensity score that provides a principled way of controlling for selection bias. Results from the two approaches converged: When background and Year 10 covariates were controlled, uncorrected apparent differences between the school types disappeared and the pattern of effects was very similar across the two methods. Overall, there was little evidence for positive effects of single-sex schooling for a broad set of outcomes in the final 2 years of high school and 2 years after graduation. We conclude with a discussion of the advantages of propensity score methods compared to ANCOVA.
Similar content being viewed by others
References
Angrist, J. D., & Pischke, J. S. (2009). Mostly harmless econometrics: An empiricist’s companion. Princeton: Princeton University Press.
Austin, P. C., Grootendorst, P., & Anderson, G. M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: A Monte Carlo study. Statistics in Medicine, 26, 734–753. doi:10.1002/sim.2580.
Baker, D. B., Riordan, C., & Schaub, M. (1995). The effects of sex-grouped schooling on achievement: The role of national context. Comparative Education Review, 39, 468–482.
Bigler, R. S., & Signorella, M. L. (2011). Single-sex education: New perspectives and evidence on a continuing controversy. Sex Roles, 65, 659–669. doi:10.1007/s11199-011-0046-x.
Billger, S. M. (2009). On reconstructing school segregation: The efficacy and equity of single-sex schooling. Economics of Education Review, 28, 393–402. doi:10.1016/j.econedurev.2007.08.005.
Burgess, S., Greaves, E., Vignoles, A., & Wilson, D. (2009). Parental choice of primary school in England: what ‘type’ of school do parents choose? (Working Paper No. 09/224). Bristol, UK: The centre for market and public organisation. Retrieved from www.bristol.ac.uk/cmpo/publications/papers/2009/wp224.pdf.
Carpenter, P., & Hayden, M. (1987). Girls’ academic achievements: Single-sex versus coeducational schools in Australia. Sociology of Education, 60, 156–167. doi:10.2307/2112273.
Caspi, A. (1995). Puberty and the gender organization of schools: How biology and social context shape the adolescent experience. In L. J. Crockett & A. C. Crouter (Eds.), Pathways through adolescence: Individual development in relation to social contexts (pp. 57–74). Mahwah: Erlbaum.
Cochran, W. G., & Rubin, D. B. (1973). Controlling bias in observational studies: A review. Sankhya-A, 35, 417–446.
Daly, P. (1996). The effects of single-sex and coeducational secondary schooling on girls’ achievement. Research Papers in Education, 11, 289–306. doi:10.1080/0267152960110306.
Daly, P., & Shuttleworth, I. (1997). Determinants of public examination entry and attainment in mathematics: Evidence on gender and gender-type of school from the 1980s and 1990s in Northern Ireland. Evaluation and Research in Education, 11, 91–101. doi:10.1080/09500799708666919.
Dehejia, R. H., & Wahba, S. (1999). Propensity score matching methods for nonexperimental causal studies: Re-evaluating the evaluation of training programs. Journal of the American Statistical Association, 94, 1053–1062. doi:10.1162/003465302317331982.
R Development Core Team (2010). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://www.R-project.org.
Diamond, A., & Sekhon, J. S. (2006). Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies. (Working paper). University of California, Berkeley. Retrieved from http://sekhon.berkeley.edu/papers/GenMatch.pdf.
Eliot, L. (2011). Single-sex education and the brain. Sex Roles, this issue. doi:10.1007/s11199-011-0037-y.
Foster, E. M. (2010). Causal inference and developmental psychology. Developmental Psychology, 46, 1454–1480. doi:10.1037/a0020204.
Gitelman, A. I. (2005). Estimating causal effects from multilevel group-allocation data. Journal of Educational and Behavioral Statistics, 30, 397–412. doi:10.3102/10769986030004397.
Hansen, B. B. (2004). Full matching in an observational study of coaching for the SAT. Journal of the American Statistical Association, 99, 609–618. doi:10.1198/016214504000000647.
Hansen, B. B., & Klopfer, S. O. (2006). Optimal full matching and related designs via network flows. Journal of Computational and Graphical Statistics, 15, 609–627. doi:10.1198/106186006X137047.
Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15, 234–249. doi:10.1037/a0019623.
Harker, R. (2000). Achievement, gender, and the single-sex/coed debate. British Journal of Sociology of Education, 21, 203–218. doi:10.1080/713655349.
Hattie, J. (2008). Visible learning. A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
Heckman, J. J., & Vytlacil, E. J. (2007a). Econometric evaluation of social programs, part I: Causal models, structural models and econometric policy evaluation. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of econometrics (Vol. 6, pt. 2, pp. 4779–4874). Amsterdam, the Netherlands: North-Holland.
Hill, J. L., Weiss, C., & Zhai, F. (2011). Challenges with propensity score strategies in a high-dimensional setting and a potential alternative. Multivariate Behavioral Research, 46, 477–513. doi:10.1080/00273171.2011.570161.
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15, 199–236. doi:10.1093/pan/mpl013.
Ho, D. E., Imai, K., King, G. & Stuart, E. A. (2011). MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software, 42, (8). Retrieved from http://www.jstatsoft.org/v42/i08/paper.
Hoffnung, M. (2011). Career and family outcomes for women graduates of single-sex versus coed colleges. Sex Roles, 65, 680–692. doi:10.1007/s11199-010-9914-z.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960. doi:10.1080/01621459.1986.10478354.
Hong, G., & Raudenbush, S. W. (2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901–910. doi:10.1198/016214506000000447.
Imbens, G. W., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615–635. doi:10.1016/j.jeconom.2007.05.001.
Jencks, C. (1985). How much do high school students learn? Sociology of Education, 58, 128–135. doi:10.2307/2112252.
Kang, J. D. Y., & Schafer, J. L. (2007). Demystifying double-robustness: A comparison of alternative strategies for estimating a population mean from incompleted data. Statistical Science, 22, 523–539. doi:10.1214/07-STS227.
LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs with experimental data. American Economic Review, 76, 604–620. doi:10.2307/1806062.
Lee, V. E. (1998). Is single-sex secondary schooling a solution to the problem of gender inequity? In American Association for University Women Educational Foundation (Ed.). Separated by sex. A critical look at single-sex education for girls (pp. 41–52). Washington, D.C.: American Association for University Women Educational Foundation.
Lee, V. E., & Bryk, A. S. (1986). Effects of single-sex schools on student achievement and attitudes. Journal of Educational Psychology, 78, 381–395. doi:10.1037/0022-0663.78.5.381.
Lee, V. E., & Bryk, A. S. (1989). Effects of single-sex schools: Response to Marsh. Journal of Educational Psychology, 81, 647–650. doi:10.1037/0022-0663.81.4.647.
Lee, V. E., & Lockheed, M. E. (1990). The effects of single-sex schooling on achievement and attitudes in Nigeria. Comparative Education Review, 34, 209–231.
Lee, V. E., & Marks, H. M. (1990). Sustained effects of the single-sex secondary school experience on attitudes, behaviors, and values in college. Journal of Educational Psychology, 82, 578–592. doi:10.1037/0022-0663.82.3.578.
LePore, P. C., & Warren, J. R. (1997). A comparison of single-sex and coeducational Catholic secondary schooling: Evidence from the National Educational Longitudinal Study of 1988. American Educational Research Journal, 34, 485–511. doi:10.3102/00028312034003485.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York: John Wiley.
Lumley, T. (2010). Survey: analysis of complex survey samples [Computer software]. R package version 3.22–4.
Mael, F., Alonso, A., Gibson, D., Rogers, K., & Smith, M. (2005). Single-sex versus coeducational schooling: A systematic review. Doc# 2005-01. Washington: Department of Education. Office of Planning, Evaluation and Policy Development.
Manski, C. F. (2010). Identification of treatment response with social interactions.Working paper. Evanston: Northwestern University, Department of Economics and Institute for Policy Research.
Marsh, H. W. (1989a). Effects of attending single-sex and coeducational high schools on achievement, attitudes, behaviors, and sex differences. Journal of Educational Psychology, 81, 70–85. doi:10.1037/0022-0663.81.1.70.
Marsh, H. W. (1989b). Effects of single-sex and coeducational schools. A reponse to Lee and Bryk. Journal of Educational Psychology, 81, 651–653. doi:10.1037/0022-0663.81.4.651.
Marsh, H. W. (1991). Public, Catholic single-sex and Catholic coeducational high schools: Their effect on achievement, affect, and behaviors. American Journal of Education, 99, 320–356.
Marsh, H. W., & Hau, K.-T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology, 32, 151–170. doi:10.1016/j.cedpsych.2006.10.008.
Marsh, H. W., Smith, I. D., Marsh, M. R., & Owens, L. (1988). The transition from single-sex to coeducational high schools: Effects on multiple dimensions of self-concept and on academic achievement. American Educational Research Journal, 25, 237–269. doi:10.3102/00028312025002237.
Marsh, H. W., Owens, L., Marsh, M. R., & Smith, I. D. (1989). The transition from single-sex to coeducational high schools: Teacher perceptions, academic achievement, and self-concept. British Journal of Educational Psychology, 59, 155–173. doi:10.1111/j.2044-8279.1989.tb03088.x.
Morgan, S. L., & Winship, C. (2007). Counterfactuals and causal inference: Methods and principles for social research. Cambridge: Cambridge University Press.
Muthén, L. K. & Muthén, B. O. (1998–2010). Mplus User’s Guide. Sixth Edition. Los Angeles, CA: Muthén & Muthén
Nagengast, B. (2009). Causal inference in multilevel designs. Unpublished doctoral dissertation. School of Social and Behavioural Sciences. Germany: Friedrich-Schiller-Universität Jena.
National Center for Educational Statistics. (1986). High school and beyond, 1980: sophomore cohort second follow-up (1984). Data file user’s manual. Ann Arbor: Inter-university Consortium for Political and Social Research.
Neyman, J. (1923/1990). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5, 465–480.
OECD. (2010). PISA 2009 results: Overcoming social background. Equity in learning opportunities and outcomes. Volume 2. Paris: Author.
Park, H., Behrman, J. R., & Choi, J. (2010). Causal effects of single-sex schools on college attendance: Random assignment in Korean high schools. PSC Working Paper Series, 15.
Pohl, S., Steiner, P. M., Eisermann, J., Soellner, R., & Cook, T. D. (2009). Unbiased causal inference from an observational study: Results of a within-study comparison. Educational Evaluation and Policy Analysis, 31, 463–479. doi:10.3102/0162373709343964.
Raudenbush, S. W. (2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29, 121–129. doi:10.3102/10769986029001121.
Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307–335. doi:10.3102/10769986020004307.
Riordan, C. (1990). Girls and boys in school: Together or separate? New York: Teachers College Press.
Riordan, C. (1994). Single-gender schools: Outcomes for African and Hispanic Americans. Research in Sociology of Education and Socialization, 10, 177–205.
Riordan, C. (1998). The future of single-sex schools. In AAUW Educational Foundation (Ed.), Separated by sex (pp. 53–62). Washington, DC: American Association for University Women Educational Foundation. Retrieved from http://www.aauw.org/research/upload/SeparatedBySex.pdf.
Rosenbaum, P. R. (1984). The consequences of adjustment for a concomitant variable that has been affected by the treatment. Journal of the Royal Statistical Society—Series A, 147, 656–666. doi:10.2307/2981697.
Rosenbaum, P. R. (1991). A characterization of optimal designs for observational studies. Journal of the Royal Statistical Society—Series B, 53, 597–610.
Rosenbaum, P. R. (2002). Covariance adjustment in randomized experiments and observational studies. Statistical Science, 17, 286–304. doi:10.1214/ss/1042727942.
Rosenbaum, P. R., & Rubin, D. B. (1983a). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55. doi:10.1093/biomet/70.1.41.
Rosenbaum, P. R., & Rubin, D. B. (1983b). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society—Series B, 45, 212–218. doi:10.2307/2345524.
Rosenbaum, P. R., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33–38. doi:10.1080/00031305.1985.10479383.
Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688–701. doi:10.1037/h0037350.
Rubin, D. B. (1977). Assignment to treatment group on the basis of a covariate. Journal of Educational Statistics, 2, 1–26. doi:10.3102/10769986002001001.
Rubin, D. B. (1978). Bayesian-inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34–58. doi:10.2307/2958688.
Rubin, D. B. (1980). Bias reduction using Mahalanobis metric matching. Biometrics, 36, 293–298. doi:10.2307/2529981.
Rubin, D. B. (1986). Statistics and causal inference—which ifs have causal answers. Journal of the American Statistical Association, 81, 961–962. doi:10.1080/01621459.1986.10478355.
Rubin, D. B. (1990a). Formal modes of statistical-inference for causal effects. Journal of Statistical Planning and Inference, 25, 279–292. doi:10.1016/0378-3758(90)90077-8.
Rubin, D. B. (1990b). Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5, 472–480.
Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169–188. doi:10.1023/A:1020363010465.
Rubin, D. B. (2005). Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association, 100, 322–331. doi:10.1198/016214504000001880.
Rubin, D. B. (2008). Comment: The design and analysis of gold standard randomized experiments. Journal of the American Statistical Association, 103, 1350–1353. doi:10.1198/016214508000001011.
Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores, relating theory to practice. Biometrics, 52, 249–264. doi:10.2307/2533160.
Rubin, D. B., & Thomas, N. (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95, 573–585. doi:10.1080/01621459.2000.10474233.
Schafer, J. L., & Kang, J. D. Y. (2008). Average causal effects from nonrandomized studies: a practical guide and simulated example. Psychological Methods, 13, 279–313. doi:10.1037/a0014268.
Senn, S., Graf, E., & Caputo, A. (2007). Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment or exposure. Statistics in Medicine, 26, 5529–5544. doi:10.1002/sim.3133.
Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating interventions. Annual Review of Psychology, 60, 607–629. doi:10.1146/annurev.psych.60.110707.163544.
Shadish, W. R., Clark, M. H., & Steiner, P. M. (2008). Can nonrandomized experiments yield accurate answers? A randomized experiment comparing random to nonrandom assignment. Journal of the American Statistical Association, 103, 1334–1343. doi:10.1198/016214508000000733.
Smithers, A., & Robinson, P. (2006). The paradox of single-sex and coeducational schooling. Buckingham: Carmichael Press.
Sobel, M. E. (2006). What do randomized studies of housing mobility demonstrate?: Causal inference in the face of interference. Journal of the American Statistical Association, 101, 1398–1407. doi:10.1198/016214506000000636.
Spielhofer, T., Benton, T., & Schagen, S. (2004). A study of the effects of school size and single-sex education in English schools. Research Papers in Education, 19, 133–159. doi:10.1080/02671520410001695407.
Steyer, R., von Davier, A. A., Nachtigall, C., & Buhl, T. (2000). Causal regression models I: Individual and average causal effects. Methods of Psychological Research Online, 5, 39–71.
Steyer, R., Nachtigall, C., Wüthrich-Martone, O., & Kraus, K. (2002). Causal regression models III: Covariates, conditional, and unconditional average causal effects. Methods of Psychological Research Online, 7, 41–68.
Stuart, E. A. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25, 1–21. doi:10.1214/09-STS313.
Stuart, E. A., & Green, K. M. (2008). Using full matching to estimate causal effects in non-experimental studies: Examining the relationship between adolescent marijuana use and adult outcomes. Developmental Psychology, 44, 395–406. doi:10.1037/0012-1649.44.2.395.
Sullivan, A. (2009). Academic self-concept, gender and single-sex schooling. British Educational Research Journal, 35, 259–288. doi:10.1080/01411920802042960.
Sullivan, A., Joshi, H., & Leonhard, D. (2010). Single-sex schooling and academic attainment at school and through the lifecourse. American Educational Research Journal, 47, 6–36. doi:10.3102/0002831209350106.
Thoemmes, F., & Kim, E. S. (2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46, 90–118. doi:10.1080/00273171.2011.540475.
U.S. Department of Education (2006). Nondiscrimination on the basis of sex in education programs or activities receiving financial assistance: final rule, Federal Register, 34 CFR Part 106, 25 October.
VanderWeele, T. J. (2008). Ignorability and stability assumptions in neighborhood effects research. Statistics in Medicine, 27, 1934–1943. doi:10.1002/sim.3139.
Watson, C. M., Quatman, T., & Edler, E. (2002). Career aspirations of adolescent girls: Effects of achievement level, grade, and single-sex school environment. Sex Roles, 46, 323–335. doi:10.1023/A:1020228613796.
West, S. G., & Thoemmes, F. (2010). Campbell’s and Rubin’s perspectives on causal inference. Psychological Methods, 15, 18–37. doi:10.1037/a0015917.
Woodward, L. J., Fergusson, D. M., & Horwood, L. J. (1999). Effects of single-sex and coeducational secondary schooling on children’s academic achievement. Australian Journal of Education, 43, 142–156.
Wooldridge, J. (2005). Fixed-effects and related estimators for correlated random-coefficient and treatment-effect panel data models. The Review of Economics and Statistics, 87, 385–390. doi:10.1162/0034653053970320.
Author note
This research was supported in part by grants to the second author from the UK Economic and Social Research Council and the King Saud University in Saudi Arabia. Requests for further information about this investigation should be sent to Benjamin Nagengast, Department of Education, Center for Educational Science and Psychology, University of Tübingen, Europastr. 6, 72072 Tübingen, Germany; E-mail: benjamin.nagengast@uni-tuebingen.de
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 2.26 mb)
Rights and permissions
About this article
Cite this article
Nagengast, B., Marsh, H.W. & Hau, KT. Effects of Single-Sex Schooling in the Final Years of High School: A Comparison of Analysis of Covariance and Propensity Score Matching. Sex Roles 69, 404–422 (2013). https://doi.org/10.1007/s11199-013-0261-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11199-013-0261-8