Abstract
Previous research suggests that racial and ethnic disparities in postsecondary STEM outcomes are rooted much earlier in the educational pipeline. One possible remedy to these disparities is participation in early STEM enrichment programs. We examine the impact of MESA, which is an early program that targets socioeconomically disadvantaged students, on outcomes that may lead students down the path to STEM. We analyze three waves of restricted nationally-representative data from the High School Longitudinal Study that trace the STEM progress of more than 25,000 students throughout high school and into their postsecondary careers. Propensity score matching models reveal that MESA participation increases students’ odds of taking AP STEM courses in high school and their aspirations for declaring a STEM major in college. However, these effects are driven primarily by black and white students, respectively. Latino and Asian students remain largely unaffected. A formal sensitivity analysis concludes that these findings are moderately robust to unobserved confounding. The results are also robust to alternative matching schemes. Collectively, the findings suggest that MESA may improve black students’ high school STEM engagement but may have little impact on black and Latino students’ STEM outcomes in college.
Similar content being viewed by others
Notes
Although the outcomes under study here are likely to invoke an image of a college-ready student, these analyses are not systematically conditioned on college-readiness or any other indicator that would limit the analysis only to students who are going to enroll (or are enrolled) in postsecondary institutions. We assess the impact of MESA on STEM outcomes for all students.
The MESA program originated in California in 1970 and has since expanded to ten states that have integrated it into their set of services for disadvantaged students. The program operates in 140 elementary schools, 470 middle schools, 359 high schools, 37 two-year institutions, and 28 four-year institutions. These MESA participation statistics are current as of 2015 and, therefore, may not necessarily represent participation at the time when HSLS respondents participated in MESA.
The counterfactual group could include students who participated in other SEPs. Given MESA’s explicit focus on STEM, however, we distinguish it among all of the other possible SEPs because of the program’s specialized attention on getting underrepresented students into the STEM college pipeline.
Pro-science climate includes whether the school has any of the following features: special focus on math or science, holds math or science fairs/workshops/competitions, offers pre-high school summer reading/math instruction for struggling 9th graders, sponsors a math or science after school program, pairs students with mentors in math or science, brings in guest speakers to talk about math or science, takes students on math- or science-relevant field trips, tells students about math/science contests/websites/blogs/other programs, requires teacher professional development in how students learn math/science, requires teacher professional development in increasing interest in math/science, raises student math/science interest/achievement in another way, offers College Board AP courses on-site, offers incentives to attract full-time high school math or science teachers, and on-site AP offerings for Calculus AB, Calculus BC, Computer Science, Computer Science A, Computer Science B, Advanced Biology, Advanced Chemistry, or Advanced Physics.
In order to control for heterogeneity in selection into the treatment and outcome that lies between schools, we included school-level variables in the matching model. This adjustment allows us to partially account for the clustered nature of the data and accurately specify the propensity score model (Arpino and Mealli 2008).
According to the Stata help file for the command pstest, “The standardized percent bias is the percent difference of the sample means in the treated and non-treated (full or matched) sub-samples as a percentage of the square root of the average of the sample variances in the treated and non-treated groups (formula from Rosenbaum and Rubin 1985).”.
We would like to note that findings for Southeast Asian American students may differ—especially given likely differences in family socioeconomic status. However, we cannot pinpoint any such differences since we did not disaggregate the data sufficiently in this regard.
References
Adams, S. (2014). The best jobs for 2014. New York: Forbes.
Afterschool Alliance. (2011). STEM learning in afterschool: an analysis of impact and outcomes. Washington, DC: Afterschool Alliance.
Allison, P. (2002). Missing data. Thousand Oaks: Sage Publications.
Arpino, B., & Mealli, F. (2008). The specification of the propensity score in multilvel observational studies. Computational Statistics & Data Analysis, 55, 1770–1780.
Aud, S., Hussar, W., Johnson, F., Kena, G., Roth, E., Manning, E., et al. (2012). The condition of education 2012 (NCES 2012-045). Washington, DC: U.S. Department of Education, National Center for Education Statistic.
Becker, S. O., & Ichino, A. (2002). Estimation of average treatment effects based on propensity scores. STATA Journal, 2, 358–377.
Bottia, M. C., Stearns, E., Mickelson, R. A., Moller, S., & Parker, A. D. (2015). The relationships among high school STEM learning experiences and students’ intent to declare and declaration of a STEM major in college. Teachers College Record, 117, 1–46.
Bureau of Labor Statistics (2013). Labor force characteristics by race and ethnicity, 2012. BLS Reports.
Caliendo, M., & Kopeinig, S. (2008). Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys, 22, 31–72.
Catsambis, S. (1995). Gender, race, ethnicity, and science-education in the middle grades. Journal of Research in Science Teaching, 32, 243–257.
Chemers, M., Zurbriggen, E., Syed, M., Goza, B., & Bearman, S. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. Journal of Social Issues, 67, 469–491.
Chen, X. (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education. In Stats in Brief (NCES 2009-161). National Center for Educational Statistics.
Coleman, J. S., Campbell, E. Q., Hobson, C., McPartland, J., Mood, A., Weinfeld, F. D., et al. (1966). Equality of educational opportunity. Washington, DC: Government Printing Office.
Correll, S. J. (2004). Constraints into preferences: Gender, status, and emerging career aspirations. American Sociological Review, 69, 93–113.
Crosnoe, R., & Schneider, B. (2010). Social capital, information, and socioeconomic disparities in math course work. The American Journal of Education, 117, 79–107.
D’Agostino, R. B. (1998). Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics in Medicine, 17, 2265–2281.
Dalton, B., Ingels, S. J., Downing, J., Bozick, R., & Owings, J. (2007). Advanced mathematics and science coursetaking in the spring high school senior classes of 1982, 1992, and 2004. Washington, DC: National Center for Educational Statistics, Institute of Education Sciences, U.S. Department of Education.
Davidson, A. (2012). Making it in America. Boston, MA: The Atlantic.
Dehejia, R. (2005). Practical propensity score matching: A reply to Smith and Todd. Journal of Economic Theory, 125, 355–364.
DiPrete, T. A., & Gangl, M. (2004). Assessing bias in the estimation of causal effects: Rosenbaum bounds on matching estimators and instrumental variables estimation with imperfect instruments (pp. 271–310). Weinheim: Wiley-VCH Verlag Gmbh.
Heckman, J. J. (2006). Skill formation and the economics of investing in disadvantaged children. Science, 312, 1900–1902.
Heckman, J. J., Ichimura, H., & Todd, P. (1998). Matching as an econometric evaluation estimator. Review of Economic Studies, 65, 261–294.
Hewson, P. W., Kahle, J. B., Scantlebury, K., & Davies, D. (2001). Equitable science education in urban middle schools: Do reform efforts make a difference?*. Journal of Research in Science Teaching, 38, 1130–1144.
Hinojosa, T., Rapaport, A., Jaciw, A., LiCalsi, C., & Zacamy, J. (2016). Exploring the foundations of the future STEM workforce: K–12 indicators of postsecondary STEM success. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southwest.
Hoepner, C. C. (2010). Advanced placement math and science courses: Influential factors and predictors for success in college STEM majors. Proquest Dissertation Publication.
Ichino, A., Mealli, F., & Nannicini, T. (2008). From temporary help jobs to permanent employment: What can we learn from matching estimators and their sensitivity? Journal of Applied Economics, 23, 305–327.
Jackson, S. A. (2003). The quiet crisis: Falling short in producing american scientific and technical talent. San Diego: Building Engineering & Science Talent (BEST).
Kaushal, N., Magnuson, K., & Waldfogel, J. (2011). How is family income related to investments in children’s schooling. In G. J. Duncan, & R. J. Murnane (Eds.) (pp. 187–205). New York: Russell Sage Foundation.
Kelly, S. (2009). The black–white gap in mathematics course taking. Sociology of Education, 82, 47–69.
Kelly, S., & Zhang, Y. (2016). Teacher support and engagement in math and science: Evidence from the high school longitudinal study. High School Journal, 99, 141–165.
Kim, C., Tamborini, C. R., & Sakamoto, A. (2015). Field of study in college and lifetime earnings in the United States. Sociology of Education, 88, 320–339.
Legewie, J., & DiPrete, T. A. (2014). The high school environment and the gender gap in science and engineering. Sociology of Education, 87, 259–280.
Leggon, C. B., & Pearson, W. (2006). Assessing programs to improve minority participation in STEM fields: What we know and what we need to know. In R. Ehrenberg, & C. V. Kuh (Eds.). Ithaca, NY: Cornell University Press.
Leuven, E., & Sianesi, B. (2003). PSMATCH2: Stata module to perform full mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing.
Lewis, J., Menzies, H., Najera, E., & Page, R. (2009). Rethinking trends in minority participation in the sciences. Science & Education, 93, 961–977.
Lucas, S. R. (1999). Tracking inequality. New York: Teachers College Press.
Moakler, M. W., & Kim, M. M. (2014). College major choice in STEM: Revisiting confidence and demographic factors. Career Development Quarterly, 62, 128–142.
Morgan, S. L. (1996). Trends in black–white differences in educational expectations: 1980–1992. Sociology of Education, 69, 308–319.
Morgan, S. L., & Harding, D. J. (2006). Matching estimators of causal effects—Prospects and pitfalls in theory and practice. Sociological Methods & Research, 35, 3–60.
Morgan, S. L., & Winship, C. (2014). Counterfactuals and causal inference: Methods and principles for social research (2nd ed.). Cambridge: Cambridge University Press.
Muller, P., Stage, F. K., & Kinzie, J. (2001). Science achievement growth trajectories: Understanding factors related to gender and racial-ethnic differences in precollege science achievement. American Educational Research Journal, 38, 981–1012.
NSF. (2014). Science and engineering indicators 2014. National Science Foundation (NSB 14-01).
NSF and NCSES. (2013). Women, minorities, and persons with disabilities in science and engineering: 2013. Alexandria, VA: National Science Foundation.
Oakes, J. (1990). Opportunities, achievement, and choice—Women and minority-students in science and mathematics (p. 16). Educ: Review of Research in Education.
Orfield, G., Frankenberg, E., Ee, J., & Kuscera, J. (2014). Brown at 60: Great progress, a long retreat and an uncertain future.
Ormerod, M. B., & Duckworth, D. (1975). Pupils’ attitudes to science. Slough: NFER.
Owens, J., & Massey, D. S. (2011). Stereotype threat and college academic performance: A latent variables approach. Social Science Research, 40, 150–166.
Press, W. H. (2013). What’s so special about science (and how much should we spend on it?). Science, 342, 817–822.
Provasnik, S., Kastberg, D., Ferraro, D., Lemanski, N., Roey, S., & Jenkins, F. (2012). Highlights from TIMSS 2011: Mathematics and science achievement of US fourth-and eighth-grade students in an international context. Washington, DC: National Center for Education Statistics.
Rakow, S. J., & Walker, C. L. (1985). The status of Hispanic American-students in science—Achievement and exposure. Science & Education, 69, 557–565.
Riegle-Crumb, C., Farkas, G., & Muller, C. (2006). The role of gender and friendship in advanced course taking. Sociology of Education, 79, 206–228.
Riegle-Crumb, C., & Grodsky, E. (2010). Racial-ethnic differences at the intersection of math course-taking and achievement. Sociology of Education, 83, 248–270.
Rosenbaum, P. (2002). Observational studies. New York: Springer.
Rosenbaum, J. E., & Harris, L. E. (2001). Low-income families in their new neighborhoods. Journal of Family Issues, 22, 183–210.
Rosenbaum, P., & Rubin, D. B. (1983a). Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. Journal of the Royal Statistical Society, Series B, 45, 212–218.
Rosenbaum, P., & Rubin, D. B. (1983b). The Central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rosenbaum, P., & Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33–38.
Royal Society. (2006). Taking a leading role—scientists survey. London: The Royal Society.
Rubin, D. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688–701.
Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.
Rubin, D. (1980). Discussion of “randomization analysis of experimental data in the fisher randomization test” by Basu. Journal of American Statistical Association, 75, 591–593.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized inference. Boston: Houghton Mifflin.
Simpkins, S. D., Price, C. D., & Garcia, K. (2015). Parental support and high school students’ motivation in biology, chemistry, and physics: Understanding differences among latino and caucasian boys and girls. Journal of Research in Science Teaching, 52, 1386–1407.
Simpson, R. D., & Oliver, J. S. (1990). A Summary of major influences on attitude toward and achievement in science among adolescent students. Science & Education, 74, 1–18.
Stake, J. E., & Mares, K. R. (2001). Science enrichment programs for gifted high school girls and boys: Predictors of program impact on science confidence and motivation. Journal of Research in Science Teaching, 38, 1065–1088.
Staniec, J. F. O. (2004). The effects of race, sex, and expected returns on the choice of college major. Eastern Economic Journal, 30, 549–562.
Steele, C., & Aronson, J. (1995). Stereotype threat and the intellectual test performance of African Americans. Journal of Personality and Social Psychology, 69, 797.
Stuart, E., & Rubin, D. B. (2008). Best practices in quasi-experimental designs: Matching methods for causal inference. In J. Osborne (Ed.) (pp. 155–176). Thousand Oaks: Sage Publications.
Summers, M. F., & Hrabowski, F. A. (2006). Preparing minority scientists and engineers. Institutions, 17, 18.
Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. T. (2006). Planning early for careers in science. Science, 312, 1143–1144.
Tyson, W., Lee, R., Borman, K. M., & Hanson, M. A. (2007). Science, technology, engineering, and mathematics (STEM) pathways: High school science and math coursework and postsecondary degree attainment. Journal of Education for Students Placed at Risk, 12, 243–270.
von Hippel, P. T. (2007). Regression with missing Y’s: An improved strategy for analyzing multiply imputed data. Sociological Methodology, 37, 83–117.
Wai, J., Lubinski, D., Benbow, C. P., & Steiger, J. H. (2010). Accomplishment in science, technology, engineering, and mathematics (STEM) and its relation to STEM educational dose: A 25-year longitudinal study. Journal of Educational Psychology, 102, 860.
Walker, C. L., & Rakow, S. J. (1985). The status of Hispanic American-students in Science—attitudes. Hispanic Journal of Behavioral Sciences, 7, 225–245.
Wang, X. (2013). Why students choose STEM Majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50, 1081–1121.
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81.
Xie, Y., & Killewald, A. (2012). Is American Science in Decline?. Cambridge: Cambridge Harvard University Press.
Yoder, B. L. (2012). Engineering by the Numbers. St. Joseph, MI: American Society of Agricultural and Biological Engineers.
Yonezawa, S., Wells, A. S., & Serna, I. (2002). Choosing tracks:“Freedom of Choice” in detracking schools. American Educational Research Journal, 39, 37–67.
Zhou, X. (2005). The institutional logic of occupational prestige ranking: Reconceptualization and reanalyses. American Journal of Sociology, 111, 90–140.
Acknowledgements
The authors would like to thank their undergraduate research assistant, Summer Lopez Colorado, for her work on developing this paper, Dr. Anna R. Haskins and Clara M. Elpi for their careful reviews of the manuscript, and the anonymous reviewers for their helpful comments. In addition, the authors thank seminar participants at the Cornell Center for the Study of Inequality for their helpful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Alvarado, S.E., Muniz, P. Racial and Ethnic Heterogeneity in the Effect of MESA on AP STEM Coursework and College STEM Major Aspirations. Res High Educ 59, 933–957 (2018). https://doi.org/10.1007/s11162-018-9493-3
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11162-018-9493-3