Abstract
Racial and ethnic disparities in STEM achievement are associated with weaker economic growth, greater social inequalities, and narrower parameters of scientific inquiry. Extant research suggests that undergraduate research experiences (URE) can reduce those disparities by enhancing perceptions of belonging and scientific self-efficacy among students from underrepresented groups. However, to date, very few studies have examined the relationship between URE and post-baccalaureate educational achievement gains among such students and those that have tend to be limited in terms of causal leverage and generalizability. In this study, we aim to make progress by analyzing data from the California State University system’s longstanding Louis Stokes Alliance for Minority Participation (CSU-LSAMP) program. Applying a quasi-experimental research design and drawing upon a large and representative sample of students whom we tracked over time, we observe that URE is strongly associated with post-baccalaureate enrollment and graduation in STEM disciplines among students from underrepresented backgrounds.
Similar content being viewed by others
Notes
The California State University, Impact of the CSU/Diversity. Available at: https://www2.calstate.edu/impact-of-the-csu/diversity.
California State University Louis Stokes Alliance for Minority Participation STEM Pathways and Research Alliance, Year Two Report, June 2020, Institute for Social Research. Available at: https://www.csus.edu/college/natural-sciences-mathematics/csu-lsamp/_internal/_documents/csu-lsamp-spara-year-two-report.pdf.
For a full CSU-LSAMP program description, see the program’s website: https://www.csus.edu/college/natural-sciences-mathematics/csu-lsamp/program-goals.html.
We were able to obtain ERS data for 80% of CSU-LSAMP participants and NSC records for 68% of CSU-LSAMP participants. ERS data could not be obtained for CSU-LSAMP participants whose WebAMP records were missing SSN. Students were matched to NSC data using name and birthdate. Students can also decline to share their data with the NSC, which accounts for the lower match rate with NSC data.
We use a simplified measure of URE, rather than one that measures differences in duration of participation (e.g., multiple years vs. one), because the matching method we employed is much more complicated if attempting to equalize three groups (non-URE, 1 year of URE, 2 or more years of URE) instead of just two, making us much more confident in the stability of the estimates when using this simplified procedure. However, it is worth pointing out that in preliminary statistical analyses in which we simply include the matching variables as model covariates, we did include a 3-point measure of URE that distinguishes those who participated in such research activities for only 1 year and those who participated for 2 or more years. Those models reveal that, generally speaking, such additional URE “dosage” slightly strengthens the size of the coefficients associated with URE participation. Thus, the estimates we report below may be considered conservative.
As a robustness test, we also estimated models using the nearest neighbor method of matching. We did so repeatedly, specifying that the procedure use 1 and 10 “nearest neighbors” to match. The results we obtained look highly similar, in terms of both substantive and statistical significance, to those we report here.
As another robustness check, in alternative model estimations, we used the complementary log-log estimator in light of the varying degrees of skew associated with our outcome variables. The results did not meaningfully differ, either substantively or statistically, in those models, so we report the probit results because probit regression is familiar to a broader audience of readers.
Rather than an HLM model assuming random campus effects, in alternative models, we included “fixed effects” dummy variables for all the campuses to account for any selection bias associated with unobserved campus-level variables. Because doing so added multicollinearity to the models, causing some of the covariates to be dropped, we do not report those results here. However, importantly, the inclusion of campus fixed effects did not substantially alter any of the results we report below.
Additional analyses reveal that URE is also associated with increases in graduation rates in non-STEM fields, but the relationship is smaller.
Unfortunately, limitations in data availability precluded us from analyzing post-graduate enrollment in STEM fields, specifically. Specifically, NSC enrollment data does not consistently provide the level of enrollment or discipline that information is only reflected in NSC records when a degree is awarded. If we had, we might have observed an even stronger relationship, given the other patterns of results we report here.
Additional analyses reveal that URE is also associated with post-graduate degree attainment in non-STEM fields, though the relationship is smaller.
References
Almukhambetova, A., & Kuzhabekova, A. (2021). Negotiating conflicting discourses. Female students’ experiences in STEM majors in an international university in Central Asia. International Journal of Science Education, 43(4), 570–593. https://doi.org/10.1080/09500693.2021.1875150
Astin, A. W. (1999). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 40, 518–529.
Austin, P. C., & Stuart, E. A. (2015). Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Statistics in Medicine, 34(28), 3661–3679. https://doi.org/10.1002/sim.6607
Barlow, A. E. L., & Villarejo, M. (2004). Making a difference for minorities: Evaluation of an educational enrichment program. Journal of Research in Science Teaching, 41(9), 861–881. https://doi.org/10.1002/tea.20029
Castleman, B. L., & Page, L. C. (2014). Summer melt: Supporting low-income students through the transition to college. Harvard Education Press https://www.hepg.org/hep-home/books/summer-melt
Chen, X. (2013). STEM attrition: College students’ paths into and out of STEM fields (NCES 2014-001). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education https://nces.ed.gov/pubs2014/2014001rev.pdf
Clewell, B. C. (2005). Final report on the evaluation of the National Science Foundation Louis Stokes Alliances for Minority Participation program: Full technical report and appendices. The Urban Institute https://www.urban.org/sites/default/files/publication/43766/411301_LSAMP_report_appen.pdf
Coley, B., & Vallas, C. (2015). Tapping into the talent: Exploring the barriers of the engineering transfer pathway (2015 ASEE Annual Conference and Exposition Proceedings). University of Virginia https://peer.asee.org/tapping-into-the-talent-exploring-the-barriers-of-the-engineering-transfer-pathway.pdf
Crawford, I., Suarez-Balcazar, Y., Reich, J., Figert, A., & Nyden, P. (1996). The use of research participation for mentoring prospective minority graduate students. Teaching Sociology, 24(3), 256. https://doi.org/10.2307/1318740
Dickson, L. (2010). Race and gender differences in college major choice. The ANNALS of the American Academy of Political and Social Science, 627(1), 108–124. https://doi.org/10.1177/0002716209348747
Dynarski, S. (2008). Building the stock of college-educated labor. Journal of Human Resources, 43(3), 576–610. https://doi.org/10.3368/jhr.43.3.576
Estrada, M., Hernandez, P. R., & Schultz, P. W. (2018). A longitudinal study of how quality mentorship and research experience integrate underrepresented minorities into STEM careers. CBE—Life Sciences Education, 17(1). https://doi.org/10.1187/cbe.17-04-0066
Fakayode, S. O., Yakubu, M., Adeyeye, O. M., Pollard, D. A., & Mohammed, A. K. (2014). Promoting undergraduate STEM education at a historically black college and university through research experience. Journal of Chemical Education, 91(5), 662–665. https://doi.org/10.1021/ed400482b
Gibson, S. M., Brinkley, K., Griggs, L. A., James, B. N., Smith, M., Schwitzerlett, M., Waller, L. M., & Hargraves, R. H. (2021). Implementing a hybrid summer transition program. Frontiers in Education, 6. https://doi.org/10.3389/feduc.2021.674337
Govindan, B., Pickett, S., & Riggs, B. (2020). Fear of the CURE: A beginner’s guide to overcoming barriers in creating a course-based undergraduate research experience. Journal of Microbiology & Biology Education, 21(2). https://doi.org/10.1128/jmbe.v21i2.2109
Haeger, H., Banks, J. E., Smith, C., & Armstrong-Land, M. (2020). What we know and what we need to know about undergraduate research. Scholarship and Practice of Undergraduate Research, 3(4), 62–69. https://doi.org/10.18833/spur/3/4/4
Hahn, E. D., & Soyer, R. (2005). Probit and logit models: Differences in the multivariate realm. The Journal of the Royal Statistical Society, Series B, 67, 1–12 https://home.gwu.edu/~soyer/mv1h.pdf
Hathaway, R. S., Nagda, B. A., & Gregerman, S. R. (2002). The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development, 43, 614–631.
Hernandez, P. R., Woodcock, A., Estrada, M., & Schultz, P. W. (2018). Undergraduate research experiences broaden diversity in the scientific workforce. BioScience, 68(3), 204–211. https://doi.org/10.1093/biosci/bix163
Hunter, A. B., Laursen, S. L., & Seymour, E. (2007). Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education, 91(1), 36–74. https://doi.org/10.1002/sce.20173
Hurtado, S., Eagan, K., Figueroa, T., & Hughes, B. (2014). Reversing underrepresentation: The impact of undergraduate research programs on enrollment in STEM graduate programs. Higher Education Research Institute.
Isphording, I., & Qendrai, P. (2019). Gender differences in student dropout in STEM. IZA. Research Reports, 87 https://docs.iza.org/report_pdfs/iza_report_87.pdf
Jones, M. T., Barlow, A. E. L., & Villarejo, M. (2010). Importance of undergraduate research for minority persistence and achievement in biology. The Journal of Higher Education, 81(1), 82–115. https://doi.org/10.1353/jhe.0.0082
Lopatto, D. (2007). Undergraduate research experiences support science career decisions and active learning. CBE—Life Sciences. Education, 6(4), 297–306. https://doi.org/10.1187/cbe.07-06-0039
Maton, K. I., Hrabowski, F. A., & Schmitt, C. L. (2000). African American college students excelling in the sciences: College and postcollege outcomes in the Meyerhoff Scholars Program. Journal of Research in Science Teaching, 37(7), 629–654. https://doi.org/10.1002/1098-2736(200009)37:7<629::aid-tea2>3.0.co;2-8
McGill, B. M., Foster, M. J., Pruitt, A. N., Thomas, S. G., Arsenault, E. R., Hanschu, J., Wahwahsuck, K., Cortez, E., Zarek, K., Loecke, T. D., & Burgin, A. J. (2021). You are welcome here: A practical guide to diversity, equity, and inclusion for undergraduates embarking on an ecological research experience. Ecology and Evolution, 11(8), 3636–3645. Portico. https://doi.org/10.1002/ece3.7321
Morgan, S., & Winship, C. (2014). Counterfactuals and causal inference: Methods and principles for social research. Cambridge University Press. https://doi.org/10.1017/CBO9781107587991
Museus, S. D., Palmer, R. T., Davis, R. J., & Maramba, D. C. (2011). Racial and ethnic minority students’ success in STEM education. ASHE Higher Education Report, 36(6), 1–140. https://doi.org/10.1002/aehe.3606
National Academies of Sciences, Engineering, and Medicine (NASEM). (2017). Undergraduate research experiences for STEM students: Successes, challenges, and opportunities. National Academies Press. https://doi.org/10.17226/24622
National Center for Science and Engineering Statistics (NCSES). (2023). Diversity and STEM: Women, minorities, and persons with disabilities 2023. National Science Foundation Alexandria, VA https://ncses.nsf.gov/pubs/nsf23315/
Okojie, F. A., Tchounwou, M., & Addison, C. (2021). A mixed methods study of factors that enhance Louis Stokes Mississippi Alliance for Minority Participation (LSMAMP) students degree attainment in STEM. The Journal of the Mississippi Academy of Sciences, 66(1), 6–27. https://doi.org/10.31753/jmas.66_106
Palid, O., Cashdollar, S., Deangelo, S., Chu, C., & Bates, M. (2023). Inclusion in practice: A systematic review of diversity-focused STEM programming in the United States. International Journal of STEM Education, 10(1). https://doi.org/10.1186/s40594-022-00387-3
Pender, M., Marcotte, D. E., Domingo, M. R. S., & Maton, K. I. (2010). The STEM pipeline: The role of summer research experience in minority students’ graduate aspirations. Education Policy Analysis Archives, 18(30), 1. https://doi.org/10.14507/epaa.v18n30.2010
President’s Council of Advisors on Science and Technology. (2012). Engage to Excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Executive Office of the President of the United States https://files.eric.ed.gov/fulltext/ED541511.pdf
Professionals Australia. (2018). All talk. Gap between policy and practice a key obstacle to gender equity in STEM. 2018 Women in STEM Professions Survey Report http://hdl.voced.edu.au/10707/468516
Rozgonjuk, D., Konstabel, K., Barker, K., Rannikmäe, M., & Täht, K. (2022). Epistemic beliefs in science, socio-economic status, and mathematics and science test results in lower secondary education: A multilevel perspective. Educational Psychology, 43(1), 22–37. https://doi.org/10.1080/01443410.2022.2144143
Russell, S. H., Hancock, M. P., & McCullough, J. (2007). Benefits of undergraduate research experiences. Science, 316(5824), 548–549. https://doi.org/10.1126/science.1140384
Seymour, E., & Hewitt, N. M. (1997). Talking about leaving (Vol. 34). Westview Press.
Villarejo, M., & Barlow, A. E. L. (2007). Evolution and evaluation of a biology enrichment program for minorities. Journal of Women and Minorities in Science and Engineering, 13(2), 119–144. https://doi.org/10.1615/jwomenminorscieneng.v13.i2.20
Watt, H. M. G., Shapka, J. D., Morris, Z. A., Durik, A. M., Keating, D. P., & Eccles, J. S. (2012). Gendered motivational processes affecting high school mathematics participation, educational aspirations, and career plans: A comparison of samples from Australia, Canada, and the United States. Developmental Psychology, 48(6), 1594–1611. https://doi.org/10.1037/a0027838
Zubair, A., & Al-Thani, N. J. (2022). Undergraduate research experience models: A systematic review of the literature from 2011 to 2021. International Journal of Educational Research, 114. https://doi.org/10.1016/j.ijer.2022.101996
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
This section describes all variables used in the analysis. Table 1 displays descriptive statistics for all variables.
Student Characteristics
-
Undergraduate research experience: undergraduate research experience outside of the classroom under the guidance of a faculty mentor. This variable is coded 0|1, with 1 representing one or more years of URE.
-
STEM bachelor’s degree: this variable is coded 0|1, with 1 representing completion of a STEM bachelor’s degree.
-
Post-baccalaureate enrollment: this variable is coded 0|1, with 1 representing enrollment following completion of a bachelor’s degree. NSC enrollment data does not consistently provide the level of enrollment or discipline; that information is only reflected in NSC records when a degree is awarded.
-
STEM graduate degree: This variable is coded 0|1, with 1 representing completion of a STEM masters or doctoral degree.
-
Major: all CSU-LSAMP students are STEM majors (or considering a STEM major). Dummy variables (0|1) were created for biology, math, physical sciences, computer science, engineering/engineering technologies, and natural resources and conservation. All other STEM majors were used as the reference category.
-
Gender: 0= male, 1 =female.
-
Pell eligibility: dummy variable is coded as 0 = not Pell eligible and 1 = Pell eligible.
-
Age at CSU entry: continuous age at CSU entry.
-
Race/ethnicity: dummy variables (0|1) were created for all race/ethnicity categories. Hispanic students of all races are coded as Hispanic. The non-Hispanic multiracial UR category includes students who identified themselves as multiracial, where at least one race category was Black, Native American, or Pacific Islander. The non-Hispanic multiracial non-UR category includes students who identified themselves as White and Asian. Non-Hispanic Whites were used as the reference category. Six non-Hispanic multiracial non-UR transfer students were dropped from the transfer student sample as none had participated in URE, and no weights could be generated for them.
-
Academic preparedness: for traditional students, SAT scores and HS GPA scores were converted to standard deviation units (with a mean of zero and a standard deviation of 1). This was accomplished by simply subtracting the mean from each observation in the variable and then dividing it by its standard deviation. Then, these two variables were summed into an index. Then, the resulting two-item index was converted to a 0-1 scale, by adding the lowest value of the variable to each observation (that brought the lowest observation up to 0) and then dividing by the highest value. For transfer students, transfer GPA was used, as SAT scores and HS GPA were not available.
-
Class at CSU entry: dummy variables (0|1) were created for each class level at entry (freshman, sophomore, junior, and senior).
Campus Variables
-
CSU-LSAMP program size: total number of program participants at each campus 2021.
-
Total enrollment: total campus enrollment in fall 2020.
-
Students in off-campus housing: proportion of students in off-campus housing.
-
Acceptance rate: proportion of students who applied who were admitted to each campus.
-
Urbanicity: dummy variables (0|1) were created for rural, suburban, and urban campuses. Rural is used as the reference category.
-
Program emphasis (URE implementation): CSU-LSAMP programs can select one of three program emphases: academic (focused student success), professionalization (focused on graduate school preparation and URE), and dual emphasis (both academic and professionalization). Only one campus had an academic focus, California Maritime Academy, a small program with very few research participants. This campus was grouped with the dual campuses in the reference category.
-
Proportion of White faculty: proportion of full-time instructional staff who were White in 2020, data comes from the IPEDS survey.
-
Proportion of White students: proportion of students who were White in 2020.
Appendix 2. Balance tables
Appendix 3. Full results
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Barker, D.C., Messier, V., Marcotte, D.E. et al. Undergraduate Research Experience and Post-graduate Achievement Among Students from Underrepresented Groups in STEM. Journal for STEM Educ Res (2023). https://doi.org/10.1007/s41979-023-00107-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s41979-023-00107-8