While increasing numbers of students have gained access to higher education during the last several decades, postsecondary persistence and academic success remain serious concerns with only about half of college entrants completing degrees. Given concerns about affordability and resources, policymakers and administrators wonder how financial aid impacts student outcomes, particularly among low-income students. We investigate this question looking at a range of outcomes beyond just academic performance by focusing on the Gates Millennium Scholars (GMS) Program, a generous grant program that provided a renewable scholarship to talented undergraduate students of color with financial need. We isolate the impact of financial aid on academic and community engagement by comparing the outcomes of GMS recipients to similar non-recipients who were likely to have comparably-high levels of motivation and potential for success. With information about the application process, we use similar applicants not selected for the award as a comparison group. We then employ a Regression Discontinuity research design to provide causal estimates of the effects of GMS. The results suggest that GMS recipients were more likely to engage with peers on school work outside of class. Additionally, GMS recipients were much more likely to participate in community service activities and marginally more likely to participate in other extracurricular activities than their non-GMS peers.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Cohort 4 data is unavailable due to limitations in the data reporting in the fall of 2003.
Cohort 1 information was not available as all but 98 of the population of 2043 non-recipients in cohort 1 were disqualified due to Pell Grant Status.
Among those in Cohort 1, the first follow-up survey (administered in the spring of the third year) yielded a response from 76.0 % of recipients and 64.2 % of non-recipients. For Cohort 2, 83.1 % of recipients and 58.6 % of non-recipients completed the first follow-up survey. For Cohort 3, 89.7 % of recipients responded in the baseline of the survey (administered in the spring of the first year), and 74.9 % of non-recipients responded. Among those in Cohort 5, 89 % of recipients responded in the baseline survey, and 73 % of non-recipients responded.
Missing ACT or SAT scores caused us to drop approximately 150 observations from cohort 1, 100 observations from cohort 2, 90 observations from cohort 3, and 100 observations from cohort 5. If a student took both exams, we use the higher of the two scores. Scores were converted into percentiles using the test score national percentiles from the student's senior year of high school.
Results available upon request.
After eliminating those students not eligible for the GMS award, the non-recipient category for students in Cohort 2 was significantly smaller than for the other cohorts. This was due to incomplete applications caused by the relative newness of the award in the second year, coupled with missing non-cognitive scores for many of the remaining non-recipients. Further inspection of the covariates, however, revealed that these students were similar to the non-recipients of other cohorts, just fewer in number.
While we did examine thresholds of achievement separately by racial group, we find that the variation in the minimum non-cognitive and SAT/ACT scores needed to be a serious candidate for the GMS award was small enough to justify using one non-cognitive and SAT/ACT threshold across all races within a cohort.
The interaction of Score and Race does not substantively change the results, and thus, we show only the coefficients for Score and Race in our regression tables.
Results from these sensitivity checks are available upon request.
This variable was not collected during the baseline survey (during the freshman year). GPAs ranged from 1 to 5.0, with only 42 students in our sample reporting GPAs greater than a 4.0. We included GPAs from 4.0 to 5.0 in the data, as some high schools report grades on a 5.0 scale.
The possible responses were: 6 = Three or more times a week; 5 = Two or 3 times a week; 4 = Once a week; 3 = Two or 3 times a month; 2 = Once a month; or 1 = Less than once a month."
For ease of interpretation, we coded these categorical variables into binary outcomes, with 1 = once a week or more and 0 = less than once a week.
The possible responses were: 5 = Very often; 4 = Often; 3 = Sometimes; 2 = Seldom; 1 = Never. This question was asked of all students in the study, regardless of whether students lived in a residence hall or not. Therefore, this question will inevitably capture some students reporting no involvement with their residence hall because, by definition, they did not reside there. However, descriptive reports from NORC indicate that a majority of students (over 60 %) lived in residence halls during their first year.
Possible responses were: 5 = Very often; 4 = Often; 3 = Sometimes; 2 = Seldom; 1 = Never.
Adelman, C. (2006). The toolbox revisited: Paths to degree completion from high school through college. Washington, DC: U.S. Department of Education.
Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25(4), 297–308.
Astin, A. W., & Sax, L. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39(3), 251–263.
Astin, A., Sax, L., & Avalos, J. (1999). Long-term effects of volunteerism during the undergraduate years. Review of Higher Education, 22(2), 187–202.
Baker, C. N. (2008). Under-represented college students and extracurricular involvement: The effects of various student organizations on academic performance. Social Psychology of Education, 11(3), 273–298.
Bean, J. P. (1990). Why students leave: Insights from research. In Don Hossler & J. P. Bean (Eds.), The strategic management of college enrollments. San Francisco: Jossey-Bass.
Becker, G. (1964). Human capital: A theoretical and empirical analysis. New York: Columbia University Press.
Becker, G. S. (1967). Human capital and the personal distribution of income. Ann Arbor: University of Michigan Press.
Ben-Porath, Y. (1967). The production of human capital and the life cycle of earnings. Journal of Political Economy, 75, 352–365.
Bettinger, E. (2004). How financial aid affects persistence. In C. M. Hoxby (Ed.), College choices: The economics of where to go, when to go, and how to pay for it. Chicago, IL: University of Chicago Press.
Board, College. (2012). Trends in college pricing. New York: Sandy Baum and Jennifer Ma.
Campbell, D. E. (2000). Social capital and service learning. PS. Political Science and Politics, 33(3), 641–645.
Castleman, B. & Long, B. T. (forthcoming). Looking beyond enrollment: The causal effect of need-based grants on college access, persistence, and graduation. Journal of Labor Economics.
Chen, R., & DesJardins, S. (2010). Investigating the impact of financial aid on student dropout risks: Racial and ethnic differences. The Journal of Higher Education, 81(2), 179–208.
Choy, S. P. (2002). Access & persistence: Findings from 10 years of longitudinal research on students. Washington, DC: American Council on Education.
Darolia, R. (2014). Working (and studying) day and night: Heterogeneous effects of working on the academic performance of full-time and part-time students. Economics of Education Review, 38, 38–50.
DeSimone, J. S. (2008). The impact of employment during school on college student academic performance (NBER working paper 14006). Cambridge, MA: National Bureau of Economic Research.
DesJardins, S. L., Ahlburg, D. A., & McCall, B. P. (2002). Simulating the longitudinal effects of changes in financial aid on student departure from college. The Journal of Human Resources, 37(3), 653–679.
DesJardins, S. L., & McCall, B. P. (2014). The impact of the Gates Millennium Scholars Program on college and post-college related choices of high ability, low-income minority students. Economics of Education Review, 38, 124–138.
DesJardins, S. L., McCall, B. P., Ott, M., & Kim, J. (2010). A quasi-experimental investigation of how the Gates Millennium Scholars Program is related to college students’ time use and activities. Educational Evaluation and Policy Analysis, 32(4), 456–475.
Dynarski, S. (2000). Hope for whom? Financial aid for the middle class and its impact on college attendance. National Tax Journal, 53(3), 629–661.
Dynarski, S. (2003). Does aid matter? Measuring the effect of student aid on college attendance and completion. American Economic Review, 93(1), 279–288.
Dynarski, S. (2008). Building the stock of college-educated labor. Journal of Human Resources, 43(3), 576–610.
Dynarski, S. & Scott-Clayton, J. (2013). Financial aid policy: Lessons from research. In L. Barrow, T. Brock, & C. E. Rouse (Eds.), Postsecondary education in the United States (pp. 67–91). Princeton, NJ: The Future of Children, 23(1). Future of Children.
Fischer, M. J. (2007). Settling into campus life: Differences by race/ethnicity in college involvement and outcomes. Journal of Higher Education, 78, 125–161.
Golden, C., Katz, L., & Kuziemko, I. (2006). The homecoming of American college women: The reversal of the college gender gap. Journal of Economic Perspectives, 20(4), 133–156.
Gray, M. J., Ondaatje, E. H., & Zakaras, L. (1999). Combining service and learning in higher education: Summary report. Santa Monica: Rand Education.
Horn, L. J., & Malizio, A. (1998). Undergraduates who work: National Postsecondary Student Aid Study, 1996. National Center for Education Statistics, U.S. Department of Education, Office of Educational Research and Improvement Publication No. NCES 98-137. Washington DC: U.S. Government Printing Office.
Hu, S. (2008). Do financial aid awards in college affect graduates’ democratic values and civic engagement? Journal of College and Character, 10(1), 1–16.
Hu, S. (2010). Scholarship awards, college choice, and student engagement in college activities: A study of high-achieving low-income students of color. Journal of College Student Development, 51(2), 150–161.
Hu, S., & St. John, E. P. (2001). Student persistence in a public higher education system: Understanding racial and ethnic differences. Journal of Higher Education, 72(3), 265–286.
Hurtado, S., Nelson-Laird, T., & Perorazio, T. (2004). The transition to college for low-income students: The impact of the Gates Millennium Scholars program. In E. P. St. John (Ed.), Readings on equal education: Improving access and college success for diverse students: Studies of the Gates Millennium Scholars Program (Vol. 20). New York: AMS Press.
Hurtado, S., Saenz, V. B., & Dar, L. (2006). Low-income students of color in higher education and the Gates Millennium Scholars. Report for the Gates Foundation.
Hurwitz, M. (2012). The impact of institutional grant aid on college choice. Educational Evaluation and Policy Analysis, 34(3), 344–363.
Imbens, G., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142(2), 615–635.
Institute for Higher Education Policy (IHEP). (2010). Expanding access and opportunity: The impact of the Gates Millennium Scholars Program. Washington, DC: IHEP.
Jacob, B., McCall, B., & Stange, K. M. (2013). College as country club: Do colleges cater to students preferences for consumption? (NBER working paper 18745. Cambridge, MA: National Bureau of Economic Research.
Kane, T. J. (2003). A quasi-experimental estimate of the impact of financial aid on college-going. Working Paper 9703. Cambridge, MA: National Bureau of Economic Research.
Kim, J. (2012). Exploring the relationship between state financial aid policy and postsecondary enrollment choices: A focus on income and race differences. Research in Higher Education, 53(2), 123–151.
Kim, J., & Sax, L. (2014). The effects of student-faculty interaction on academic self-concept: Does academic major matter? Research in Higher Education, 55(8), 780–809.
Knapp, L. G., Kelly-Reid, J. E., & Ginder, S. A. (2012). Enrollment in postsecondary institutions, fall 2011; financial statistics, fiscal year 2011; and graduation rates, selected cohorts, 2003–2008. Washington, DC: National Center for Education Statistics, U.S. Department of Education.
Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., & Gonyea, R. M. (2008). Unmasking the effects of student engagement on first-year college grades and persistence. Journal of Higher Education, 79, 540–563.
Kuh, G. D., & Hu, S. (2001). The effects of student faculty interaction in the 1990s. The Review of Higher Education, 24(3), 309–332.
Kuh, G. D., Schuh, J. H., Whitt, E. J., Andreas, R., Lyons, J., Strange, C. C., et al. (1991). Involving colleges: Successful approaches to fostering student learning and development outside the classroom. San Francisco: Jossey-Bass Publishers.
Leppel, K. (2002). Similarities and differences in the college persistence of men and women. The Review of Higher Education, 25(4), 433–450.
Lesik, S. A. (2008). Studying the effectiveness of programs and initiatives in higher education using the regression-discontinuity design. In J. C. Smart (Ed.), Higher education: Handbook of theory and research. Dordrecht: Springer.
Leslie, L., & Brinkman, P. (1987). Student price response in higher education. Journal of Higher Education, 58, 181–204.
Long, B. T. (2004). The role of perceptions and information in college access: An exploratory review of the literature and possible data sources. The Education Resources Institute (TERI).
Long, B. T. (2008). What is known about the impact of financial aid? Implications for policy. National Center for Postsecondary Research (NCPR) Working Paper, Teachers College, Columbia University.
Long, B. T., & Riley, E. (2007). Financial aid: A broken bridge to college access. Harvard Educational Review, 77(1), 39–63.
Markus, G. B., Howard, J. P. F., & King, D. C. (1993). Integrating community service and classroom instruction enhances learning: Results from an experiment. Educational Evaluation and Policy Analysis, 15(4), 410–419.
McCall, B. P., & Bielby, R. M. (2012). Regression Discontinuity Design: Recent developments and a guide to practice for researchers in higher education. In J. C. Smart & M. B. Paulson (Eds.), Higher education: Handbook of theory and research (Vol. 27). Dordrecht: Springer.
McCrary, J. (2008). Manipulation of the running variable in the regression discontinuity design: A density test. Journal of Econometrics, 142(2), 714–798.
Melguizo, T. (2010). Are students of color more likely to graduate from college if they attend more selective institutions? Evidence from a cohort of recipients and non-recipients of the Gates Millennium Scholarship Program. Educational Evaluation and Policy Analysis, 32(2), 230–248.
Milem, J. F., & Berger, J. B. (1997). A modified model of college student persistence: Exploring the relationship between Astin’s theory of involvement and Tinto’s theory of student departure. Journal of College Student Development, 38(4), 387–400.
National Center for Education Statistics (NCES). (2002). Student background and institutional differences in reasons for early departure, 1996–98. Postsecondary Education Descriptive Analysis Reports. Office of Educational Research and Improvement.
National Center for Education Statistics (NCES). (2012). Digest of education statistics 20011 (NCES 2012-001). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC: U.S. Government Printing Office.
National Opinion Research Corporation (NORC). (2004). Gates Millennium Scholars (GMS) Tracking and Longitudinal Study: 2003–2004. Final Report, Cohort 3. Chicago, IL: NORC.
Nonis, S. A., & Hudson, G. I. (2006). Academic performance of college students: Influence of time spent studying and working. Journal of Education for Business, 81(3), 151–159.
Nora, A., Barlow, L., & Crisp, G. (2006). Examining the tangible and psychosocial benefits of financial aid with student access, engagement, and degree attainment. American Behavioral Scientist, 49(12), 1636–1651.
Oseguera, L., Denson, N., & Hurtado, S. (2008). Hispanic students and the Gates Millennium Scholarship Program: Promising results extending to the third college year. Journal of College Student Retention, 10(3), 307–338.
Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students: Findings and insights from twenty years of research. San Francisco: Jossey-Bass.
Perna, L. W. (Ed.). (2010). Understanding the working college student. Sterling, VA: Sylus Publishing.
Ramsey, J. (2010). Expanding access and opportunity: The impact of the Gates Millennium Scholars Program. Institute for Higher Education Policy.
Scott-Clayton, J. (2011). On money and motivation: A quasi-experimental analysis of financial incentives for college achievement. Journal of Human Resources, 46(3), 614–646.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton-Mifflin.
St. John, E. P. (1990). Price response in enrollment decisions: An analysis of the High School and Beyond sophomore cohort. Research in Higher Education, 31(2), 161–176.
St. John, E. P. & Chung, C. G. (2002). The impact of GMS on financial access: Analyses of the 2000 freshman cohort. Report prepared for the Bill & Melinda Gates Foundation.
St. John, E. P., Hu, S., Simmons, A., Carter, D. F., & Weber, J. (2004). What difference does it make? The influence of college major field on persistence by African American and White students. Research in Higher Education, 45(3), 202–232.
St. John, E. P., Paulsen, M. B., & Carter, D. F. (2005). Diversity, college costs, and postsecondary opportunity: An examination of the financial nexus between college choice and persistence for African Americans and Whites. The Journal of Higher Education, 76, 545–569.
Stinebrickner, R., & Stinebrickner, T. R. (2003). Working during school and academic performance. Journal of Labor Economics, 21(2), 473–491.
Stratton, L. S., O’Toole, D. M., & Wetzel, J. N. (2007). Are the factors affecting dropout behavior related to initial enrollment intensity for college undergraduates? Research in Higher Education, 48(4), 453–485.
Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45, 89–125.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition- (2nd ed.). Chicago: University of Chicago Press.
Trent, W. T., & St. John, E. P. (Eds.). (2008). Resources, assets, and strengths among successful diverse students: Understanding the contributions of the Gates Millennium Scholars Program.
Umbach, P. D., & Wawrzynski, M. R. (2005). Faculty do matter: The role of college faculty in student learning and engagement. Research in Higher Education, 46(2), 153–184.
Walpole, M. (2003). Socioeconomic status and college: How SES affects college experiences and outcomes. The Review of Higher Education, 27(1), 45–73.
Wetzel, J., O’Toole, D., & Peterson, S. (1999). Factors affecting student retention probabilities. Journal of Economics and Finance, 23(1), 45–55.
We thank the Bill & Melinda Gates Foundation and The Institute for Higher Education Policy for their support and the data. We also thank Melissa Bert for early help with the data and comments, and participants at the Student Financial Aid Research Network (SFARN) and Association for the Study of Higher Education (ASHE) conferences who provided useful insights on the mechanisms and methodologies in this paper. The views contained herein are not necessarily those of the Bill & Melinda Gates Foundation. All errors, omissions, and conclusions are our own.
About this article
Cite this article
Boatman, A., Long, B.T. Does Financial Aid Impact College Student Engagement?. Res High Educ 57, 653–681 (2016). https://doi.org/10.1007/s11162-015-9402-y
- Financial aid
- Student engagement
- College success
- Community service