Graduate and doctoral schools around the world struggle to shorten the long time to degree and to prevent high dropout rates. While most of previous research studied individual determinants of PhD completion, we analyze the impact of two structural reforms of the doctoral program on thesis completion at a selective European graduate school. Exploiting a unique PhD dataset covering 30 entry cohorts, we identify reform effects on PhD outcomes using an interrupted time-series regression design. We find that the first reform improved timely completion rates by between 10 and 15 percentage points (according to the specific outcome), whereas the second reform increased completion rates by between 9 and 20 percentage points. Additionally, each reform reduced dropout rates by 7 percentage points. The results are robust to various sensitivity checks. At the end, we discuss lessons learned for those in charge of graduate schools and/or PhD programs.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
A commonly heard light-bulb joke about PhD programs. For a recent report, see “The Long-Haul Degree”. By Patricia Cohen in the New York Times, April 16, 2010. Accessible online at http://www.nytimes.com/2010/04/18/education/edlife/18phd-t.html?pagewanted=all.
This number normally includes the time to finish a masters’ degree.
Some exceptions exist in the United States, notably Ehrenberg et al. (2007) and Groen et al. (2008), which evaluated the impact of the Andrew W. Mellon Foundation’s Graduate Education Initiative (GEI) in the 1990s that provided funding to improve the quality of PhD programs in humanities and related social sciences.
EUI Strategic Review, November 1991, “Beyond Maintenance”. Although this review was published later than the year 1990, the recommendations would affect students starting in the 1989/90 student cohort.
“EUI Strategic Review. November 2001 Enhancing and Enlarging. The Future EUI”. The main results of the review were put in place for the student cohort that had started in the year 2000.
See for instance the position paper issued by the Coimbra group on doctoral education, a leading coalition of research universities. http://www.coimbra-group.eu/uploads/2010-2011/DoctoralProgrammesPositionPaper.pdf.
Our results do not change substantively when we apply the time-restrictions more sharply (i.e. 4 years is 48 months, 5 years is 60 months, and 6 years is 72 months).
Unfortunately we were not able to retrieve the departmental assessment of candidates in the selection procedure. We are therefore not able to include any measures of academic ability in our list of covariates. Nevertheless, since the process is so selective, the variance in academic ability may not be all that big.
More precisely, Y t is the predictive margin of the outcome of interest, which correspond to the covariate-adjusted predicted probability of graduating within 4, 5 and 6 years and to dropout from the program. More details are provided below.
One might argue that interpreting test statistics makes little sense as we have total population data of EUI students from 1976 to 2006. However, the students who effectively studied at the EUI might be seen at as a sample from a universe of students who are eligible for the EUI. Elements of chance in the recruitment resulting from arbitrariness in applying and admitting then creates a mechanism of chance sampling from that super-population, that is, if recruitment were repeated hypothetically different student populations may have been created. Thus, one could interpret standard errors as measures for the variability of the hypothetical distribution of coefficients that could have been produced by other potential recruitment outcomes under the assumption that the null hypothesis is true. Nonetheless, we do not over-interpret test statistics in our setting and use them only in a heuristic way.
For checking robustness, we applied a more flexible local linear regression to our data, using optimal bandwidth (Imbens and Kalyanaraman 2012) and lower degree polynomials (Gelman and Imbens 2014). Results (available from the authors upon request) were substantially very similar to the models presented in the article.
Detailed results from the logistic regression models from which the predictive margins were derived are available from the authors upon request.
Indeed, the coefficients for REF1 × TIME reported in Table 3, which measure the difference in the slopes after and before the first reform, are not statistically significant.
This finding comes from the 2012 Alumni Survey conducted by the EUI on a sample of 945 former PhD students in order to investigate their following occupational placement.
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.
Abadie, A., Diamond, A., & Hainmueller, J. (2015). Comparative politics and the synthetic control method. American Journal of Political Science, 59(2), 495–510.
Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate. Research in Higher Education, 27(1), 3–14.
Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral study. Research in Higher Education, 31(4), 369–385.
Ballarino, G., & Colombo, S. (2010). Occupational outcomes of PhD graduates in Northern Italy. Italian Journal of Sociology of Education, 2(2), 149–171.
Baum, C. F., & Schafer, M. E. (2013). actest: Stata module to perform Cumby-Huizinga general test for autocorrelation in time series. Statistical Software Components s457668, Boston College Department of Economics. http://ideas.repec.org/c/boc/bocode/s457668.html.
Beekhoven, S., De Jong, U., & Van Hout, H. (2002). Explaining academic progress via combining concepts of integration theory and rational choice theory. Research in Higher Education, 43(5), 577–600.
Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348–355.
Bhaskaran, K., Gasparrini, A., Hajat, S., Smeeth, L., & Armstrong, B. (2013). Time series regression studies in environmental epidemiology. International Journal of Epidemiology, 42(4), 1187–1195.
Booth, A. L., & Satchell, S. E. (1995). The hazards of doing a PhD: An analysis of completion and withdrawal rates of British PhD students in the 1980s. Journal of the Royal Statistical Society. Series A (Statistics in Society), 158(2), 297–318.
Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press.
Breen, R., & Goldthorpe, J. H. (1997). Explaining educational differentials: Towards a formal rational action theory. Rationality and society, 9(3), 275–305.
Byrne, J., Jørgensen, T., & Loukkola, T. (2013). Quality assurance in doctoral education—results of the ARDE project. Brussels: European University Association.
Council of Graduate Schools. (2010). Ph.D. completion and attrition: Policies and practices to promote student success. Washington DC: CGS.
Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974.
Cumby, R. E., & Huizinga, J. (1992). Investigating the correlation of unobserved expectations: Expected returns in equity and foreign exchange markets and other examples. Journal of Monetary Economics, 30(2), 217–253.
De Valero, Y. F. (2001). Departmental factors affecting time-to-degree and completion rates of doctoral students at one land-grant research institution. Journal of Higher Education, 72(3), 341–367.
Ehrenberg, R. G., Jakubson, G. H., Groen, J. A., So, E., & Price, J. (2007). Inside the black box of doctoral education: What program characteristics influence doctoral students’ attrition and graduation probabilities? Educational Evaluation and Policy Analysis, 29(2), 134–150.
Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and completion probabilities? Journal of Human Resources, 30(3), 581–609.
Frasier, H. (2013). An analysis of institutional characteristics that contribute to extended time to doctoral degree. Unpublished doctoral dissertation, University of Maryland, College Park.
Gelman, A., & Imbens, G. (2014). Why high-order polynomials should not be used in regression discontinuity designs. NBER Working Paper No. 20405. doi:10.3386/w20405.
Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments. The Journal of Higher Education, 76(6), 669–700.
Gottman, J. M. (1981). Time-series analysis. A comprehensive introduction for social scientists. New York: Cambridge University Press.
Graubard, B. I., & Korn, E. L. (1999). Predictive margins with survey data. Biometrics, 55(2), 652–659.
Greenland, S. (2004). Model-based estimation of relative risks and other epidemiologic measures in studies of common outcomes and in case-control studies. American Journal of Epidemiology, 160(4), 301–305.
Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. Cornell Higher Education Research Institute, mimeo.
Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in graduate education. Economics of Education Review, 27(2), 111–124.
Groenvynck, H., Vandevelde, K., & Van Rossem, R. (2013). The PhD track: Who succeeds, who drops out? Research Evaluation, 22(4), 199–209.
Hansen, W. L. (1990). Educating and training new economics PhDs: How good a job are we doing? American Economic Review, 80(2), 437–450.
Heckman, J., & Hotz, V. J. (1989). Choosing among alternative nonexperimental methods for estimating the impact of social programs: The case of manpower training. Journal of the American Statistical Association, 84(408), 862–874.
Hovdhaugen, E. (2011). Do structured study programmes lead to lower rates of dropout and student transfer from university? Irish Educational Studies, 30(2), 237–251.
Imbens, G., & Kalyanaraman, K. (2012). Optimal bandwidth choice for the regression discontinuity estimator. The Review of Economic Studies, 79(3), 933–959.
Linden, A. (2015). Conducting interrupted time-series analysis for single-and multiple-group comparisons. Stata Journal, 15(2), 480–500.
Linden, A. (2017). A comprehensive set of postestimation measures to enrich interrupted time-series analysis. Stata Journal, 17(1), 73–88.
Lott, J. L., Gardner, S., & Powers, D. A. (2009). Doctoral student attrition in the STEM fields: An exploratory event history analysis. Journal of College Student Retention: Research, Theory and Practice, 11(2), 247–266.
Lovitts, B. E. (2001). Leaving the ivory tower: The causes and consequences of departure from doctoral study. Lanham: Rowman & Littlefield.
Lunneborg, C. E., & Lunneborg, P. W. (1973). Doctoral study attrition in psychology. Research in Higher Education, 1(4), 379–387.
Mangematin, V. (2000). PhD job market: Professional trajectories and incentives during the PhD. Research Policy, 29(6), 741–756.
McKnight, S., McKean, J. W., & Huitema, B. E. (2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5, 87–101.
National Science Foundation. (2013). Doctorate Recipients from U.S. Universities 2012. Special Report NSF 14-305. Arlington, VA. Retrieved September 3, 2017 from http://www.nsf.gov/statistics/sed/digest/2012.
Price, J. (2006). Does a spouse slow you down? Marriage and graduate student outcomes. Cornell University, School of Industrial and Labor Relations site. Retrieved September 3, 2017 from http://digitalcommons.ilr.cornell.edu/workingpapers/147.
Rodriguez-Planas, N. (2012). Longer-term impacts of mentoring, educational services, and learning incentives: Evidence from a randomized trial in the United States. American Economic Journal: Applied Economics, 4(4), 121–139.
Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of doctoral degrees. Research in Higher Education, 39(3), 319–335.
Shadish, W.R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Wadsworth Cengage learning.
Siegfried, J. J., & Stock, W. A. (2001). So you want to earn a Ph.D. in economics? How long do you think it will take? Journal of Human Resources, 36(2), 364–378.
Smallwood, S. (2004). Doctor dropout. The Chronicle of Higher Education, 16th January. Retrieved December, 2015.
StataCorp. (2015). Stata Statistical Software: Release 14. College Station: StataCorp LP.
Stock, W. A., Finegan, T. A., & Siegfried, J. J. (2006). Attrition in economics Ph.D. programs. The American Economic Review, 36(2), 458–466.
Taljaard, M., McKenzie, J. E., Ramsay, C. R., & Grimshaw, J. M. (2014). The use of segmented regression in analysing interrupted time series studies: An example in pre-hospital ambulance care. Implementation Science, 9(1), 77.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (Rev ed.). Chicago: University of Chicago Press.
Van de Schoot, R., Yerkes, M. A., Mouw, J. M., & Sonneveld, H. (2013). What took them so long? Explaining PhD delays among doctoral candidates. PLoS ONE, 8(7), e68839.
Van der Haert, M., Arias Ortiz, E., Emplit, P., Halloin, V., & Dehon, C. (2014). Are dropout and degree completion in doctoral study significantly dependent on type of financial support and field of research? Studies in Higher Education, 39(10), 1–25.
Van Ours, J. C., & Ridder, G. (2003). Fast track or failure: A study of the graduation and dropout rates of PhD students in economics. Economics of Education Review, 22(2), 157–166.
Wagner, A. K., Soumerai, S. B., Zhang, F., & Ross-Degnan, D. (2002). Segmented regression analysis of interrupted time series studies in medication use research. Journal of Clinical Pharmacy and Therapeutics, 27(4), 299–309.
Wao, H. O., & Onwuegbuzie, A. J. (2011). A mixed research investigation of factors related to time to the doctorate in education. International Journal of Doctoral Studies, 6, 115–134.
Williams, R. (2012). Using the margins command to estimate and interpret adjusted predictions and marginal effects. Stata Journal, 12(2), 308–331.
Wright, T., & Cochrane, R. (2000). Factors influencing successful submission of PhD theses. Studies in Higher Education, 25(2), 181–195.
Zhou, E. & Okahana, H. (2016). The role of department supports on doctoral completion and time-to-degree. Journal of College Student Retention: Research, Theory & Practice. doi:10.1177/1521025116682036
We would like to thank the European University Institute for providing us with the data and Ken Hulley for precious guidance on the information contained in the dataset. We would like to thank the participants to our seminar at the EUI’s Economics Department (2015) and at the COMPIE Conference at the Catholic University of Milan (2016) for useful comments. We would also like to express our gratitude to Hans-Peter Blossfeld, Andrea Ichino, Michael Grätz, Hannes Kröger, Loris Vergolini, and Nadir Zanini, as well as two reviewers who provided stimulating remarks on a previous version of this article. All remaining errors are our own.
About this article
Cite this article
Geven, K., Skopek, J. & Triventi, M. How to Increase PhD Completion Rates? An Impact Evaluation of Two Reforms in a Selective Graduate School, 1976–2012. Res High Educ 59, 529–552 (2018). https://doi.org/10.1007/s11162-017-9481-z
- Doctoral program
- Interrupted time-series analysis