Using predictive modelling to identify students at risk of poor university outcomes

Abstract

Predictive modelling is used to identify students at risk of failing their first-year courses and not returning to university in the second year. Our aim is twofold. Firstly, we want to understand the factors that lead to poor first-year experiences at university. Secondly, we want to develop simple, low-cost tools that would allow universities to identify and intervene on vulnerable students when they first arrive on campus. This is why we base our analysis on administrative data routinely collected as part of the enrollment process from a New Zealand university. We assess the ‘target effectiveness’ of our model from a number of perspectives. This approach is found to be substantially more predictive than a previously developed risk tool at this university. For example, observations from validation samples in the top decile of risk scores account for nearly 28 % of first-year course non-completions and 22 % of second-year student non-retentions at this university.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2

Notes

  1. 1.

    For example, previous studies have analyzed the reasons behind rising university dropout rates in France (Gury 2011), Italy (Di Pietro 2004), South Africa (Bokana 2010) and the UK (Johnes and McNabb 2004).

  2. 2.

    There are only eight universities in New Zealand. All of them are publically funded. The average enrolment level was slightly less than 17,000 in 2012. The University of Auckland was the largest with over 32,000 students and Lincoln University was the smallest with less than 4,000 students.

  3. 3.

    We do not distinguish in this analysis between course dropouts (i.e., individuals who discontinued study prior to the end of the semester) and course failures (i.e., individuals who continued to the end of the semester, completed all assessments, but failed the course). This is largely because of the government reporting requirements in New Zealand that emphasize non-completion outcomes as a result of either process.

  4. 4.

    Possible explanations for dropout behaviour include students struggling academically at university, transferring to other institutions, leaving for employment opportunities, etc. It should be noted that we have no information in the database on the reasons why individuals may have failed to return to this university at the beginning of the second year.

  5. 5.

    For more information on the NCEA system, see http://www.nzqa.govt.nz/qualifications-standards/qualifications/ncea/.

  6. 6.

    For more information on the process used to determine school deciles see

    http://www.minedu.govt.nz/NZEducation/EducationPolicies/Schools/SchoolOperations/Resourcing/OperationalFunding/Deciles/HowTheDecileIsCalculated.aspx.

  7. 7.

    We excluded all students from our sample who had studied previously at another university. These ‘university transfers’ may not be directly comparable to students who enter university Bachelor degree programmes for the first time.

  8. 8.

    Universities consider a range of factors in granting this Special Admission status. The applicant must be at least 20 years old by the time that they enroll at university. Previous educational records, work and training experiences, English language skills, and motivations for study are also considered.

  9. 9.

    Evidence from our sample suggests that those entering university through this Special Admission status are at higher risk of poor university outcomes on a number of dimensions compared to those with University Entrance. They are relatively more likely to be Pacifica or Māori (27.9 vs. 20.0 %), male (41.4 vs. 34.8 %), studying part-time (45.8 vs. 17.9 %), and originally from schools in the bottom three deciles (13.7 vs. 11.9 %).

  10. 10.

    It is unclear why contact hours were not reported for some of these courses. This could be related to the nature of some of these courses. For example, a course may have less formal scheduled contact hours if it involves largely ‘independent study’.

  11. 11.

    The typical university baccalaureate programme in New Zealand is completed in 3 years of full-time study.

  12. 12.

    The marginal effects could also be calculated at the sample means for the explanatory variables. For continuous functions in large samples, this technique yields similar results to the sample mean for the individual marginal effects.

References

  1. Angrist, J., & Lavy, V. (1999). Using Maimonides’s rule to estimate the effect of class size on children’s academic achievement. Quarterly Journal of Economics, 114, 533–575.

    Article  Google Scholar 

  2. Bai, J., & Maloney, T. (2006). Ethnicity and academic success at university. New Zealand Economic Papers, 40(2), 181–213.

    Article  Google Scholar 

  3. Belloc, F., Maruotti, A., & Petrella, L. (2010). University drop-out: An Italian experience. Higher Education, 60, 127–138.

    Article  Google Scholar 

  4. Betts, J. R., & Morell, D. (1999). The determinants of undergraduate grade point average: The relative importance of family background, high school resources, and peer group effects. Journal of Human Resources, 34(2), 268–293.

    Article  Google Scholar 

  5. Billings, J., Blunt, I., Steventon, A., Georghiou, T., Lewis, G., & Bardsley, M. (2012). Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30). BMJ Open, 2(4), e001667.

    Article  Google Scholar 

  6. Bokana, K. G. (2010). The attrition crisis in South African universities: How to keep students on the graduation path. Journal of Interdisciplinary Economics, 22(3), 181–201.

    Google Scholar 

  7. Cohn, E., Cohn, S., Balch, D. C., & Bradley, J. (2004). Determinants of undergraduate GPAs: SAT scores, high-school GPA and high school rank. Economics of Education Review, 23, 577–586.

    Article  Google Scholar 

  8. Cyrenne, P., & Chan, A. (2012). High school grades and university performance: A case study. Economics of Education Review, 31, 524–542.

    Article  Google Scholar 

  9. Di Pietro, G. (2004). The determinants of university dropout in Italy: A bivariate probability model with sample selection. Applied Economics Letters, 11(3), 187–191.

    Article  Google Scholar 

  10. Ficano, C. C. (2012). Peer effects in college academic outcomes: Gender matters! Economics of Education Review, 31, 1102–1115.

    Article  Google Scholar 

  11. Fredriksson, P., Öckert, B., & Oosterbeek, H. (2014). Inside the black box of class size: Mechanisms, behavioral responses, and social background. IZA Discussion Paper No. 8019, Institute for the Study of Labor, Bonn, Germany.

  12. Grayson, J. P. (1998). Racial origin and student retention in a Canadian university. Higher Education, 36, 323–352.

    Article  Google Scholar 

  13. Gury, N. (2011). Dropping out of higher education in France: A micro-economic approach using survival analysis. Education Economics, 19(1), 51–64.

    Article  Google Scholar 

  14. Ishitani, T. T. (2006). Studying attrition and degree completion behaviour among first-generation college students in the United States. Journal of Higher Education, 77(5), 861–885.

    Article  Google Scholar 

  15. Johnes, G., & McNabb, R. (2004). Never give up on the good times: Student attrition in the UK. Oxford Bulletin of Economics and Statistics, 66(1), 23–47.

    Article  Google Scholar 

  16. Kerkvliet, J., & Nowell, C. (2005). Does one size fit all? University differences in the influence of wages, financial aid and integration on student retention. Economics of Education Review, 24, 85–95.

    Article  Google Scholar 

  17. Krueger, A. B. (2003). Economic considerations and class size. Economic Journal, 113, F34–F63.

    Article  Google Scholar 

  18. Mastekaasa, A., & Smeby, J. C. (2008). Educational choice and persistence in male- and female-dominated fields. Higher Education, 55, 189–202.

    Article  Google Scholar 

  19. Montmarquette, C., Mahseredjian, S., & Houle, R. (2001). The determinants of university dropouts: A bivariate probability model with sample selection. Economics of Education Review, 20, 475–484.

    Article  Google Scholar 

  20. New Zealand Ministry of Education. (2004). Retention, completion and progression in tertiary education 2003. Wellington: Ministry of Education.

    Google Scholar 

  21. Ost, B. (2010). The role of peers and grades in determining major persistence in the sciences. Economics of Education Review, 29(6), 923–934.

    Article  Google Scholar 

  22. Rask, K. (2010). Attrition in STEM fields at a liberal arts college: The importance of grades and pre-collegiate preferences. Economics of Education Review, 29, 892–900.

    Article  Google Scholar 

  23. Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417–458.

    Article  Google Scholar 

  24. Robst, J., Keil, J., & Russo, D. (1998). The effect of gender composition of faculty on student retention. Economics of Education Review, 17(4), 429–439.

    Article  Google Scholar 

  25. Rodgers, T. (2013). Should high non-completion rates amongst ethnic minority students be seen as an ethnicity issue? Evidence from a case study of a student cohort from a British university. Higher Education, 66(5), 535–550.

    Article  Google Scholar 

  26. Singell, L. D. (2004). Come and stay a while: Does financial aid affect retention conditioned on enrolment at a large public university? Economics of Education Review, 23, 459–471.

    Article  Google Scholar 

  27. Stratton, L. S., O’Toole, D. M., & Wetzel, J. N. (2008). A multinomial logit model of college stopout and dropout behaviour. Economics of Education Review, 27, 319–331.

    Article  Google Scholar 

  28. Vaithianathan, R., Maloney, T., Putnam-Hornstein, E., & Jiang, N. (2013). Using predictive modelling to identify children in the public benefit system at high risk of substantiated maltreatment. American Journal of Preventive Medicine, 45(3), 354–359.

    Article  Google Scholar 

  29. Wetzel, J. N., O’Toole, D. M., & Peterson, S. (1999). Factors affecting student retention probabilities: A case study. Journal of Economics and Finance, 23(1), 45–55.

    Article  Google Scholar 

Download references

Acknowledgments

Access to the data used in this study was provided by a public university in New Zealand for the agreed purposes of this research project. The interpretations of the results presented in this study are those of the authors and do not reflect the views of this anonymous university. We thank Gail Pacheco and the anonymous referees of this journal for helpful suggestions on earlier drafts of this paper.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tim Maloney.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Jia, P., Maloney, T. Using predictive modelling to identify students at risk of poor university outcomes. High Educ 70, 127–149 (2015). https://doi.org/10.1007/s10734-014-9829-7

Download citation

Keywords

  • Educational finance and efficiency
  • Resource allocation
  • Predictive risk modelling
  • University dropout behavior
  • New Zealand

JEL Classification

  • I21
  • I22
  • I28