Skip to main content

Early predictors of need for remediation in the Australian general practice training program: a retrospective cohort study

Abstract

Underperforming trainees requiring remediation may threaten patient safety and are challenging for vocational training programs. Decisions to institute remediation are high-stakes—remediation being resource-intensive and emotionally demanding on trainees. Detection of underperformance requiring remediation is particularly problematic in general (family) practice. We sought to establish early-training assessment instruments predictive of general practice (GP) trainees’ subsequently requiring formal remediation. We conducted a retrospective cohort study of trainees from a large Australian regionally-based GP training organization. The outcome factor was requirement for formal remediation. Independent variables were demographic factors and a range of formative assessments conducted immediately prior to or during early-stage training. Analyses employed univariate and multivariate logistic regression of each predictor assessment modality with the outcome, adjusting for potential confounders. Of 248 trainees, 26 (10.5 %) required formal remediation. Performance on the Colleague Feedback Evaluation Tool (entailing feedback from a trainee’s clinical colleagues on clinical performance, communication and probity) and External Clinical Teaching Visits (half-day sessions of the trainee’s clinical consultations observed directly by an experienced GP), along with non-Australian primary medical qualification, were significantly associated with requiring remediation. There was a non-significant trend for association with performance on the Doctors Interpersonal Skills Questionnaire (patient feedback on interpersonal elements of the consultation). There were no significant associations with entry-selection scores or formative exam or assessment scores. Our finding that ‘in vivo’ assessments of complex behaviour, but not ‘in vitro’ knowledge-based assessments, predict need for remediation is consistent with theoretical understanding of the nature of remediation decision-making and should inform remediation practice in GP vocational training.

This is a preview of subscription content, access via your institution.

References

  • Bahrami, J. (1997). Remediation. British Medical Journal, 315, S2–S7114.

    Article  Google Scholar 

  • Barrett, A., Galvin, R., et al. (2015). A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees. Systematic Reviews, 4, 65.

    Article  Google Scholar 

  • Beaman, M., & King, J. (2009). Supporting professional performance: Managing trainees in difficulty South West Peninsula Deanery NHS Report.

  • Bingham, C., & Crampton, R. (2011). A review of prevocational medical trainee assessment in New South Wales. Medical Journal of Australia, 195, 410–412.

    Article  Google Scholar 

  • Black, D., & Welch, J. (2009). The under-performing trainee-concerns and challenges for medical educators. The Clinical Teacher, 6, 79–82.

    Article  Google Scholar 

  • Borkett-Jones, H., & Morris, C. (2010). Managing the trainee in difficulty. British Journal of Hospital Medicine, 71, 286–289.

    Article  Google Scholar 

  • Byrne, P. (2012). Are they safe in there? Patient safety and trainees in the practice. Australian Family Physician, 41, 26–29.

    Google Scholar 

  • Campbell, J., Narayanan, A., Burford, B., & Greco, M. (2010). Validation of a multi-source feedback tool for use in general practice. Education for Primary Care, 21, 165–179.

    Article  Google Scholar 

  • Cook, D. A., Beckman, T. J., Mandrekar, J. N., & Pankratz, V. S. (2010). Internal structure of mini-CEX scores for internal medicine residents: Factor analysis and generalizability. Advances in Health Sciences Education, 15, 633–645.

    Article  Google Scholar 

  • Dijksterhuis, F., Scheele, F., Schuwirth, L., et al. (2009). Progress testing in postgraduate medical education. Medical Teacher, 31, e464–e468.

    Article  Google Scholar 

  • Durning, S., et al. (2011). Viewing ‘strugglers” through a different lens: How a self regulated learning perspective can help medical educators with assessment and remediation. Academic Medicine, 86, 1–8.

    Article  Google Scholar 

  • Dwyer, A. J., Morely, P., Reid, E., & Angelatos, C. (2011). Distressed doctors: A hospital based support program for poorly performing and “at risk” junior medical staff. Medical Journal of Australia, 194, 466–469.

    Google Scholar 

  • Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51, 380–417.

    Article  Google Scholar 

  • Epstein, R. (2007). Assessment in medical education. New England Journal of Medicine, 356, 387–396.

    Article  Google Scholar 

  • Evans, D., Alstead, E., & Brown, J. (2010). Applying your clinical skills to students and trainees in academic difficulty. The Clinical Teacher, 7, 230–235.

    Article  Google Scholar 

  • Firth-Cozens, J., & King, J. (2006). Are psychological factors linked to performance? In J. Cox, J. King, A. Hutchinson, & P. McAvoy (Eds.), Understanding doctors’ performance (pp. 61–77). Oxford: Radcliffe Press.

    Google Scholar 

  • Garibaldi, R. A., Trontell, M. C., Waxaman, H., et al. (1994). The in-training examination in internal medicine. Annals of Internal Medicine, 121, 117–123.

    Article  Google Scholar 

  • Gladman, G. (2011). The Identification of general practice registrar needing assistance. The Australasian Medical Journal, 4, 308–314.

    Article  Google Scholar 

  • Hauer, K. E., Ciccone, A., Henzel, T. R., Katsufrakis, P., Miller, S. H., Norcross, W. A., et al. (2009). Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Academic Medicine, 84, 1822–1832.

    Article  Google Scholar 

  • Hays, R., & Peterson, L. (1996). An evaluation of visiting clinical teachers in general practice training. Education for General Practice, 7, 54–58.

    Google Scholar 

  • Hays, R., & Wellard, R. (1998). In-training assessment in postgraduate training for general practice. Medical Education, 32, 507–513.

    Article  Google Scholar 

  • Ingham, G., Morgan, S., et al. (2015). Are GP supervisors confident they can assess registrar competence and safety, and what methods do they use? Australian Family Physician, 44, 154–158.

    Google Scholar 

  • Kefalas, B., Usherwood, T., & Knight, A. (2005). External clinical teaching visits in the regionalised environment. Australian Family Physician, 34(Suppl 1), 10–12.

    Google Scholar 

  • Kimatian, S. J., & Lloyd, S. H. (2008). Remediation and due process for trainees: Setting the stage for future success. International Anesthesiology Clinics, 46, 113–125.

    Article  Google Scholar 

  • Larkins, S., Spillman, M., Vanlint, J., & Hays, R. (2003). Stress, personal and educational problems in vocational training. Australian Family Physician, 32, 473–476.

    Google Scholar 

  • Leap, L. L., & Fromson, J. A. (2006). Problem doctors: Is there a system-level solution? Annals of Internal Medicine, 144, 107–115.

    Article  Google Scholar 

  • Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65, S63–S67.

    Article  Google Scholar 

  • Miller, A., & Archer, J. (2010). Impact of workplace based assessment on doctors’ education and performance: A systematic review. British Medical Journal, 341, c5064.

    Article  Google Scholar 

  • Mitchell, C., Bhat, S., Herbert, A., & Baker, P. (2011). Workplace-based assessments of junior doctors: Do score predict training difficulties. Medical Education, 45, 1190–1198.

    Article  Google Scholar 

  • Mugford, B. (2003). Performance and remediation in general practice training. Australian Family Physician, 32, 544–546.

    Google Scholar 

  • Nair, B. R., Alexander, H., McGrath, B. P., et al. (2008). The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates. Medical Jounal of Australia, 189(3), 159–161.

    Google Scholar 

  • Paice, E., & Orton, V. (2005). Managing the trainee doctor in difficulty. In A. Hastie, I. Hastie, & N. Jackson (Eds.), Postgraduate medical education and training: A guide for primary and secondary care (pp. 165–175). Oxford: Radcliffe.

    Google Scholar 

  • Papadakis, M. A., Arnold, G. K., Blank, L. L., Holmboe, E. S., & Lipner, R. S. (2008). Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Annals of Internal Medicine, 148, 869–876.

    Article  Google Scholar 

  • Reamy, B., & Harman, J. (2006). Residents introuble: An in depth assessment of the 25 year experience of a single family medicine residency. Family Medicine, 38, 252–257.

    Google Scholar 

  • Riebschleger, M. P., & Haftel, H. M. (2013). Remediation in the context of the competencies: A survey of pediatrics residency program directors. Journal of Graduate Medical Education, 5, 60–63.

    Article  Google Scholar 

  • Roberts, N. K., Williams, R. G., Klingensmith, M., et al. (2012). The case of the entitled resident: A composite case study of a resident performance problem syndrome with interdisciplinary commentary. Medical Teacher, 34, 1024–1032.

    Article  Google Scholar 

  • Shah, S., Knowlden, F. R., Harris, M., & Yeo, G. (1997). General practice trainees in the borderline zone of competence: The experience of remediation at the regional general practice training and education centre. Education for General Practice, 8, 102–112.

    Google Scholar 

  • van der Vleuten, C. P. M., Schuwirth, L. W. T., Scheele, F., Driessen, E. W., & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice & Research Clinical Obstetrics & Gynaecology, 24, 703–719.

    Article  Google Scholar 

  • Van der Vleuten, C., Schuwirth, L., et al. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34, 205–214.

    Article  Google Scholar 

  • Violato, C., Lockyer, J., & Fidler, H. (2008). Changes in performance: A 5-year longitudinal study of participants in a multi-source feedback programme. Medical Education, 42, 1007–1013.

    Article  Google Scholar 

  • Weizberg, M., Smith, J. L., Murano, T., Silverberg, M., & Santen, S. A. (2015). What does remediation and probation status mean? A survey of emergency medicine residency program directors. Academic Emergency Medicine, 22, 113–116.

    Article  Google Scholar 

  • Williams, R. G., Roberts, N. K., Schwind, C. J., & Dunningham, G. L. (2009). The nature of general surgery resident performance problems. Surgery, 145, 651–658.

    Article  Google Scholar 

  • Winston, K. A., Van der Vleuten, C. P., & Scherpbier, A. J. (2014). Prediction and prevention of failure: An early intervention to assist at risk medical students. Medical Teacher, 36, 25–31.

    Article  Google Scholar 

  • Yao, D. C., & Wright, S. M. (2000). National survey of internal medicine residency program directors regarding problem residents. JAMA, 284, 1099.

    Article  Google Scholar 

  • Yao, D. C., & Wright, S. M. (2001). The challenge of problem residents. Journal of General Internal Medicine, 16, 486–492.

    Article  Google Scholar 

  • Zsambok, C. E., & Klein, G. (2014). Naturalistic Decision Making: Where are we now? In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (3rd ed., p. 19). New York: Psychology Press.

    Google Scholar 

Download references

Acknowledgments

This project was funded by an Education Research Grant, 2015, Australian Government Department of Health.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Parker Magin.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Magin, P., Stewart, R., Turnock, A. et al. Early predictors of need for remediation in the Australian general practice training program: a retrospective cohort study. Adv in Health Sci Educ 22, 915–929 (2017). https://doi.org/10.1007/s10459-016-9722-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-016-9722-5

Keywords

  • Family practice
  • General practice
  • Education, medical, graduate
  • Professional competence
  • Remedial teaching
  • Educational measurement