Abstract
Underperforming trainees requiring remediation may threaten patient safety and are challenging for vocational training programs. Decisions to institute remediation are high-stakes—remediation being resource-intensive and emotionally demanding on trainees. Detection of underperformance requiring remediation is particularly problematic in general (family) practice. We sought to establish early-training assessment instruments predictive of general practice (GP) trainees’ subsequently requiring formal remediation. We conducted a retrospective cohort study of trainees from a large Australian regionally-based GP training organization. The outcome factor was requirement for formal remediation. Independent variables were demographic factors and a range of formative assessments conducted immediately prior to or during early-stage training. Analyses employed univariate and multivariate logistic regression of each predictor assessment modality with the outcome, adjusting for potential confounders. Of 248 trainees, 26 (10.5 %) required formal remediation. Performance on the Colleague Feedback Evaluation Tool (entailing feedback from a trainee’s clinical colleagues on clinical performance, communication and probity) and External Clinical Teaching Visits (half-day sessions of the trainee’s clinical consultations observed directly by an experienced GP), along with non-Australian primary medical qualification, were significantly associated with requiring remediation. There was a non-significant trend for association with performance on the Doctors Interpersonal Skills Questionnaire (patient feedback on interpersonal elements of the consultation). There were no significant associations with entry-selection scores or formative exam or assessment scores. Our finding that ‘in vivo’ assessments of complex behaviour, but not ‘in vitro’ knowledge-based assessments, predict need for remediation is consistent with theoretical understanding of the nature of remediation decision-making and should inform remediation practice in GP vocational training.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bahrami, J. (1997). Remediation. British Medical Journal, 315, S2–S7114.
Barrett, A., Galvin, R., et al. (2015). A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees. Systematic Reviews, 4, 65.
Beaman, M., & King, J. (2009). Supporting professional performance: Managing trainees in difficulty South West Peninsula Deanery NHS Report.
Bingham, C., & Crampton, R. (2011). A review of prevocational medical trainee assessment in New South Wales. Medical Journal of Australia, 195, 410–412.
Black, D., & Welch, J. (2009). The under-performing trainee-concerns and challenges for medical educators. The Clinical Teacher, 6, 79–82.
Borkett-Jones, H., & Morris, C. (2010). Managing the trainee in difficulty. British Journal of Hospital Medicine, 71, 286–289.
Byrne, P. (2012). Are they safe in there? Patient safety and trainees in the practice. Australian Family Physician, 41, 26–29.
Campbell, J., Narayanan, A., Burford, B., & Greco, M. (2010). Validation of a multi-source feedback tool for use in general practice. Education for Primary Care, 21, 165–179.
Cook, D. A., Beckman, T. J., Mandrekar, J. N., & Pankratz, V. S. (2010). Internal structure of mini-CEX scores for internal medicine residents: Factor analysis and generalizability. Advances in Health Sciences Education, 15, 633–645.
Dijksterhuis, F., Scheele, F., Schuwirth, L., et al. (2009). Progress testing in postgraduate medical education. Medical Teacher, 31, e464–e468.
Durning, S., et al. (2011). Viewing ‘strugglers” through a different lens: How a self regulated learning perspective can help medical educators with assessment and remediation. Academic Medicine, 86, 1–8.
Dwyer, A. J., Morely, P., Reid, E., & Angelatos, C. (2011). Distressed doctors: A hospital based support program for poorly performing and “at risk” junior medical staff. Medical Journal of Australia, 194, 466–469.
Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51, 380–417.
Epstein, R. (2007). Assessment in medical education. New England Journal of Medicine, 356, 387–396.
Evans, D., Alstead, E., & Brown, J. (2010). Applying your clinical skills to students and trainees in academic difficulty. The Clinical Teacher, 7, 230–235.
Firth-Cozens, J., & King, J. (2006). Are psychological factors linked to performance? In J. Cox, J. King, A. Hutchinson, & P. McAvoy (Eds.), Understanding doctors’ performance (pp. 61–77). Oxford: Radcliffe Press.
Garibaldi, R. A., Trontell, M. C., Waxaman, H., et al. (1994). The in-training examination in internal medicine. Annals of Internal Medicine, 121, 117–123.
Gladman, G. (2011). The Identification of general practice registrar needing assistance. The Australasian Medical Journal, 4, 308–314.
Hauer, K. E., Ciccone, A., Henzel, T. R., Katsufrakis, P., Miller, S. H., Norcross, W. A., et al. (2009). Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Academic Medicine, 84, 1822–1832.
Hays, R., & Peterson, L. (1996). An evaluation of visiting clinical teachers in general practice training. Education for General Practice, 7, 54–58.
Hays, R., & Wellard, R. (1998). In-training assessment in postgraduate training for general practice. Medical Education, 32, 507–513.
Ingham, G., Morgan, S., et al. (2015). Are GP supervisors confident they can assess registrar competence and safety, and what methods do they use? Australian Family Physician, 44, 154–158.
Kefalas, B., Usherwood, T., & Knight, A. (2005). External clinical teaching visits in the regionalised environment. Australian Family Physician, 34(Suppl 1), 10–12.
Kimatian, S. J., & Lloyd, S. H. (2008). Remediation and due process for trainees: Setting the stage for future success. International Anesthesiology Clinics, 46, 113–125.
Larkins, S., Spillman, M., Vanlint, J., & Hays, R. (2003). Stress, personal and educational problems in vocational training. Australian Family Physician, 32, 473–476.
Leap, L. L., & Fromson, J. A. (2006). Problem doctors: Is there a system-level solution? Annals of Internal Medicine, 144, 107–115.
Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65, S63–S67.
Miller, A., & Archer, J. (2010). Impact of workplace based assessment on doctors’ education and performance: A systematic review. British Medical Journal, 341, c5064.
Mitchell, C., Bhat, S., Herbert, A., & Baker, P. (2011). Workplace-based assessments of junior doctors: Do score predict training difficulties. Medical Education, 45, 1190–1198.
Mugford, B. (2003). Performance and remediation in general practice training. Australian Family Physician, 32, 544–546.
Nair, B. R., Alexander, H., McGrath, B. P., et al. (2008). The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates. Medical Jounal of Australia, 189(3), 159–161.
Paice, E., & Orton, V. (2005). Managing the trainee doctor in difficulty. In A. Hastie, I. Hastie, & N. Jackson (Eds.), Postgraduate medical education and training: A guide for primary and secondary care (pp. 165–175). Oxford: Radcliffe.
Papadakis, M. A., Arnold, G. K., Blank, L. L., Holmboe, E. S., & Lipner, R. S. (2008). Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Annals of Internal Medicine, 148, 869–876.
Reamy, B., & Harman, J. (2006). Residents introuble: An in depth assessment of the 25 year experience of a single family medicine residency. Family Medicine, 38, 252–257.
Riebschleger, M. P., & Haftel, H. M. (2013). Remediation in the context of the competencies: A survey of pediatrics residency program directors. Journal of Graduate Medical Education, 5, 60–63.
Roberts, N. K., Williams, R. G., Klingensmith, M., et al. (2012). The case of the entitled resident: A composite case study of a resident performance problem syndrome with interdisciplinary commentary. Medical Teacher, 34, 1024–1032.
Shah, S., Knowlden, F. R., Harris, M., & Yeo, G. (1997). General practice trainees in the borderline zone of competence: The experience of remediation at the regional general practice training and education centre. Education for General Practice, 8, 102–112.
van der Vleuten, C. P. M., Schuwirth, L. W. T., Scheele, F., Driessen, E. W., & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice & Research Clinical Obstetrics & Gynaecology, 24, 703–719.
Van der Vleuten, C., Schuwirth, L., et al. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34, 205–214.
Violato, C., Lockyer, J., & Fidler, H. (2008). Changes in performance: A 5-year longitudinal study of participants in a multi-source feedback programme. Medical Education, 42, 1007–1013.
Weizberg, M., Smith, J. L., Murano, T., Silverberg, M., & Santen, S. A. (2015). What does remediation and probation status mean? A survey of emergency medicine residency program directors. Academic Emergency Medicine, 22, 113–116.
Williams, R. G., Roberts, N. K., Schwind, C. J., & Dunningham, G. L. (2009). The nature of general surgery resident performance problems. Surgery, 145, 651–658.
Winston, K. A., Van der Vleuten, C. P., & Scherpbier, A. J. (2014). Prediction and prevention of failure: An early intervention to assist at risk medical students. Medical Teacher, 36, 25–31.
Yao, D. C., & Wright, S. M. (2000). National survey of internal medicine residency program directors regarding problem residents. JAMA, 284, 1099.
Yao, D. C., & Wright, S. M. (2001). The challenge of problem residents. Journal of General Internal Medicine, 16, 486–492.
Zsambok, C. E., & Klein, G. (2014). Naturalistic Decision Making: Where are we now? In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (3rd ed., p. 19). New York: Psychology Press.
Acknowledgments
This project was funded by an Education Research Grant, 2015, Australian Government Department of Health.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Magin, P., Stewart, R., Turnock, A. et al. Early predictors of need for remediation in the Australian general practice training program: a retrospective cohort study. Adv in Health Sci Educ 22, 915–929 (2017). https://doi.org/10.1007/s10459-016-9722-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-016-9722-5