Advances in Health Sciences Education

, Volume 22, Issue 4, pp 915–929 | Cite as

Early predictors of need for remediation in the Australian general practice training program: a retrospective cohort study

  • Parker Magin
  • Rebecca Stewart
  • Allison Turnock
  • Amanda Tapley
  • Elizabeth Holliday
  • Nick Cooling
Article

Abstract

Underperforming trainees requiring remediation may threaten patient safety and are challenging for vocational training programs. Decisions to institute remediation are high-stakes—remediation being resource-intensive and emotionally demanding on trainees. Detection of underperformance requiring remediation is particularly problematic in general (family) practice. We sought to establish early-training assessment instruments predictive of general practice (GP) trainees’ subsequently requiring formal remediation. We conducted a retrospective cohort study of trainees from a large Australian regionally-based GP training organization. The outcome factor was requirement for formal remediation. Independent variables were demographic factors and a range of formative assessments conducted immediately prior to or during early-stage training. Analyses employed univariate and multivariate logistic regression of each predictor assessment modality with the outcome, adjusting for potential confounders. Of 248 trainees, 26 (10.5 %) required formal remediation. Performance on the Colleague Feedback Evaluation Tool (entailing feedback from a trainee’s clinical colleagues on clinical performance, communication and probity) and External Clinical Teaching Visits (half-day sessions of the trainee’s clinical consultations observed directly by an experienced GP), along with non-Australian primary medical qualification, were significantly associated with requiring remediation. There was a non-significant trend for association with performance on the Doctors Interpersonal Skills Questionnaire (patient feedback on interpersonal elements of the consultation). There were no significant associations with entry-selection scores or formative exam or assessment scores. Our finding that ‘in vivo’ assessments of complex behaviour, but not ‘in vitro’ knowledge-based assessments, predict need for remediation is consistent with theoretical understanding of the nature of remediation decision-making and should inform remediation practice in GP vocational training.

Keywords

Family practice General practice Education, medical, graduate Professional competence Remedial teaching Educational measurement 

Notes

Acknowledgments

This project was funded by an Education Research Grant, 2015, Australian Government Department of Health.

References

  1. Bahrami, J. (1997). Remediation. British Medical Journal, 315, S2–S7114.CrossRefGoogle Scholar
  2. Barrett, A., Galvin, R., et al. (2015). A BEME (Best Evidence in Medical Education) systematic review of the use of workplace-based assessment in identifying and remediating poor performance among postgraduate medical trainees. Systematic Reviews, 4, 65.CrossRefGoogle Scholar
  3. Beaman, M., & King, J. (2009). Supporting professional performance: Managing trainees in difficulty South West Peninsula Deanery NHS Report.Google Scholar
  4. Bingham, C., & Crampton, R. (2011). A review of prevocational medical trainee assessment in New South Wales. Medical Journal of Australia, 195, 410–412.CrossRefGoogle Scholar
  5. Black, D., & Welch, J. (2009). The under-performing trainee-concerns and challenges for medical educators. The Clinical Teacher, 6, 79–82.CrossRefGoogle Scholar
  6. Borkett-Jones, H., & Morris, C. (2010). Managing the trainee in difficulty. British Journal of Hospital Medicine, 71, 286–289.CrossRefGoogle Scholar
  7. Byrne, P. (2012). Are they safe in there? Patient safety and trainees in the practice. Australian Family Physician, 41, 26–29.Google Scholar
  8. Campbell, J., Narayanan, A., Burford, B., & Greco, M. (2010). Validation of a multi-source feedback tool for use in general practice. Education for Primary Care, 21, 165–179.CrossRefGoogle Scholar
  9. Cook, D. A., Beckman, T. J., Mandrekar, J. N., & Pankratz, V. S. (2010). Internal structure of mini-CEX scores for internal medicine residents: Factor analysis and generalizability. Advances in Health Sciences Education, 15, 633–645.CrossRefGoogle Scholar
  10. Dijksterhuis, F., Scheele, F., Schuwirth, L., et al. (2009). Progress testing in postgraduate medical education. Medical Teacher, 31, e464–e468.CrossRefGoogle Scholar
  11. Durning, S., et al. (2011). Viewing ‘strugglers” through a different lens: How a self regulated learning perspective can help medical educators with assessment and remediation. Academic Medicine, 86, 1–8.CrossRefGoogle Scholar
  12. Dwyer, A. J., Morely, P., Reid, E., & Angelatos, C. (2011). Distressed doctors: A hospital based support program for poorly performing and “at risk” junior medical staff. Medical Journal of Australia, 194, 466–469.Google Scholar
  13. Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51, 380–417.CrossRefGoogle Scholar
  14. Epstein, R. (2007). Assessment in medical education. New England Journal of Medicine, 356, 387–396.CrossRefGoogle Scholar
  15. Evans, D., Alstead, E., & Brown, J. (2010). Applying your clinical skills to students and trainees in academic difficulty. The Clinical Teacher, 7, 230–235.CrossRefGoogle Scholar
  16. Firth-Cozens, J., & King, J. (2006). Are psychological factors linked to performance? In J. Cox, J. King, A. Hutchinson, & P. McAvoy (Eds.), Understanding doctors’ performance (pp. 61–77). Oxford: Radcliffe Press.Google Scholar
  17. Garibaldi, R. A., Trontell, M. C., Waxaman, H., et al. (1994). The in-training examination in internal medicine. Annals of Internal Medicine, 121, 117–123.CrossRefGoogle Scholar
  18. Gladman, G. (2011). The Identification of general practice registrar needing assistance. The Australasian Medical Journal, 4, 308–314.CrossRefGoogle Scholar
  19. Hauer, K. E., Ciccone, A., Henzel, T. R., Katsufrakis, P., Miller, S. H., Norcross, W. A., et al. (2009). Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Academic Medicine, 84, 1822–1832.CrossRefGoogle Scholar
  20. Hays, R., & Peterson, L. (1996). An evaluation of visiting clinical teachers in general practice training. Education for General Practice, 7, 54–58.Google Scholar
  21. Hays, R., & Wellard, R. (1998). In-training assessment in postgraduate training for general practice. Medical Education, 32, 507–513.CrossRefGoogle Scholar
  22. Ingham, G., Morgan, S., et al. (2015). Are GP supervisors confident they can assess registrar competence and safety, and what methods do they use? Australian Family Physician, 44, 154–158.Google Scholar
  23. Kefalas, B., Usherwood, T., & Knight, A. (2005). External clinical teaching visits in the regionalised environment. Australian Family Physician, 34(Suppl 1), 10–12.Google Scholar
  24. Kimatian, S. J., & Lloyd, S. H. (2008). Remediation and due process for trainees: Setting the stage for future success. International Anesthesiology Clinics, 46, 113–125.CrossRefGoogle Scholar
  25. Larkins, S., Spillman, M., Vanlint, J., & Hays, R. (2003). Stress, personal and educational problems in vocational training. Australian Family Physician, 32, 473–476.Google Scholar
  26. Leap, L. L., & Fromson, J. A. (2006). Problem doctors: Is there a system-level solution? Annals of Internal Medicine, 144, 107–115.CrossRefGoogle Scholar
  27. Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65, S63–S67.CrossRefGoogle Scholar
  28. Miller, A., & Archer, J. (2010). Impact of workplace based assessment on doctors’ education and performance: A systematic review. British Medical Journal, 341, c5064.CrossRefGoogle Scholar
  29. Mitchell, C., Bhat, S., Herbert, A., & Baker, P. (2011). Workplace-based assessments of junior doctors: Do score predict training difficulties. Medical Education, 45, 1190–1198.CrossRefGoogle Scholar
  30. Mugford, B. (2003). Performance and remediation in general practice training. Australian Family Physician, 32, 544–546.Google Scholar
  31. Nair, B. R., Alexander, H., McGrath, B. P., et al. (2008). The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates. Medical Jounal of Australia, 189(3), 159–161.Google Scholar
  32. Paice, E., & Orton, V. (2005). Managing the trainee doctor in difficulty. In A. Hastie, I. Hastie, & N. Jackson (Eds.), Postgraduate medical education and training: A guide for primary and secondary care (pp. 165–175). Oxford: Radcliffe.Google Scholar
  33. Papadakis, M. A., Arnold, G. K., Blank, L. L., Holmboe, E. S., & Lipner, R. S. (2008). Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Annals of Internal Medicine, 148, 869–876.CrossRefGoogle Scholar
  34. Reamy, B., & Harman, J. (2006). Residents introuble: An in depth assessment of the 25 year experience of a single family medicine residency. Family Medicine, 38, 252–257.Google Scholar
  35. Riebschleger, M. P., & Haftel, H. M. (2013). Remediation in the context of the competencies: A survey of pediatrics residency program directors. Journal of Graduate Medical Education, 5, 60–63.CrossRefGoogle Scholar
  36. Roberts, N. K., Williams, R. G., Klingensmith, M., et al. (2012). The case of the entitled resident: A composite case study of a resident performance problem syndrome with interdisciplinary commentary. Medical Teacher, 34, 1024–1032.CrossRefGoogle Scholar
  37. Shah, S., Knowlden, F. R., Harris, M., & Yeo, G. (1997). General practice trainees in the borderline zone of competence: The experience of remediation at the regional general practice training and education centre. Education for General Practice, 8, 102–112.Google Scholar
  38. van der Vleuten, C. P. M., Schuwirth, L. W. T., Scheele, F., Driessen, E. W., & Hodges, B. (2010). The assessment of professional competence: Building blocks for theory development. Best Practice & Research Clinical Obstetrics & Gynaecology, 24, 703–719.CrossRefGoogle Scholar
  39. Van der Vleuten, C., Schuwirth, L., et al. (2012). A model for programmatic assessment fit for purpose. Medical Teacher, 34, 205–214.CrossRefGoogle Scholar
  40. Violato, C., Lockyer, J., & Fidler, H. (2008). Changes in performance: A 5-year longitudinal study of participants in a multi-source feedback programme. Medical Education, 42, 1007–1013.CrossRefGoogle Scholar
  41. Weizberg, M., Smith, J. L., Murano, T., Silverberg, M., & Santen, S. A. (2015). What does remediation and probation status mean? A survey of emergency medicine residency program directors. Academic Emergency Medicine, 22, 113–116.CrossRefGoogle Scholar
  42. Williams, R. G., Roberts, N. K., Schwind, C. J., & Dunningham, G. L. (2009). The nature of general surgery resident performance problems. Surgery, 145, 651–658.CrossRefGoogle Scholar
  43. Winston, K. A., Van der Vleuten, C. P., & Scherpbier, A. J. (2014). Prediction and prevention of failure: An early intervention to assist at risk medical students. Medical Teacher, 36, 25–31.CrossRefGoogle Scholar
  44. Yao, D. C., & Wright, S. M. (2000). National survey of internal medicine residency program directors regarding problem residents. JAMA, 284, 1099.CrossRefGoogle Scholar
  45. Yao, D. C., & Wright, S. M. (2001). The challenge of problem residents. Journal of General Internal Medicine, 16, 486–492.CrossRefGoogle Scholar
  46. Zsambok, C. E., & Klein, G. (2014). Naturalistic Decision Making: Where are we now? In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (3rd ed., p. 19). New York: Psychology Press.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.School of Medicine and Public HealthUniversity of NewcastleCallaghanAustralia
  2. 2.General Practice Training Valley to CoastMayfieldAustralia
  3. 3.Tropical Medical TrainingTownsvilleAustralia
  4. 4.School of MedicineUniversity of TasmaniaHobartAustralia
  5. 5.Clinical Research Design, IT and Statistical Support UnitHunter Medical Research InstituteNewcastleAustralia

Personalised recommendations