Advances in Health Sciences Education

, Volume 17, Issue 2, pp 183–194

Toward meaningful evaluation of medical trainees: the influence of participants’ perceptions of the process

Reflections

Abstract

An essential goal of evaluation is to foster learning. Across the medical education spectrum, evaluation of clinical performance is dominated by subjective feedback to learners based on observation by expert supervisors. Research in non-medical settings has suggested that participants’ perceptions of evaluation processes exert considerable influence over whether the feedback they receive actually facilitates learning, but similar research on perceptions of feedback in the medical setting has been limited. In this review, we examine the literature on recipient perceptions of feedback and how those perceptions influence the contribution that feedback makes to their learning. A focused exploration of relevant work on this subject in higher education and industrial psychology settings is followed by a detailed examination of available research on perceptions of evaluation processes in medical settings, encompassing both trainee and evaluator perspectives. We conclude that recipients’ and evaluators’ perceptions of an evaluation process profoundly affect the usefulness of the evaluation and the extent to which it achieves its goals. Attempts to improve evaluation processes cannot, therefore, be limited to assessment tool modification driven by reliability and validity concerns, but must also take account of the critical issue of feedback reception and the factors that influence it. Given the unique context of clinical performance evaluation in medicine, a research agenda is required that seeks to more fully understand the complexity of the processes of giving, receiving, interpreting, and using feedback as a basis for real progress toward meaningful evaluation.

Keywords

Evaluation Feedback Medical students Perceptions Residents 

References

  1. Accreditation Council for Graduate Medical Education. (2007). Common program requirements (Revised July 1, 2007). Accessed December 16, 2009. http://www.acgme.org.
  2. Awad, S. S., Liscum, K. R., Aoki, N., Awad, S. H., & Berger, D. H. (2002). Does the subjective evaluation of medical student surgical knowledge correlate with written and oral exam performance? Journal of Surgical Research, 104, 36–39.CrossRefGoogle Scholar
  3. Barclay, L. J., Skarlicki, D. P., & Pugh, S. D. (2005). Exploring the role of emotions in injustice perceptions and retaliation. Journal of Applied Psychology, 90(4), 629–643.CrossRefGoogle Scholar
  4. Barrows, H. S. (1986). The scope of clinical education. In J. A. D. Cooper (Ed.), Clinical education of medical students. Journal of Medical Education, 61(9 Part 2), 23–33.Google Scholar
  5. Bing-You, R. G., Bertsch, T., & Thompson, J. A. (1998). Coaching medical students in receiving effective feedback. Teaching and Learning in Medicine, 10(4), 228–231.CrossRefGoogle Scholar
  6. Bing-You, R. G., & Paterson, J. (1997). Feedback falling on deaf ears: Residents’ receptivity to feedback tempered by sender credibility. Medical Teacher, 19(1), 40–44.CrossRefGoogle Scholar
  7. Bing-You, R. G., & Trowbridge, R. L. (2009). Why medical educators may be failing at feedback. JAMA, 302(12), 1330–1331.CrossRefGoogle Scholar
  8. Boehler, M. L., Rogers, D. A., Schwind, C. J., Mayforth, R., Quin, J., Williams, R. G., et al. (2006). An investigation of medical student reactions to feedback: A randomised controlled trial. Medical Education, 40, 746–749.CrossRefGoogle Scholar
  9. Brett, J. F., & Atwater, L. E. (2001). 360° feedback: Accuracy, reactions, and perceptions of usefulness. Journal of Applied Psychology, 86(5), 930–942.CrossRefGoogle Scholar
  10. Chou, S., Lockyer, J., Cole, G., & McLaughlin, K. (2008). CanMEDS evaluation in Canadian postgraduate training programmes: Tools used and programme director satisfaction. Medical Education, 42, 879–886.CrossRefGoogle Scholar
  11. Claridge, J. A., Calland, J. F., Chandrasekhara, V., Young, J. S., Sanfey, H., & Schirmer, B. D. (2003). Comparing resident measurements to attending surgeon self-perceptions of surgical educators. American Journal of Surgery, 185, 323–327.CrossRefGoogle Scholar
  12. Colletti, L. M. (2000). Difficulty with negative feedback: Face-to-face evaluation of junior medical student clinical performance results in grade inflation. Journal of Surgical Research, 90(1), 82–87.CrossRefGoogle Scholar
  13. Daelmans, H. E. M., Overmeer, R. M., van der Hem-Stokroos, H. H., Scherpbier, A. J. J. A., Stehouwer, C. D. A., & van der Vleuten, C. P. M. (2006). In-training assessment: Qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Medical Education, 40, 51–58.CrossRefGoogle Scholar
  14. Davis, M. H., & Harden, R. M. (2003). Competency-based assessment: Making it a reality. Medical Teacher, 25(6), 565–568.CrossRefGoogle Scholar
  15. Davis, D. A., Mazmanian, P. E., Fordis, M., et al. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA, 296, 1094–1102.CrossRefGoogle Scholar
  16. De Cremer, D., van Knippenberg, B., van Knippenberg, D., Mullenders, D., & Stinglhamber, F. (2005). Rewarding leadership and fair procedures as determinants of self-esteem. Journal of Applied Psychology, 90(1), 3–12.CrossRefGoogle Scholar
  17. Dipboye, R. L., & de Pontbriand, R. (1981). Correlates of employee reactions to performance appraisals and appraisal systems. Journal of Applied Psychology, 66(2), 248–251.CrossRefGoogle Scholar
  18. Dudek, N. L., Marks, M. B., & Regehr, G. (2005). Failure to fail: The perspectives of clinical supervisors. Academic Medicine, 80(10 Suppl), S84–S87.CrossRefGoogle Scholar
  19. Duffield, K. E., & Spencer, J. A. (2002). A survey of medical students’ views about the purposes and fairness of assessment. Medical Education, 36(9), 879–886.CrossRefGoogle Scholar
  20. Ende, J. (1983). Feedback in clinical medical education. JAMA, 250(6), 777–781.CrossRefGoogle Scholar
  21. Epstein, R. M. (2007). Assessment in medical education. New England Journal of Medicine, 356(4), 387–396.CrossRefGoogle Scholar
  22. Eva, K. W., & Regehr, G. (2005). Self-assessment in the health professions: A reformulation and research agenda. Academic Medicine, 80(10 Suppl), S46–S54.CrossRefGoogle Scholar
  23. Evans, R., Elwyn, G., & Edwards, A. (2004). Review of instruments for peer assessment of physicians. BMJ, 328(7450), 1240.CrossRefGoogle Scholar
  24. Fromme, H. B., Karani, R., & Downing, S. M. (2009). Direct observation in medical education: Review of the literature and evidence for validity. Mount Sinai Journal of Medicine, 76(4), 365–371.CrossRefGoogle Scholar
  25. Gray, J. D. (1996). Global rating scales in residency education. Academic Medicine, 71(1 Suppl), S55–S63.CrossRefGoogle Scholar
  26. Haber, R. J., & Avins, A. L. (1994). Do ratings on the American board of internal medicine resident evaluation form detect differences in clinical competence? Journal of General Internal Medicine, 9, 140–145.CrossRefGoogle Scholar
  27. Hasley, P. B., & Arnold, R. M. (2009). Summative evaluation on the hospital wards. What do faculty say to learners? Advances in Health Sciences Education: Theory and Practice, 14(3), 431–439. Epub 2008 Jun 5.CrossRefGoogle Scholar
  28. Higgins, R. S. D., Bridges, J., Burke, J. M., O’Donnell, M. A., Cohen, N. M., & Wilkes, S. B. (2004). Implementing the ACGME general competencies in a cardiothoracic surgery residency program using 360-degree feedback. Annals of Thoracic Surgery, 77, 12–17.CrossRefGoogle Scholar
  29. Higgins, R., Hartley, P., & Skelton, A. (2001). Getting the message across: The problem of communicating assessment feedback. Teaching in Higher Education, 6(2), 269–274.CrossRefGoogle Scholar
  30. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behaviour in organizations. Journal of Applied Psychology, 64, 349–371.CrossRefGoogle Scholar
  31. Isaacson, J. H., Posk, L. K., Litaker, D. G., & Halperin, A. K. (1998). Resident perception of the evaluation process (abstract). Journal of General Internal Medicine, 10(suppl), 89.Google Scholar
  32. Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254–284.CrossRefGoogle Scholar
  33. Kolars, J. C., McDonald, F. S., Subhiyah, R. G., & Edson, R. S. (2003). Knowledge base evaluation of medicine residents on the gastroenterology service: Implications for competency assessments by faculty. Clinical Gastroenterology and Hepatology, 1, 64–68.CrossRefGoogle Scholar
  34. Krupat, E., & Dienstag, J. L. (2009). Commentary: Assessment is an educational tool. Academic Medicine, 84(5), 548–550.CrossRefGoogle Scholar
  35. Kwolek, C. J., Donnelly, M. B., Sloan, D. A., Birrell, S. N., Strodel, W. E., & Schwartz, R. W. (1997). Ward evaluations: Should they be abandoned? Journal of Surgical Research, 69(1), 1–6.CrossRefGoogle Scholar
  36. Lawler, E. E. (1967). The multitrait-multirater approach to measuring managerial job performance. Journal of Applied Psychology, 51, 369–381.CrossRefGoogle Scholar
  37. Liason Committee on Medical Education. (2008). Current LCME accreditation standards: Structure and function of a medical school (Revised June, 2008). Accessed December 16, 2009. http://www.lcme.org.
  38. Meyer, H. H. (1975). The pay-for-performance dilemma. Organizational Dynamics, 3, 39–50.CrossRefGoogle Scholar
  39. Meyer, H. H., Kay, E., & French, J. R. P. (1965). Split roles in performance appraisal. Harvard Business Review, 42(1), 123–129.Google Scholar
  40. Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63–S67.CrossRefGoogle Scholar
  41. Mowday, R. T. (1983). Beliefs about the causes of behaviour: The motivational implications of attribution processes. In R. M. Steers & L. W. Porter (Eds.), Motivation and work behavior (3rd ed.). New York: McGraw-Hill.Google Scholar
  42. Pasquina, P. F., Kelly, S., & Hawkins, R. E. (2003). Assessing clinical competence in physical medicine and rehabilitation residency programs. American Journal of Physical Medicine and Rehabilitation, 82, 473–478.Google Scholar
  43. Pearce, J. L., & Porter, L. W. (1986). Employee responses to formal performance appraisal feedback. Journal of Applied Psychology, 71(2), 211–218.CrossRefGoogle Scholar
  44. Sargeant, J., Mann, K., & Ferrier, S. (2005). Exploring family physicians’ reactions to multisource feedback: Perceptions of credibility and usefulness. Medical Education, 39, 497–504.CrossRefGoogle Scholar
  45. Sargeant, J., Mann, K., Sinclair, D., van der Vleuten, C., & Metsemakers, J. (2008). Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Advances in Health Sciences Education Theory Practice, 13, 275–288.CrossRefGoogle Scholar
  46. Sender Liberman, A., Liberman, M., Steinert, Y., McLeod, P., & Meterissian, S. (2005). Surgery residents and attending surgeons have different perceptions of feedback. Med Teach, 27(5), 470–472.CrossRefGoogle Scholar
  47. Short, J. P. (1993). The importance of strong evaluation standards and procedures in training residents. Academic Medicine, 68(7), 522–525.CrossRefGoogle Scholar
  48. Sumit, K. De., Henke, P. K., Ailawadi, G., Dimick, J. B., & Colletti, L. M. (2004). Attending, house officer, and medical student perceptions about teaching in the third-year medical school general surgery clerkship. Journal of the American College of Surgeons, 199, 932–942.CrossRefGoogle Scholar
  49. The Royal College of Physicians and Surgeons of Canada. (2006). General standards of accreditation (Revised June 2006). Accessed December 16, 2009. http://www.rcpsc.edu.
  50. Thompson, P. H., & Dalton, G. W. (1970). Performance appraisal: Managers beware. Harvard Business Review, 48, 149–157.Google Scholar
  51. Turnbull, J., Gray, J., & MacFadyen, J. (1998). Improving in-training evaluation programs. Journal of General Internal Medicine, 13, 317–323.CrossRefGoogle Scholar
  52. Watling, C. J., Kenyon, C. F., Schulz, V., et al. (2010). First do no harm: Challenges to faculty engagement in the resident in-training evaluation process. Academic Medicine, 85, 7.Google Scholar
  53. Watling, C. J., Kenyon, C. F., Zibrowski, E. M., et al. (2008). Rules of engagement: Residents’ perceptions of the in-training evaluation process. Academic Medicine, 83(10 Suppl), S97–S100.CrossRefGoogle Scholar
  54. Yorke, M. (2003). Formative assessment in higher education: Moves toward theory and the enhancement of pedagogic practice. Higher Education, 45, 477–501.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.Department of Clinical Neurological Sciences, Schulich School of Medicine and DentistryUniversity of Western OntarioLondonCanada
  2. 2.Centre for Education Research and Innovation, Schulich School of Medicine and DentistryUniversity of Western OntarioLondonCanada
  3. 3.London Health Sciences Centre, Victoria HospitalLondonCanada

Personalised recommendations