Skip to main content
SpringerLink
Log in

The Impact of Resident- and Self-Evaluations on Surgeon’s Subsequent Teaching Performance

  • Published:
World Journal of Surgery Aims and scope Submit manuscript

Abstract

Background

This study evaluates how residents’ evaluations and self-evaluations of surgeon’s teaching performance evolve after two cycles of evaluation, reporting, and feedback. Furthermore, the influence of over- and underestimating own performance on subsequent teaching performance was investigated.

Methods

In a multicenter cohort study, 351 surgeons evaluated themselves and were also evaluated by residents during annual evaluation periods for three subsequent years. At the end of each evaluation period, surgeons received a personal report summarizing the residents’ feedback. Changes in each surgeon’s teaching performance evaluated on a five-point scale were studied using growth models. The effect of surgeons over- or underestimating their own performance on the improvement of teaching performance was studied using adjusted multivariable regressions.

Results

Compared with the first (median score: 3.83, 20th to 80th percentile score: 3.46–4.16) and second (median: 3.82, 20th to 80th: 3.46–4.14) evaluation period, residents evaluated surgeon’s teaching performance higher during the third evaluation period (median: 3.91, 20th to 80th: 3.59–4.27), p < 0.001. Surgeons did not alter self-evaluation scores over the three periods. Surgeons who overestimated their teaching performance received lower subsequent performance scores by residents (regression coefficient b: −0.08, 95 % confidence limits (CL): −0.18, 0.02) and self (b: −0.12, 95 % CL: −0.21, −0.02). Surgeons who underestimated their performance subsequently scored themselves higher (b: 0.10, 95 % CL: 0.03, 0.16), but were evaluated equally by residents.

Conclusions

Residents’ evaluation of surgeon’s teaching performance was enhanced after two cycles of evaluation, reporting, and feedback. Overestimating own teaching performance could impede subsequent performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Eva KW, Regehr G (2005) Self-assessment in the health professions: a reformulation and research agenda. Acad Med 80(10 Suppl):S46–S54

    Article  PubMed  Google Scholar 

  2. Eva KW, Regehr G (2008) “I’ll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof 28(1):14–19

    Article  PubMed  Google Scholar 

  3. Sargeant J, Armson H, Chesluk B et al (2010) The processes and dimensions of informed self-assessment: a conceptual model. Acad Med 85(7):1212–1220

    Article  PubMed  Google Scholar 

  4. Stalmeijer RE, Dolmans DH, Wolfhagen IH et al (2010) Combined student ratings and self-assessment provide useful feedback for clinical teachers. Adv Health Sci Educ Theory Pract 15(3):315–328

    Article  PubMed  PubMed Central  Google Scholar 

  5. Boerebach BC, Arah OA, Busch OR et al (2012) Reliable and valid tools for measuring surgeons’ teaching performance: residents’ vs. self evaluation. J Surg Educ 69(4):511–520

    Article  PubMed  Google Scholar 

  6. van der Leeuw R, Lombarts K, Heineman MJ et al (2011) Systematic evaluation of the teaching qualities of obstetrics and gynecology faculty: reliability and validity of the SETQ tools. PLoS One 6(5):e19142

    Article  PubMed  PubMed Central  Google Scholar 

  7. Beckman TJ, Ghosh AK, Cook DA et al (2004) How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 19(9):971–977

    Article  PubMed  PubMed Central  Google Scholar 

  8. Eva KW, Armson H, Holmboe E et al (2012) Factors influencing responsiveness to feedback: on the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract 17(1):15–26

    Article  PubMed  PubMed Central  Google Scholar 

  9. Mann K, van der Vleuten C, Eva K et al (2011) Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict. Acad Med 86(9):1120–1127

    Article  PubMed  Google Scholar 

  10. van der Leeuw R, Slootweg IA, Heineman MJ et al (2013) Explaining how faculty act upon residents’ feedback to improve their teaching performance. Med Educ 47:1089–1098

    Article  PubMed  Google Scholar 

  11. van Roermund T, Schreurs ML, Mokkink H et al (2013) Qualitative study about the ways teachers react to feedback from resident evaluations. BMC Med Educ 13(1):98

    Article  PubMed  PubMed Central  Google Scholar 

  12. Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84(2):191–215

    Article  PubMed  CAS  Google Scholar 

  13. Kluger AN, DeNisi A (1996) The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 119(2):254–284

    Article  Google Scholar 

  14. Korman AK (1970) Toward an hypothesis of work behavior. J Appl Psychol 54(1):31–41

    Article  Google Scholar 

  15. Atwater L, Roush P, Fischthal A (1995) The influence of upward feedback on self- and follower ratings of leadership. Pers Psychol 48(1):35–59

    Article  Google Scholar 

  16. Atwater LE, Ostroff C, Yammarino FJ, Fleenor JW (1998) Self-other agreement: does it really matter? Pers Psychol 51(3):577–598

    Article  Google Scholar 

  17. Brett JF, Atwater L (2001) 360° Feedback: accuracy, reactions, and perceptions of usefulness. J Appl Psychol 86(5):930–942

    Article  PubMed  CAS  Google Scholar 

  18. Johnson JW, Ferstl KL (1999) The effects of interrater and self-other agreement on performance improvement following upward feedback. Pers Psychol 52(2):271–303

    Article  Google Scholar 

  19. Ostroff C, Atwater LE, Feinberg BJ (2004) Understanding self-other agreement: a look at rater and ratee characteristics, context, and outcomes. Pers Psychol 57(2):333–375

    Article  Google Scholar 

  20. Smither JW, London M, Vasilopoulos NL, Reilly RR, Millsap RE, Salvemini N (1995) An examination of the effects of an upward feedback program over time. Pers Psychol 48(1):1–34

    Article  Google Scholar 

  21. Arah OA, Hoekstra JBL, Bos AP et al (2011) New tools for systematic evaluation of teaching qualities of medical faculty: results of an ongoing multi-center survey. PLoS One 6(10):e25983

    Article  PubMed  CAS  PubMed Central  Google Scholar 

  22. Lombarts KM, Bucx MJ, Arah OA (2009) Development of a system for the evaluation of the teaching qualities of anesthesiology faculty. Anesthesiology 111(4):709–716

    Article  PubMed  Google Scholar 

  23. van der Leeuw RM, Overeem K, Arah OA et al (2013) Frequency and determinants of residents’ narrative feedback on the teaching performance of faculty: narratives in numbers. Acad Med 88:1324–1331

    Article  PubMed  Google Scholar 

  24. Van Buuren S, Groothuis-Oudshoorn K (2011) MICE: multivariate imputation by chained equations in R. J Stat Softw 45(3):1–67

    Google Scholar 

  25. Singer JD (1998) Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J Educ Behav Stat 23(4):323–355

    Article  Google Scholar 

  26. Twisk JWR (2006) Applied multilevel analysis: a practical guide. Cambridge University Press, Cambridge

    Book  Google Scholar 

  27. Robins JM, Hernán MA (2009) Estimation of the causal effects of time-varying exposures. Longitudinal data analysis. CRC Press, Boca Raton, pp 553–599

    Google Scholar 

  28. Robins JM, Hernan MA, Brumback B (2000) Marginal structural models and causal inference in epidemiology. Epidemiology 11(5):550–560

    Article  PubMed  CAS  Google Scholar 

  29. Ivers N, Jamtvedt G, Flottorp S et al (2012) Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 6:CD000259

    PubMed  Google Scholar 

  30. Overeem K, Wollersheim H, Driessen E et al (2009) Doctors’ perceptions of why 360° feedback does (not) work: a qualitative study. Med Educ 43(9):874–882

    Article  PubMed  Google Scholar 

  31. Sargeant J, Mann K, Sinclair D et al (2008) Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Adv Health Sci Educ Theory Pract 13(3):275–288

    Article  PubMed  Google Scholar 

  32. Sargeant JM, Mann KV, van der Vleuten CP et al (2009) Reflection: a link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract 14(3):399–410

    Article  PubMed  Google Scholar 

  33. Brutus S, Fleenor JW, McCauley CD (1999) Demographic and personality predictors of congruence in multisource ratings. J Manag Dev 18:417–435

    Article  Google Scholar 

  34. Davis DA, Mazmanian PE, Fordis M et al (2006) Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 296(9):1094–1102

    Article  PubMed  CAS  Google Scholar 

  35. Sargeant J, McNaughton E, Mercer S et al (2011) Providing feedback: exploring a model (emotion, content, outcomes) for facilitating multisource feedback. Med Teach 33(9):744–749

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all surgeons and residents for their participation in this study. We also want to thank Anne Huitema for her assistance with the data preparation and analysis and http://medox.nl for the maintenance and optimization of the web-based SETQ application.

Funding

This study is part of the research project Quality of Clinical Teachers and Residency Training, which is co-financed by the Dutch Ministry of Health; the Academic Medical Center, Amsterdam; and the Faculty of Health and Life Sciences of the University of Maastricht. Funders had no role in the study design, data collection, data analysis, data interpretation, or writing of the report.

Disclosures

None.

Ethical Approval

Waiver of ethical approval was provided by the institutional review board of the Academic Medical Center of the University of Amsterdam, Amsterdam, The Netherlands.

Author information

Authors and Affiliations

  1. Center for Evidence-Based Education, Academic Medical Center, University of Amsterdam, Meibergdreef 9, PO Box 22700, 1100 DE, Amsterdam, The Netherlands

    Benjamin C. M. Boerebach, Onyebuchi A. Arah, Maas Jan Heineman & Kiki M. J. M. H. Lombarts

  2. Department of Epidemiology, School of Public Health, University of California, Los Angeles (UCLA), Los Angeles, CA, USA

    Onyebuchi A. Arah

  3. UCLA Center for Health Policy Research, Los Angeles, CA, USA

    Onyebuchi A. Arah

  4. Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    Maas Jan Heineman

  5. Department of Surgery, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands

    Olivier R. C. Busch

Authors
  1. Benjamin C. M. Boerebach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Onyebuchi A. Arah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maas Jan Heineman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olivier R. C. Busch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kiki M. J. M. H. Lombarts
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin C. M. Boerebach.

Appendix

Appendix

See Tables 4, 5, 6 and Fig. 2.

Table 4 Scales and items of the SETQ
Full size table
Table 5 Regression coefficients and 95 % CLs for the associations between the key predictors and subsequent teaching performance, weighted by inverse-probability-of-censoring weights
Full size table
Table 6 Regression coefficients and 95 % CLs for the associations between the key predictors and subsequent teaching performance, stratified by gender
Full size table
Fig. 2
figure 2

Sex-specific effects of surgeon under- or overestimating own teaching performance on subsequent resident and own evaluations of teaching performance

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boerebach, B.C.M., Arah, O.A., Heineman, M.J. et al. The Impact of Resident- and Self-Evaluations on Surgeon’s Subsequent Teaching Performance. World J Surg 38, 2761–2769 (2014). https://doi.org/10.1007/s00268-014-2655-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00268-014-2655-3

Keywords

Search

Navigation