The Impact of Feedback of Surgical Outcome Data on Surgical Performance: A Systematic Review

Abstract

Background

Increasing patient demands, costs and emphasis on safety have led to performance tracking of individual surgeons. Several methods of using these data, including feedback have been proposed. Our aim was to systematically review the impact of feedback of outcome data to surgeons on their performance.

Study design

MEDLINE, Embase, PsycINFO, AMED and the Cochrane Database of Systematic Reviews (from their inception to February 2013) were searched. Two reviewers independently reviewed citations using predetermined inclusion and exclusion criteria. Forty two data-points per study were extracted.

Results

The search strategy yielded 1,531 citations. Seven studies were eligible comprising 18,632 cases or procedures by 52 surgeons. Overall, feedback was found to be a powerful method for improving surgical outcomes or indicators of surgical performance, including reductions in hospital mortality after CABG of 24 % (P = 0.001), decreases of stroke and mortality following carotid endarterectomy from 5.2 to 2.3 %, improved ovarian cancer resection from 77 to 85 % (P = 0.157) and reductions in wound infection rates from 14 to 10.3 %. Improvements in performance occurred in concert with reduced costs: for hepaticojejunostomy, implementation of feedback was associated with a decrease in overall hospital costs from $24,446 to $20,240 (P < 0.01). Similarly, total cost of carotid endarterectomy and following management decreased from $13,344 to $9548.

Conclusions

The available literature suggests that feedback can improve surgical performance and outcomes; however, given the heterogeneity and limited number of studies, in addition to their non-randomised nature, it is difficult to draw clear conclusions from the literature with regard to the efficacy of feedback and the specific nuances required to optimise the impact of feedback. There is a clear need for more rigorous studies to determine how feedback of outcome data may impact performance, and whether this low-cost intervention has potential to benefit surgical practice.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. 1.

    Fraher EP, Knapton A, Sheldon GF, Meyer A, Ricketts TC (2013) Projecting surgeon supply using a dynamic model. Ann Surg 257(5):867–872

    Article  PubMed  Google Scholar 

  2. 2.

    Lyu H, Wick EC, Housman M, Freischlag J, Makary MA (2013) Patient satisfaction as a possible indicator of quality surgical care. JAMA Surg 148(4):362–367

    Article  PubMed  Google Scholar 

  3. 3.

    Cooper M, Makary MA (2012) A comprehensive unit-based safety program (CUSP) in surgery: improving quality through transparency. Surg Clin North Am 92(1):51–63

    Article  PubMed  Google Scholar 

  4. 4.

    Cleveland Clinic. Treatment Outcomes 2011 (http://my.clevelandclinic.org/heart/about-heart-vascular-institute/treatment-outcomes.aspx)

  5. 5.

    Shahian DM, Jacobs JP, Edwards FH, Brennan JM, Dokholyan RS, Prager RL et al (2013) The society of thoracic surgeons national database. Heart 99:1494–1501. http://www.ncbi.nlm.nih.gov/pubmed/23335498

  6. 6.

    Godlee F (2012) Measure your team’s performance, and publish the results. BMJ 345:e4590

    Article  Google Scholar 

  7. 7.

    Royal College of Surgeons of England. RCS hails landmark publication of individual surgeons’ outcomes data. 2013 (https://www.rcseng.ac.uk/news/rcs-hails-landmark-publication-of-individual-surgeons2019-outcomes-data#.U-UoVRaOdss)

  8. 8.

    Khuri SF, Daley J, Henderson W, Hur K, Demakis J, Aust JB, Chong V, Fabri PJ, Gibbs JO, Grover F, Hammermeister K, Irvin G 3rd, McDonald G, Passaro E Jr, Phillips L, Scamman F, Spencer J, Stremple Jf (1998) The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg 228(4):491–507

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  9. 9.

    van de Ridder JM, Stokking KM, McGaghie WC, ten Cate OT (2008) What is feedback in clinical education? Med Educ 42(2):189–197

    Article  PubMed  Google Scholar 

  10. 10.

    Kluger A, DeNisi A (1996) The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 119(2):254–284

    Article  Google Scholar 

  11. 11.

    Bienstock JL, Katz NT, Cox SM, Hueppchen N, Erickson S, Puscheck EE et al (2007) To the point: medical education reviews—providing feedback. Am J Obstet Gynecol 196(6):508–513

    Article  PubMed  Google Scholar 

  12. 12.

    Hewson MG, Little ML (1998) Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med 13(2):111–116

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  13. 13.

    Cantillon Peter, Sargeant Joan (2008) Giving feedback in clinical settings. BMJ 337:a1961

    Article  PubMed  Google Scholar 

  14. 14.

    O’Connor GT, Plume SK, Olmstead EM, Morton JR, Maloney CT, Nugent WC et al (1996) A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. The Northern New England Cardiovascular Disease Study Group. JAMA 275(11):841–846

    Article  PubMed  Google Scholar 

  15. 15.

    Findlay JM, Nykolyn L, Lubkey TB, Wong JH, Mouradian M, Senthilselvan A (2002) Auditing carotid endarterectomy: a regional experience. Can J Neurol Sci 29(4):326–332

    Article  PubMed  Google Scholar 

  16. 16.

    Olcott CIV, Mitchell RS, Steinberg GK, Zarins CK (2000) Institutional peer review can reduce the risk and cost of carotid endarterectomy. Arch Surg 135(8):939–942

    Article  CAS  PubMed  Google Scholar 

  17. 17.

    Reilly J, McIntosh J, Currie K (2002) Changing surgical practice through feedback of performance data. J Adv Nurs 38(6):607–614

    Article  PubMed  Google Scholar 

  18. 18.

    Rodriguez JF, Trobo AR, Garcia MV, Martinez MJ, Millan CP, Vazquez MC et al (2006) The effect of performance feedback on wound infection rate in abdominal hysterectomy. Am J Infect Control 34(4):182–187

    Article  PubMed  Google Scholar 

  19. 19.

    Aletti GD, Dowdy SC, Gostout BS, Jones MB, Stanhope RC, Wilson TO et al (2009) Quality improvement in the surgical approach to advanced ovarian cancer: the Mayo Clinic experience. J Am Coll Surg 208(4):614–620

    Article  PubMed  Google Scholar 

  20. 20.

    Pitt HA, Murray KP, Bowman HM, Coleman J, Gordon TA, Yeo CJ et al (1999) Clinical pathway implementation improves outcomes for complex biliary surgery. Surgery 126(4):751–756 discussion 756–8

    Article  CAS  PubMed  Google Scholar 

  21. 21.

    Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB (2006) Systematic review of the literature on assessment, feedback and physicians’ clinical performance: BEME Guide No. 7. Med Teach 28(2):117–128

    Article  PubMed  Google Scholar 

  22. 22.

    Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD (2006) Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 15(6):433–436

    Article  PubMed Central  PubMed  Google Scholar 

  23. 23.

    Rogers DA, Regehr G, Howdieshell TR, Yeh KA, Palm E (2000) The impact of external feedback on computer-assisted learning for surgical technical skill training. Am J Surg 179(4):341–343

    Article  CAS  PubMed  Google Scholar 

  24. 24.

    Miller DC, Murtagh DS, Suh RS, Knapp PM, Schuster TG, Dunn RL et al (2011) Regional collaboration to improve radiographic staging practices among men with early stage prostate cancer. J Urol 186(3):844–849

    Article  PubMed  Google Scholar 

  25. 25.

    Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, et al. (2009) Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev (2):CD003030

Download references

Financial Disclosure and Products Statement

None of the participating authors has a conflicting financial interest related to the work detailed in this manuscript, nor do any of the authors maintain a financial stake in any product, device or drug cited in this report.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Abhishek Trehan.

Additional information

Mahiben Maruthappu and Abhishek Trehan have equally contributed to the development of this manuscript.

Appendix: search strategy

Appendix: search strategy

Data extraction from reviewed studies

Study design

Author Year Prospective Retrospective Experimental Observational Cross-sectional Longitudinal Study design Number of study periods Total study duration (years)
Aletti et al. [19] 2008 1 0 1 0 0 1 Before–after study 2 6
Cornelius Olcott IV et al. [16] 2000 1 0 1 0 0 1 Before–after study 5 6
Findlay et al. [15] 2002 1 (in part) 1 (in part) 1 0 0 1 Before–after study 4 3
O’ Connor et al. [14] 1996 1 0 1 0 0 1 Before–after study 2 6
Pitt et al. [20] 1999 1 0 1 0 0 1 Before–after study 3 5
Reilly et al. [17] 2002 1 0 1 0 0 1 Before–after study 2 4
Rodriguez et al. [18] 2005 1 0 1 0 0 1 Before–after study 2 5

Study participants

Reference Country Hospital setting Surgeon speciality Surgical procedure No. of participating surgeons No. of cases in period 1 No. of cases in period 2 No. of cases in period 3 No. of cases in period 4 No. of cases in period 5 Total no. of cases
Aletti et al. [19] USA 1 Gynaecology Surgical management of ovarian cancer 105 132     237
Cornelius Olcott IV et al. [16] USA 1 Neurosurgery, general and vascular Carotid endarterectomy 10 105 188 181 154 135 763
Findlay et al. [15] Canada 1 Neurosurgery, general and vascular Carotid endarterectomy 10 291 184 249 222   946
O’ Connor et al. [14] USA 1 Cardiothoracic Coronary artery bypass grafting 23 6,638 6,488     13,126
Pitt et al. [20] USA 1 Pancreatobiliary Complex biliary surgery (choledochojejunostomy, cholangiojejunostomy, or hepaticojejunostomy) 4 plus ‘several others’ 66 107 166    339
Reilly et al. [17] Scotland 1 General All general surgical procedures 5     2241
Rodriguez et al. [18] Spain 1 Gynaecology Hysterectomy     980

Feedback

Reference Content of feedback = outcomes Qualitative oral feedback by observer Other information disseminated with feedback/other parallel measures Feedback dissemination—written Feedback dissemination—clinical session/meeting Feedback provided with literature outcomes Feedback provided with outcomes of overall hospital and/or other surgeons Frequency of feedback
Aletti et al. [19] 1 0 1 0 1 1 1 “Periodically”
Cornelius Olcott IV et al. [16] 1 0 0 1 1 0 1 Annually
Findlay et al. [15] 1 0 1 1 0 0 0 Essentially once a year
O’ Connor et al. [14] 1 0 1 1 1 0 1 Three times per year
Pitt et al. [20] 1 0 0 0 0 Once (after period “2″)
Reilly et al. [17] 1 0 0 0 0 Monthly
Rodriguez et al. [18] 1 0 0 1 1 0 1 Annually

Outcomes and improvement

Reference Measured outcome—Mortality Measured outcome—appropriateness if surgery Measured outcome—Morbidity Measured outcome—Infection rate Measured outcome—operative time Measured outcome—length of hospital stay Measured outcome—Hospital charges Length of follow up (days) Intervention following feedback Improvement in outcomes Improvement in outcomes (but not statistically significant) No improvement in outcomes
Aletti et al. [19] 1 0 1 0 1 1 0 30 (morbidity); 90 (mortality) 0 1 0
Cornelius Olcott IV et al. [16] 1 0 1 0 0 1 1 30 1 0 0
Findlay et al. [15] 1 1 1 0 0 0 0 30 1 0 0
O’ Connor et al. [14] 1 0 0 0 0 0 0 30 1 0 0
Pitt et al. [20] 1 0 1 0 0 0 1 1 0 0
Reilly et al. [17] 0 0 0 1 0 0 0 30 “Implementation of recommendations for change” 0 1 0
Rodriguez et al. [18] 0 0 0 1 0 0 0 30 1 0 0

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Maruthappu, M., Trehan, A., Barnett-Vanes, A. et al. The Impact of Feedback of Surgical Outcome Data on Surgical Performance: A Systematic Review. World J Surg 39, 879–889 (2015). https://doi.org/10.1007/s00268-014-2897-0

Download citation

Keywords

  • Coronary Artery Bypass Graft
  • Hospital Mortality
  • Carotid Endarterectomy
  • Quality Improvement Program
  • Wound Infection Rate