The Impact of Feedback of Surgical Outcome Data on Surgical Performance: A Systematic Review

Abstract

Background

Increasing patient demands, costs and emphasis on safety have led to performance tracking of individual surgeons. Several methods of using these data, including feedback have been proposed. Our aim was to systematically review the impact of feedback of outcome data to surgeons on their performance.

Study design

MEDLINE, Embase, PsycINFO, AMED and the Cochrane Database of Systematic Reviews (from their inception to February 2013) were searched. Two reviewers independently reviewed citations using predetermined inclusion and exclusion criteria. Forty two data-points per study were extracted.

Results

The search strategy yielded 1,531 citations. Seven studies were eligible comprising 18,632 cases or procedures by 52 surgeons. Overall, feedback was found to be a powerful method for improving surgical outcomes or indicators of surgical performance, including reductions in hospital mortality after CABG of 24 % (P = 0.001), decreases of stroke and mortality following carotid endarterectomy from 5.2 to 2.3 %, improved ovarian cancer resection from 77 to 85 % (P = 0.157) and reductions in wound infection rates from 14 to 10.3 %. Improvements in performance occurred in concert with reduced costs: for hepaticojejunostomy, implementation of feedback was associated with a decrease in overall hospital costs from $24,446 to $20,240 (P < 0.01). Similarly, total cost of carotid endarterectomy and following management decreased from $13,344 to $9548.

Conclusions

The available literature suggests that feedback can improve surgical performance and outcomes; however, given the heterogeneity and limited number of studies, in addition to their non-randomised nature, it is difficult to draw clear conclusions from the literature with regard to the efficacy of feedback and the specific nuances required to optimise the impact of feedback. There is a clear need for more rigorous studies to determine how feedback of outcome data may impact performance, and whether this low-cost intervention has potential to benefit surgical practice.

Appendix: search strategy

Data extraction from reviewed studies

Study design

Author	Year	Prospective	Retrospective	Experimental	Observational	Cross-sectional	Longitudinal	Study design	Number of study periods	Total study duration (years)
Aletti et al. [19]	2008	1	0	1	0	0	1	Before–after study	2	6
Cornelius Olcott IV et al. [16]	2000	1	0	1	0	0	1	Before–after study	5	6
Findlay et al. [15]	2002	1 (in part)	1 (in part)	1	0	0	1	Before–after study	4	3
O’ Connor et al. [14]	1996	1	0	1	0	0	1	Before–after study	2	6
Pitt et al. [20]	1999	1	0	1	0	0	1	Before–after study	3	5
Reilly et al. [17]	2002	1	0	1	0	0	1	Before–after study	2	4
Rodriguez et al. [18]	2005	1	0	1	0	0	1	Before–after study	2	5

Study participants

Reference	Country	Hospital setting	Surgeon speciality	Surgical procedure	No. of participating surgeons	No. of cases in period 1	No. of cases in period 2	No. of cases in period 3	No. of cases in period 4	No. of cases in period 5	Total no. of cases
Aletti et al. [19]	USA	1	Gynaecology	Surgical management of ovarian cancer	–	105	132				237
Cornelius Olcott IV et al. [16]	USA	1	Neurosurgery, general and vascular	Carotid endarterectomy	10	105	188	181	154	135	763
Findlay et al. [15]	Canada	1	Neurosurgery, general and vascular	Carotid endarterectomy	10	291	184	249	222		946
O’ Connor et al. [14]	USA	1	Cardiothoracic	Coronary artery bypass grafting	23	6,638	6,488				13,126
Pitt et al. [20]	USA	1	Pancreatobiliary	Complex biliary surgery (choledochojejunostomy, cholangiojejunostomy, or hepaticojejunostomy)	4 plus ‘several others’	66	107	166			339
Reilly et al. [17]	Scotland	1	General	All general surgical procedures	5	–	–				2241
Rodriguez et al. [18]	Spain	1	Gynaecology	Hysterectomy	–	–	–				980

Feedback

Reference	Content of feedback = outcomes	Qualitative oral feedback by observer	Other information disseminated with feedback/other parallel measures	Feedback dissemination—written	Feedback dissemination—clinical session/meeting	Feedback provided with literature outcomes	Feedback provided with outcomes of overall hospital and/or other surgeons	Frequency of feedback
Aletti et al. [19]	1	0	1	0	1	1	1	“Periodically”
Cornelius Olcott IV et al. [16]	1	0	0	1	1	0	1	Annually
Findlay et al. [15]	1	0	1	1	0	0	0	Essentially once a year
O’ Connor et al. [14]	1	0	1	1	1	0	1	Three times per year
Pitt et al. [20]	1	0	0	–	–	0	0	Once (after period “2″)
Reilly et al. [17]	1	0	0	–	–	0	0	Monthly
Rodriguez et al. [18]	1	0	0	1	1	0	1	Annually

Outcomes and improvement

Reference	Measured outcome—Mortality	Measured outcome—appropriateness if surgery	Measured outcome—Morbidity	Measured outcome—Infection rate	Measured outcome—operative time	Measured outcome—length of hospital stay	Measured outcome—Hospital charges	Length of follow up (days)	Intervention following feedback	Improvement in outcomes	Improvement in outcomes (but not statistically significant)	No improvement in outcomes
Aletti et al. [19]	1	0	1	0	1	1	0	30 (morbidity); 90 (mortality)	–	0	1	0
Cornelius Olcott IV et al. [16]	1	0	1	0	0	1	1	30	–	1	0	0
Findlay et al. [15]	1	1	1	0	0	0	0	30	–	1	0	0
O’ Connor et al. [14]	1	0	0	0	0	0	0	30	–	1	0	0
Pitt et al. [20]	1	0	1	0	0	0	1	–	–	1	0	0
Reilly et al. [17]	0	0	0	1	0	0	0	30	“Implementation of recommendations for change”	0	1	0
Rodriguez et al. [18]	0	0	0	1	0	0	0	30	–	1	0	0