To the Editor,

The growing use of electronic medical records has provided a wealth of clinical data from which objective indicators of the quality of perioperative care can be drawn. Development of such indicators of quality and performance could result in potential methods to support more meaningful professional development. Unfortunately, optimal indicators of the quality of anesthetic care and how to use such data to provide formal performance feedback remain ill-defined.1

In an effort to identify clinical indicators and provide relevant performance feedback to anesthesiologists at The Ottawa Hospital, we developed the 2013 Performance Assessment. The data fed back were extracted from a pre-existing prospectively collected data set from our local electronic medical record system (Ottawa Health Services Network Research Ethics Board (REB) 20120828-01H). Each anesthesiologist was provided data relating to their individual clinical practice. Metrics in the report included demographic characteristics of their practice, postanesthesia care unit (PACU) outcomes (i.e., pain, nausea, length of stay), and practice information relevant to those outcomes (e.g., analgesia, antiemetic use). Summary data on the same variables for the department as a whole were provided as a comparator. A sample 2013 Performance Assessment and the 2016 update can be viewed at 10.6084/m9.figshare.4291157.v1.

To determine user-acceptability of the assessment and to highlight areas of potential improvement for future assessments/feedback, a post-implementation survey was administered to all recipients of a 2013 Performance Assessment (REB 20140438-01H). This 12-item survey, validated using Burns’ clinical sensibility testing tool2 in a pilot user group, was administered in in both electronic and paper formats. All eligible participants were given a copy of their personal 2013 Assessment along with a notice of the survey. Two e-mail invitations and distribution of a paper questionnaire were sent during the following six-week period. The close-ended responses were assigned a numerical value to generate increased response frequency. Open-ended or free-text responses were categorized into general themes.

Among the 76 eligible faculty, 52 (70%) submitted completed questionnaires, all electronic. Respondent perceptions of the performance assessment tool are demonstrated in the Table. Among the 52 (69%) participants, 36 (69%) agreed that this type of feedback is useful for professional development, suggesting a reasonable level of user acceptability. Whereas 34 of the 52 (65%) respondents indicated that the results of the performance assessment would influence their practice, 42 of the 52 (80%) respondents were neutral or disagreed that the tool was an effective measure of performance, and 22 of the 52 (42%) respondents were neutral or disagreed with questioning about physician autonomy. The latter responses suggested that some participants were uneasy with the assessment process and its potential implications.

Table Respondent perceptions of the 2013 performance assessment at The Ottawa Hospital
Full size table

Free-text responses addressed the frequency of resident inclusion in cases, frequency of adverse events, patient temperature on arrival in the PACU, and exclusion of specific practices and process-of-care descriptors, such as the patient’s age, sex, and frequency of specific drug usage. Overall, 36 of the 52 (70%) respondents thought that all existing data items should be included in future feedback.

Our high survey completion rate suggests that physician performance feedback is of interest to anesthesiologists at The Ottawa Hospital. Our results indicated reasonable user acceptance of the performance feedback tool and identified several areas for future improvement. Survey responses were incorporated into subsequent annual iterations of the performance assessment. We hope to repeat the survey process and further improve the acceptability and utility efficacy of this form of feedback. We are considering the addition of a reflection to make this activity eligible for Royal College of Physicians and Surgeons Maintenance of Certification Section 3 credit.