Introduction

Succinct and thorough operative reports are key to communicating intraoperative events and providing crucial information for future operative planning [1]. Accurate and comprehensive dictation is similarly necessary for medicolegal purposes and to ensure accurate coding and billing for appropriate reimbursement [2, 3]. Furthermore, operative documentation serves as an important cognitive task tool for residents, helping them understand and integrate the reasons for, and performance of, surgical procedures [4].

The three aspects of operative reports: patient care, billing, and learning, are sometimes in conflict with each other. While residents may learn from practicing dictation, prior studies have criticized the accuracy and completeness of resident-dictated operative notes, noting that key information was missing in up to 76% of cases [2, 5]. Operative notes are utilized to generate codes for billing purposes [6]. Without directed education, residents may not understand how a hospital coding team analyzes reports to generate codes for billing. As a result, deficiency in resident dictations has been estimated to reduce reimbursement by nearly 10% [2].

The Joint Commission and Centers for Medicare and Medicaid Services (CMS) require certain data in operative documentation, but there is no universally recognized operative note format that includes all required components [7]. While the Accreditation Council for Graduate Medical Education (ACGME) and ABS have set professional competency and operative experience requirements for graduation from residency and board certification in general surgery, there exist no specific requirements related to operative dictation [8, 9]. As such, even though most residents and program directors favor structured teaching on operative dictation, less than 25% of surgical residencies provide formal education on dictation skills [1, 5]. Feedback is rare and most residents rely on old operative dictations from fellow residents and attendings as resources to learn the skill [1, 10,11,12]. Clearly, there is a gap that needs to be addressed in trainee education, as operative reporting is an essential skill of the practicing surgeon.

On July 1, 2021, our university-based surgery department implemented a policy requiring attending surgeons to dictate all operative reports, due to concerns about note accuracy, as well as missed revenue opportunities. The policy change afforded a unique opportunity to directly examine differences in operative dictation among surgical trainees and faculty. The purpose of this study was to (1) assess the financial impact of resident operative dictation without structured education and (2) to identify educational opportunities for future intervention.

Methods

Core general surgery procedures performed at our university-affiliated medical center by the Department of Surgery between July 1, 2020 and June 30, 2022 were identified from billing data. Core procedures were selected based on the ACGME-defined category minimums for operative experience and are listed in Table 1. Operations with current procedural terminology (CPT) codes for core procedures, unique patient identification numbers, encounter numbers, and dates were included. Operations were defined as all procedures performed on the same day, by the same attending surgeon, on the same patient. Operations from the following divisions were included: emergency general surgery/trauma, elective/gastrointestinal surgery, colorectal surgery, surgical oncology, and vascular surgery. The thoracic and cardiac surgery divisions were excluded given the lack of resident-dictated operative notes. Fellow-dictated notes and operative notes from surgeons who were not present for the whole study period were excluded. Additionally, operations with coded modifiers 62, 80, or 82 (indicating the necessity of more than one attending surgeon and shared billing) were excluded.

Table 1 Differences in RVU distributions among 18 general surgery operations pre-and post-policy
Full size table

Surgeon name, CPT code, modifier codes, and relative value units (RVUs) were recorded for each case. The primary outcome was the difference in RVU distributions between fiscal year (FY) 2021 and 2022. Specific operations were then analyzed to determine differences in RVU distributions between fiscal years. This study was deemed exempt from full review by our institutional review board (2023E0045).

During and prior to the study period, residents had not received structured education on operative note dictation. Operative coding for billing was performed by trained salaried operative coders, which was consistent between the pre- and post-policy periods. Institutional policy required submission of all operative notes within 24 h of the operation, including cosigning by the attending if the note is initially authored by a resident. This was consistent between the pre- and post-policy periods studied.

Quantitative analysis

The 18 core general surgery operations were summarized using descriptive statistics. The Kolmogorov–Smirnov (KS) non-parametric test was performed to examine differences between pre- and post-policy RVU distributions. The KS test was chosen given its sensitivity and applicability to non-normal distributions; the KS statistic summarizes the difference between the pre- and post-policy cumulative distributions. The threshold for statistical significance was set at p < 0.05. Statistical analysis was performed utilizing the SciPy Python package [13] and reviewed by a statistics expert (DW).

Qualitative document analysis

Operative reports were analyzed utilizing an explanatory sequential design. Based on the operation-specific billing comparisons, six operations were chosen to represent general surgery cases: two favored pre-policy documentation (partial colectomy, open ventral hernia repair), two favored post-policy documentation (minimally invasive ventral hernia repair, melanoma excision), and two had no statistically significant billing differences (laparoscopic cholecystectomy, mastectomy). We selected resident-dictated operative reports from these operations in FY2021, extracted the resident dictations and attending revisions from the electronic medical record, and paired them with attending-dictated operative reports from FY2022. Reports were selected through convenience sampling and were matched on core CPT code (defined as the primary procedure performed in the operation), attending surgeon, and elective or non-elective status. The operation type, date of surgery, core CPT code, additional CPT codes, modifier codes, resident training level, and attending surgeon were recorded. Targeted thematic content analysis was performed on operative reports to compare resident dictation, attending revisions of resident dictation, and attending dictation.

Two coders (TW, JC) performed content analysis using a deductive and inductive approach [14,15,16,17]. Codes were additionally reviewed by a qualitative methodology expert (EH) and refined with ongoing analysis. Based on our quantitative results, we theorized that there would be identifiable billable differences; additional codes and themes were generated as coding progressed. Initial codes fell into “billable differences” and “clinical differences”. With further analysis, additional key concepts were identified, refined into codes, and sorted into the parent- and child-codes described in this study. Inter-coder reliability was reassessed after every 20 documents reviewed, with re-coding of prior documents after codebook modification and agreement over code application. Theoretical saturation was achieved when document review yielded no further modifications to theories or codes [18].

Frequency of codes (billable notations, surgical or clinical reasoning, and technical details) were compared between resident dictation, attending revisions of resident dictation, and attending dictation using resident dictations as the reference group. Document analysis and codebook generation were performed in Microsoft Office 365 (Microsoft, Redmond, WA).

Results

Quantitative analysis

A total of 16,233 operations with 28,560 billed CPT codes were included in the study. One-half (50%, n = 8173) were resident-dictated notes with attending revisions from FY2021. The remaining half (50%, n = 8060) were attending-dictated operative notes from FY2022 (Fig. 1). Service intensity and procedural time, represented by modifier 22 code applications, were similar between the two fiscal years (FY2021 3.53% versus FY2022 3.74%, p = 0.304).

Fig. 1
figure 1

Included operations by fiscal year: number and percentage of total

Full size image

RVU distributions of all operations between the two fiscal years demonstrated a KS = 0.026 (p = 0.009) difference favoring attending-only dictations (Fig. 2). This translated into an overall difference of 1016 RVUs or $32,928 using 2021 Medicare reimbursement rates of $32.41 per RVU [19].

Fig. 2
figure 2

RVU distribution by fiscal year displayed as number of cases per RVU. FY2021 in blue and FY2022 in red

Full size image

The KS test was then applied to 18 operations across multiple specialties that a general surgery resident would be expected to be able to perform for board certification [8, 20]. Among these procedures, partial colectomy (p = 0.043) and open ventral hernia repair (p = 0.004) had higher RVU distributions pre-policy (Table 1). Melanoma excision (p = 0.009), minimally invasive ventral hernia repair (p = 0.008), parathyroidectomy (p = 0.000), anorectal abscess incision and drainage (p = 0.003), and anorectal exam under anesthesia (p = 0.029) had higher RVU distributions post-policy (Table 1). No RVU distribution differences were seen in the remaining operations between fiscal years (p > 0.05) (Table 1).

Qualitative document analysis

Operative notes for the following six representative operative procedures were selected for in-depth document analysis: partial colectomy, open ventral hernia repair, minimally invasive ventral hernia repair, melanoma excision, laparoscopic cholecystectomy, and mastectomy (Table 2). Most analyzed resident-dictated notes were initially dictated by post-graduate year 4 or 5 residents (range: 70–90%), except for melanoma excision (33%).

Table 2 Characteristics of operative notes analyzed by operation
Full size table

Three major themes were identified in the operative dictations: billable items, clinical or surgical reasoning, and technical details. Billable items could be described in the procedure list or body of the report, clinical or surgical reasoning could apply to preoperative planning or intraoperative judgment, and technical details described specific aspects of surgical action or anatomy encountered at the time of the operation. The final codebook is summarized in Table 3.

Table 3 Major themes and codes in operative dictations
Full size table

Both residents and attending surgeons included material in operative reports encompassing each theme. Sample quotations of each code, organized by author type, are portrayed in Table 4. Compared with resident-dictated operative notes, 43.7% of attending revisions and attending-dictated notes contained more billable notations, 27.7% contained clinical or surgical reasoning components, and 25.9% contained technical details. Thematically, billable items and clinical or surgical reasoning differences were considered most influential in changing the meaning of the operative note. This difference was most pronounced in procedures with a significantly higher RVU distribution in post-policy. Specifically, attending-dictated operative reports were more thorough in detailing preoperative indications for surgery and intraoperative decision-making for more complex operations.

Table 4 Demonstrative quotations of each code organized by author type
Full size table

Discussion

Our Department of Surgery’s 2021 transition from resident-dictated operative notes to attending-only dictations provided a unique opportunity to study the financial impact of resident dictation on billing accuracy during an era when no formal education on procedural billing was provided to trainees. We noted a small but statistically significant billing difference favoring the attending dictation (post-policy) period, which resulted in a calculated $32,928 annual cost difference. In-depth analysis of operative reports dictated by residents, attending revisions of resident-dictated notes, and attending-dictated notes revealed three key themes: billable items, clinical or surgical reasoning, and technical details. Of these, billable items and clinical or surgical reasoning were thematically the most influential on billing or clinical ramifications of operative reports. Attending surgeon revisions and attending-dictated notes more frequently contained these elements. These results provide a clear path for educational intervention.

A few studies have investigated the financial impact of resident dictation across an entire surgical department. Novitsky et al. noted that deficiencies in resident dictations reduced reimbursement by $18,200 or 9.7% in 2005 [2]. However, their study was limited to dictations from senior surgical residents (PGY 3–5) and compared these unedited resident dictations directly to attending dictations. Additionally, their study evaluated 50 operative reports over a span of 2 weeks, further limiting interpretation of their results. Although our study similarly noted reduced reimbursements with resident dictations, the difference was much smaller at 2%. Furthermore, we were able to identify which specific operations contributed most to these differences. Our study was also unique in its ability to identify more in-depth differences between resident and attending operative dictations. Rather than limiting our investigation to quality indicators, the exploratory and inductive nature of our targeted thematic analysis allowed us to identify novel themes (i.e., billable differences, and clinical and surgical reasoning) that may serve as targets for future educational interventions [3, 4, 7].

A more recent study noted no association between the number, proportion, or types of operations senior residents dictated with first-time pass rates of both the American Board of Surgery Qualifying Exam and the Certifying Exam [21]. However, this study solely focused on the number of dictations rather than the quality of dictations. Given the lack of formal teaching on operative dictation skills, residents often have to rely on senior residents or attendings’ operative notes [1]. The impact of training progression on operative note accuracy and quality remains controversial. Zwintscher et al. noted a positive correlation between training level and improved operative note completion [7]. Porterfield et al. on the other hand, noted the opposite, whereby first-year residents had more complete documentation, presumably due to an “inverted U relationship” in data recall within the field of medicine [3]. There is no spontaneous transition from being a resident to becoming an attending that confers knowledge of accurate procedural billing phrases, though increasing awareness of the operative note as a medicolegal document may be related to more thorough documentation of clinical and surgical reasoning. One-third of attending dictations continue to remain deficient in reporting quality indicators [4]. Relying solely on other surgeons’ operative notes thereby only perpetuates the low quality of operative dictation, highlighting the importance of intentional training on operative documentation [3, 4].

There is an unmet need for residency curricula to prepare general surgery residents for “real world”, non-clinical skills, including navigating healthcare systems, finance, and membership23. A 2019 study of program directors and chief residents in general surgery identified “coding and billing” as the most important non-clinical topic that should be formally taught during residency. Some studies have incorporated more formal educational sessions on operative dictation with notable improvements [1, 5, 10, 11, 22]. Interventions have ranged from teaching sessions with accompanying examples to the incorporation of synoptic reports. While dictation templates may provide organization, structure, and reminders of required content, synoptic reporting may also rob learners of the necessary recall, organizing, and explanation of procedures that help them better understand and integrate the reasons for, and performance steps of, operative procedures. In addition, the long-term effects of these educational interventions are unknown. However, by focusing on broader themes (i.e., billable differences, and clinical and surgical reasoning), as identified within this study, we would enhance learners' foundational knowledge of both the necessary components in operative dictations and the indications and steps of procedures. Clinical and surgical reasoning differences between attending- and resident-authored notes, in particular, may reveal deficits in the “following” of intraoperative decision-making by the trainee. When identified, these pose valuable opportunities for discussion to enhance resident understanding of decision-making.

Salaried, trained billing coders applied codes to all operations during both pre- and post-policy periods, but our study is limited by our inability to adjust for coder experience, as some billable items were notably not billed when they were described in the body of the operative report but not listed in the procedure list. In addition, divisions with recent faculty turnover were underrepresented (e.g., one division in which 5 attending surgeons were included and 13 attendings excluded). There may have also been confounders between the fiscal years studied. For example, while the overall case number and complexity were similar between the two fiscal years, the impact of the COVID-19 pandemic on elective surgeries during FY2022 may have inadvertently impacted case distributions. The greater sensitivity of the KS test may also make our study more prone to Type I errors. Finally, we note that the fiscal comparison performed was between attending-edited resident dictations and attending dictations; there would likely have been even higher billing differences in the absence of attending review of resident dictations.

This study is one of the first to evaluate the financial impact of resident operative dictation as well as qualitatively explore differences between resident and attending dictations. Differences remained minimal and primarily involved billable items and clinical and surgical reasoning. Future educational interventions should therefore focus on these themes. For example, clinical leaders in each subspecialty should share significant billable items for residents to note during their operative dictations. Clinical and surgical reasoning should be discussed before, during, and after procedures to advance residents’ clinical reasoning skills. Once residents have dictated or written operative reports, attending revisions should be discussed and presented directly as feedback. We recommend incorporating billable components and clinical reasoning into educational curricula on operative documentation, alongside a formal feedback mechanism, to serve simultaneous purposes of strengthening resident education while increasing operative dictation accuracy.

Conclusion

Operative dictation is a valuable educational tool for residents to review operative steps and clinical decision-making. Mixed-method analysis of differences between resident- and attending-dictated operative notes demonstrated meaningful differences in billable items and clinical and surgical reasoning. Incorporating formal resident education emphasizing billable items and clinical reasoning in operative dictations is necessary to prepare residents for independent practice.