Introduction

Thyroid nodules are a common finding, with a lifetime incidence of approximately 19–35 % in adults when using the aid of ultrasonography [1]. Approximately 3–7 % of all thyroid nodules are diagnosed as malignant by cytology obtained via fine-needle aspiration (FNA) [2]. FNA and cytology are the recommended next steps to determine the nature of any suspicious thyroid nodule. The Bethesda System of Reporting Thyroid Cytopathology classifies thyroid nodules into six different diagnostic categories: (I) nondiagnostic or unsatisfactory, (II) benign, (III) atypia of undetermined significance or follicular lesion of undetermined significance (AUS), (IV) follicular neoplasm or suspicious for a follicular neoplasm (SFN), (V) suspicious for malignancy, or (VI) malignant [2]. AUS and SFN are grouped as “indeterminate” findings according to the Bethesda system and are observed in 15 to 30 % of all thyroid nodule biopsies via FNA [3–6].

According to the Bethesda consensus, AUS can be managed with a repeat FNA, while SFN warrants surgical resection [2]. In cases of AUS, if the repeated FNA reconfirmed an indeterminate finding or malignancy, surgery should follow [5]. Ultimately, about 74 % of patients who are diagnosed as indeterminate at initial FNA will require surgery [7]. Surgical pathology report, however, demonstrated that 70 to 85 % of these nodules are benign, yet these patients are exposed to the risk of surgical complications, cost, and decreased quality of life from surgery, without improving overall survival [2, 5, 8–10].

A relatively novel alternative approach to evaluate indeterminate cytology of thyroid nodules is the Afirma® Gene Expression Classifier (AGEC), which utilizes genomic data to molecularly categorize thyroid nodules. The AGEC compares the mRNA expression of 167 genes. Out of these 167 genes, 142 are involved in a proprietary algorithm to identify benign gene expression patterns while the remaining 25 genes are used to identify possible rare neoplasms [11]. The AGEC, with a sensitivity of 90 % in evaluating malignancies in AUS and SFN and a negative predictive value of 95 and 94 % in evaluating malignancies in AUS and SFN, respectively, was developed to optimize surgical selection by excluding benign nodules in patients with indeterminate diagnosis [12].

Studies which compare the outcomes of similar patient populations managed with and without using AGEC in one clinical entity remain limited. This study had two objectives: first, to find whether implementing AGEC was associated with decreased rate of surgical recommendations and decreased cost, and the second, to evaluate whether the incidence of thyroid malignancy diagnosed by surgery has increased after AGEC implementation compared to before AGEC implementation in patients with indeterminate diagnosis of thyroid nodules.

Methods

Subjects

All patients who underwent thyroid nodule FNA between January 1, 2004 and June 30, 2014 were screened. Electronic medical records at the Kansas University-Wichita (KU) Endocrine clinic were reviewed for data extraction.

Instruments

At the KU Endocrine clinic, AGEC was utilized in thyroid cancer management starting in June, 2012. Before AGEC was implemented, thyroid nodule FNA specimens were analyzed locally. After implementation of AGEC, FNA specimens were analyzed at Veracyte laboratory, Austin, TX. At both locations, the FNA specimens were analyzed and reported as per the Bethesda classification system. After June 2012, if FNA specimens yielded an indeterminate diagnosis, it was followed by AGEC analysis. The data were abstracted from medical charts from December, 2014 to February, 2015. The study was approved by the University of Kansas Institutional Review Board.

Procedures

Patients were classified into two cohorts: (1) patients with indeterminate cytopathology findings managed before implementation of the AGEC test and (2) patients with indeterminate cytopathology findings managed with AGEC testing.

Analysis

We evaluated the proportion of patients whom surgery was recommended and who actually underwent surgery following the indeterminate diagnosis before and after implementation of AGEC testing, percentage of patients whom surgery was not recommended due to AGEC test results, and proportion of surgical pathology diagnosed as malignant and financial cost for evaluation and initial management of patients with indeterminate diagnosis before and after AGEC implementation.

Basic costs were estimated by using two variables: cost per outpatient surgery including less than 24 h hospital stay and cost for AGEC analysis. Variables such as cost due to surgical complications, readmission, intensive care unit use, and post-surgical follow-up and cost for medications such as lifelong thyroid hormone and calcium supplements were not included in this basic cost estimation.

Data were analyzed using the SAS version 9.3 (SAS Institute, Cary, NC). Descriptive statistics were presented as frequencies and proportions for categorical variables. Chi-square analysis was conducted to assess the proportion of surgical recommendations, proportion of actual surgical procedures, proportion of AUS and SFN diagnoses, and incidence of malignant diagnosis by surgical pathology in pre- and post-implementation of AGEC. All statistical analyses were two sided. p values of <0.05 were considered to be statistically significant.

Results

The medical records query identified 299 patients, of which 288 patients were diagnosed using the Bethesda classification system for thyroid cytopathology. In the remaining 11 patients, the FNA was performed at an outside facility and results were not available or were not reviewed at our institution. The pre-AGEC cohort included 113 patients. The median age of the pre-AGEC group was 57 years (range 17–74 years) and the majority was female (88 %). The mean nodule size was 2 cm. The post-AGEC cohort included 175 patients with a median age of 50.5 years (range 14–89 years). The majority was female (91 %). The mean nodule size was 2.3 cm. Twenty-seven patients (24 %) in the pre-AGEC cohort had indeterminate diagnosis via FNA compared to 34 patients (19 %) in the post-AGEC cohort (Table 1).

Table 1 Patient demographic information
Full size table

Among the 27 patients in the pre-AGEC implementation cohort with indeterminate FNA specimens, surgery was recommended in 22 (81.5 %) patients, with 18 (67 %) of those patients actually undergoing surgical resection of the thyroid nodule. Out of the 4 patients in whom surgery was recommended, 2 patients did not undergo the procedure due to underlying medical comorbidities unrelated to thyroid malignancy and personal preference, respectively. Information about whether surgery was performed was missing in the remaining 2 patients (Fig. 1). Among the 34 patients in the post-AGEC implementation cohort with indeterminate FNA specimens, 17 (50 %) were recommended for surgery due to “suspicious” AGEC result, with 16 (47 %) of those patients undergoing the procedure. One patient did not undergo surgery due to relocation from Kansas. In the remaining 17 patients, surgery was not recommended due to “benign” AGEC result and no surgery was performed (Fig. 2). Therefore, surgical recommendation rate for patients with indeterminate diagnosis before and after AGEC implementation was 81.5 and 50 % (p = 0.01), respectively, and actual performed surgery rate before and after AGEC implementation was 67 and 47 % (p = 0.12), respectively (Fig. 3).

Fig. 1
figure 1

Flow diagram before AGEC implementation. Flow diagram representing the number of indeterminate diagnosis, number of surgical recommendations, and number of surgeries and surgical pathology diagnosis in the patients cohort before AGEC implementation

Full size image
Fig. 2
figure 2

Flow diagram after AGEC implementation. Flow diagram representing the number of indeterminate diagnosis, number of surgical recommendations, and number of surgeries and surgical pathology diagnosis in the patients cohort after AGEC implementation

Full size image
Fig. 3
figure 3

Surgical recommendation rate for patients with indeterminate diagnosis in pre- and post-AGEC implementation. The difference between surgical recommendation rates before and after AGEC implementation in patients with indeterminate diagnosis

Full size image

In the pre-AGEC implementation cohort, 12 out of 15 (80 %) available surgical pathology reports showed a benign lesion, while the remaining 3 (20 %) showed a malignant lesion. In contrast, in the post-AGEC implementation cohort, only 2 out of the available 14 (14.3 %) surgical pathology reports showed a benign lesion while the remaining 12 (85.7 %) were malignant by the surgical pathology diagnosis (p < 0.01) (Fig. 4).

Fig. 4
figure 4

Surgical pathology diagnosis in patients with indeterminate diagnosis in pre- and post-AGEC implementation. The difference between the incidence of malignant diagnosis by surgical pathology before and after AGEC implementation in patients with indeterminate diagnosis

Full size image

Before AGEC was implemented, a patient who had an indeterminate diagnosis would have spent an average of $10,994 to undergo thyroid nodule evaluation and initial treatment. After AGEC implementation, a patient with an indeterminate diagnosis spent an average of $9946 to undergo thyroid nodule evaluation and initial treatment (Table 2).

Table 2 Cost analysis
Full size table

AUS was diagnosed in 11 out of 27 (40.7 %) patients and 24 out of 34 (70.6 %) patients, before and after AGEC implementation, respectively, representing a 73.5 % increase in AUS diagnosis after AGEC implementation. SFN was diagnosed in 16 out of 27 (59.3 %) patients and 10 out of 34 (29.4 %) patients, before and after AGEC implementation, respectively, resulting in a 50.4 % decrease in SFN diagnosis after AGEC implementation. The p value derived from chi-square analysis for AUS and SFN diagnosis before and after AGEC implementation was 0.02. Surgical recommendation rate for AUS was 6 of 11 (54.5 %) and 11 of 24 (45.8 %) (p = 0.63), before and after AGEC implementation, respectively. For SFN, surgical recommendation rate was 16 of 16 (100 %) and 6 of 10 (60 %) (p < 0.01), before and after AGEC implementation, respectively.

Discussion

Our study evaluated whether implementing AGEC in clinic practice decreases the number of unwarranted surgical recommendations and its associated cost and increases the incidence of malignant diagnosis by surgical pathology in patients with thyroid nodules of indeterminate diagnosis.

After the implementation of AGEC, there was a statistically significant decrease in surgical recommendation rate. Given the similarity of the two patient cohorts with respect to clinical location, incidence of indeterminate diagnosis, and patient demographics, it is evident that implementation of AGEC has decreased the rate of surgical recommendations. The number of actual surgeries performed differs from the recommended surgical rates due to underlying comorbidities unrelated to thyroid malignancy, personal preference, relocation from Kansas, or missing data. Surgical recommendation rates are not subject to patients’ variables and individual choices as the actual surgical procedure rate. In the study by Yang et al., the percentages of surgical procedure rate in patients with AUS diagnosis decreased from 63 to 35 % after the implementation of AGEC and similar findings were also obtained for SFN in a single institution [11]. Our study found a similar decrease in surgical recommendation rate in patients with indeterminate diagnosis and is one of the early studies to compare the rate of surgical recommendation and actual commencement of surgery before and after AGEC implementation in similar groups of patients.

Before AGEC was implemented, only 20 % of the surgical pathology specimens had a malignant diagnosis, with the remaining 80 % of the indeterminate thyroid nodules having a benign diagnosis. This data parallels the studies done by Baloch et al. and Goldstein et al. in two major university hospitals where it was found that benign diagnosis rate made by surgical pathology ranged from 70 to 85 % in patients with indeterminate thyroid nodule findings [8, 9]. After the implementation of AGEC, incidence of malignant diagnosis by surgical pathology increased to 85.7 from 20 %, demonstrating the suitability of AGEC as a screening tool. Studies done by Li et al. and Yang et al. also found that AGEC implementation has increased malignant diagnosis by surgical pathology from 35 to 50–54 %, but the large increase that our study found was unique [11, 13]. As it is evident, this may also suggest that many patients with indeterminate diagnosis underwent surgical evaluation before AGEC was implemented with little or no benefit and to recognize this fact may prove vital to uplift patient management and wellbeing.

Cost analysis was done by using surgical recommendation rates as this eliminates confounders. AGEC implementation has reduced the cost associated with evaluation and initial treatment of indeterminate thyroid nodules as it reduces the number of surgical recommendations. The cost analysis suggested that, by implementing AGEC, $1048 per patient could be saved in evaluation and initial management of patients with an indeterminate diagnosis. Prior study by Tuggle et al. has shown that approximately 15 % of patients above 65 years of age require re-hospitalization for complications related to thyroidectomy [14]. If other variables such as cost of readmission after thyroidectomy, lifelong thyroid hormone supplementation, surgical complications, and post-surgical follow-up had been incorporated in our cost estimate, it would further increase the cost savings by implementing AGEC. Thus, the cost savings demonstrated by our study is conservative and likely underestimates it. Past studies which incorporated variables such as cost for lifelong medications and post-surgical follow-up have estimated a cost benefit ranging from $1453 per patient in a course of 5 years to $2600 per patient [13, 15].

There was no statistically significant difference in the incidence of indeterminate diagnosis before and after AGEC implementation suggesting that the patient population before and after AGEC implementation was similar. Additionally, the incidence of indeterminate diagnosis in our study is comparable with previous studies ranging from 15 to 30 % of the total thyroid nodule biopsies via FNA [3–6]. Given the similarity of the two cohorts with respect to patient demographics and incidence of indeterminate diagnosis before and after AGEC implementation, this data may also suggest less bias to increase indeterminate diagnosis by using AGEC.

Although it was not included in the research questions, the results further found a difference in incidence of AUS and SFN before and after AGEC implementation. After AGEC implementation, diagnosis of AUS increased by 73.5 % and diagnosis of SFN decreased by 50.4 %. This might be due to overdiagnosis of SFN in the past as AUS was only introduced to the medical community in 2007 by the Bethesda classification system [2].

The study also suggested that there was a difference in the number of surgical recommendation rates between patients who were diagnosed as AUS and SFN in pre- and post-AGEC implementation periods. Before AGEC was implemented, surgical recommendation for SFN was about two times higher than that for AUS. The reasoning behind the higher rates of surgical recommendation among patients with SFN is the higher perceived rate of malignancy, 15–30 %, compared to patients with the diagnosis of AUS which was thought to have a malignancy rate of 5–15 % [2]. The surgical recommendation rate has decreased by 40 % in patients with SFN diagnosis after AGEC implementation. This may also suggests that physicians are more comfortable in recommending against surgery in patients who had SFN diagnosis with benign AGEC findings. This study showed that AGEC implementation has had much more impact on FNA specimens with SFN diagnosis as it was able to reclassify 40 % of FNA specimens with SFN diagnosis as benign. Similar results were seen by Faquin et al. where AGEC reclassified 52 % of FNA specimens with SFN diagnosis as benign [16]. Further, Alexander et al. also demonstrated the large impact of AGEC on SFN category [12, 17].

The main limitations of our research study are the small sample size and the single-center patient cohort. Though there are similarities with respect to age, gender, nodule size, and incidence of indeterminate diagnosis between the two cohorts, the retrospective cohort study design may still introduce confounding factors and biases. Further, FNA specimens were analyzed by more than one pathologist over the study period, thus contributing to interobserver variability in diagnosing thyroid malignancy. This was minimized by adhering to the guidelines under the Bethesda System of Reporting Thyroid Cytopathology, but still, interobserver variability may not have been fully eliminated. It is reassuring to see that our data paralleled with other studies regarding the incidence of indeterminate diagnosis [6]. Further, the incidence of malignant diagnosis by surgical pathology in pre-AGEC implementation cohort also matched the data from previous studies [8, 9]. Therefore, the results of this study may prove applicable to a bigger population. Also, our study is one of the few studies which recognized the outcomes of AGEC implementation by comparing similar patient cohorts before and after AGEC implementation in one clinical entity.

Conclusion

The number of surgical recommendations in patients with indeterminate diagnosis decreased after AGEC implementation. The incidence of malignant diagnosis of indeterminate thyroid nodules by surgical pathology after AGEC implementation was significantly higher than the incidence of malignant diagnosis before AGEC implementation. Implementation of AGEC also showed to be cost-effective in evaluation and initial management of patients with indeterminate diagnosis. The immediate benefits of routine AGEC testing of cytologically indeterminate thyroid nodules instead of routine diagnostic surgery to patients and healthcare systems are supported by our findings.