Opening

A recent article was published in the Journal of Otolaryngology—Head & Neck Surgery titled “Risk of diabetes in patients with sleep apnea: comparison of surgery versus CPAP in a long-term follow-up study” by O’Connor-Reina et al., highlighting a strong interest in the potential benefits of upper airway surgeries (UAS) in reducing risk of diabetes in patients with obstructive sleep apnea (OSA) [1]. This study utilized a federated de-identified database, TriNetX, that compared the rates of new-onset diabetes and mortality in OSA patients treated with UAS to continuous positive airway pressure (CPAP). The idea behind this study was interesting and contributed to the ongoing discourse on the different treatments of OSA. To further this discussion, we would like to point out some limitations that warrant attention and caution interpretation of the findings by physicians and patients.

Selection bias

First, it is possible that the findings were confounded by selection bias: i.e. patients in the CPAP cohort had significantly more comorbidities than those who underwent UAS, and were therefore more likely to also develop diabetes. As seen in Table 4 of the study, 33.20% of the CPAP group suffered from obesity compared to 18.00% in the UAS cohort and 49.60% of CPAP patients had hypertension compared to 16.50% of the surgery group. This was a consistent trend for all baseline comorbidities in the study. Even though the study mentioned the significant “differences between age, sex and the presence of comorbidity between both cohorts before matching,” this point may need further emphasis. Risk factors for developing diabetes include obesity, a sedentary lifestyle, and lack of physical activity [2,3,4]. Furthermore, a sedentary lifestyle and inactivity are also correlated to cardiovascular co-morbidities such as hypertension and dyslipidemia [5]. These are important risk factors that should be considered in propensity score matching while building cohorts. The authors noted that all cohorts were matched for age, sex, and co-morbidities, but the large difference in co-morbidities in the CPAP group indicated that these patients likely had other co-morbidities and behavioral factors that were not considered [6]. While authors matched for many relevant comorbidities, the results may still be biased due to the nature of the retrospective study.

Secondly, we agree with the authors that for a multi-year retrospective study, it is imperative to account for follow-up time in selecting cohorts. O’Connor-Reina et al. ensured that all patients had 5-year follow-up from the date of OSA diagnosis, but we believe that selecting the index event as the diagnosis date of OSA, instead of time of CPAP initiation or surgery date, was another limitation. Index events describe the initial occurrence or presentation of a medical condition. It marks the beginning of a diagnosis or treatment and is different for each patient [7, 8]. Since O’Connor-Reina et al. included data “obtained from up to 20 years ago,” a patient diagnosed with OSA in 2003 may have contributed data to this study during 2003–2008. However, it is possible that they didn’t receive UAS until 2020. Therefore, they would have been incorrectly included in the surgical cohort in this study. As such, we suggest the index date should be set to the date of procedure to compare the efficacy of two treatments [9, 10].

Coding and cohort queries

The ICD-10-PCS codes for Upper Airway Surgeries listed in Table 1 in the O'Connor-Reina study may nonspecifically represent surgeries for OSA or other diagnoses. Upper airway surgeries have many indications in both children and adults, including chronic bacterial tonsillitis, chronic ear infections, etc. [11,12,13]. Therefore, it is difficult to be certain that documented upper airway surgeries were intended to treat OSA and not other conditions, as the authors noted in the limitations section. One way to minimize this confounding variable is to set a time relation by linking the diagnosis code of OSA within the same day that patients receive UAS treatments. This way, the specificity of the targeted patient population can be improved.

In reviewing the diagnosis codes utilized to build cohorts, we noticed that the cohorts may not be entirely composed of patients of interest. When capturing patients who were prescribed CPAP, the authors used the codes ICD-10-PCS 5A09357, ICD-10-PCS 5A09457, and ICD-10-PCS 5A09557, noting that two of the codes mandate continuous use for more than 24 h which would not be home CPAP patients (Table 1). In the United States, ICD-10-PCS codes are used only to classify procedures performed in an inpatient setting [14]. Inpatient CPAP treatment is indicated for respiratory distress syndrome and respiratory failure, suggesting a sicker patient population [15,16,17,18]. Since O’Connor-Reina et al. utilized the Global Collaborative Network in TriNetX, which captures patients globally, this study likely included hospitalized patients with serious conditions. Even though OSA treatment using CPAP therapy may require hospital titration, solely using ICD-10-PCS codes may exclude the general OSA patients using CPAP at home [19]. This has significant implications on the analysis of comorbidities and mortality. On this note, studies have used CPT codes that more appropriately characterizes regular CPAP use because they require physicians to order a CPAP machine and perform face-to-face patient care like mask fitting, titration pressure, and instruction on how to use the machine [20, 21]. That said, we recognize the challenges in gathering the correct codes for CPAP treatment. Since there is no unifying guideline delineating the coding differences across multiple countries, it is difficult to perfectly identify patients receiving CPAP treatment for OSA in a global retrospective study. We also acknowledge that ICD-10-PCS codes are also primarily used in Europe, while CPT codes are the standard for billing in the United States. As there is no translation for CPT into PCS codes for US data, researchers should be careful in choosing the appropriate codes for their study.

Table 1 ICD code for CPAP use. Reproduced from O’Connor-Reina et al. [1]
Full size table

OSA treatment efficacy and TriNetX limitations

One of the advantages of large retrospective databases like TriNetX is its ability to provide a bigger sample size, allowing researchers to study rare diseases on a large scale. However, one of their drawbacks is the dependence on diagnosis and procedural codes. As the the authors pointed out in the limitations section, TriNetX “records for CPAP did not include data for the adherence and acceptance of this therapy.” Studies have shown short term CPAP treatment or poor CPAP adherence does not result in decreased diabetic rates, while CPAP adherence and consistent long term CPAP treatments are associated with decreased risk of diabetes [22,23,24,25,26]. Therefore, without a means to accurately measure patient compliance, the results might be biased. Despite a large difference in risk development of new diabetes diagnosis between CPAP and UAS, readers should take cautions when concluding that surgery has a clinically significant role.

Updated risk of diabetes methods and results

We developed more comprehensive and balanced cohorts to study the risk of diabetes in patients with OSA. The TriNetX database was queried to identify OSA patients over 18 years of age (ICD-10 G47.30 and G47.33). To ensure surgeries were indicated for OSA, patients with head and neck neoplasms were excluded. Patients with a BMI ≥ 35 kg/m2 were recommended to undergo weight loss surgery prior to UAS [27]. Thus, these patients were also excluded from the study. Two cohorts were built based on the ICD and CPT codes to ensure comprehensive coverage of CPAP use or UAS (Table 2). In the CPAP cohort, patients with any head and neck surgeries were excluded. Likewise, any patients in the UAS cohort who received CPAP after the procedure were excluded. Only patients with at least 5 years of follow-up after treatment date were included in the study. To balance for confounding variables, 1:1 propensity score matching was performed on patient demographics (age, sex, race, ethnicity) and co-morbidities (Tables 3 and 4). Baseline characteristic comparison and relative risk analysis were performed. Kaplan–Meier analysis was used to estimate 5-year “survival rate” of not developing diabetes. We would caution the interpretation of this analysis as it does not represent survival, but instead depicts developing the outcome of interest—in this case, developing diabetes [28].

Table 2 Updated ICD-10 and CPT codes of CPAP and UAS
Full size table
Table 3 Diagnosis characteristics used in propensity score match between CPAP and UAS cohorts
Full size table
Table 4 Demographic and clinical characteristics of the study population (n = 65,881)
Full size table

Of all patients greater than 18 years of age who had at least 5 years of follow-up after treatment initiation, there were 59,787 and 9224 patients in the CPAP and UAS cohorts, respectively. After 1:1 propensity score matching and excluding patients who did not satisfy inclusion criteria, there were 6651 patients in each cohort. The mean age at index was 46 years of age. Both groups included about 58% male with 64% white, 15% black, and 8% Hispanic. There were 491 (10.3%) patients in the CPAP group and 404 (7.9%) patients in the UAS group with new onset diabetes (Fig. 1). Within the CPAP group, the number of patients with a new diagnosis of diabetes after treatment in the UAS group was significantly lower than the CPAP group (risk ratio RR = 1.31, 95% confidence interval CI = [1.15, 1.48], p < 0.0001). Figure 2 shows a 5-year probability of not developing diabetes in CPAP vs UAS cohort (87.7% vs 91.2%, hazard ratio = 1.43, 95% CI = [1.25, 1.63]).

Fig. 1
figure 1

Risk analysis after excluding patients with the outcome (Diabetes) prior to time window

Full size image
Fig. 2
figure 2

Kaplan Meier plot comparing outcome of diabetes after five years of follow up in both cohorts

Full size image

Our results showed that with a comprehensive usage of codes and extensive co-morbidities matching, there is still a statistically significant reduction in the risk of developing new diabetes in the UAS cohort compared to CPAP group, though the absolute risk difference may not be as clinically relevant.

Conclusion

In conclusion, we feel strongly that there are limitations in the study published by O'Connor-Reina et al. which bias the comparison of UAS versus CPAP. We commend the authors for studying an important topic and we appreciate their consideration of the points we have made here. By facilitating a balanced discussion on this topic, we can advance the understanding and management of OSA.