Highlights

  • Thrombosis risk assessment remains an ongoing challenge for acutely ill hospitalized medical patients.

  • In this external validation study, the ADA (Age–D-dimer–Albumin) score demonstrated good calibration but suboptimal discrimination for predicting thrombosis among medical inpatients.

  • The ADA score was modestly superior to the IMPROVE score in discriminating venous thrombosis, suggesting that the ADA score can be considered as a risk assessment tool.

Introduction

Thrombosis is the leading cause of death worldwide and the common underlying mechanism of myocardial infarction (MI), ischemic stroke (IS), and venous thromboembolism (VTE) [1]. For acutely ill hospitalized medical patients, several VTE risk assessment models have been developed to identify high-risk subsets that may benefit from thromboprophylaxis [2].

The development of a risk assessment model can be divided into three stages [3]. In the first stage (i.e., derivation), factors that demonstrate predictive ability are identified from the study cohort, and a multivariable model comprised of the identified predictors is constructed. During this stage, the model performance is assessed by metrics of calibration (i.e., the agreement between the observed risk and the predicted risk) and discrimination (i.e., the ability to differentiate events from non-events) [4]. In the second stage (i.e., validation), the reproducibility of model performance is examined in clinically relevant settings or populations. In the third stage (i.e., impact analysis), the change in physician behaviors, patient outcomes, and healthcare costs associated with the utilization of a model is evaluated. The three stages provide the evidence base that determines whether a model can be applied to clinical practice.

In light of the elevated thrombotic risk among patients hospitalized with coronavirus infectious disease-19 (COVID-19) pneumonia [5], the ADA (Age–D-dimer–Albumin) score identifies high-risk patients for arterial and venous thrombosis in the COVID-19 setting [6] It remains unclear whether the ADA score can be used for thrombotic risk prediction in non-COVID-19 medical inpatients and whether it is comparable to the IMPROVE (International Medical Prevention Registry on Venous Thromboembolism) score [7] Our study aimed to externally validate the ADA score in a medically ill population from the APEX (Acute Medically Ill VTE Prevention with Extended Duration Betrixaban) trial [8].

Methods

Study population

The APEX trial was a multinational, randomized, double-blind, active-controlled Phase 3 clinical trial that compared extended-duration betrixaban (80 mg once daily for 35 to 42 days) to shorter-duration enoxaparin (40 mg once daily for 10 ± 4 days) among acutely ill hospitalized patients at risk for VTE [8]. Patients were considered for participation if they were: (1) hospitalized for an acute medical illness, including acute decompensated heart failure, acute respiratory failure, acute infection, acute rheumatic disorder, acute disabling ischemic stroke; (2) age ≥ 75 years, age 60 to 74 years with D-dimer ≥ 2 × the upper limit of normal (ULN), or age 40 to 59 years with D-dimer ≥ 2 × ULN and a history of VTE or cancer; (3) anticipated severe immobilization for ≥ 24 h followed by moderate or severe immobilization for three or more days; and (4) anticipated hospitalization for three or more days. Key exclusion criteria were: (1) a recent history of clinically significant bleeding or severe trauma; (2) requiring major surgery or invasive procedure within three months; (3) end-stage renal disease with creatinine clearance < 15 mL/min or requiring dialysis; (4) anticipated need for prolonged anticoagulation or contraindication to anticoagulation; and (5) concomitant dual antiplatelet therapy with any two of the following: aspirin, dipyridamole, or any thienopyridine (i.e., clopidogrel, prasugrel, ticlopidine, and ticagrelor). All randomized patients with non-missing baseline D-dimer or albumin measurements and evaluable endpoints (N = 7,119) were included in the analysis.

All study procedures involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments. Informed consent was obtained from all individual participants included in the APEX trial.

Study Endpoints

In the present analysis, the occurrence of the following endpoints through the end of the study at 77 days was evaluated: (1) arterial and venous thrombosis (defined as the composite of cardiovascular death [CVD], MI, IS, VTE-related death [VD], or VTE); (2) arterial thrombosis (defined as the composite of CVD, MI, or IS); (3) venous thrombosis (defined as the composite of VD or VTE). CVD was defined as cardiovascular death related to MI or IS. VTE was defined as non-fatal pulmonary embolism or symptomatic deep vein thrombosis. All events, except for MI, were adjudicated by an independent clinical events committee blinded to thromboprophylaxis allocation.

Risk Assessment Models

The performance of two risk assessment models (ADA [Age–D-dimer–Albumin] score and IMPROVE [International Medical Prevention Registry on Venous Thromboembolism] score) was examined in the present study [6, 7].

The ADA score was developed to predict arterial and venous thrombosis in adult patients hospitalized with laboratory-confirmed COVID-19 and severe acute respiratory syndrome coronavirus 2-related pneumonia. It was derived and internally validated from an observational retrospective multi-center cohort study [6]. The ADA score was calculated by the following formula: ADA score = 0.3 * age in years + 0.004 * D-dimer in ng/mL − 0.5 * albumin in g/L. To obtain an ADA score ranging from 0 to 100, min-max scaling transformation was applied, where the following ranges defined the minimum and maximum values: 18 to 100 years for age; 1 to 4750 ng/mL for D-dimer; 1 to 100 g/L for albumin. The optimal cut-off for discriminating patients with vs. without thrombotic events using the ADA score was 49, as proposed by the original derivation study.

The IMPROVE score was developed to predict venous thrombosis in acutely ill hospitalized medical patients and was derived from an observational prospective international multi-center cohort study and externally validated in non-COVID-19 medically ill patients [7, 9]. The IMPROVE score was calculated by adding 3 points for previous VTE, 2 points for known thrombophilia, 2 points for lower-limb paralysis, 2 points for cancer, 1 point for immobilization ≥ 7 days, 1 point for intensive care unit or coronary care unit stay, and 1 point for age > 60 years. The IMPROVE score ≥ 2 corresponded to a VTE event rate of ≥ 1%, as recommended for warranting pharmacological thromboprophylaxis by the American College of Chest Physicians guidelines [7]. As prespecified in the Statistical Analysis Plan, the IMPROVE score was calculated to categorize study participants into low- vs. high-risk groups in the APEX trial. Therefore, the IMPROVE score was selected as the comparator in the present analysis.

Statistical analysis

Demographic, clinical characteristics, treatment, and laboratory measurements were expressed as mean and standard deviation (SD) or median and interquartile range (IQR) for continuous variables according to their distribution, and count and percentages for categorical ones. The comparison was made between patients who experienced thrombotic events and those who did not experience thrombotic events using the χ2 test for categorical variables and Student’s t-test or Mann–Whitney U test for continuous variables as appropriate.

We assessed the calibration and discrimination of the ADA score and the IMPROVE score. Detailed statistical approaches are provided in the Appendix. In brief, calibration was assessed with the Hosmer–Lemeshow test by grouping cases into deciles of risk. Calibration plots were constructed to compare predicted risks to observed risks. Discrimination was assessed with c statistic (i.e., area under the receiver operating characteristic [ROC] curve). Comparative discrimination between the ADA vs. IMPROVE score was assessed with ∆ c statistic (i.e., the difference in c statistic), integrated discrimination improvement (IDI), and continuous net reclassification improvement (NRI). Lastly, patients were stratified into the at-risk category vs. low-risk category at the relevant decision cutoffs (i.e., 49 points for the ADA score and 2 points for the IMPROVE score). Event rates between at-risk vs. low-risk categories according to the ADA score and the IMPROVE score were compared. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of both scores were also calculated.

All analyses were performed independently by the PERFUSE Study Group using the SAS software version 9.4 (SAS Institute, Inc., Cary, North Carolina, United States).

Results

Among 7,119 hospitalized medical patients, 209 (2.9%) experienced thrombosis up to 77 days of follow-up. There were 120 (1.7%) arterial thromboses, including 65 (0.9%) cardiovascular deaths, 34 (0.5%) MI events, and 49 (0.7%) IS events. There were 96 (1.3%) venous thromboses, including 40 (0.6%) VTE-related deaths and 60 (0.8%) VTE events (i.e., non-fatal pulmonary embolism and symptomatic deep vein thrombosis).

The baseline characteristics of study participants are summarized in Table 1. Compared to those who did not have thrombosis, patients who had thrombosis were more likely to have creatinine clearance < 90 mL/min, acute ischemic stroke as the admitting diagnosis, lower-limb paralysis, and intensive or coronary care unit stay. In addition, the median of D-dimer is higher (1820 vs. 1280 ng/mL), and the median of albumin is lower (37 vs. 39 g/L) among patients who had thrombosis vs. those who did not have thrombosis. The proportion of patients who received betrixaban was higher in the group that did not develop thrombosis than in the group that developed thrombosis (50.2% vs. 37.8%; p < 0.001). The finding was consistent with the previous studies that supported the superior efficacy of betrixaban over enoxaparin in preventing venous or arterial thrombosis events [10, 11].

Table 1 Baseline characteristics of study participants
Full size table

The performance of the ADA and IMPROVE scores are summarized in Table 2. With respect to calibration, the ADA score demonstrated good calibration for both arterial and venous thrombosis, whereas the IMPROVE score had adequate calibration for venous thrombosis (p > 0.05 from the Hosmer–Lemeshow test). Calibration plots of the ADA score and the IMPROVE score are provided in Figure S1 to Figure S6. The discriminative ability of the ADA score was generally comparable to the IMPROVE score, as indicated by the c statistic. For discriminating arterial and venous thrombosis (Fig. 1), there was no significant difference between the ADA score vs. IMPROVE score (c statistic = 0.620 [95% CI: 0.582 to 0.657] vs. 0.590 [95% CI: 0.556 to 0.624]; ∆ c statistic = 0.030 [95% CI: −0.022 to 0.081]; p = 0.255). Similarly, for discriminating arterial thrombosis (Fig. 2), there was no significant difference between the ADA score vs. IMPROVE score (c statistic = 0.582 [95% CI: 0.534 to 0.629] vs. 0.609 [95% CI: 0.564 to 0.653]; ∆ c statistic = −0.027 [95% CI: −0.091 to 0.036]; p = 0.397). For discriminating venous thrombosis (Fig. 3), the ADA score demonstrated a modestly greater ability compared to the IMPROVE score (c statistic = 0.664 [95% CI: 0.607 to 0.722] vs. 0.573 [95% CI: 0.521 to 0.624]; ∆ c statistic = 0.091 [95% CI: 0.011 to 0.172]; p = 0.026). In addition, the ADA score demonstrated a greater ability to separate the predicted probabilities between patients with thrombosis vs. those without thrombosis compared to the IMPROVE score assessed by IDI. There was no significant difference in classification accuracy as assessed by NRI.

Table 2 Comparison of model performance between the ADA score and the IMPROVE score in predicting thrombosis
Full size table
Fig. 1
figure 1

Receiver operating characteristic curve of the ADA vs. IMPROVE score in predicting arterial and venous thrombosis

Full size image
Fig. 2
figure 2

Receiver operating characteristic curve of the ADA vs. IMPROVE score in predicting arterial thrombosis

Full size image
Fig. 3
figure 3

Receiver operating characteristic curve of the ADA vs. IMPROVE score in predicting venous thrombosis

Full size image

A comparison of event rates between at-risk vs. low-risk categories according to the ADA and IMPROVE scores was summarized in Table 3. Patients stratified into the at-risk category by the ADA score had a significantly higher risk for all thrombotic endpoints than those in the low-risk category (OR = 2.295 [95% CI: 1.736 to 3.033] for arterial and venous thrombosis; OR = 1.817 [95% CI: 1.266 to 2.609] for arterial thrombosis; OR = 3.013 [95% CI: 1.978 to 4.590] for venous thrombosis). Similarly, except for IS and VTE, patients in the at-risk category by the IMPROVE score had a higher risk than those in the low-risk category (OR = 2.328 [95% CI: 1.762 to 3.075] for arterial and venous thrombosis; OR = 2.840 [95% CI: 1.978 to 4.079] for arterial thrombosis; OR = 1.903 [95% CI: 1.259 to 2.874] for venous thrombosis).

Table 3 Comparison of event rate between at-risk vs. low-risk categories according to the ADA score (ADA ≥ 49 vs. ADA < 49) or the IMPROVE score (IMPROVE ≥ 2 vs. IMPROVE < 2)
Full size table

Sensitivity, specificity, PPV, and NPV of the ADA and IMPROVE scores were summarized in Table 4. For arterial and venous thrombosis, the sensitivity and specificity of the ADA score vs. IMPROVE score were 0.579 (95% CI: 0.512 to 0.646) vs. 0.440 (95% CI: 0.373 to 0.507) and 0.625 (95% CI: 0.614 to 0.637) vs. 0.747 (95% CI: 0.737 to 0.758), respectively. For arterial thrombosis, the sensitivity and specificity of the ADA score vs. IMPROVE score were 0.525 (95% CI: 0.436 to 0.614) vs. 0.492 (95% CI: 0.402 to 0.581) and 0.622 (95% CI: 0.610 to 0.633) vs. 0.746 (95% CI: 0.736 to 0.756), respectively. For venous thrombosis, the sensitivity and specificity of the ADA score vs. IMPROVE score were 0.646 (95% CI: 0.550 to 0.742) vs. 0.396 (95% CI: 0.298 to 0.494) and 0.623 (95% CI: 0.612 to 0.634) vs. 0.744 (95% CI: 0.734 to 0.754), respectively. Both scores had similarly low PPV and high PPV for predicting thrombosis.

Table 4 Sensitivity, specificity, PPV, and NPV of the ADA score and the IMPROVE score in predicting thrombosis
Full size table

Discussion

In this external validation study of acutely ill hospitalized medical patients from the APEX trial, the ADA score demonstrated good calibration but suboptimal discrimination for detecting thrombosis. Additionally, the ADA score was modestly superior to the IMPROVE score in discriminating venous thrombosis, suggesting that the ADA score can be considered a VTE risk assessment tool. At the relevant decision cutoffs, both scores demonstrated significant risk differences between the at-risk vs. low-risk category. Of note, the ADA score had higher sensitivity but lower specificity than the IMPROVE score for predicting arterial or venous thrombosis (except for CVD), suggesting that the ADA score likely intercepts a more general population. However, low PPV and high NPV were consistently observed for all endpoints from both scores, indicating that a small proportion of the high-risk patients experienced thrombosis, and most low-risk patients did not experience thrombosis. This may be explained by the relatively low event rates of thrombosis (1.7% for arterial thrombosis and 1.3% for venous thrombosis) in the study population with the universal provision of pharmacological thromboprophylaxis.

The present study also revealed weaker discrimination of the ADA score compared to the derivation study (c statistic = 0.620 vs. 0.752) [6]. This may be attributed to differences in outcome incidence and case mix (i.e., distribution of predictor values) between the derivation and validation cohorts. Note that the ADA score was derived from hospitalized COVID-19 patients with a much higher incidence of thrombotic events (16.3%). The difference in case mix between COVID-19 and non-COVID-19 patients was further complicated by the enrichment strategy of the APEX trial, which preferentially enrolled patients with baseline D-dimer ≥ 2 × ULN or age ≥ 75 years as these patients were more likely to derive clinical benefit from thromboprophylaxis.

Risk scores provide a personalized risk assessment to assist patients and clinicians in making shared health decisions. In the context of acutely ill hospitalized medical patients, utilization of VTE risk scores may assist decision-making in thromboprophylaxis and reduce under-prophylaxis in high-risk patients or over-prophylaxis in low-risk patients. While the study validates the usefulness of the ADA score and the IMPROVE score, several limitations should be considered. First, the study was a post hoc analysis of a population that agreed to participate in a clinical trial with specific enrollment criteria. The APEX trial enrolled medical inpatients at increased risk of VTE that may benefit from pharmacological thromboprophylaxis. In contrast, the real-world settings include patients with a heterogeneous predisposition to VTE that may or may not be eligible for pharmacological thromboprophylaxis. Given the difference in VTE risk profile, the generalizability to real-world patient populations remains uncertain. Second, asymptomatic deep vein thrombosis was assessed by trial-mandated compression ultrasound but not included as a component of thrombosis in the present analysis. Third, albumin level may not be measured upon hospital admission. Compared to albumin, hemoglobin may be more available and has demonstrated a comparable magnitude of association with symptomatic VTE in the APEX population [12, 13]. Future research should explore whether other laboratory tests can be used when albumin measurement is not obtained. Last, MI events were investigator-reported without adjudication by an independent clinical events committee.

Conclusion

Among acutely ill hospitalized medical patients enrolled in the APEX trial, the ADA score demonstrated good calibration but suboptimal discrimination for predicting thrombosis. The findings support the use of either the ADA or IMPROVE score for thrombosis risk assessment. The applicability of the ADA score to non-COVID-19 populations warrants further research.