Predictors of 30-Day Unplanned Readmission After Carotid Artery Stenting Using Artificial Intelligence

This article has been updated

Abstract

Introduction

This study aimed to describe the rates and causes of unplanned readmissions within 30 days following carotid artery stenting (CAS) and to use artificial intelligence machine learning analysis for creating a prediction model for short-term readmissions. The prediction of unplanned readmissions after index CAS remains challenging. There is a need to leverage deep machine learning algorithms in order to develop robust prediction tools for early readmissions.

Methods

Patients undergoing inpatient CAS during the year 2017 in the US Nationwide Readmission Database (NRD) were evaluated for the rates, predictors, and costs of unplanned 30-day readmission. Logistic regression, support vector machine (SVM), deep neural network (DNN), random forest, and decision tree models were evaluated to generate a robust prediction model.

Results

We identified 16,745 patients who underwent CAS, of whom 7.4% were readmitted within 30 days. Depression [p < 0.001, OR 1.461 (95% CI 1.231–1.735)], heart failure [p < 0.001, OR 1.619 (95% CI 1.363–1.922)], cancer [p < 0.001, OR 1.631 (95% CI 1.286–2.068)], in-hospital bleeding [p = 0.039, OR 1.641 (95% CI 1.026–2.626)], and coagulation disorders [p = 0.007, OR 1.412 (95% CI 1.100–1.813)] were the strongest predictors of readmission. The artificial intelligence machine learning DNN prediction model has a C-statistic value of 0.79 (validation 0.73) in predicting the patients who might have all-cause unplanned readmission within 30 days of the index CAS discharge.

Conclusions

Machine learning derived models may effectively identify high-risk patients for intervention strategies that may reduce unplanned readmissions post carotid artery stenting.

Central Illustration

Figure 2: ROC and AUPRC analysis of DNN prediction model with other classification models on 30-day readmission data for CAS subjects

Plain Language Summary

We present a novel deep neural network-based artificial intelligence prediction model to help identify a subgroup of patients undergoing carotid artery stenting who are at risk for short-term unplanned readmissions. Prior studies have attempted to develop prediction models but have used mainly logistic regression models and have low prediction ability. The novel model presented in this study boasts 79% capability to accurately predict individuals for unplanned readmissions post carotid artery stenting within 30 days of discharge.

FormalPara Key Summary Points
We present a novel deep neural network-based artificial intelligence prediction model to help identify a subgroup of patients undergoing carotid artery stenting who are at risk for short-term unplanned readmissions.
Prior studies have attempted to develop prediction models but have used mainly logistic regression models and have low prediction ability.
The novel model presented in this study boasts 79% capability to accurately predict individuals for unplanned readmissions post carotid artery stenting within 30 days of discharge.

Digital Features

This article is published with digital features, including a summary slide and plain language summary, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.14198939.

Introduction

Strokes secondary to thromboembolism from an atherosclerotic plaque at the carotid bifurcation or the internal carotid artery (ICA) account for 10–15% of all strokes worldwide, and carotid artery stenting (CAS) in a timely fashion may help prevent such strokes [1, 2]. Unplanned readmissions due to all causes are an important aspect from a patient care perspective and quality metric under the Patient and Protection and Affordable Care Act. Prediction of early readmission is crucial in helping plan the delivery of healthcare services and identifying high-risk patients for intervention strategies to reduce readmissions and provide cost conscientious care to the community [3, 4].

Most of the prior studies evaluating carotid revascularization have focused on comparing the CAS with carotid end-arterectomy (CEA) in terms of outcomes, including stroke, major adverse cardiovascular and cerebrovascular events (MACCE), death, or readmission. Furthermore, these studies and others evaluating readmissions used data before 2014 when the International Classification of Diseases, 9th Revision (ICD-9) was in practice. To the best of our knowledge, there is no study that has specifically evaluated CAS for early readmissions using the strength of the International Classification of Diseases, 10th Revision (ICD-10) diagnosis codes and for more recent patient-level data [5, 6].

The specificity of diagnostic and procedure codes has immensely improved with the advent of the ICD-10 codes, which can give us better insight into the causes and associations of unplanned readmissions [7]. With the advent of machine learning methods, there has been a gradual adaptation of those in healthcare, and they are also found to be superior to standard prediction rules for hospital readmissions [8]. We present a nationwide evaluation of early unplanned readmission and an artificial intelligence-assisted readmission prediction model for patients undergoing CAS.

The purpose of this study was to describe the rates and causes of unplanned readmissions within 30 days following CAS, and we aimed at using artificial intelligence machine learning analysis to develop a robust prediction model for short-term readmissions.

Methods

The US Nationwide Readmission Database (NRD) is a nationally representative sample of all-age, all-payer discharges from US nonfederal hospitals produced by the Healthcare Cost and Utilization Project (HCUP) of the Agency for Healthcare Research and Quality (AHRQ) [9]. This database is composed of discharge-level hospitalization data from 28 geographically dispersed states across the USA. It has approximately 18 million discharges for the year 2017 (weighted estimated to roughly 36 million discharges). The dataset used in the present study represents 60% of the US population and 58.2% of all US hospitalizations.

Individual patients in the NRD are assigned up to 40 diagnosis codes and 25 procedure codes for each hospitalization. We defined CAS with the procedure codes from ICD-10 listed in Table 1 in the supplementary material. The primary outcome was first unplanned readmission within 30 days of the index discharge after the first CAS. If a patient had multiple CAS in a year, only the first intervention was used for the analysis. The cohort patients admitted in the month of December for the index admission were also excluded, as they may not have 30 days of follow-up, leading to immortal time bias. All ICD-10 and Clinical Classification Software Refined (CCSR) codes used in this study are presented in Table 2 in the supplementary material. Statistical analysis was performed using IBM SPSS Statistics V26 using two-sided tests and a significance of less than 0.05. We used the Pearson–chi-square test for categorical variables, Mann–Whitney U test for continuous variables with no readmission as the reference group, and logistic regression for predictions.

Compliance with Ethics Guidelines

This study was performed utilizing publicly available datasets and hence does not require IRB review under 45 CFR 46. In addition, the Institutional Review Board (IRB) at the University of South Alabama approved the study for exempt status.

Model Development

A prediction model was developed in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) initiative guidelines [10].

Data Preprocessing

The preprocessing of CAS readmission records was conducted to make the data at hand represent nationwide readmission rates. The data at hand involved 9019 individual patients (weighted analysis to 16,827 patients for national representation).

Description of Outcome Variable

The outcome of interest was readmission (yes/no) within 30 days from discharge after CAS. Among all subjects undergoing CAS, 7.4% of subjects were readmitted, while 93.6% were not readmitted.

Description of Predictor Variables

A total of 42 clinically pertinent variables were included for model development. These included patient demographics, insurance status, hospital bed size, teaching hospital teaching status, length of stay, and relevant comorbidities. We performed exploratory analysis (Table 1) by evaluating all categorical and continuous variables. Additionally, we evaluated for missing data and zero variances and found no zero variance categorical predictors along with any missing values among all the predictors. The continuous variables were standardized to zero mean and unit variance. The categorical variables were dummy coded using one-hot encoding, which were then used as inputs for the prediction model.

Table 1 Baseline characteristics and procedure-related factors during index admission for CAS

Model Specification

Deep neural network (DNN) is a machine learning algorithm that processes data with complex architectures to perform classification and build prediction models. Briefly, DNN is an artificial neural network with one input layer, at least four hidden layers, and one output layer [11]. Fully connected DNN conventionally uses a feed forward learning mechanism where the information flows from the input layer, through the hidden layers, towards the output layer to learn complex data patterns, extract important features, and make predictions. DNN uses nonlinear transformations to construct such prediction models [12, 13]. To reveal patterns from large datasets and investigate those patterns with biological questions of interest, a key factor is to set up a DNN model properly. For this end, we choose the proper choice of hyper-parameters, such as the number of hidden layers and the type of loss function using a grid search using Grid_SearchCV [14] function on a range of values of the hyper-parameters and then choose the combination which results in the best accuracy. In our model setup, we used (1) hidden layers ranging between 1 and 100 with a step size of 1 and between two loss functions “adam” and “rmsprop”, where “adam” function performed the best and with four as the number of hidden layers. (2) To choose the number of nodes on each hidden layer since we had 91 input variables after dummy coding, we chose the number of nodes to be 128 per hidden layer, as the rule of thumb that recommends the number of chosen nodes to be less than twice the size of the input. (3) For nonlinear activation function to the hidden layers rectified linear unit (relu) was used and sigmoid function as the output layer’s activation function.

Evaluation Strategy

Addressing Class Imbalance

Only 7.4% of the CAS patients were readmitted within 30 days in our study, highlighting a very low readmission rate as often seen with medical data. Particularly, in our dataset, we observed that the proportion of non-readmitted patients (93.6%) was 13 times more than that of readmitted patients (7.4%), clearly making our data highly class imbalanced. To address this class imbalance problem, we adjusted the minority class (patients who were readmitted) weight to 13 times the majority class’s weight (patients who were not readmitted). This adjustment inherently constructs a model with better generalization. These class weights are used during the DNN model’s training to increase the misclassification cost of minority class samples. This makes the training process pay more attention towards the minority class samples, thus increasing the sensitivity of the prediction model.

Incorporating AUPRC as a Metric of Performance Evaluation

Inferring model performance from area under receiver operating characteristic curve (AUROC) can be deceptive in imbalanced datasets like ours where only 7.4% of patients were readmitted while 93.6% were not readmitted. Specifically, in an imbalanced setup, since the number of negative samples (non-readmitted patients) is very large, the false positive rate increases more slowly because the true negatives would probably be very high and make this metric smaller. Therefore, a receiver operating characteristic (ROC) curve can have a better result while misclassifying most or all the minority class. However, precision is not affected by a large number of negative samples, which is because it measures the number of true positives out of the samples predicted as positives. Precision is more focused on the positive class than in the negative class; it actually measures the probability of correct detection of positive values, while false positive rate (FPR) and true positive rate (TPR) (ROC metrics) measure the ability to distinguish between the classes. In contrast, area under the precision recall curve (AUPRC) scores are specially designed to detect rare events and are appropriate in these scenarios as they particularly show a classifier having a low performance if it is misclassifying most or all the minority class [15, 16].

Splitting Data for Training Models

We divided the dataset into three parts, 70% for training, 10% for validation, and 20% for testing. Firstly, we build the model using 70% of the data by obtaining the optimized weight of each node. Secondly, to evaluate our built model’s fit, we apply the model on a validation set to check the prediction accuracy, to which we found it to be at 90.86%. Finally, the remaining 20% of data was used to test the performance of the model. The performance of the model was evaluated using accuracy, precision, sensitivity, specificity, and AUROC. Additionally, we used ELI5 to predict the importance weight of each feature. To determine variable importance, ELI5 uses the idea of permutation importance, which randomly shuffles the values of a single feature and makes a prediction using the shuffled dataset. The prediction accuracy using the shuffled dataset and original dataset are then compared to enumerate the performance degradation. This process is conducted till the importance of all the features is computed.

Results

Population Characteristics and Descriptive Results

The NRD included 18,882 individual patients undergoing CAS from January through December 2017 among hospitalized patients 18 years of age and older. We excluded those who died during the index hospitalization (n = 251) and were discharged after November 30, or had missing data on length of stay. The final study sample included 16,745 patients who were discharged alive after index CAS from January through November 2017.

After index CAS, 7.4% patients (n = 1233) had 30-day readmission. Notably, 8.6% of patients returned with acute cerebral infarction, 4.2% acute hemorrhagic cerebrovascular disease, and 4.7% of patients died during the unplanned 30-day readmission encounter. Table 1 gives a detailed synopsis of the baseline and procedure-related factors associated with early readmission. Forest plot analysis of comorbidities and procedure-related factors affecting 30-day readmission is presented (Fig. 1).

Fig. 1
figure1

Forest plot analysis of comorbidities and procedure-related factors affecting 30-day readmission after carotid artery stenting

Cardiac causes made up 39.8% of all readmissions. Ten leading causes and frequencies of primary diagnosis category for readmissions encounters based on Clinical Classification Software Refined (CCSR) categories are presented in Table 2. Total charges of care for the index and 30-day unplanned admissions amount to over $1.9 billion and $66 million, respectively.

Table 2 Causes and frequencies of primary diagnosis category for readmissions encounters [based on the primary Clinical Classification Software Refined (CCSR)]

Performance Comparison of Deep Neural Network with Other Machine Learning Models

We evaluated the proposed DNN model’s performance with four other frequently used machine learning algorithms in medical data, such as logistic regression, random forest, decision tree, and support vector machine. Firstly, we found that its accuracy was 92.57% for logistic regression, with AUROC of 0.68 and AUPRC of 0.14. Secondly, the accuracy was 55.26% for random forest, with AUROC and AUPRC of 0.611 and 0.367, respectively. Thirdly, for the decision tree, the accuracy was 78.19%, with AUROC of 0.588 and AUPRC of 0.269. Lastly, for the support vector machine, the accuracy was 70.35% with AUROC of 0.67 and AUPRC of 0.14. DNN, on the other hand, produced an accuracy of 87.43% but notably had a higher AUROC of 0.79 (validation 0.73) and AUPRC of 0.383 compared to all other models (Table 3). A graphical representation of the performance (AUROC and AUPRC) of the proposed DNN prediction model with other classification models on 30-day readmission data for CAS subjects is shown in Fig. 2.

Table 3 Machine learning algorithms and accuracy in predicting early readmission post CAS
Fig. 2
figure2

ROC and AUPRC analysis of DNN prediction model with other classification models on 30-day readmission data for CAS subjects. Plot of prediction capability of machine learning models

Predictors of 30-Day Readmission and Costs

To identify the variables that had a higher contribution to our DNN models predictive power, we applied ELI5 to identify such variables. Specifically, using Eli5, we obtained importance weights for each of the 42 model variables used in this analysis and then ranked them using their respective percentile score. Variables with their importance weight ranked in the top 20 percentiles (ranging from 0.0116 to 0.0067) were then selected as the most influential/important variables in our model (Table 4). Following this procedure, we obtained ten most important variables to predict 30-day readmission (Fig. 3).

Table 4 Important score of each variable in early readmission post CAS dataset
Fig. 3
figure3

Bar graph diagram showing relative importance of predictors for unplanned readmission

Discussion

This study, which examines the rate and costs of 30-day readmissions after index CAS, finds that 7.4% of patients get readmitted within 30 days of discharge after undergoing CAS. The major causes for 30-day unplanned readmission were septicemia or cerebral infarction/hemorrhagic cerebrovascular bleed. Using machine learning approaches, we can develop a risk prediction model that can identify patients at high risk of unplanned readmissions with a C-statistic of 0.802 using DNN, to the best of our knowledge, the first of its kind in many ways. This is the most contemporary analysis looking at 30-day unplanned readmissions. The first study uses nationally representative data to develop risk prediction models using advanced machine learning like DNNs for CAS. On the basis of AUROC and AUPRC metrics, DNN shows superior performance to commonly used statistical or machine learning methods in modeling CAS readmission rates.

Prior studies looking at readmissions have shown a variety of readmission rates, including vascular interventions in Medicare patients (24%) [17], endovascular aortic aneurysm repair (10.2%) [18], lower extremity bypass (14.8%) [19], endovascular or surgical revascularization for chronic mesenteric ischemia (19.5%) [20], and revascularization for critical limb ischemia (20.4%) [21]. Looking specifically at the readmission rates in the CAS population, most prior studies have compared CEA readmission rates versus CAS. These studies have demonstrated rates of 12.0% and 8.3% for Medicare-only and nationally representative data, respectively, for the CAS cohorts [5, 22]. Other studies have also shown similar readmission rates for CAS patients in the range of 9.6% in the Pennsylvania Health Care Cost Containment Council study by Hintze et al. [23], 10.75% by Greenleaf et al. [24], and 11.11% by Galinanes et al. [25]. All these prior studies have used patient-level data from before 2015 when ICD-9 was in use. Our study gives a glimpse into the most contemporary nationally representative data using ICD-10 codes. In addition, our national evaluation of unplanned 30-day readmissions after CAS has several key findings. Our observation showed that 7.4% of patients undergoing CAS had unplanned readmissions within 30 days of hospital discharge. The decline in readmission rates observed in our study, as compared to aforementioned prior studies, may be related to increased operator and/or hospital experience or may be due to strict inclusion/exclusion criteria employed in our study [26, 27].

In our study, sepsis was found to be one of the leading causes for readmission post CAS. Interestingly literature review showed that postoperative surgical-site infection, sepsis/septic shock, pneumonia, and urinary tract infection are known associations with readmissions after CEA [28]. Quiroz et al. looked into hidden readmissions after CEA and CAS, and found infectious etiologies amounting to 9.9% as a cause for readmission (wound complication 3.7%, sepsis 3.1%, urinary tract infection/pyelonephritis 0.5%, and other infections 2.6%). This proves that the infection/sepsis rates found in our study were not in excess of those in the existing literature [29].

There are over 40 models for predicting short- and long-term outcomes after carotid revascularization [30]. However, the prediction models available have the following potential limitations. Firstly, most of the models have used patient databases that are not representative of the national population. The models have used logistic regression, and none of them have used artificial intelligence to improve the quality of predictions. The existing models have used data from the ICD-9 era wherein the specificity of diagnostic codes was significantly inferior to ICD-10 codes. None of the short-term models have looked at all-cause readmissions and can only predict stroke or death. C-statistic (or area under the curve) is considered an important discriminating factor for the accuracy of a prediction model with scores of < 0.50, > 0.50–0.70, > 0.70–0.80, < 0.80–0.90, and > 0.90 representing no, poor, low, excellent, and outstanding discrimination, respectively [31, 32]. Volkers et al. presented an excellent external validation study evaluating 30 prediction models for CEA, CAS, or both and found that not a single model had C-statistic over 0.67 (poor prediction capability) during external validation proving that although there are many models to choose from none of them truly qualify for being substantially useful by clinicians [30].

Despite having a plethora of existing state-of-the-art statistical and machine learning methods in modeling readmission rates, it is noteworthy that our novel implementation of DNN helped us build a model with increased predictive power and, at the same time, facilitated the identification of features that are clinically both relevant and important to their association with the event of readmission post CAS. It was also interesting to see that although DNN had a lower or similar level of accuracy compared to traditional methods, the score of our model with respect to performance metrics such as AUROC and AUPRC was significantly higher. One plausible reasoning for such an observation stems from the fact that the accuracy of existing models such as logistic regression is heavily biased towards the majority class’s proportion as opposed to the minority class samples. Although it enriches the accuracy score, this inherent bias has downstream consequences in high misclassification rates, subsequently resulting in low predictive power [33]. In contrast, DNN is agnostic to such biases. It uses the entire dataset to find out the complex patterns between the variables and then further utilizes this pattern to classify the outcome labels even if the data is highly imbalanced. Therefore, on the basis of our strong evidence, DNN should serve as a premium choice in building more robust and adaptive predictive models for accurate predictions in complex data architectures such as 30-day readmission post CAS.

Patients with comorbidities, including depression, heart failure, cancer, in-hospital bleeding, and coagulation disorders, were the strongest predictors of readmission based on logistic regression, as shown in Fig. 1. The logistic regression analysis did not include hospital-level factors like teaching status, control ownership, or hospital location. Also evident was that most comorbidities had overlapping power, which would make it challenging to develop a robust prediction model using this analysis alone. This meant that logistic regression-based prediction models would not perform well in a clinical setting. Our study further improved upon this and used DNN and hospital-level data to identify novel predictors for early readmission, as shown in Fig. 3. The DNN model provided a zoomed-in view with refined results and showed that factors like hospital rural–urban designation, control/ownership, and teaching status form one of the strongest predictors in addition to newly identified comorbidities to identify patients at risk of early readmissions post CAS. Patients with these comorbidities were more likely to get readmitted, which may have been due to disease progression. Further prospective research would be needed to determine real impact and causal associations.

Reasons for readmission based on the primary diagnosis code were septicemia (8.6%), cerebral infarction (8.6%), heart failure (5.9%), acute hemorrhagic cerebrovascular disease (4.2%), acute renal failure (4.2%), and gastrointestinal hemorrhage (3.6%). These further affirm the need to develop robust prediction models to help decrease unplanned readmissions and comorbidities. Interestingly, in our study, patients with different insurance status (Medicare, Medicaid, or private) and hospital bed size had no significant impact on unplanned readmissions. In contrast, patients with higher scores on mortality/severity of illness subclass of APRDRG scores had worse outcomes in terms of all-cause readmissions. Patients treated at private hospitals compared to government, nonfederal hospitals, and those at metropolitan hospitals were at increased risk for unplanned readmissions. This may be attributed to the difference in the practice patterns at different hospital locations or subtypes or secondary to the number of cases being done at that location and physician experience [26, 27].

Multiple prediction models have been developed in the past, mostly looking at outcomes like recurrent stroke, myocardial infarction, or death. Unfortunately, none of them have a prediction tool to help with short-term readmission risk due to all-cause readmissions. The studies also lack prediction power, especially when evaluated with external validation. A study by Volkers et al. presented an excellent external validation study evaluating 30 prediction models for CEA, CAS, or both and found that not a single model had AUC over 0.67 during external validation [30]. This further proves the point that although there are many models to choose from, none of them truly qualify for being substantially useful in current day practice. Our prediction model is novel in many ways. First, it is the first model to use nationally representative data in the contemporary ICD-10 era and uses machine learning models to predict all-cause unplanned short-term readmissions. The AUC score of 0.79 for DNN is very robust in predicting our primary outcome.

Study Limitations

NRD is in a format of annualized data with a maximum follow-up of 1 year. As with any observational data, the results do not suggest a causal relationship as there can be other unmeasured confounders. The NRD database does not provide pharmacological data/lesion-level data that may impact readmissions. Lastly, the presented risk scores have not been externally validated and currently stand applicable only to the US population.

Conclusion

Our analysis suggests that 7.4% of patients get readmitted within 30 days of discharge after undergoing CAS with septicemia or cerebral infarction/hemorrhagic cerebrovascular bleed as the major causes of all unplanned readmissions. We demonstrate that using machine learning approaches, and we are able to develop a risk prediction model that is able to identify patients at high risk of unplanned readmissions with a C-statistic of 0.802 using DNN. Our work is an exemplar of how machine learning techniques can be used to identify patients at high risk of unplanned readmission for targeted interventions, which, if efficacious, may represent significant healthcare savings to the wider healthcare economy. We plan to acquire funding to develop an easy-to-use online tool, and a software plug-in for existing electronic medical record software to allow for quick assessment of readmission risk in patients undergoing CAS.

Change history

  • 06 May 2021

    The original article has been updated due to update in figshare link.

References

  1. 1.

    Flaherty ML, Kissela B, Khoury JC, et al. Carotid artery stenosis as a cause of stroke. Neuroepidemiology. 2013;40:36–41. https://doi.org/10.1159/000341410.

    Article  PubMed  Google Scholar 

  2. 2.

    Bonati LH, Lyrer P, Ederle J, Featherstone R, Brown MM. Percutaneous transluminal balloon angioplasty and stenting for carotid artery stenosis. Cochrane Database Syst Rev. 2012;2012:CD000515.

    Google Scholar 

  3. 3.

    Goldfield NI, McCullough EC, Hughes JS, et al. Identifying potentially preventable readmissions. Health Care Financ Rev. 2008;30(1):75–91.

    PubMed  PubMed Central  Google Scholar 

  4. 4.

    Rosenbaum S. The Patient Protection and Affordable Care Act: implications for public health policy and practice. Public Health Rep. 2011;126(1):130–5.

    Article  Google Scholar 

  5. 5.

    Al-Damluji MS, Dharmarajan K, Zhang W, et al. Readmissions after carotid artery revascularization in the Medicare population. J Am Coll Cardiol. 2015;65(14):1398–408. https://doi.org/10.1016/j.jacc.2015.01.048.

    Article  PubMed  PubMed Central  Google Scholar 

  6. 6.

    Galiñanes EL, Dombroviskiy VY, Hupp CS, Kruse RL, Vogel TR. Evaluation of readmission rates for carotid endarterectomy versus carotid artery stenting in the U.S. Medicare population. Vasc Endovascular Surg. 2014;48:217–23.

    Article  Google Scholar 

  7. 7.

    CDC. International Classification of Diseases, (ICD-10-CM/PCS) Transition. https://www.cdc.gov/nchs/icd/icd10cm_pcs_background.htm. Accessed 20 July 2020.

  8. 8.

    Morgan DJ, Bame B, Zimand P, et al. Assessment of machine learning vs standard prediction rules for predicting hospital readmissions. JAMA Netw Open. 2019;2(3):e190348–e190348.

    Article  Google Scholar 

  9. 9.

    Agency for Healthcare Research and Quality. NRD database documentation. https://www.hcup-us.ahrq.gov/db/nation/nrd/nrddbdocumentation.jsp. Accessed 06 Feb 2020.

  10. 10.

    Moons KG, Altman DG, Reitsma JB, Collins GS. Transparent reporting of a multivariate prediction model for individual prognosis or development initiative. New guideline for the reporting of studies developing, validating, or updating a multivariable clinical prediction model: The TRIPOD Statement. Adv Anat Pathol. 2015;22(5):303–5. https://doi.org/10.1097/PAP.0000000000000072.

    Article  PubMed  Google Scholar 

  11. 11.

    Heba M, El-Dahshan EA, El-Horbaty EM, et al. Classification using deep learning neural networks for brain tumors. Future Comput Inform J. 2018;3(1):68–71. https://doi.org/10.1016/j.fcij.2017.12.001.

    Article  Google Scholar 

  12. 12.

    Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88. https://doi.org/10.1016/j.media.2017.07.005.

    Article  PubMed  Google Scholar 

  13. 13.

    Ravì D, Wong C, Deligianni F, et al. Deep learning for health informatics. IEEE J Biomed Health Inform. 2017;21(1):4–21. https://doi.org/10.1109/JBHI.2016.2636665.

    Article  PubMed  Google Scholar 

  14. 14.

    Syarif I, Prugel-Bennett A, Wills G. SVM parameter optimization using grid search and genetic algorithm to improve classification performance. TELKOMNIKA. 2016;14(4):1502–9. https://doi.org/10.12928/TELKOMNIKA.v14i4.3956.

    Article  Google Scholar 

  15. 15.

    Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;10(3):e0118432. https://doi.org/10.1371/journal.pone.0118432.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  16. 16.

    Guang-Hui Fu, Feng Xu, Zhang B-Y, Yi L-Z. Stable variable selection of class-imbalanced data with precision-recall criterion. Chemom Intell Lab Syst. 2017;171:241–50. https://doi.org/10.1016/j.chemolab.2017.10.015.

    CAS  Article  Google Scholar 

  17. 17.

    Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360:1418–28. https://doi.org/10.1056/NEJMsa0803563.

    CAS  Article  PubMed  Google Scholar 

  18. 18.

    Atti V, Nalluri N, Kumar V, et al. Frequency of 30-day readmission and its causes after endovascular aneurysm intervention of abdominal aortic aneurysm (from the Nationwide Readmission Database). Am J Cardiol. 2019;123:986–94. https://doi.org/10.1016/j.amjcard.2018.12.006.

    Article  PubMed  Google Scholar 

  19. 19.

    Jones CE, Richman JS, Chu DI, Gullick AA, Pearce BJ, Morris MS. Readmission rates after lower extremity bypass vary significantly by surgical indication. J Vasc Surg. 2016;64:458–64. https://doi.org/10.1016/j.jvs.2016.03.422.

    Article  PubMed  Google Scholar 

  20. 20.

    Lima FV, Kolte D, Louis DW, et al. Thirty-day readmission after endovascular or surgical revascularization for chronic mesenteric ischemia: insights from the nationwide readmissions database. Vasc Med. 2019;24:216–23. https://doi.org/10.1177/1358863X18816816.

    Article  PubMed  Google Scholar 

  21. 21.

    Kolte D, Kennedy KF, Shishehbor MH, et al. Thirty-day readmissions after endovascular or surgical therapy for critical limb ischemia: analysis of the 2013 to 2014 nationwide readmissions databases. Circulation. 2017;136:167–76. https://doi.org/10.1161/CIRCULATIONAHA.117.027625.

    Article  PubMed  Google Scholar 

  22. 22.

    Lima FV, Kolte D, Kennedy KF, et al. Thirty-day readmissions after carotid artery stenting versus endarterectomy: analysis of the 2013–2014 nationwide readmissions database. Circ Cardiovasc Interv. 2020;13(4):e008508. https://doi.org/10.1161/CIRCINTERVENTIONS.119.008508.

    Article  PubMed  Google Scholar 

  23. 23.

    Hintze AJ, Greenleaf EK, Schilling AL, Hollenbeak CS. Thirty-day readmission rates for carotid endarterectomy versus carotid artery stenting. J Surg Res. 2019;235:270–9. https://doi.org/10.1016/j.jss.2018.10.011.

    Article  PubMed  Google Scholar 

  24. 24.

    Greenleaf EK, Han DC, Hollenbeak CS. Carotid endarterectomy versus carotid artery stenting: no difference in 30-day postprocedure readmission rates. Ann Vasc Surg. 2015;29(7):1408–15. https://doi.org/10.1016/j.avsg.2015.05.013.

    Article  PubMed  Google Scholar 

  25. 25.

    Galinanes EL, Dombroviskiy VY, Hupp CS, Kruse RL, Vogel TR. Evaluation of readmission rates for carotid endarterectomy versus carotid artery stenting in the US Medicare population. Vasc Endovascular Surg. 2014;48:217e223.

    Article  Google Scholar 

  26. 26.

    Poorthuis MHF, Brand EC, Halliday A, Bulbulia R, Bots ML, de Borst GJ. High operator and hospital volume are associated with a decreased risk of death and stroke following carotid revascularization: a systematic review and meta-analysis: authors’ reply. Ann Surg. 2018;269:631–41. https://doi.org/10.1097/SLA.0000000000002880.

    Article  Google Scholar 

  27. 27.

    Kim LK, Yang DC, Swaminathan RV, et al. Comparison of trends and outcomes of carotid artery stenting and endarterectomy in the United States, 2001 to 2010. Circ Cardiovasc Interv. 2014;7:692–700. https://doi.org/10.1161/CIRCINTERVENTIONS.113.001338.

    Article  PubMed  Google Scholar 

  28. 28.

    Rambachan A, Smith TR, Saha S, Eskandari MK, Bendok BR, Kim JY. Reasons for readmission after carotid endarterectomy. World Neurosurg. 2014;82(6):e771–6. https://doi.org/10.1016/j.wneu.2013.08.020.

    Article  PubMed  Google Scholar 

  29. 29.

    Quiroz HJ, Martinez R, Parikh PP, et al. Hidden readmissions after carotid endarterectomy and stenting. Ann Vasc Surg. 2020;68:132–40. https://doi.org/10.1016/j.avsg.2020.04.025.

    Article  PubMed  Google Scholar 

  30. 30.

    Volkers EJ, Algra A, Kappelle LJ, et al. Prediction models for clinical outcome after a carotid revascularization procedure. Stroke. 2018;49(8):1880–5. https://doi.org/10.1161/STROKEAHA.117.020486.

    Article  PubMed  PubMed Central  Google Scholar 

  31. 31.

    Hosmer DW, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd ed. Hoboken: Wiley; 2013.

    Book  Google Scholar 

  32. 32.

    Lloyd-Jones D. Cardiovascular risk prediction: basic concepts, current status, and future directions. Circulation. 2010;121:1768–77.

    Article  Google Scholar 

  33. 33.

    Ottenbacher KJ, Smith PM, Illig SB, Linn RT, Fiedler RC, Granger CV. Comparison of logistic regression and neural networks to predict rehospitalization in patients with stroke. J Clin Epidemiol. 2001;54(11):1159–65. https://doi.org/10.1016/s0895-4356(01)00395-x.

    CAS  Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank the Healthcare Utilization Project (HCUP) for providing this publicly available data. The authors also thank all participants of the HCUP project.

Funding

No funding or sponsorship was received for this study or publication of this article.

Authorship

All named authors meet the International Committee of Medical Journal Editors (ICMJE) criteria for authorship for this article, take responsibility for the integrity of the work as a whole, and have given their approval for this version to be published.

Authorship Contributions

Dr. AA, Dr. NA, and Dr. GCF conceptualized and designed the study, carried out the initial analysis and interpretation, coordinated data collection and prepared the manuscript. Dr. AA, Dr. SC, Dr. AR, and Mr. RC helped in the formation of study design, data analysis, artificial intelligence model development, and interpretation and critically reviewed the manuscript for important intellectual content. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

Disclosures

Gregg C. Fonarow reports consulting for Abbott, Amgen, AstraZeneca, Bayer, Edwards, Janssen, Medtronic, Merck, and Novartis. Amod Amritphale, Ranojoy Chatterjee, Suvo Chatterjee, Nupur Amritphale, Ali Rahnavard, G. Mustafa Awan, and Bassam Omar have nothing to disclose.

Compliance with Ethics Guidelines

This study was performed utilizing publicly available datasets and hence does not require IRB review under 45 CFR 46. In addition, the study was approved for exempt status by the Institutional Review Board at the University of South Alabama.

Data Availability

The data used in this study is publicly available from the Healthcare Utilization Project at https://www.hcup-us.ahrq.gov/. All codes used for data extraction/analysis have been provided in the supplementary material.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Amod Amritphale.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 18 KB)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Amritphale, A., Chatterjee, R., Chatterjee, S. et al. Predictors of 30-Day Unplanned Readmission After Carotid Artery Stenting Using Artificial Intelligence. Adv Ther 38, 2954–2972 (2021). https://doi.org/10.1007/s12325-021-01709-7

Download citation

Keywords

  • Readmission
  • Carotid artery stenting
  • Artificial intelligence
  • Machine learning