Background

In the previous decade, incidence rates of diabetes-associated amputations have significantly decreased [1–5]. However, a major lower extremity amputation (LEA) is associated with a reduction of long-term survival and quality of life [6, 7].

In Spain, more than 5,000 major LEAs are performed yearly, 57% in patients with type 2 diabetes (T2DM) [8]. Despite improvements in long-term outcome, in-hospital mortality (IHM) following major LEA in patients with T2DM has remained stable from 2001 to 2012 with figures of 9.7% and 9.5% respectively [8, 9].

Outcome prediction is important in the clinical decision-making process. Older age, renal failure and congestive heart failure have been identified as independent factors for operative mortality following major LEA in Veterans Administration patients [10]. Wise et al. (2016) reported that preoperative septic shock and thrombocytopenia are independent risk factors for 30-day mortality after major LEA [11]. Other authors have described that age and vascular disease in patients with T2DM are important predictors that contribute to increase the mortality after major LEA [12]. However, to our knowledge, there are no studies investigating the predictive factors of in-hospital operative mortality in T2DM patients following major LEA.

Artificial neural networks (ANN) have been used to predict the risk of post-operative events, including survival, exploring complex relationships between preoperative variables survival in different surgical settings and are increasingly being used in complex medical decision making [13–16].

The aim of our study was to use ANNs analysis to estimate predictive factors of IHM in patients with T2DM after major LEA in Spain. We assessed the performance indices for ANN models in four scenarios: i) with all discharge diagnosis included in the database; ii) with all discharge diagnosis included in the database, excluding infectious diseases; iii) comorbidities included in the Charlson Comorbidities Index (CCI) and iv) comorbidities included in the Elixhauser Comorbidity Index (ECI).

Methods

We have been used the Spanish national hospital database, from 2003 to 2013 to identify predictive factors of IHM in type 2 diabetic patients after major LEA. The database contained variables like sex, date of birth, admission date, discharge diagnoses (up to 14), procedures performed during the hospital stay (up to 20), length of hospital stay (LOHS) and IHM [17]. Diagnosis and procedures are codified according to the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM). Details on this database have been published previously [18, 19].

The study included all procedures of major amputation, defined as any LEA through or proximal to the ankle joint (ICD-9-CM codes 84.13–84.17) in discharges with a T2DM diagnosis (ICD-9-CM codes 250.x0; 250.x2) in any diagnosis position. We excluded all traumatic major LEAs (ICD-9-CM codes 895–897; 905.9; 928–929; 959 in any diagnosis field) and patients with type 1 diabetes (T1DM) (ICD-9-CM codes 250.x1; 250.x3).

The study sample included 40,857 patients who undergone major amputation procedure between January 1, 2003 and December 31, 2013.

Development of the ANN models

To provide a good quality predictive model, the database has been randomly splitted in three independent subsets: 60% of the entire database for training the neural network (n = 24,513 cases), 20% for selecting those models with better generalization capabilities (n = 8,172 cases) and 20% for testing the model (n = 8,172 cases). The independent variables were age, sex, comorbidity and LOHS, and the dependent variable was outcome (IHM).

The ANN used in this study was a standard feed-forward, back-propagation neural network in which each input layer received information from the data, then it passed through the hidden layers and, finally, it arrived to the output layer. We used the quasi-Newton method in order to carry out the learning process (training algorithm) [20] and we applied model selection to find the optimal number of neurons in the hidden layer [21].

We elaborated four models as follow: model 1) including all discharge diagnosis in the database; model 2) all discharge diagnosis included in the database, except infectious diseases (ICD-9-CM codes 001 to 139); model 3) comorbidities included in the CCI [22] and finally; model 4) comorbidities included in the ECI [22].

Statistical analyses

The unit of analysis was the individual patient with T2DM undergoing major LEA. A descriptive analysis was performed for all variables. Categories variables are shown as total numbers or proportions. Significant predictors were identified by bivariate analysis using chi square test, Student's t-test or Mann–Whitney U test when appropriate (p < 0.05, 2-tailed).

The predictive capabilities of the four models was analyzed using the area under the ROC curve (AUC) and the performance indices for ANN models described in Table 3 [23]. In this context, discrimination model is which to ability to distinguish patients who died from those who survived. Finally, once the model is trained, it can be studied the importance of the input variables by performing a sensitivity analysis. The sensitivity of an input variable is calculated dividing the sum squared error of the neural network responses without using the input variable by the sum squared error of the neural network responses using all the inputs. In this way, if the error without the input variable is lower than the error using all the inputs, the value of the sensitivity will be lower than 1 and if it is greater, the sensitivity will be greater than 1, as described previously by Shi et al. [24].

To estimate the 95% confidence interval and to compare AUC we used EPIDAT 4.2 software applying the methods described by DeLong et al. [25, 26].

The design of the four ANNs was performed using the predictive analytics solution Neural Designer, developed by the authors of this work. Statistical analyses were performed using the Stata MP version 10.1 statistical package (StataCorp LP, College Station, TX).

Sensitivity analysis

In order to assess the possible misclassification of T1DM patients as T2DM or vice versa in the database the predictive capabilities of the CCI and the ECI were analyzed estimating the AUC including all patients with any diabetes ICD-9-CM codes (250.xx), beside Type 1 o Type 2.

Results

Between January 1, 2003 and December 31, 2013, a total of 40,857 non-traumatic major LEAs (26,222 in men and 14,635 in women) corresponding to T2DM were identified in Spain. Almost 82% of the patients have ≥65 years of age (mean age 73.77 years, SD 10.26 years).

We found that 10.0% (n = 4,078) of the patients who died during the admission in hospital. The mean age of patients who died during their stay was significantly higher (76.58 years) than those survived (73.47 years).

The mean (SD) LOHS was similar, 23.10 (22.17) days in patients who died during their hospitalization and 23.78 (22.09) days in those who survive after the procedure (P > 0.05).

Table 1 shows the demographic characteristics and prevalence of comorbidities included in the CCI according to IHM after a major LEA in patients with T2DM.

Table 1 Demographic characteristics and prevalence of comorbidities included in the Charlson Comorbidity Index according to IHM due to major LEA in patients with T2DM in Spain, 2003-2013
Full size table

We found that 68.83% of the patients had peripheral vascular disease, 17.49% have renal disease and 11.52% have chronic pulmonary disease these were the three more common chronic conditions.

In our study IHM in women with T2DM was more common than in diabetic men (11.7% vs. 9.0%, p < 0.05). IHM was significantly higher in older patients (13.4% in ≥80 years vs. 4.1% in <50 years). Patients with myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, chronic pulmonary disease, peptic ulcer disease, mild liver disease, renal disease, moderate or severe liver disease or metastatic solid tumour had significantly higher IHM than those without these conditions (Table 1). Over 20% of the patients suffering moderate or severe liver disease (23.8%) and congestive heart failure (23.3%) died in the hospital after a major LEA.

As can be seen in Table 2 when we analyse comorbidity included in the ECI we find that peripheral vascular disease and hypertension (47% complicated and 1.8% uncomplicated) were the more prevalent diseases in diabetic patients who underwent LEAs. All patients with any ECI (except those with paralysis, hypothyroidism, peptic ulcer disease excluding bleeding, AIDS/HIV, rheumatoid arthritis/collagen vascular diseases, obesity, blood loss anemia, alcohol abuse or drug abuse) have significantly higher IHM than those without these conditions (Table 2). Beside congestive heart failure (23.3%) patients suffering fluid and electrolyte disorders (29.9%), coagulopathy (24.8%) and pulmonary circulation disorders (21.6%) showed the highest IHM.

Table 2 Prevalence of comorbidities included in the Elixhauser Comorbidity Index according to IHM due to major LEA in patients with T2DM in Spain, 2003-2013
Full size table

The ANN-based approaches provided the 3-layer networks and the relative weights of neurons used for predicting IHM. Table 3 shows the ANN models in terms of binary classification tests. The data analysis showed that ANN model 3 and ANN model 4 have the best accuracy rate (0.8612 and 0.8307) and ECI model performed better in terms of sensitivity, specificity and precision than CCI model (0.7634 vs 0.7444; 0.9602 vs 0.9121; 0.9511 vs 0.888, respectively).

Table 3 Comparison of performance indices for Artificial Neural Network models when using new data sets for predicting in hospital mortality rate in patients with T2DM after major LEA in Spain
Full size table

Compared with CCI model, ECI model showed a significantly better (p = 0.043) area under the ROC curve (91.7% [95% CI 90.3–93.0] vs 88.9% [95% CI; 87.5–90.2]) as can been seen in Fig. 1. The area under the curve for models 1 and 2 were 0.87 and 0.867, respectively, showing that these models are less accurate to predict IHM after major LEAS using ANN.

Fig. 1
figure 1

Area under the ROC curve of the Charlson comorbidities index model and the Elixhauser comorbidities index model in predicting the in-hospital mortality rate after major LEA in patients with T2DM in Spain

Full size image

The selection data set was also used to calculate the variable sensitive ratio (VSR). Table 4 presents the VSR values for the outcome variable (IHM) regarding CCI and ECI comorbidities. In the ECI network for predicting IHM, the most sensitive parameter was age (VSR 1.451) followed by female sex (VSR 1.433), congestive heart failure (VSR 1.341), renal failure (VSR 1.274) and chronic pulmonary disease (VSR 1.266). Age, female sex, myocardial infarction, renal disease and congestive heart failure were the predictors of IHM with the CCI network. All VSR exceeded 1, which indicated that the network performs better when all variables were considered.

Table 4 Global sensitivity analysis of the Charlson comorbidities index model and the Elixhauser comorbidities index model in predicting the in-hospital mortality rate after major LEA in patients with T2DM in Spain
Full size table

The sensitivity analysis, including all diabetic patients beside if codified as Type 1(n = 1531; 3.61%) or Type 2 (n = 40,857; 96.39%), showed that the AUC for the ECI was 90.1% (95% CI 88.9–91.3) and for the CCI was 86.9% (95% CI 85.7%–88.2%) (p = 0.026).

Discussion

To our knowledge, this study is the first to use a nationwide population-based discharge database to train and test an ANN for predicting factors of IHM in type 2 diabetic patients after major LEAs. We assessed and compared the two most commonly used comorbidity risk adjustment models in surgery, the Charlson and Elixhauser measures, regarding their ability to predict in-hospital death [22].

Like in other surgical procedures it is necessary to appropriately adjust for T2DM patient risk, recognizing that the underlying nature of some patients’ conditions may make them more likely than others to experience poor outcomes. Using ANNs we found that the ECI is a superior comorbidity risk-adjustment model for major lower extremity amputation in-hospital mortality compared with the CCI measure. Although differences in the area under the ROC curve between the two comorbidity-based measures were small, it has been noted that even slight improvements in area under the ROC curve for such indexes can translate into quantificable reductions in confounding bias [27]. Overall, the area under the ROC curve for inpatient mortality for the Charlson and Elixhauser comorbidity-based measures in our study were high and comparable to those described in other patient population [28, 29].

Major LEA in T2DM patients is associated with significant mortality. Our IHM of 10.0% is consistent with previously reported data ranging from 7% to 12.4% [7, 10, 30–33]. Investigation of underlying influences from health services, surgical decisions and patient motivations behind decisions to amputate might help to explain some of the differences in post-operative mortality rates between studies [34].

The effect of advancing age on IHM is also highlighted in this study, as has been well established by other authors [7, 10, 30]. Older age is associated with high prevalence of comorbid conditions, mainly cardiovascular and renal diseases, both of which are also associated with higher mortality rates [32, 35].

Despite having lower incidence rates of major LEAs several studies, including ours, suggest that women have higher mortality rates associated with diabetes-related LEAs [36, 37]. In US have reported that women had higher IHM associated with diabetes-related LEAS (37.7 IHM/1000 amputations vs. 29.7/1000 amputations [38]. Some previous research suggests higher LEA-associated cardiovascular disease mortality risk, including atherosclerotic complications, among women with LEAs [35, 38–40].

Several studies have evaluated specific comorbid diseases associated with reduced survival after major LEA [30–35, 38, 40]. Patients who needed major amputations suffered from a more severe cardiovascular disease than others [36]. Previous studies in Spain, significant risk factors for IHM were cardiac and respiratory complications and first episode of amputation [41]. Schofield et al. concluded that diabetic amputees have a greater risk of heart failure (OR: 2.26; 95% CI 1.12–4.57) further amputation and death than nondiabetic amputees [12] and highlighted the need for a more aggressive approach in the management of cardiovascular risk factors in those who undergo amputation and have diabetes. A recent study in the Canadian population reported that predictors of prolonged hospital stay included cardiovascular risk factors, such as diabetes, hypertension, ischemic heart disease, congestive heart failure or hyperlipidemia [42].

Different studies have identified renal disease as a risk factor for a significantly lower long-term survival after a major LEA [6, 36, 43–46]. Recently, Assi et al. reported that patients with chronic kidney disease have higher risk (OR 2.27; 95% CI 1.02–5.06) of postoperative long-term mortality and concluded that how CKD contributes to worse survival is unclear [31].

Other factor that has been found associated to mortality after amputations among people with diabetes was chronic pulmonary disease. Shah et al. demonstrated that several independent factors were detrimental to survival (at 24 months) including chronic obstructive pulmonary disease (hazard ratio [HR] 1.82, P = 0.002), dialysis dependence (HR 2.50, P < 0.001), high cardiac risk (HR 2.20, P < 0.001), and guillotine amputation (HR 2.49, P = 0.004) [33].

The strength of our study lies in its large sample size and in the use of ANN models. Different studies indicated that ANN models can simultaneously process numerous variables and can consider outliers and nonlinear interactions among variables. Therefore, whereas conventional statistics reveal parameters that are significant only for the overall population, the ANN model includes parameters that are significant at the individual level even if they are not significant in the overall population [13–16].

One of the major limitations of our study is that our data source was the CMBD, which contains administrative discharge data for hospitalizations in Spain and uses information the physician included in the discharge report. In our data we have no information about detailed clinical variables like glycemic control or lifestyle variables like smoking habits. Nevertheless, the CMBD, are periodically audited and the validity of our dataset has been assessed and shown to be useful for health research [47].

Another possible limitation is that a misclassification bias may occur and some patients suffering T2DM may have been recorded as T1DM or vice versa. In our investigation we excluded T1DM patients because in our database only 1531 patients had the codes for T1DM recorded, representing the 3.61% of all mayor amputations analyzed. Furthermore, patients with T1DM were significantly younger (mean age 66.56 years vs. 73.77 years; p < 0.001) and had less co-morbidities (mean CCI 1.25 vs. 1.39 and mean ECI 1.90 vs. 2.27; both p < 0.001) than those with T2DM. However, the results of the sensitivity analysis showed very similar results suggesting that the misclassification bias would not affect the main conclusions of our study.

Conclusions

In conclusion, using ANN models we found that ECI method is a superior comorbidity risk-adjustment model for major LEA survival prediction in patients with T2DM than Charlson comorbidity. The global sensitivity analysis with the ECI model showed that the more predictors of IHM after major LEA procedure were older age followed by female sex, congestive heart failure, renal failure, chronic pulmonary disease, liver disease and metastatic cancer. The predictors analyzed in this study could be addressed in preoperative and postoperative health care of patients with T2DM candidates for major LEA.