Introduction

Severe vitamin D deficiency (SVDD), defined as a 25-hydroxy-vitamin D [25(OH)D] concentration below 12 ng/mL (or 30 nmol/L), is highly prevalent in patients admitted to intensive care units (ICUs) and is associated with adverse outcomes [1,2,3]. The prevalence of SVDD in ICUs typically ranges from 20 to 70% [4]. In Taiwan, the prevalence of vitamin D deficiency [i.e., 25(OH)D level below 20 ng/mL or 50 nmol/mL] in the general population ranges from 20 to 40% [5,6,7]; however, there are little data about the prevalence of SVDD. Additionally, our previous multicenter observational study reported a higher prevalence of vitamin D deficiency of 59% and a prevalence of SVDD of 18% in critically ill patients in Taiwan [8]. The study also revealed strong associations of vitamin D deficiency with longer duration of ventilator use and greater length of ICU stay [8].

Supplementation of vitamin D in patients with critical illness has been reported to be safe [9]. According to the 2019 European Society for Clinical Nutrition and Metabolism guidelines for clinical nutrition in the ICU [10], administering a single high dose of vitamin D3 (500,000 UI) in patients with vitamin D deficiency is recommended within 1 week of ICU admission. However, vitamin D testing for every ICU patient is not a routine practice and may be impractical and too expensive in many countries. Therefore, developing a prediction model for SVDD to determine which patient would benefit most from vitamin D tests and supplementation is essential.

Several models for predicting vitamin D deficiency have been created for the general population [11,12,13,14] but not patients admitted to ICUs. To facilitate decision making on vitamin D supplementation in an intensive care setting, this multicenter cohort study developed and validated a score-based model for predicting SVDD in patients with critical illness.

Methods

Study design

This study was based on the data obtained in our previous multicenter, prospective, observational study [8]. It was approved by the Research Ethics Committee of National Taiwan University Hospital (approval number: 202203073RIND) and registered on the ClinicalTrials.gov protocol registration system (ID: NCT05376774). This study was conducted in eight ICUs at four hospitals in northern Taiwan between October 2017 and July 2020. We included surgical ICUs (SICUs), medical ICUs (MICUs), and mixed ICUs with both postoperative patients and medical cases. To perform temporal validation, the data were divided into a derivation cohort (the first 77% of the data set) and a validation cohort (the remaining 23% of the data set) on the basis of the date of enrollment. To cover all seasons, the validation cohort included patients over a year (i.e., August 2019 to July 2020). The models were developed and validated in accordance with the recommendations established in the Transparency Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) initiative [15].

Study sample

The inclusion and exclusion criteria were the same as in our previous study [8]. Patients admitted to the ICUs were eligible for enrollment. Patients were excluded if they were younger than 20 years old; had a body mass index lower than 18 kg/m2; had severe anemia (i.e., a hemoglobin level less than 7 g/dL); received an additional vitamin D supplement greater than 3,000 IU/day within 4 weeks of the study; were previously admitted to an ICU within the preceding 3 months; or had a diagnosis of hyperparathyroidism, rickets, or liver cirrhosis (Child–Pugh C). Because the study was a retrospective analysis of deidentified collected data, informed consent was not required.

Predictor selection and processing

We selected 15 variables as candidate predictors on the basis of our clinical judgment and a literature review [11,12,13]. To facilitate practical application of the models, we excluded variables that are not routinely recorded or tested, such as C-reactive protein and total serum calcium levels. Imbalanced categorical predictors with percentages smaller than 10% or greater than 90% were excluded to prevent overfitting [16]. Ultimately, six categories of candidate predictors were incorporated into our models: general characteristics, comorbidities, indication of ICU admission, enrollment season, vital signs, and laboratory findings. All the data were collected upon enrollment. Multiple imputation was conducted using the “mice” package to address missing data for the potential predictors [17]. Five imputed data sets were created; the imputation methods consisted of predictive mean matching for continuous predictors and logistic regression for binary predictors. All statistical analyses were performed using the five imputed data sets, and the estimates were combined in accordance with the guidelines proposed by Marshall et al. [18].

Outcome definition

The outcome variable, SVDD, was defined as a serum 25(OH)D level of < 12 ng/mL (or 30 nmol/L). In our previous study, blood samples were obtained upon enrollment, and serum 25(OH)D level was measured using the commercially available TOTAL Liaison chemiluminescence assay (Liaison, Diasorin, Saluggia, Italy) [19].

Statistical analysis

We express categorical variables as percentages and continuous variables as medians. The Shapiro–Wilk test was used to test normality. For comparisons between the derivation and validation cohorts, we performed the Mann–Whitney test for non-normally distributed continuous variables, Student’s t test for normally distributed continuous variables, and the Chi-squared test for categorical variables. A P value of < 0.05 was considered statistically significant.

We first fit a multivariable logistic regression (MLR) model in the derivation cohort. The relationships between heart rate, age, and outcome could be nonlinear; this was enabled by the use of restricted cubic splines [20]. To construct a simple prediction score (i.e., the SVDD score), we included predictors that were significantly associated with SVDD in the MLR model [21,22,23] or predictors that strongly influenced the model, that is, that had a standardized odds ratio greater than 1.2 or less than 0.8. A reduced MLR model was created using these included predictors, and SVDD scores were estimated using the reduced MLR model.

To convert the SVDD scores into integers, the regression β coefficients were multiplied by 5 and rounded to the nearest integer [22, 23]. We wanted a score of 0 for the lowest-risk group. By grouping each continuous predictor into convenient intervals, such as intervals of 10 mmHg for mean blood pressure, an individual's score increased by an integer amount for each risk factor level above the lowest-risk category [21]. The total number of points was the value of the final SVDD score. Additionally, estimated probabilities of each SVDD score were obtained using logistic regression.

The performance of the MLR model and the SVDD score were evaluated in both the derivation and validation cohorts by using metrics that represent discrimination and calibration. Discrimination was assessed using the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPRC). Calibration was assessed using calibration plots, Brier scores, and Hosmer and Lemeshow goodness-of-fit tests. Post hoc recalibrations were performed by adjusting the intercept because the predicted probability was underestimated, which resulted from differences in the overall incidence of SVDD between the derivation and validation cohorts [15, 24].

Patients were divided into three risk groups: very low risk, low risk, and medium-to-high risk groups. The groups were based on the estimated probabilities [25]. We also developed web and mobile phone applications with which clinicians can calculate SVDD scores and conveniently interpret the risk stratification. All statistical analyses were conducted using R (R version 4.1.3; R Foundation for Statistical Computing, Vienna, Austria).

Results

Patient enrollment and characteristics

We divided 662 patients from our previous study into a derivation cohort with 510 patients (from October 2017 to July 2019) and a validation cohort with 152 patients (from August 2019 to July 2020; Fig. 1). Table 1 shows the patient characteristics of both cohorts. The significant differences found in the validation cohort compared with the derivation cohort were a lower percentage of neurological indication of ICU admission (3.3% vs. 11.3%), a lower percentage of enrollment in spring (13.2% vs. 28.6%), a higher percentage of enrollment in fall (30.9% vs. 20.4%), a lower median total calcium level (2.07 vs. 2.10 mmol/L), and a higher median C-reactive protein level (11.7 vs. 6.7 mg/L). The prevalence of SVDD was 16.3% and 21.7% in the derivation and validation cohorts (P = 0.154), respectively.

Fig. 1
figure 1

Scheme of the development and validation of the prediction model. Abbreviations: MLR, multivariable logistic regression; SVDD, severe vitamin D deficiency

Full size image
Table 1 Baseline characteristics and risk factors in derivation and validation cohorts
Full size table

MLR analyses and development of prediction models

Additional file 1: Table S1 summarizes the result of the MLR analysis. Restricted cubic splines were applied to age and heart rate. The spline variables were prepared using four knots set at the 5th, 35th, 65th, and 95th percentiles of the variables. Significant contributions were made by age, gender, heart rate, sepsis, albumin level, and mean arterial pressure. Additionally, postoperation, that is entering the ICU after having received surgery, and enrollment season greatly influenced the model. These eight predictors and their β coefficients in the reduced MLR model were then used to determine the SVDD scores. Table 2 presents the score chart. The SVDD score was defined as the sum of the points from each variable.

Table 2 SVDD score chart
Full size table

Performance analyses

SVDD score had an AUROC of 0.751 [95% confidence interval (CI) 0.694–0.809] in the derivation cohort and 0.848 (95% CI 0.781–0.914) in the validation cohort, neither of which were significantly different from the AUROC of the MLR models. The AUPRC of SVDD score was 0.439 (95% CI 0.381–0.491) in the derivation cohort and 0.619 (95% CI 0.577–0.669) in the validation cohort. The calibration plots for the MLR model and SVDD score presented in Additional file 1: Figs. S1 and S2 indicated acceptable calibration in the derivation cohort. However, the calibration plots for SVDD score in the validation cohort revealed a general trend of underestimation and required recalibration with the intercept. We obtained the recalibrated intercept of 0.273 (P = 0.203). The performance of the recalibrated model is illustrated in Additional file 1: Fig. S2.

Application of SVDD score prediction model

Figure 2 shows the SVDD score predictions. The estimated probability of SVDD was calculated as follows:

$$\frac{1}{{1 + e^{{ - \left[ { - 6.58 + 0.21 \times SVDD\;score} \right]}} }}.$$
Fig. 2
figure 2

Recalibrated predicted probabilities of SVDD for each SVDD score. Abbreviations: SVDD, severe vitamin D deficiency

Full size image

The recalibrated probability of SVDD was calculated as follows:

$$\frac{1}{{1 + e^{{ - \left[ {0.273 - 6.58 + 0.21 \times SVDD\;score} \right]}} }}.$$

The recalibrated probability of SVDD was grouped into three categories: very low risk (≤ 15%), with an SVDD score of 0–22; low risk (15–30%), with an SVDD score of 23–26; and medium-to-high risk (≥ 30%), with an SVDD score of ≥ 27 (Fig. 3). Applying the risk group classification to the validation cohort revealed favorable discrimination and an AUROC of 0.812 (95% CI 0.741–0.883). When using a score of 24 as the threshold for predicting SVDD, the sensitivity was 86% (95% CI 70–96%), the specificity was 65% (95% CI 55–73%), the positive predictive value was 40% (95% CI 28–41%), and the negative predictive value was 94% (95% CI 90–98%). Additional file 1: Figs. S3 and S4 show the web [26] and mobile phone applications for the SVDD scores.

Fig. 3
figure 3

Probability of SVDD in accordance with risk groups in the validation cohort. Abbreviations: SVDD, severe vitamin D deficiency

Full size image

Discussion

This multicenter cohort study constructed a score-based model for predicting SVDD in patients with critical illness. Independent predictors of SVDD include age, gender, sepsis, postoperation, season, heart rate, mean arterial pressure, and albumin level. The SVDD score demonstrated favorable performance, with its AUROC being 0.848 and exhibited good calibration after recalibration. Our model can predict SVDD in patients with critical illness by calculating a simple SVDD score and can assist with screening high-risk patients who may benefit from vitamin D supplementation [27].

The SVDD score is an easy-to-use scoring tool and is based on information routinely available in ICUs. A patient’s risk of SVDD is quantified by simply using an SVDD scoring chart (Table 2) and the predicted probability of each SVDD score (Fig. 2); complex computer calculations are not required. We also developed web and mobile phone application that had an SVDD score calculator; clinicians can use the web or application to conveniently assess a patient’s risk of SVDD. Clearly defined risk groups were established and demonstrated to have favorable discrimination ability. In some countries, vitamin D level tests are time-consuming and expensive; the proposed SVDD score can facilitate vitamin D supplementation for patients with critical illness and reduce the money spent on vitamin D tests. It has wide applicability in general ICU practice. Kheir et al. had proposed a single-center study about a dynamic nomogram predicting SVDD at ICU admission [28]. In comparison with our SVDD score, their model depended on complex computer calculation, and the predictors included other clinical scores that needs further calculation, such as Sequential Organ Failure Assessment score.

Considering the population of patients with critical illness and the feasibility of use of the prediction model in ICUs, we excluded some predictors that are commonly included in vitamin D deficiency models, such as suntan use, fatty fish consumption, or lifestyle. Body mass index was a potential predictor but found to not be significantly associated with SVDD. Our results revealed that female gender, sepsis, hypoalbuminemia, and high mean arterial pressure are significantly associated with SVDD. These findings are consistent with those of other studies [29,30,31,32]. Moreover, age and heart rate had nonlinear relationships with SVDD in our prediction models. In other studies [11,12,13,14], age has often been dichotomized using variable cutoffs, although the TRIPOD guidelines strongly discourage the dichotomization of continuous predictors [15]. Further studies are necessary to investigate the mechanism or possible confounding effects of this nonlinear relationship. In our study, postoperation was a protective predictor for SVDD. We suggest that medical cases have more comorbidities than postoperative patients, and multimorbidity may be a risk factor of vitamin D deficiency. Further studies are warranted to investigate SVDD in patients admitted to SICU or MICU.

The strengths of this study include a multicenter design, the use of predictors that can feasibly be determined in an ICU setting, and strict adherence to the TRIPOD guidelines. The limitations of this study are a small sample, few events per predictor in the MLR model, missing values for some of the laboratory data, and heterogeneity of patients from different types of ICUs. Moreover, the prediction model lacks external validity, and the model may not be applicable in countries at different latitudes or in specialized ICUs. Recalibrations may be required for new study populations and settings. Future studies are warranted to externally validate the SVDD prediction model.

Conclusions

Our study establishes an easy-to-use SVDD score for predicting SVDD in patients admitted to ICUs. This SVDD score is the first vitamin D deficiency prediction score that is specialized to patients with critical illness. Future studies in different countries and geographic locations are necessary to externally validate the model.