Dear Editor,

The coronavirus disease 2019 (COVID-19) pandemic stretched intensive care unit (ICU) capacity to its limits worldwide. While rationing ICU beds and making ethical judgments related to the appropriateness of the initiation of life-sustaining therapies have been a routine part of intensive care for many years [1, 2], COVID-19 brought the unique need to triage patients in overwhelmed ICUs. While guidelines do exist, ICU admission policies vary between countries, institutions and individual intensivists [3]. Furthermore, while guidelines recommend a multidisciplinary approach with respect to withdrawing life-sustaining therapies, triage decisions in contrast are often made by individual physicians after hours, during times of crisis and at the bedside.

To improve this triage process, we explored the option to implement a decision support modality based on behavioural artificial intelligence technology (BAIT), a technique originating from discrete choice modelling [4]. With discrete choice modelling, expertise can be codified to provide insight in often implicit decision trade-offs (Background Information, Supplementary Information). We aimed to develop a model that explicates the implicit conditions Dutch intensivists use to determine ICU eligibility of COVID-19 patients.

In a number of brainstorm sessions, we determined the factors intensivists might use to evaluate eligibility for ICU treatment and mechanical ventilation of COVID-19 patients. By using choice experiments containing 25 hypothetical ICU admission scenarios designed for maximum information content, we collected a set of expert decisions from both senior intensivists as well as fellows in Intensive Care medicine from hospital A (OLVG) (Table S1; Fig. S1, Supplementary Information) using a web application. By applying logistic regression, these observed decisions were used to create a model containing the weights that ICU physicians implicitly assign to various factors. To determine interhospital variability, the same process was repeated for hospital B (Amsterdam UMC). Model performance was reported as McFadden’s \({\rho }^{2}\) using both the models from the individual hospitals as well as a combined model from both hospitals. \({\rho }^{2}\) is defined as the extent to which the estimated model outperforms a model with no information content (i.e., with all weights zero): a \({\rho }^{2}\) of zero means no improvement relative to such a null-model and its theoretical maximum of one means that every single choice made has been correctly predicted with full certainty by the model [5].

The choice experiments collected a total number of 1000 decisions from 13 intensivists and four fellows of hospital A and 17 intensivists and six fellows of hospital B. The most important parameters in determining ICU admission eligibility and initiation of mechanical ventilation for both hospitals are shown in Fig. 1. Though there were differences, most features were similar across both hospitals in terms of their relative importance: clinical frailty score, age, cognitive comorbidity, admission capacity and patients’ own ideas on ICU treatment. The empirical performance of the models in both hospitals was similar and the combined model showed a similar performance as the individual models (Table S2, Supplementary Information), further suggesting consistency between choices made by intensivists of both hospitals.

Fig. 1
figure 1

Factors determining ICU eligibility and initiation of mechanical ventilation for models for hospital A, B and hospitals A and B combined. The coloured bars show the relative weight or contribution of a factor (i.e. the importance, from 0 to 100%) in determining ICU eligibility and/or initiation of mechanical ventilation

Full size image

In conclusion, we showed that BAIT is a feasible technique to gain insights into the triage decision process. Optimisation of the model with both retrospective and prospective validation will be our next steps towards clinical implementation. Since cultural perspectives vary between institutes and influence triage decisions, our approach should be used to develop tailored models for individual hospitals. Digitally accessing the model by an individual intensivist, delivers the collective opinion of the local group of peers, making the implicit explicit.