Background

Adults requiring admission to an acute care hospital who are at risk of delirium comprise a heterogeneous group including those undergoing major surgery [1], older adults with non-surgical indications for acute hospital admission such as pneumonia and urosepsis [2], and in recent times the SARS-CoV-2 virus [3]. Delirium is a syndrome characterized by fluctuating mental status with marked inattention and other cognitive disturbance [4] that is attributable to one or more etiologies. Post-operative delirium is a common complication, with prevalence as high as 50% depending on surgery type and patient risk [2]. Among older adults admitted to an acute care hospital ward, 1 in 5 experience delirium [5].

The consequences of delirium are serious and include neurocognitive disturbance and cognitive decline [6], prolonged hospitalization [7], discharge to post-acute care facilities, increased caregiver burden [8], decreased functional status [9, 10], adverse events such as falls [11], and mortality [12, 13]. As delirium persists, the risk of mortality at 6 months increases [14]. Many patients whose hospitalization is complicated by delirium never return to baseline functional status. Delirium poses substantial additional costs to healthcare systems; with US healthcare costs attributable to delirium estimated to exceed $182 billion annually [15, 16].

While strong evidence indicates delirium is partially preventable through multi-component nonpharmacologic approaches [17], pharmacological prevention or treatment strategies have yet to be proven effective [18, 19]. Disparate outcome selection in trials evaluating the same intervention is an important barrier to effectively synthesizing study results, precluding the ability to developing evidence-based practices and policies [20, 21]. Core outcomes sets (COS) are an agreed-upon minimum set of outcomes to be measured and reported in all studies relating to a specific health condition [22]. COS offer a solution to reducing heterogeneity of trial outcome selection. Therefore, our objective was to undertake a rigorous international consensus process for a COS for trials of interventions, designed to prevent and/or treat delirium, for adults requiring an acute care hospital admission, but who do not require intensive care unit (ICU) admission. We elected to develop a single COS for prevention and treatment trials as many evaluate an intervention as a continuum of prevention to treatment, particularly in participant groups (those receiving anesthesia or sedation, or those with concomitant cognitive issues) in whom early confirmation of delirium is challenging. We excluded trials conducted in the ICU in this COS as we hypothesized outcomes specific to critically ill patients such as ventilation duration might be considered important.

Methods

We followed Core Outcome Measures in Effectiveness Trials (COMET) guidelines [23] for this COS development study and report on it in accordance with Core Outcome Set–STAndards for Reporting [24]. To commence the item generation process required for a COS, we conducted a systematic review of outcomes reported in published trials (1980 to December 2016) and registered trial protocols (January 2014 to December 2016) via (1) search term development in collaboration with two senior information specialists and conduct of the search across ten publication databases and grey literature; (2) two authors independently screening citations and extracting data on study characteristics, outcomes, and measures (with a third author as arbiter if needed); and (3) assignment of outcomes according to COMET taxonomy [25]. We included randomized, quasi-randomized, or non-randomized intervention studies of pharmacological (e.g., haloperidol) or non-pharmacological (e.g., reorientation, music) interventions for delirium prevention, treatment, or both, conducted in adults or children experiencing an acute hospital admission. We excluded studies conducted in ICUs and those reporting interventions to treat pediatric or adult agitation on emergence from general anesthesia. In addition, our item generation process included semi-structured qualitative interviews exploring outcomes important to delirium survivors and family members.

Item reduction and consensus methods comprised a two-round, web-based modified Delphi consensus process. To gain final consensus, this Delphi process was followed by an in-person consensus meeting, hosted by the European Delirium Association, using a modified nominal group technique [26].

Recruitment of participants for qualitative interviews, Delphi panel, and consensus meeting

We sought a purposive and international sample from three stakeholder groups: (1) clinical researchers, (2) clinicians, and (3) delirium survivors and family. We recruited delirium survivor and family participants using a multi-modal strategy, including a designated study Twitter account, snowballing (i.e., research participants passing on recruitment materials to other potential participants), and personal contacts. Our multi-modal strategy to recruit expert clinicians and delirium researchers included recruitment flyers sent through membership lists of the American Delirium Society and Australian Delirium Association and to attendees of the European Delirium Association 2019 meeting (in-person consensus meeting), announcements at the American Delirium Society 2019 meeting, personalized recruitment emails sent to corresponding authors of studies included in our systematic review, flyers posted in UK National Health Service organizations, snowballing, and personal contacts.

Semi-structured interviews

Semi-structured interviews with delirium survivors and family members were conducted by telephone by a single experienced interviewer (LR). The interview guide incorporated COMET plain language [27] to orient participants to the terms “study outcomes” and “COS.” Interviews were audio recorded, transcribed verbatim, and content analyzed by one author [28].

Delphi methods

Item reduction for identified outcomes occurred via de-duplication (i.e., removing redundant outcomes), removing outcomes related to aggregate population data rather than individual patient outcomes (e.g., number of patients receiving analgesia), and grouping similar outcomes [29]. We grouped into a single outcome those describing adverse events, side effects, and complications, and those describing study-related feasibility or process outcomes. As more items are associated with lower COS Delphi response rates [30], we further reduced outcomes by removing those identified in < 5% of studies, unless specifically mentioned in survivor/family member interview transcripts. The final list of outcomes was then reviewed for wording clarity (with lay descriptions of medical terms to aid understanding) and for domain grouping.

To conduct the Delphi, we used the bespoke DelphiManager software, Version 4 (COMET Initiative, Liverpool, UK). Participants were directed to self-select their key stakeholder group (i.e., patient/family; clinician; researcher) and to score the importance of each outcome for COS inclusion, without consideration of measurability or feasibility. Importance was scored using the Grading of Recommendations Assessment, Development and Evaluations (GRADE) Scale [31]. This is a 9-point Likert scale with scores 1 to 3 considered not important, 4 to 6 important but not critical, and 7 to 9 as critical for inclusion. This scoring method is recommended by COMET to facilitate maximum discrimination between questionnaire items [32, 33]. Participants were provided an “Unable to Score” response option and the opportunity to suggest additional outcomes. To avoid presentation bias, the DelphiManager software randomized outcome domain presentation order.

For Delphi round 1 scores, we calculated mean and standard deviation (SD) of GRADE importance scores and determined the proportion of participants rating each outcome with scores of 7 to 9 (critically important), 4 to 6 (important but not critical), and 1 to 3 (not important) for the entire expert panel, and separately for each stakeholder group. Additional suggested outcomes were deduplicated and worded appropriately for inclusion in round 2. For round 2, participants received their own round 1 scores and summarized scores, with visual representation using histograms. Participants were asked to re-score outcome importance. If a participant changed their scoring so that it moved into a new category (e.g., from “important but not critical” to “critical for inclusion”), participants were requested to provide a free-text reason for this change. For both rounds, we sent three email reminders regarding completion using the DelphiManager software.

In-person consensus meeting and nominal group technique

To inform our in-person consensus meeting, we calculated mean (SD) Delphi round 2 importance scores and determined the proportion of participants rating each outcome as critical for inclusion overall and by stakeholder group. As recommended by COMET [22], outcomes brought to the consensus meeting met the following criteria: scored as “critical for inclusion” by ≥ 70% of respondents and “not important” by < 15% considering all participants and for each of the three key stakeholder groups. No outcome that was rated by < 70% of participants as critically important overall or within a stakeholder group was brought forward to the consensus meeting.

For pragmatic reasons, we timed our consensus meeting with the 2019 European Delirium Association annual conference. We provided an overview of our meeting’s aim and structure and the Delphi results. We provided the importance scoring for the outcomes by stakeholder group to consensus meeting participants, for consideration during outcome ranking. Using nominal group technique methods, we held iterative rounds of small group and then whole group discussion. To avoid negating the Delphi process, participants were not permitted to suggest new outcomes. Participants ranked outcomes from most critical to least critical for COS inclusion at the end of each discussion.

The study was funded by the Canadian Institutes of Health Research. It received approval from the Research Ethics Boards of the University of Toronto, King’s College London, Sunnybrook Health Sciences Centre (Toronto, Canada), and the UK Health Research Authority (HRA) and Health and Care Research Wales (HCRW). Ethics approvals to recruit via social media, snowballing, and networking methods enabled recruitment from multiple countries including the USA, Europe, Asia, Oceania, and South America. Written informed consent was obtained from all study participants. The Del-CORs project is registered with the COMET initiative (http://www.comet- initiative. org/studies/details/796). We previously published the study protocol [34].

Results

Item generation via systematic review

We screened 18,933 citations, identified and extracted data pertaining to study outcomes and measures from 183 studies meeting our inclusion criteria (Fig. 1). Of the 183 included studies, 150 (88%) recruited older adults only (classified according to the study author’s participant description); most (109/183, 60%) were in post-operative patients. Delirium prevention was the primary intervention aim for 125 (68%) studies (Table 1). We extracted information on 79 outcomes reported in more than one study. These included 8 delirium-specific outcomes (Additional File Table 1) and 71 other outcomes categorized using COMET taxonomy [25] (Additional File Table 2).

Fig. 1
figure 1

PRISMA flow diagram

Full size image
Table 1 Systematic review study characteristics
Full size table

Item generation via interviews with delirium survivors and family members

We recruited 18 delirium survivors or family members from Canada, USA, and UK. From these interviews, 30 potential outcomes were identified (Table 2). The most commonly identified outcomes in the interview dataset were (1) “emotional distress, i.e., fear and anxiety related to delirium symptoms such as delusions” and (2) “delirium severity” (both identified by 50% of interview participants). Only two outcomes, “ability to return to previous lifestyle/work” and “subsyndromal delirium” (named as such by a family member with medical background) were not identified in the systematic review and brought forward for consideration for inclusion in the round 1 Delphi.

Table 2 Outcomes identified by delirium survivor and family interview participants
Full size table

Consensus building

Deduplication decisions (Additional File Table 3) resulted in selection of 31 outcomes for the Delphi round 1 (see outcomes listed in Additional File Table 4). We recruited 110 participants for the Delphi international expert panel; 15 (14%) were delirium survivors or family members of whom 7 were also healthcare providers or researchers (Table 3). Of the 31 outcomes provided in round 1, 20 (65%) met a priori consensus criteria for COS inclusion considering all participant responses, 8 (26%) by all three stakeholder groups (Additional file 1: Table 4). Compared to clinicians or survivors/family, researchers were less likely to consider outcomes critical for inclusion that related to emotional well-being, sleep, or agitation and its management (i.e., physical restraint).

Table 3 Round 1 Delphi participants
Full size table

For Delphi round 2, 8 additional outcomes were included based on suggestions in round 1. These included (1) development of incontinence, (2) nutritional status, (3) workload, (4) use of sitters, (5) family caregiver burden, (6) staff satisfaction, (7) new onset dementia, and (8) ability to participate in rehabilitation. Of the 110 round 1 participants, 77 (70%) participated in round 2. Of the 39 outcomes provided, 22 (56%) met consensus criteria for inclusion in the COS considering all participant responses, 17 (44%) by all three stakeholder groups (Table 4). Of the 8 added outcomes, none met inclusion criteria.

Table 4 Round 2 Delphi scores
Full size table

Twelve experts (including 1 delirium survivor) participated in the in-person consensus meeting. After the first round of small and then large group discussion using nominal group technique ranking exercises, 6 of the 17 (35%) outcomes were excluded. “Falls and other injuries” and “agitation” were voted out by one small group but not the second. On further discussion and ranking, these outcomes as well as mortality (causes of falls and mortality were considered multi-factorial and not delirium specific) were excluded. Delirium occurrence and reoccurrence was collapsed into a single outcome. The 6 outcomes selected for the COS for trials of interventions to prevent and/or treat delirium in adults requiring an acute care hospital admission comprised: (1) delirium occurrence and reoccurrence, (2) delirium severity, (3) delirium duration, (4) cognition, (5) emotional distress, and (6) health-related quality of life (Fig. 2).

Fig. 2
figure 2

Development of COS for trials of interventions to prevent or treat delirium in hospitalized adults

Full size image

Discussion

This study is the first to establish a COS for use in future efficacy and effectiveness trials of interventions focused on preventing and/or treating delirium experienced by adults required an acute care hospital admission, without an ICU admission. This study rigorously followed established COMET methods for COS development. We used systematic review methods and qualitative interviews with delirium survivors and family members to generate a comprehensive list of potential outcomes. Robust consensus building (Delphi and nominal group technique) methods were then used to establish six outcomes for inclusion in the final COS. This COS addresses one of the priority areas identified by the 2019 Scientific Think Tank report from NIH-funded Network for Investigation of Delirium: Unifying Scientists (NIDUS) group [35] and an Agency for Healthcare Research and Quality report recommendation [36]. The use of this COS in future trials of hospitalized adults is endorsed by the American and Australian Delirium Societies and European Delirium Association.

This COS includes three outcomes that characterize delirium itself, i.e., “occurrence and reoccurrence,” “severity,” and “duration.” Measurement of delirium occurrence, an umbrella term for incidence and prevalence, is imperative to determine the effect of interventions designed to prevent delirium. Delirium reoccurrence is relevant to trials of interventions designed to treat delirium. Fear of delirium reoccurrence was emphasized in interviews with delirium survivors and family members further highlighting the emotional distress associated with delirium. Inclusion of delirium severity reflects growing interest in its measurement as a way of understanding symptom burden, risk stratification, selection of appropriate interventions, clinical course, and prognosis [37]. Greater delirium severity is associated with worse outcomes, such as discharge to a post-acute healthcare facility and death. Similarly prolonged delirium duration is associated with worse outcomes after hospital discharge [38].

Inclusion of “cognition” and “health-related quality of life” as core outcomes is understandable considering that delirium is associated with long-term cognitive decline, including incident dementia in older adults, and has an adverse impact on health-related quality of life [6]. Similarly, inclusion of “emotional distress” is reasonable as delirium can cause considerable subjective distress such as fear, anxiety, depression, and post-traumatic stress disorder [39]. Emotional distress at the time of delirium may manifest as psychomotor agitation, an outcome that was ranked highly in the Delphi rounds despite being voted out of the final COS. Emotional distress and burden to family members is also substantial [40]; however, researcher participants in our study, in particular, rated this outcome as unimportant for COS inclusion. However, the COS represents the minimum set of outcomes recommended for inclusion in all studies; hence, these outcomes can be included in future studies despite not included in the final COS.

A group with high delirium prevalence is the critically ill [41]. Our group recently completed a COS for future trials of interventions for preventing or treating delirium experienced by critically ill adults [42]. Interestingly, the outcomes selected for this COS, that used the same processes but an expert panel with experience of critical care, were similar, with five outcomes included in both COS. Rather than “delirium duration,” participants in the critical care COS selected the outcome “time to delirium resolution” as this was felt more indicative of the end of delirium. Mortality was included as a seventh outcome which likely reflects the increased risk of death of critically ill patients [43]. Ongoing work for both these COS is to establish the measurement instruments and the time horizon for measurement for each of the outcomes.

Strengths of this study are inclusion of interviews with delirium survivors and family members in the item generation phase, a relatively large and international stakeholder panel, and adherence to COMET COS development methods. Study limitations include exclusion of non-English studies reporting outcomes as well as observational studies and non-English speaking research participants. International representation in the interview phase was limited to three nations. The majority of published studies included in our systematic review phase included post-operative patients which may limit the generalizability of our findings. Our decision to exclude outcomes identified in < 5% of studies with our systematic review may have led to exclusion of important outcomes. However, this was mitigated by their inclusion if mentioned in survivor/family member interviews and enabling Delphi round 1 participants to suggest additional outcomes for consideration. We did not ask Delphi participants to identify themselves as a survivor or family member. As such, we cannot confirm if we have sufficient representation of these distinct groups. Due to the timing of the Delphi and the in-person consensus meeting, we were unable to conduct a third Delphi round confirming importance scores of the eight additional outcomes suggested in round 1. Although patient and family representation in the Delphi was reasonable, only one delirium survivor was able to attend the in-person consensus meeting.

Conclusion

With development of this COS, we seek to promote standardized outcome selection and reporting in future trials of interventions to prevent and/or treat delirium in adults experiencing an acute care hospital admission and without an ICU admission. We anticipate widespread dissemination and adoption of this COS will facilitate faster detection of effective interventions to prevent or treat delirium due to enhanced ability to pool trial data, ultimately improving patient outcomes. Further work is now needed to operationally define the six core outcomes that will include consensus in selecting validated measurement instruments, the time horizon for measurement, analysis metrics, and method of aggregation.