Background

Critically ill patients have prolonged physical, cognitive, and mental issues, termed post-intensive care syndrome (PICS) [1]. In a previous study, approximately 60% of patients exhibited a PICS symptom 6 months after hospital discharge [2]. Similarly, PICS was observed in approximately 60% of patients with Coronavirus Disease 2019 (COVID-19) 6 months after hospital discharge [3]. PICS symptoms are obstacles to daily life and returning to work, and, thus, decrease quality of life (QOL) [4]. It is important to assess PICS after intensive care and follow-up the screened population [5].

The concept of PICS was proposed in 2013 by Needham et al. [6]. Following this proposal, research on PICS increased. Turnbull et al. conducted a scoping review on instruments to assess outcomes between 1970 and 2013 [7] and showed that numerous PICS assessment tools were used without properly standardized recommendations. Needham et al. performed an international modified Delphi consensus study to identify PICS assessment instruments for acute respiratory failure survivors [8]. This international modified Delphi consensus study did not reach a consensus for various instruments, and the Society of Critical Care Medicine proposed the recommendation of PICS assessment instruments [5]. Other groups also proposed recommendations for PICS assessment instruments [9,10,11,12].

Despite these recommendations, recent studies used different PICS assessment instruments [2, 13]. Some studies employed the Short-Memory Questionnaire (SMQ) to assess cognitive impairments [2, 3], which was not included in previous recommendations. A meta-analysis revealed variations in PICS assessment instruments among studies [13]. Therefore, it is still unclear how frequently the recommended PICS assessment instruments are used in research and clinical practice. Ten years have passed since the scoping review by Turnbull et al. [7], and, thus, an update to investigate how PICS assessment instruments have been used is required.

We conducted a two-step process involving a scoping review and modified Delphi method to investigate the recommendations of PICS assessment instruments. We initially performed a scoping review to identify, which PICS assessment instruments are frequently used. We then held a modified Delphi meeting on the screened instruments to create recommendations in clinical practice. Since clinical usefulness is based not only on the frequency of use, but various aspects are also important for identifying the appropriate recommendation to assess PICS, including scientific evidence, convenience, such as online or telephone assessments, time for the assessment, and cost. Therefore, in this Delphi process, we did not focus on a specific condition, but rather on various diseases, environments, and evaluators for applications under any condition.

Methods

Study design

The present study aimed to reach to a consensus on PICS assessment instruments. The study design was based on a scoping review and modified Delphi method (Fig. 1). The scoping review was conducted to identify eligible assessment methods that have frequently been reported in research. The Delphi method has been used to evaluate methods recommended by members with extensive experience on PICS. Since the scoping review and Delphi method were used, we did not obtain approval from an ethics committee. The present study was registered as a clinical trial (UMIN Clinical Trials Registry: 000049634).

Fig. 1
figure 1

Illustration of the scoping review and modified Delphi consensus process. CENTRAL: Cochrane Central Register of Controlled Trials, MEDLINE: Medical Literature Analysis and Retrieval System Online, CINAHL: Cumulative Index to Nursing and Allied Health Literature, ADL: activities of daily living, QOL: quality of life, JSICM: the Japanese Society of Intensive Care Medicine, PICS: post-intensive care syndrome

Full size image

Scoping review

In this scoping review based on Arksey and O’Malley’s 5-stage framework and PRISMA [14, 15], we searched for common evaluation instruments on the physical, cognitive, mental health, QOL, ADL, other, and family domains of ICU survivors. Since a previous scoping review searched until 2013 [7], we investigated the following databases between 2014 and 2022: Cochrane Central Register of Controlled Trials (CENTRAL) in the Cochrane Library, Medical Literature Analysis and Retrieval System Online (MEDLINE) via PubMed, and Cumulative Index to Nursing and Allied Health Literature (CINAHL). The key search terms are listed in the Additional file 1. We did not ask the authors of original studies for unpublished or additional data.

Data extraction and study selection in the scoping review

After record identification, data were exported into Endnote, and duplicates were deleted. After data were imported into Rayan from Endnote, 1st and 2nd screenings were conducted by four reviewers with reliable interrater reproducibility (κ value of 0.78 [95% CI: 0.68 to 0.87]), based on a previous study [16]. The 1st screening was conducted from the titles and abstracts of each set of retrieved data. We included studies on adult ICU survivors (≥ 18 years of age) and their family members at hospital discharge or thereafter. The study design included retrospective, observational, and randomized controlled trials regardless of any intervention. We excluded reviews, protocols, trial registries, case reports, conference abstracts, and studies in languages other than English. In the case of a disagreement between reviewers, a third reviewer adjudicated when needed. The 2nd screening was conducted on full texts. We included studies that assessed any PICS outcome at hospital discharge or thereafter. Exclusion criteria were classified into different languages, designs, populations, and outcomes. Data from eligible articles were extracted into the data collection format by reviewers. Data were separately input into categories at or after hospital discharge. Extraction was based on methods to assess the following outcomes: (1) physical function, (2) cognitive function, (3) mental health, (4) ADL, (5) QOL, (6) other, and (7) family domains.

The Delphi consensus process

A three-round, modified Delphi consensus process was conducted to identify desirable instruments to assess PICS. Based on the scoping review, we selected outcome evaluation instruments used more than two times. Voting was conducted three times online by the 23 members of the Japanese Society of Intensive Care Medicine PICS committee and working group members, consisting of 14 physicians, 6 physiotherapists, and 3 nurses. The evaluation of these instruments was based on the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist [17]. The information provided was summarized and attached in the Additional file 1. Voters rated outcome assessment instruments using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) scale as follows: not important (score: 1–3), important, but not critical (4–6), and critical (7–9) [18]. The evaluation was based on the clinical usefulness of the PICS evaluation, including scientific evidence and the published frequency assessed through the scoping review. Scoring included the usefulness of both screening and follow-ups. In the objective evaluation, we examined the assessment consensus as follows: When (evaluation timing): at and after hospital discharge, Where: hospital, outpatients’ clinic, and telephone interviews, Who (person to use the scores): not specific and includes voters or other staff, Whom (patients to be assessed): the general population and not limited to a specific disease, How: convenience, time for the assessment, and cost. Convenience for use in Japan, such as a verified Japanese version, was not taken into consideration. A consensus was defined as ≥ 70% of respondents rating critical and ≤ 15% of respondents rating not important in the 3rd round of the modified Delphi meeting. Scoring results and comments were provided to the voters in rounds 2 and 3, and voters reevaluated outcome assessment instruments.

Results

Literature Search

The PRISMA Flow Diagram in Fig. 2 shows the article selection process in the scoping review. The search strategy identified 6972 records, of which 754 were included in the analysis. Among 754 included studies, 114 (15%) were related to COVID-19. The timing of the PICS evaluation varied among studies, and multiple follow-ups were counted separately. Among 995 follow-ups, evaluations were frequently conducted after 3 months in 253 (25%), after 12 months in 212 (21%), and after 6 months in 204 (21%). Follow-ups were counted from hospital discharge in 239 (42%), ICU discharge in 211 (37%), and ICU admission in 120 (21%).

Fig. 2
figure 2

Flowchart for selecting articles in the systematic review. CENTRAL, Cochrane Central Register of Controlled Trials; MEDLINE, Medical Literature Analysis and Retrieval System Online; CINAHL, Cumulative Index to Nursing and Allied Health Literature

Full size image

Data extraction

All 754 records were shown in the Additional file 1: Table S1. Among 754 records, we extracted PICS assessment instruments, which are shown in Additional file 1: Tables S2 and S3. We included the following number of PICS assessment instruments from those used more than 2 times: (1) 23 items in physical function, (2) 14 in cognitive function, (3) 24 in mental health, (4) 13 in ADL, (5) 9 in QOL, (6) other: 3 in sleep, 4 in pain, 2 in fatigue, and 2 other items, and (7) 13 in family (Table 1).

Table 1 Extracted PICS assessment after hospital discharge
Full size table

The Delphi consensus

The results of 3 rounds of the modified Delphi consensus are shown in Table 1. We ultimately included 20 PICS assessment instruments (Table 2, [19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54]): (1) 3 items in physical function: the 6-min walk test, Medical Research Council (MRC) score, and grip strength, (2) 3 items in cognitive function: the Montreal Cognitive Assessment (MoCA), Mini-Mental State Examination (MMSE), and Short Memory Questionnaire (SMQ), (3) 3 items in mental health: the Hospital Anxiety and Depression Scale (HADS), Impact of Event Scale-Revised (IES-R), and Patient Health Questionnaire-9 (PHQ-9), (4) 3 items in ADL: the Barthel Index, Instrumental Activities of Daily Living (IADL), and Functional Independence Measure (FIM), (5) 5 items in QOL: Short Form-36 (SF-36), EQ-5D-5L (EuroQol-5Dimension-5Level), EQ-5D-3L (EuroQol-5Dimension-3Level), EQ-VAS (EuroQol-Visual Analog Scale), and SF-12, (6) 1 item in sleep: the Pittsburgh Sleep Quality Index (PSQI), 1 item in pain: Brief Pain Inventory, and (7) 3 items in family: SF-36, HADS, and IES-R.

Table 2 Summary of assessment instruments for post-intensive care syndrome
Full size table

Discussion

We herein conducted a scoping review and used the modified Delphi method to reach a consensus on recommendations of PICS assessment instruments. In the scoping review, 107 PICS assessment instruments were identified from the order of instrument use frequency between 2014 and 2022. After the modified Delphi meeting, we reached a consensus on 20 PICS assessment instruments for the physical, cognitive, mental, ADL, QOL, sleep, pain, and family domains. Three assessment instruments were included in each physical, cognitive, mental, ADL, QOL, and PICS-F assessment.

The assessment of physical function included the 6-min walk test, MRC score, and grip strength. The 6-min walk test was included as a PICS assessment instrument because of its confirmed validity [55, 56]; it is included in the Society of Critical Care Medicine’s consensus [5]. The advantage of using the MRC score is its convenience without the need for instruments. It is a valid and reliable tool for assessing prolonged physical impairments under careful training [57]. Grip strength allows for a quick evaluation of muscle strength and only requires a grip dynamometer. It is important to note that grip strength reflects not only grip strength, but also the whole-body strength and QOL of patients [58, 59].

Cognitive function assessments included MoCA, MMSE, and SMQ. MoCA is the most widely used for cognitive assessments, with several international consensuses [5, 60]. MMSE is often used in PICS assessments [61, 62], but may require the proper selection of a follow-up population [63]. Although SMQ was not included in previous international consensuses [5, 60], there are several positive features for its use. SMQ is a valid and reliable tool that correlates with MMSE [64]. It may also be assessed through a telephone interview or questionnaire [2], which is important for the continued follow-up of critically ill patients [65].

Mental health assessments included HADS, IES-R, and PHQ-9. HADS had the highest score in the modified Delphi meeting for anxiety and depression assessments, which is consistent with previous international consensuses [5, 60]. IES-R is used to assess PTSD [66], and is valid for evaluations of acute lung injury survivors [67]. IES-R is also recommended in the Society of Critical Care Medicine’s consensus with a clear cutoff value threshold [5]. PHQ-9 is the tool recommended by the American Heart Association to evaluate depression [36, 68] and has also been used in prolonged depression symptom assessments after COVID-19 [69]. IES-6 was not included as a recommendation in this study, but it was in the Society of Critical Care Medicine’s consensus. The infrequent use of IES-6 may have contributed to this difference.

The assessment of ADL included the Barthel Index, IADL, and FIM. In this study, the Barthel index was the most widely used scale for the ADL assessment [70]. IADL evaluates more complex skills in daily life. In a previous study, only 13% of survivors of critical illness were able to drive after hospital discharge [71]. These high-level skills are important for reintegration into society, as assessed by IADL. FIM includes communication and social cognitive functions [72]. The assessment of FIM is useful for monitoring changes in the status of a patient [73].

The assessment of QOL included SF-36, EQ-5D-5L, 3L, VAS, and SF-12. Although SF-36 involves questions on many items and requires a usage fee, it is the most widely used tool to assess the QOL of patients, reflecting the reliability and validity of this score [74]. SF-12 was included as a recommendation in this study. SF-12 is brief, but not inferior to SF-36 [75, 76]. Since numerous questionnaires may be stressful for patients [77], the brief version will be useful for future clinical use. EQ-5D was included as a recommendation, similar to a previous consensus [5]. The assessment of EQ-5D had a high score in the Delphi meeting in the following order: 5 levels, visual analog scale, 3 levels.

Other domains included sleep and pain. Many ICU survivors develop sleep disturbance and approximately 40% have chronic pain after hospital discharge [78, 79]. Therefore, these assessments are important. The assessment of sleep disturbance using PSQI [80] and pain with the Brief Pain Inventory [81] were previously recommended. It is important to note that the assessment of pain is also included in other QOL scores. These domains have been attracting increasing attention as symptoms included in PICS [82]. The management of sleep and pain symptoms warrants further study [83].

Family members of ICU survivors may also develop prolonged mental illness [84]. Family members experience anxiety, depression, and PTSD, which decrease QOL [85]. In the assessment of PICS-F, we recommend the use of HADS for anxiety and depression, IES-R for PTSD, and SF-36 for QOL. Few consensuses have been reported regarding PICS-F assessment tools [86]. Assessment tools varied among previous studies [87]. Therefore, this recommendation will contribute to the mental health issues and QOL of family members as well as ICU survivors.

Among 20 PICS assessment instruments, the following instruments had the highest scores in the final round of the modified Delphi consensus process in each domain of PICS-F: the MRC score in physical function, MoCA in cognitive function, HADS in mental health, the Barthel index in ADL, EQ-5D-5L in QOL, PSQI in sleep, Brief Pain Inventory in pain, and HADS in mental health. Because we finally identified 3 PICS assessment instruments in each domain, the single recommendation based on the highest scores may be more helpful for the PICS assessment in the future research. However, it is important to be aware that each instrument has different advantages and disadvantages and difficult to recommend a single assessment tool for each PICS domain.

Our final goal of providing PICS assessment instruments is the complete reintegration of ICU survivors into society. PICS may result in financial hardship, social isolation, suicide, and unemployment. As the first step for prevention, early identification is mandatory using these PICS assessment instruments. Furthermore, we need to provide continuous assessments and necessary interventions, including rehabilitation, nutrition management, and cognitive and psychological interventions. Although further studies are needed to validate these instruments for the assessment of PICS, these recommendations will contribute to preventing and managing PICS in ICU survivors and family members for their future reintegration into society.

Limitations

This study has several limitations. The scoping review and Delphi consensus were conducted in Japan. Although generalizability may be limited, we did not consider convenience for use of the Delphi consensus in Japan in order to maintain its validation in other countries. Furthermore, some recommendations were consistent with previous consensuses [5, 8]. Another limitation is that Delphi meeting members did not include patients, their family, or various health care providers; however, we included physiotherapists and nurses as well as physicians. Moreover, since we did not include sleep or pain in the scoping review formula, their frequency of use may have been underestimated. We also conducted the modified Delphi method on scores used more than two times. Therefore, new scores used less than three times may not have been included in the present study. Recent scores, such as PCL-5, were not included in this consensus study partly due to their infrequent use in the scoping review. In addition the classification of instruments was difficult. We classified EQ-5D-5L, EQ-5D-3L, and EQ-5D-VAS as different scores, but did not divide the brief pain inventory into a short- or full-length version. This type of classification may have affected the results of the scoping review.

Conclusions

We conducted a scoping review and the modified Delphi method to clarify the recommendations of instruments to assess PICS. Based on the results obtained, we recommended 20 PICS assessment instruments in the physical, cognitive, mental, ADL, QOL, sleep, pain, and family (PICS-F) domains. Further studies are needed to validate these instruments for PICS assessments.