Introduction

ICU survivorship may come at a price—the price of cognitive [1] and physical dysfunction [2], psychiatric and psychological problems [3], financial and work-related shortcomings [4] and healthcare consumption [5].

To identify and describe these problems, the intensive care community uses a synthesis of tests, examinations and questionnaires depending on the context: SF-36 and EQ-5D are the most commonly used questionnaires for measuring health-related quality of life [6, 7], but concerns regarding their ability to identify issues valued by ICU survivors have been raised [8]. Within the domains of physical, cognitive and mental health, the concept of PICS (post-intensive care syndrome) points out directions for investigations rather than provides scales. All three of these domains have numerous specific measurements, with for example at least 26 different tools to measure functional outcome [9]. Furthermore, discriminating post-ICU issues from prevalent psychological and physical ill-being in the general population is challenging when problems overlap [10, 11].

We hypothesized that a questionnaire mainly based on interviews with ICU survivors would contain a majority of issues experienced after intensive care, as well as carry a discriminative capacity to identify those issues with a magnitude distinct from a non-ICU-treated population. Influenced by advances made in oncology [12], we applied an established qualitative methodology [13, 14], using interviews to identify and extract issues from survivors. In an attempt to encase the full extent of the problems, interviews were not limited to particular areas.

Our aim was to develop a provisional questionnaire based on the content of such interviews, test its practicality in a scientific setting, as well as its ability to identify differences in the magnitude of issues between ICU survivors and non-ICU-treated controls. This would be a first step toward a questionnaire for long-term follow-up after intensive care. It could be useful to healthcare providers in post-ICU clinics as well as in primary care to identify clinical problems in need of specific treatments, consulting or referral to rehabilitation. In a research setting, it could be practical as an outcome measurement when evaluating issues and their trajectories.

Comparing the results from an ICU survivor group with those from a non-ICU-treated group gives two advantages. First, a comparison between an ICU survivor group and a non-ICU-treated group will aid a future reduction in the number of items. Second, by being able to measure the degree to which issues are related to intensive care and not to problems common in a non-ICU-treated population, the questionnaire could be used to identify domains suitable for interventional trials.

Methods

Interviews and development of a provisional questionnaire

Methodological framework and considerations

We took a pragmatic approach inspired by our earlier experience of developing instruments in oncology based on interviews [13, 14]. Thus, the method applied in this study follows recommendations of EORTC (European Organization of Research and Treatment of Cancer) and the Division of Clinical Cancer Epidemiology, Gothenburg, Sweden [15, 16].

While the methodology has similarities with Grounded Theory such as using data saturation as an endpoint and the parallel process of data collection and analyses, there are important differences. For example, we use an interviewer with clinical experience and domain knowledge. As recommended by the EORTC findings from the literature, other scales and questionnaires may be shown to the interviewee to evoke further thoughts in the second part of the interview. In summary, our methodology aims at creating an as comprehensive list of symptoms and issues as possible, where keeping the exact wording of the interviewees is important to minimize interpretations.

Setting and study population

Our post-ICU (16-bed mixed ICU in a university hospital) plans a scheduled visit six months after ICU discharge for survivors with an ICU length of stay of at least 72 h. Survivors may also contact the clinic for a visit or be invited at the discretion of the post-ICU clinic nurses. All survivors visiting the post-ICU clinic between February and May in 2015, with at least six months from ICU discharge, were eligible for the study.

Sampling strategies

Using a purposive, maximum variation sampling approach, potentially “information-rich” interviewees representative for different ages, gender, admission diagnoses, ICU length of stay, time from ICU discharge and postal areas as a marker of socioeconomic status were invited to participate [17]. They were selected by one of the researchers (J.M.), who had not been involved in their care but met them at their visit to the post-ICU clinic to gain trust and explain the study. Sample size was based on data saturation—the point where no new information emerged [18]. To confirm saturation, we decided a priori to continue the data collection for an additional three interviews.

Interviews

Interviews were conducted either in the post-ICU clinic or in their home, based on the interviewee’s own choice. There was no time limitation for the interviews, and participants were interviewed only once. Using a semi-structured technique, we explored their current situation as well as symptoms, difficulties, quality-of-life issues and social effects arising at any point after ICU discharge. All interviews started with the question “We are asking for your help in creating a questionnaire which will be used to identify and follow the experiences of patients who have survived intensive care. I would like to ask you a few things about your health. Can you tell me about the experiences you may have had as a result of your intensive care stay, and the time between discharge and today?”. While initial questions were open-ended, as interviews progressed details about findings were sought for. Once the interviewee could think of nothing further, domains and issues from previous interviews, literature or other scales and questionnaires were discussed (Additional file 1: Table S1). Examples of interview questions, probes and prompts can be seen in Additional file 4: Figure S1. Field notes were taken by the interviewer and read to the interviewee at the end of the interview to allow for comments or corrections.

Data analysis

In parallel with conducting the interviews, already transcribed interviews were independently analyzed by two of the researchers (J.M. and A-C.W.). Analyses were made manually: Long quotes were shortened while preserving the core meaning of the issues [16]. All extracted issues were categorized into domains, and duplicates were removed. To ensure that important issues were retained as well as to minimize the risk of recall bias, issues only had to be mentioned once to be included in the provisional questionnaire. No items were excluded in this phase since an item reduction will be performed at a later stage. The remaining issues were rephrased as questions, where care was taken to maintain the wording used by the interviewee.

At the time of data analysis, the first researcher (J.M.), a male intensive care physician, had two years of experience in qualitative research and several years of experience with post-ICU care. The second researcher (A-C.W.), a female gynaecological oncologist, had ten years of experience in qualitative research in the Division of Clinical Cancer Epidemiology, Gothenburg University, Sweden and 14 years of experience in the EORTC Quality of Life Group.

Additional questions

Composite questions about domain-specific quality of life and domain-specific future concerns were added at the end of each domain (How much do you think problems within [domain] affects your quality of life? For the past month, have you been worried about your future regarding [domain]?). Empty space and a request for missing issues or other comments were provided after each domain.

Questions regarding demographics and comorbidities were added at the end of the questionnaire.

Response scales

The response scales used are based on the established experience of the Division of Clinical Cancer Epidemiology, Gothenburg, Sweden, and were created to match each conceptual entity as closely as possible using incidence, prevalence, intensity and agreement when applicable (Table 1) [13].

Table 1 Examples of wordings and response scales of questions asked in the questionnaire
Full size table

Because our interest lies in long-term effects, the time frame asked about in most response scales was "the last month." This would also minimize the problem of recall bias. Care was taken not to overlap between alternatives and to include "Not applicable" if needed.

Content validity and cognitive interviews

Evidence of content validity as a measurement property refers to the extent of which an instrument contains the relevant aspects of the construct it intends to measure, in our case “issues experienced after intensive care” [19]. However, the content is not only the issues raised within the questions, but also such aspects as the wording of questions, the clarity of instructions and proper response scales [18]. In accordance with the ISPOR (International Society for Pharmacoeconomics and Outcomes Research) guidelines on testing for evidence of content validity, all questions were tested with cognitive interviews on additional ICU survivors chosen with the same criteria as the initial interviewees, and with the same saturation-based sample size [20]. These interviews were recorded as well. The cognitive interviews were the final opportunity to make content changes before administering the questionnaire to a larger group and included appropriate response scales and recall period. The aim was to ensure that the questions were conceptually clear, easily understood, perceived as relevant and to make sure no important issues are missing. Interviewees were initially instructed to complete the questionnaire while thinking aloud, but as the two first interviewees failed to follow these instructions, we changed to a retrospective probing technique, where questions were asked after finishing each domain, in line with EORTC’s guidelines when a questionnaire has a substantial number of questions [15].

Application of the questionnaire

Eligible patients were all adult ICU survivors admitted between February 2013 and December 2015 to one of three mixed ICUs in Sahlgrenska University Hospital, Gothenburg, Sweden (in total 31 ICU beds), and with a minimum ICU length of stay of 72 h. They all had been discharged from the ICU between six months and three years prior to the study. Exclusion criteria were primary neurological/neurosurgical reason for admission, limited understanding of Swedish as judged by study personnel, no Swedish personal identity number, no Swedish address or phone number or a secret Swedish personal identity. We obtained a non-ICU-treated control group from the Swedish Population Register, matched for age and sex with respect to ICU survivors having returned a completed questionnaire. For the version of the questionnaire addressing the control group, we removed all questions requiring a previous ICU stay (e.g., Have you had difficulties describing your ICU experiences?) and added one question checking for previous intensive care. Exclusion criteria for the control group were previous ICU stay or a limited understanding of Swedish.

All eligible participants received an initial letter with information about the study, and within a week they received a phone call asking for participation. The questionnaire was sent together with a pre-paid return envelope, and reminder phone calls were made if the questionnaire was not returned within two weeks. The questionnaire was sent to the ICU survivors between April 2016 and October 2017, and to the control group between March 2017 and December 2017.

Statistical analysis

Univariate descriptive statistics are presented as frequencies and percentages for all categorical variables. Continuous variables were screened for normality using Shapiro-Wilks (p > 0.05) and box-plots. For non-normally distributed continuous variables, median and range or median and interquartile range (IQR) are reported. Bivariable comparisons were made between ICU survivors and the control group for all ordered categorical variables and continuous variables in order to identify differences between the groups by applying the Mann–Whitney U test. These results are presented as means and mean rank sums and the associated p-value calculated in the Mann–Whitney U test. In addition, all the bivariable comparisons for ordered categorical variables were analyzed with Fisher's exact test as a robustness check. Dichotomous variables were also assessed with Fisher's exact test. All tests were two-tailed, and significance level was set to 0.05.

Questionnaires were scanned with Remark Office OMR (Remark Office OMR 10, Gravic Inc, Malvern, USA). Statistical analyses were performed using the IBM SPSS v26 package (IBM SPSS v26 Statistics, IBM, Armonk, USA).

Results

Interviews and development of a provisional questionnaire

Study population

The median age of the interviewees was 55.5 years (range 20–82), and 33% were females. The interviews took place at a median of 14.7 months (range 7.6–68.0) after ICU discharge. The median ICU length of stay was 4.9 days (range 1.7–76.1), and the median SAPS 3 score was 57.5 (range 24–81). Seventy per cent were treated with mechanical ventilation for a median time of five days (range 1–62). The most common primary diagnosis was infection/sepsis (18.8%), followed by trauma and cardiac arrest as second and third most common (both 12.5%) diagnosis (Table 2).

Table 2 Clinical characteristics of ICU survivors and interviewees
Full size table

Interviews

All invited patients accepted to be interviewed. In total, 32 interviews including six cognitive interviews were performed. Ten of the interviews were conducted in the presence of a partner. Apart from one interview conducted in the interviewee’s home and one conducted in a public location, all interviews were conducted in the post-ICU clinic. The average time of interviews was 49 min (range 15–113). Minor language corrections were made based on the cognitive interviews, but no new issues were identified. No question was considered upsetting, no response scale was changed, and no time frame was adjusted.

Data analysis

Quotes from the interviews generated 437 issues. By removing duplicates and similarities, they were reduced to 195 unique issues (Additional file 5: Figure S2). These were rephrased as questions and categorized into 13 domains: cognition, fatigue, physical health, pain, psychological health, activities of daily living, sleep, appetite and alcohol, sexual health, sensory functions, gastrointestinal functions, urinary functions and work life.

Additional questions

For the questionnaire, 31 composite questions regarding domain-specific quality of life and domain-specific future worries were added. Twelve questions from other scales and questionnaires were considered relevant by the interviewees and were added: All three questions from AUDIT-C [21], four questions from the KATZ-ADL index [22], four questions from the Work Ability Index [23] and one question about the ability to walk for six minutes [24]. The distribution of questions is shown in Table 3. In the version of the questionnaire for the control group, twenty questions requiring a previous ICU-stay were removed.

Table 3 Domains and number of questions in the questionnaire sent to ICU survivors
Full size table

Response scales

A majority of questions was measured on an ordered category scale: 113 questions on a 6-point scale, 91 questions on a 5-point scale, eight questions on a 4-point scale, two questions on a 3-point scale. Twenty-two questions were measured on a dichotomous scale and two questions were quantitative. Higher scores indicated higher levels of difficulties or problems except in eleven reversely coded questions where higher scores indicated lower levels of problems (e.g., Do you have the ability to look forward to things?; No—Rarely—Sometimes—Quite often—Very often—All the time).

Application of the questionnaires

A total of 518 ICU survivors and 289 controls received a questionnaire. Among these, 395 ICU survivors and 195 controls returned a completed questionnaire, the return rates being 76.2% and 85.3%, respectively (Fig. 1). The most commonly stated reason for declining participation among ICU survivors was family members declining (1.8%; 10 of 567) and among controls the reason was "No time" (2.2%; 6 of 276).

Fig. 1
figure 1

CONSORT diagram of screening, recruitment, follow-up and reasons for non-participation

Full size image

The most frequent ICU admission diagnosis for survivors was infection/sepsis (27.8%; n = 110) followed by trauma (13.4%; n = 53) and respiratory failure (10.9%; n = 43). Median SAPS 3 score was 59 (range 16–100), and median ICU length of stay was 5.6 days (range 3.0–78.6). Most ICU survivors were mechanically ventilated (78.5%), with a median time of 4.0 days (range 0–74). The representation of the major diagnosis groups was fairly similar between the ICU survivors and the interviewees (Table 3).

Demographics and comorbidities

There were no differences in age and gender between the ICU survivors and the control group (Table 4). While there were no differences in educational levels between the two groups, significantly more ICU survivors were on sick leave/sickness benefit compared to the control group (p = 0.000). The ICU survivors were also sicker compared with controls, differing significantly in 13 of 22 comorbidities (Table 5). Cardiovascular disease (hypertension, angina pectoris, myocardial infarction and heart failure) was more common among the ICU-survivors as was respiratory disease and pulmonary embolus. The ICU survivors suffered more often from depression and anxiety. Diabetes, kidney disease and bowel disease were also more common in this group. The need for walking aids or wheelchair due to physical impairment occurred only among ICU survivors as did having amputated limb(s).

Table 4 Demographics for ICU survivors and controls at the time of answering the questionnaire
Full size table
Table 5 History of comorbidities for ICU survivors and controls at the time of answering the questionnaire
Full size table

Symptoms and burden of disease

At the time of completing the questionnaire, which was between six months and three years after discharge, the ICU survivors differed significantly in a majority of questions across all domains when compared with the control group (Additional file 2: Table S2a and Table 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18). Examples are:

Table 6 Means and mean rank sums for the domain on cognition—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 7 Means and mean rank sums for the domain on fatigue—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 8 Means and mean rank sums for the domain on physical health—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 9 Means and mean rank sums for the domain on pain—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 10 Means and mean rank sums for the domain on psychological health—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 11 Means and mean rank sums for the domain on ADL—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 12 Means and mean rank sums for the domain on sleep—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 13 Means and mean rank sums for the domain on appetite and alcohol use—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 14 Means and mean rank sums for the domain on sexual health—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 15 Means and mean rank sums for the domain on sensory functions—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 16 Means and mean rank sums for the domain on gastrointestinal functions—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 17 Means and mean rank sums for the domain on urinary functions—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table
Table 18 Means and mean rank sums for the domain on work life—Comparison between ICU survivors and controls at the time of completing the questionnaire
Full size table

Cognitive difficulties affecting quality of life such as losing the thread easily, think you have done something you haven’t, mistaken which day of the week it is, or difficulties taking initiatives.

Fatigue was a common symptom often with the need for a daytime rest. It could be tough getting started doing things, difficult finishing things due to feeling exhausted or doing things under pressure of time or multitasking. Getting tired from reading, or from conversation between more than two people could affect work and limit social activities.

Physical health in general was more often affected among ICU survivors. They could suffer from reduced body feeling, muscle weakness in arms or legs, dizziness when standing up, losing balance easily, difficulties climbing stairs, unsteady gait, contractures and shortness of breath, many of them limiting physical activities.

Pain was reported from different parts of the body or as general body pain by survivors making painkillers necessary for managing ADL or to get sufficient sleep.

Psychological health problems were also overrepresented among the survivors. It made them cry more easily. Feeling low-spirited or depressed, or suffering from panic attacks, were also more common as was feelings of hopelessness and feelings of life being meaningless. Many suffered from low self-confidence.

Activities of daily living (ADL) were often more difficult for the survivors. They could need help with things like getting dressed, moving from bed to chair, visiting the toilet, shopping, cooking and doing housework. Help with medication and managing bills was also more common.

Sleep could be affected in many ways, at worst as nightmares.

Poor appetite, bothersome thirst and difficulties chewing more often affected quality of life for the ICU-survivors. Together with reduced taste, mouth dryness, mouth soreness or mouth pain, swallowing were difficult and made it easier to choke.

Sexual health issues like libido and sex life were less satisfactory in ICU-survivors.

Work life differed between the two groups. For the survivors the capacity for work was negatively affected both due to physical demands and psychological demands. Work problems as well as financial problems were more common among ICU-survivors.

A complete list of all questions and their response rates is shown in Additional file 2: table S2a/Additional file 3: S2b. All continuous variables were found to deviate from normality in both groups. No additional valuable information was added to the space after each domain.

Discussion

Our study describes a first step toward an intensive care long-term follow-up questionnaire with the capacity to detect the burden of ICU survivorship and the effect on quality of life. By creating a questionnaire from interviews with ICU survivors and testing it on ICU survivors and a non-ICU-treated control group, we were able to show that the questionnaire contained most issues experienced by survivors and was able to identify differences between the two groups. While issues found in our interviews may apply to a general population to a certain degree, it is our belief that many might worsen after intensive care. A comparison with a non-ICU-treated control group may help describe to what extent intensive care can be attributed to a change in magnitude rather than simply describe a prevalence.

At a stakeholders' conference in 2010, the concept of PICS was created to enclose impairments in mental health, cognition and physical functions [6]. At a follow-up conference in 2012, the PICS group pointed out the need for outcome assessment tools created with qualitative methods [25]. Several groups have addressed this issue, either by developing new instruments or by examining the evidence of content validity of existing ones. Jeong and Kang reported the development and validation of a questionnaire specifically for the three domains of PICS, using a methodology similar to ours [26]. In 2018, Nedergaard et al. interviewed 18 ICU survivors and extracted the most important issues [27]. Although symptoms from the PICS domains were well represented, additional symptoms were also considered important, for example incontinence, short temper and the feeling of being isolated. Furthermore, large differences between patients and clinicians when ranking the importance of symptoms have been found in areas as diverse as bariatric surgery [28], diabetes [29] and aphasia [30]. These findings would argue toward instruments developed with input from former patients.

Regarding measuring HRQoL (Health-Related Quality of Life), SF-36 and EQ-5D are currently the most commonly used tools after intensive care. Lim et al. extracted post-ICU issues from 30 ICU survivors and let the same patients compare these issues with SF-36 and EQ-5D [8]. Of the domains identified as relevant by the ICU survivors, only one was considered adequately covered by SF-36 or EQ-5D. The remaining domains were either inadequately covered or completely missing, suggesting that the use of either of these instruments as a measurement of post-ICU HRQoL will miss important issues. In another study, Jensen et al. were unable to show improvement in HRQoL measured by SF-36 after their ICU recovery program and recommend new instruments to be developed and validated to assess the particular HRQoL problems of post-ICU patients [31].

Our questionnaire focuses on different measures of ICU survivorship: First of all, it covers the areas of PICS; physical, cognitive and mental health. Secondly, it describes quality-of-life related to the survivor's health status. Further, areas not covered by PICS or SF-36/EQ-5D such as dysphagia [32], joint contractions [33], sleep disturbances [34] and personal finances [35] are included, all previously described problems after intensive care.

Strengths

The response rate of 76.2% from the ICU survivors and 85.3% from the control group indicates not only the usability of the questionnaire in a trial context, but that questions were considered relevant. Participants did not provide any additional issues in the comment areas in the questionnaire, only encouraging comments, arguing toward evidence of content validity in our questionnaire rather than "questionnaire fatigue."

The development of the questionnaire follows international recommendations for development of patient-reported outcome measures [36]. Choosing interviewees purposively instead of in a consecutive order has been the most effective for reaching data saturation with minimum sample size in simulations [37] Data saturation is the most commonly used delimiter for sample size but not randomizing the order of the interviews poses a hypothetical risk of affecting the saturation point [38]. Therefore, we decided à priori to set the sample size to when three consecutive interviews did not provide any new information.

We took several steps to show evidence of content validity: First, we based this provisional questionnaire mainly on issues reported by ICU survivors themselves. Second, all interviewees were read the field notes to ensure our proper understanding of issues. Third, we used cognitive interviews, and finally we allowed all participants to add potentially missing issues in the quantitative phase.

Limitations

This being a provisional questionnaire awaiting further analyses, there are some limitations. Even if a high response rate indicates the relevance of the questions asked, this questionnaire will be reduced. Differences in comorbidities between ICU survivors and the control group, where 13 of 22 differed significantly, align with previous findings that chronic comorbidities are common in ICU patients [39]. We do not know to what extent these comorbidities explain the differences between the two groups. The questionnaire has not been developed from, nor tested on, patients with a shorter time from ICU discharge than six months; hence, our questionnaire may miss issues that resolve completely within this time frame. Nor was the questionnaire developed for patients with an ICU length-of-stay shorter than 72 h or with neurological/neurosurgical primary diagnoses, and results cannot be generalized to these groups. We cannot exclude that interviewees may have forgotten issues experienced between ICU discharge and the interview, and thus our questionnaire cannot claim to be comprehensive. However, by including all issues appearing in interviews, no matter how uncommon, we have attempted to minimize the impact of potential recall bias. In the second phase, recall bias was accounted for by using ‘the last month’ as time frame in the provisional questionnaire. Although we have shown that a majority of issues differed significantly in magnitude in comparison with a non-ICU-treated control group, we do not know to what extent these differences were already prevalent before intensive care. Regarding internal validity, there was a difference in age between the interviewees and the cohort groups. However, ranges of age, SAPS score etc. did not differ markedly. Finally, we cannot exclude a selection bias with regard to patients who chose not to participate or who we were unable to reach.

Conclusions

This study describes the development of a provisional questionnaire for long-term health-related quality of life and burden of disease after intensive care. This first version, based mainly on issues from interviews with ICU survivors, clearly identified burden of disease affecting multiple domains in a large group of patients. The next steps in order to make this questionnaire a useful tool for follow-up after intensive care include further statistical analyses including psychometric properties and reduction in the number of questions.