Background

Each year, injuries resulting from physical trauma cause worldwide over five million deaths [1]. Tens of millions of people survive an injury and may be confronted with physical or psychosocial problems due to trauma [1]. Between 25% [2]–76% [3] of patients report psychosocial problems as early as 2 weeks after injury. In addition, 7% [4]–25% [5] has psychiatric comorbidity between 3 and 12 months following injury. It is important to recognize psychosocial problems post-injury, since such problems may negatively impact physical recovery [6, 7] and patients’ quality of life (QoL) [8,9,10,11,12,13]. Psychosocial screening not only prevents problems from escalating, but may also improve communication between patients and health care providers (HCPs) and is time saving because the information provided by screening creates the opportunity to focus on issues that are important for patients [14]. Systematic screening may assist in early detection of psychosocial problems and has received much attention in oncological care [15,16,17], but not yet in trauma care. Furthermore, there is no psychosocial screening instrument currently available for an adult trauma population. Existing screening instruments are specifically developed for and validated among cancer patients [15,16,17]. Some of those questionnaires also measure physical problems [17], which may interfere with the detection of psychosocial problems [18]. Therefore, a psychosocial screening instrument should preferably only contain psychosocial problems. Existing questionnaires that are sometimes used in clinical practice mainly focus on psychological problems such as depressive and anxiety symptoms (e.g., the Hospital Anxiety and Depression Scale [19]) or post-traumatic stress symptoms (PTSS) (e.g., the Impact of Events Scale [20]). Yet, injured patients may also experience other psychosocial problems, such as impaired social life [21].

Recently, the Psychosocial Screening Instrument for physical Trauma patients (PSIT) was developed, a self-report instrument which screens for several psychosocial problems after injury. To develop the PSIT, first a systematic review was conducted to generate a comprehensive list of psychosocial problems following physical trauma (submitted). Second, focus groups with trauma patients and HCPs were organized to ask patients which psychosocial problems they experienced and to ask patients and HCPs feedback on the problems list resulting from the review and which problems they perceived as most important (submitted). Whereas studies most frequently have assessed symptoms of depression, post-traumatic stress, and anxiety [22,23,24], our systematic review and focus groups revealed that trauma patients can experience these but also other psychosocial problems following their trauma, such as a decreased self-esteem [25] and sexual problems [26]. Therefore, these problems were also included in the preliminary version of the PSIT. The aim of this study was to finalize the PSIT and examine its psychometric properties.

Method

Participants

Patients were eligible if they were 18 years or older and admitted to a ward or the Intensive Care Unit (ICU) of the ETZ Hospital, a level I trauma center in the Netherlands, from October 2016 to September 2017. Patients were invited using the Brabant Trauma Registry (BTR) database. Exclusion criteria were (i) severe cognitive impairment (e.g., dementia) and (ii) insufficient knowledge of the Dutch language. The Medical Ethical Committee Brabant approved the study. The data were collected between October 2017 and March 2018.

Procedure

Eligible participants received written explanation about the study and contact details of one of the researchers. When a patient was willing to participate, he/she was asked to sign an informed consent form, complete the questionnaires, and return all documents together in a return envelope. Patients who did not return the questionnaires were called to remind them of the study and, if they were unreachable, they received a reminder by post. After approximately 2 weeks, patients who completed the first set of questionnaires were sent the PSIT again, with a request to complete this instrument a second time to establish test-retest reliability. This approach was chosen because a smaller sample size is needed to examine test-retest reliability compared with other psychometric properties [27]. Participation was voluntarily.

Measures

Demographic and clinical information

The following variables were derived from the patient database: sex, date of birth, date of hospital admission, injury cause, injury mechanism, injury severity score (ISS), and whether they were admitted to the ICU. Patients were asked to provide the following demographic information: level of education, living situation (e.g., alone or with a partner), and whether they currently had a paid job (yes/no). Furthermore, to gain insight in pre-existing psychosocial problems, patients were asked whether they experienced psychological problems before the trauma (yes/no) and if they could briefly describe those problems (if applicable), and if they ever received counseling for psychological problems (yes/no). In addition, patients were asked if they currently received counseling for psychological problems (yes/no).

Psychosocial screening instrument for physical trauma patients (PSIT)

The PSIT is a recently developed Dutch psychosocial screening instrument for adult trauma patients. The preliminary PSIT consists of 20 items and covers the following topics: anxiety symptoms (2 items), mood disturbances (2 items), sexual problems (1 item), impaired body image (1 item), loneliness (1 item), feeling burdensome to others (1 item), inadequate social support (1 item), decreased self-confidence (1 item), employment-related problems (1 item), post-traumatic stress symptoms (3 items), impairments in social activities/leisure time (1 item), frustration (1 item), disappointment (1 item), powerlessness (1 item), anger (1 item), and relationship issues (1 item). This preliminary version of the PSIT ended with an open-ended question to provide patients the opportunity to indicate any other psychosocial problem or problems that they experienced. Each item can be answered on a 4-point Likert scale from 0 (not at all) to 3 (very much). After completion of the PSIT, patients were asked whether they found one or more items confusing or difficult (if yes, which and why), whether they missed a topic (if yes, which topic), and whether they had any remarks about the PSIT.

Patient health questionnaire-9 (PHQ-9)

The PHQ-9 is a 9-item measure to assess depressive symptoms. It is considered a suitable questionnaire to screen for depressive symptoms following injury [28]. Each symptom can be rated from 0 (not at all) to 3 (nearly every day) [29]. The total score ranges from 0 to 18. A score of at least 10 is indicative of depressive symptoms [30,31,32,33]. The PHQ-9 has shown good psychometric properties in several trauma populations [30, 31, 34, 35].

Impact of events scale-revised (IES-R)

The IES-R consists of 22 items and measures three symptom clusters of PTSS, namely intrusive, avoidance and hyperarousal symptoms [36]. Each symptom can be rated from 0 (not at all) to 4 (extremely). Scores can range from 0 to 88 and a score of 33 or higher represents the most appropriate cut-off value of PTSS [37]. Studies in several trauma populations have shown good psychometric properties [20, 37, 38].

State-trait anxiety inventory - state anxiety subscale (STAI-S)

The STAI-S is a 20-item questionnaire which measures state anxiety [39]. Each item ranges from 1 (almost never) to 4 (almost always). Despite limited research on useful cut-off values, a score of 40 or higher has been reported to reflect anxiety symptoms [40, 41]. Studies have shown that the STAI is a reliable instrument in several populations [39, 41].

Rosenberg self-esteem scale (RSES)

The RSES has 10 items and is a self-report instrument to assess global self-esteem [42]. Responses range from 1 (strongly disagree) to 4 (strongly agree). Although it has been stated that scores should preferably be analyzed in a continuous manner, scores below 15 reflect low self-esteem [43]. The RSES has good psychometric properties [42].

World Health Organization quality of life assessment instrument - Bref (WHOQOL-Bref)

The WHOQOL-Bref consists of 26 items and is the short form of the WHOQOL-100 which is developed to assess QoL [44]. Scores are calculated for one facet (Overall QoL and general health) and four domains (Physical Health, Psychological Health, Social Relationships, and Environment) [45]. Higher scores indicate good QoL [46]. The WHOQOL-Bref is a valid and reliable measure to assess QoL in patients with TBI [47] and SCI [48].

Sample size

Several recommendations exist regarding the minimum sample size needed to assess psychometric properties of an instrument [27, 49, 50]. Studies using Monte Carlo simulations revealed that a minimum of 300 participants is required for exploratory studies [50]. Specifically, to reach good test-retest reliability (i.e., intraclass correlation coefficient or ICC ≥ 0.80), a minimum sample size of 50 is advised [27]. To obtain a representative sample and to account for drop-out, we aimed to include at least 80 patients for the test-retest analysis.

Statistical analyses

To compare responders and non-responders on demographic and clinical characteristics, chi-squared and Mann-Whitney U tests were calculated. Descriptive statistics were used to create an overview of the sample characteristics. The distribution of item scores on the PSIT was explored with regard to kurtosis and skewness and by performing frequency analyses. Moreover, the presence of floor and ceiling effects was assessed using frequency analyses. Next, principal components analysis (PCA) was used to examine the internal structure of the PSIT. Appropriateness of PCA was checked using the Kaiser-Meyer-Olkin measure (KMO), which should be at least 0.06, and by Bartlett’s test of sphericity, which should be statistically significant [51]. Oblique rotation was done because correlation coefficients of the components were > 0.3 [51]. Items were considered for deletion if cross-loadings were ≥ 0.3 [27] and loadings on any of the components < 0.4 [49, 52, 53]. To assess whether the data fits the established structure, confirmatory factor analysis (CFA) was performed. Goodness of fit was tested by using the comparative fit index (CFI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). The following cut-off values were used for these measures: CFI ≥ 0.95, RMSEA ≤ 0.06, and SRMR ≤ 0.08 [27, 54]. Subsequently, presence of floor and ceiling effects were present if at least 15% of patients reported either the lowest or highest possible score on the total PSIT and subscales [55].

Reliability was measured by examining internal consistency and test-retest reliability. Internal consistency was assessed using Cronbach’s alpha coefficients (α) and values of at least 0.70 reflect satisfactory internal consistency [27]. Test-retest reliability was assessed by calculating the ICC (two-way mixed effects model, single measure) and should be at least 0.80 [27].

To examine construct validity, Spearman’s rho correlation coefficients were calculated between the PSIT subscales and the additional questionnaires. A priori, five hypotheses were formulated (Table 1). Instruments measuring a similar construct (i.e., convergent validity) should show an r ≥ 0.50, dissimilar but related constructs should show 0.30 > r < 0.50, and unrelated constructs should show r < 0.30 [27, 56]. Construct validity is considered to be good if ≥ 75% of the hypotheses are supported by the results, moderate if 50–75% of the hypotheses are supported, and poor if ≤ 50% of the hypotheses are supported [57].

Table 1 A priori formulated hypotheses to evaluate construct validity
Full size table

Receiver operating characteristics (ROC) analyses were performed to evaluate the ability of the PSIT to detect patients with psychosocial problems [58]. The area under the curve (AUC) should be at least 0.7 [27]. Furthermore, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for each potentially appropriate cut-off value, based on the ROC analyses. The most appropriate cut-off value corresponds with optimum sensitivity and specificity, which can be expressed by the Youden’s Index (J) [58]. J is a measure of diagnostic accuracy which can be calculated by the formula J = (sensitivity + specificity) - 1 [58]. CFA was conducted using IBM AMOS version 24. All other data analyses were done using IBM SPSS version 24.

Results

Patient characteristics

The BTR database contained 1729 trauma patients admitted to the ETZ from October 2016 through September 2017. Patients were excluded if they had died (n = 78), had insufficient knowledge of the Dutch language (n = 63), had severe cognitive disorders such as dementia (n = 116), did not have an injury after all according to the electronical medical file (n = 5), or if their address was unknown or incomplete (n = 6). Furthermore, 13 patients were registered twice in the BTR database. In total, 1448 eligible patients were invited to participate of which 364 returned the questionnaires (response rate: 25.1%). The PSIT was completed a second time by 128 patients (response rate: 78.5%; Fig. 1). There was no difference between responders and non-responders regarding ISS (Median = 5 for both groups, Mann-Whitney U = 173,292, p = 0.14), gender (χ2 (1, n = 1448) = 0.43, p = 0.50), injury cause (χ2 (7, n = 1346) = 9.25, p = 0.24), and ICU admission (χ2 (1, n = 1448) = 1.20, p = 0.27; Table 2). However, responders were slightly older (Median = 64.4) compared to non-responders (Median = 62.0) (Mann-Whitney U = 181,211, p = 0.02) but this was a small effect (r = 0.06). In addition, patients with penetrating injury were less likely to respond, although the effect size was small (χ2 (1, n = 1448) = 5.95, p = 0.02, phi = −0.06). Table 3 presents the demographic and clinical characteristics of the patients in the total group and of the patients included in the test-retest analysis.

Fig. 1
figure 1

Flow chart of participant selection

Full size image
Table 2 Demographic and clinical characteristics of the responders and non-responders
Full size table
Table 3 Demographic and clinical characteristics of the patients
Full size table

Internal structure

Initial PCA revealed three components with an Eigenvalue > 1, but there were several items with high cross-loadings which hampered interpretation of the structure. After an iterative process in which these items were deleted one by one and PCA was repeated, five items were deleted in the following order: ‘feelings of loneliness’, ‘problems with work/finances’, ‘feeling like a burden’, ‘excessive worrying’, and ‘more emotional’. The remaining 15 items loaded each on one component with loadings ≥ 0.4, thus revealing a simple and interpretable structure. The three components explained 64.5% of the variance and were labeled (1) Negative affect, (2) Anxiety and PTSS, and (3) Social and self-image (Table 4).

Table 4 Final results principal components analysis with oblique rotationa
Full size table

Initial CFA revealed an acceptable model fit (χ2 (87) = 240.55, CFI = 0.95, RMSEA = 0.07, and SRMR = 0.05). To improve the model fit, two correlations of two error terms were added to the model (‘Intimacy/sexuality’ with ‘Attractiveness’; ‘Re-experiencing symptoms’ with ‘Feeling upset with memories’). This resulted in an excellent model fit (χ2 (85) = 191.58, CFI = 0.96, RMSEA = 0.06, and SRMR = 0.04) (Fig. 2). Additional file 1: Table S1 presents for each item of the final PSIT the missing rates, distribution of responses, kurtosis, and skewness. The final PSIT and its instructions are presented in Additional file 2.

Fig. 2
figure 2

Final confirmatory factor model PSIT

Full size image

Reliability

A high Cronbach’s alpha was found for the total PSIT (15 items, α = 0.92), subscale 1 (Negative affect, 7 items, α = 0.91), subscale 2 (Anxiety and PTSS, 4 items, α = 0.77), and subscale 3 (Social and self-image, 4 items, α = 0.79) (Table 5). Patients completing the PSIT twice returned the second instrument on average within 19.5 ± 6.8 days. The ICC was 0.86 (95% confidence interval (CI) = 0.81–0.90), reflecting a good test-retest reliability.

Table 5 Cronbach’s alpha coefficients and floor and ceiling effects of the total PSIT and the subscales
Full size table

Floor and ceiling effects

No ceiling effects were found (Table 5). Floor effects were observed for every subscale of the PSIT, namely 26.9% for Negative affect (minimum (min) - maximum (max): 0–21), 20.3% for Anxiety and PTSS (min - max: 0–12), and 47% for Social and self-image (min - max: 0–12). There were no floor effects regarding the total PSIT (9.9%) (min - max: 0–45).

Table 6 Spearman’s rho correlations coefficients between the subscales of the PSIT and between the PSIT and the additional questionnaires
Full size table

Construct validity

All correlations between the subscales of the PSIT and the additional questionnaires were statistically significant at the p < 0.01 level (Table 6). Ten of 12 correlations (83.3%) were as expected, confirming four of the five a priori formulated hypotheses (80%). This result indicates a good construct validity.

ROC analyses and cut-off values

Figures 3a to c present the AUC curves for each subscale of the PSIT. Each scale has a high diagnostic accuracy showing an AUC of 0.92 for Negative affect (standard error = 0.02, 95%CI = 0.87–0.96, p < 0.01), 0.88 for Anxiety and PTSS (standard error = 0.02, 95%CI = 0.84–0.92, p < 0.01), and 0.92 for Social and self-image (standard error = 0.03, 95%CI = 0.86–0.98, p < 0.01). Table 7 shows per PSIT subscale the sensitivity, specificity, J, PPV, and NPV for each potential cut-off value. A cut-off score of 7 on Negative affect resulted in a sensitivity of 89.6% and a specificity of 83.4%; a cut-off value of 3 on Anxiety and PTSS showed a sensitivity of 94.4% and specificity of 90.3%; and a cut-off value of 4 on Social and self-image had a sensitivity of 85.7% and a specificity of 90.7%.

Fig. 3
figure 3

a Area Under the Curve (AUC) of subscale 1 of the PSIT versus the PHQ-9. b Area Under the Curve (AUC) of subscale 2 of the PSIT versus the STAI-S and IES-R. c Area Under the Curve (AUC) of subscale 3 of the PSIT versus the RSES

Full size image
Table 7 Cut-off value analyses for each subscale of the PSIT
Full size table

Feedback PSIT

Thirty-four patients (9.3%) reported that they found one of the questions in the PSIT confusing or difficult. One patient required assistance to complete the PIST, another patient found the item regarding re-experiencing symptoms ambiguous, and a third patient was confused regarding the difference between ‘frustration’ and ‘disappointment’. The most common remarks were that patients found the questions confronting (n = 7) and that some of the experienced problems were not related to the trauma (n = 5). In other words, only three patients had difficulty with interpreting one or more items of the PSIT. Therefore, it was decided that it was not needed to change the wording of the items or the response options.

Thirty patients (8.2%) stated that they missed a topic in the PSIT, most often related to physical or cognitive problems (n = 11) and less often to psychosocial problems (e.g., ‘feeling unhappy’, n = 2). Since the goal of the PSIT is to screen for psychosocial problems, the suggested topics were not included in the final PSIT. The optional open-ended question was retained to provide patients the opportunity to write an experienced problem not listed in the PSIT.

Discussion

The aim of the current study was to finalize the PSIT, a recently developed psychosocial screening instrument for adults following physical trauma, and to examine its psychometric properties. After PCA and CFA, the final PSIT consists of 15 items covering three subscales and one optional open-ended question to provide patients the opportunity to report any other problem they might have (Additional file 2). This study indicates that the PSIT is an easy to complete, reliable and valid self-report psychosocial screening instrument. Less than 10% of patients indicated difficulties with one or more items, but this was most often related to finding the questions confronting and only three patients actually had difficulty with interpreting one or more items of the PSIT. This supports the notion that the PSIT is easy to complete and, therefore, no changes were made to the wording of the items and response options. In addition, few patients missed a topic in the PSIT. Suggestions for additional topics were most often related to physical or cognitive problems. As such problems can be reflections of psychosocial problems (e.g., concentration problems [59]), and the PSIT is intended to assess psychosocial problems, no additional items were included.

For nearly each item on the PSIT, missing values were below 2%. Only one item had a higher percentage of missing values, namely ‘relationship issues’ (3.8%). It is plausible that patients did not answer this item because they did not have a romantic relationship, since several patients had written down that they were single. Nevertheless, this missing rate (3.8%) is still far below the threshold of a problematic missing rate of 15% or more [27].

All subscales of the PSIT had floor effects. A disadvantage of floor effects is that discrimination between patients without psychosocial problems is not possible [27]. However, the PSIT is meant to result in the differentiation in patients who do and who do not experience psychosocial problems. Any attempt to discriminate within the group of patients without problems is not possible. Therefore, floor effects are not considered problematic [27].

As expected, strong correlations were found between the subscales of the PSIT. Research shows that psychosocial problems can be related or co-existing [60,61,62]. Consequently, it was expected that the scales of the PSIT would be interrelated. Nonetheless, PCA revealed a three-component structure with an excellent model fit as demonstrated by CFA.

Concerning the construct validity, only one hypothesis for the third subscale of the PSIT (Social and self-image) could not be confirmed. Moderate correlations were found between this subscale and the RSES and domain 3 of the WHOQOL-Bref, while high correlations were expected. This could be explained by the fact that this PSIT subscale contains items related to self-confidence and social problems and therefore measures a slightly broader construct than the other two instruments, which are focused on either self-esteem (the RSES [42]) or social relationships (domain 3 of the WHOQOL-Bref [46]).

The current study has some limitations. First, response bias might have occurred as only 25.1% of the eligible trauma patients responded to the questionnaire. Analyses revealed that younger age and penetrating injury were associated with being a non-responder, although effect sizes for these variables were small. Responders and non-responders were comparable on other characteristics (gender, ISS, ICU admission, injury cause). The majority of eligible patients were not reachable. Patients declining participation and willing to provide the reason often indicated that they were not interested because they were participants in other studies, they did not experience any psychosocial problems, or they found the questionnaire too long and/or burdensome. The response rate for the second PSIT (to assess test-retest reliability) was higher, namely 78.5%. This group completed the first questionnaire and was therefore already willing to participate in this study. Second, 63 patients were excluded based on their insufficient knowledge of the Dutch language. Yet, this is only 3.6% of the total trauma population, implying a relatively low risk for language or cultural bias.

Future research should explore whether the established cut-off values are useful in clinical practice and how the referral system could be organized. For instance, to whom should referral occur (e.g., psychologist, medical social work)? Another relevant research area is appropriate timing of psychosocial screening (e.g., 1 week, 2 months post-injury). Moreover, future studies might consider exploring how the PSIT can be best implemented in trauma care. Once these questions are addressed, the PSIT could be translated in different languages to assess its cross-cultural validity.

This study also has a number of clinical implications. While various questionnaires and screening instruments are available, these mainly assess depressive and anxiety symptoms (such as the Hospital Anxiety and Depression Scale [19]), or PTSS (such as the Impact of Events Scale [20]). The PSIT is the first psychosocial screening instrument for adult trauma patients which covers a range of all relevant psychosocial problems in one instrument. Although the literature increasingly advocates to monitor trauma patients’ wellbeing, the focus is primarily on depressive symptoms, post-traumatic stress symptoms, and anxiety symptoms [24]. The PSIT screens for these symptoms but also other psychological and social problems relevant to trauma patients. HCPs in trauma care now have a tool to systematically screen for psychosocial problems, which is short and easy to complete. The proposed cut-off values provide criteria by which patients should be referred for psychosocial aftercare.

Conclusion

In conclusion, this study showed that the PSIT is a reliable, valid, and easy to complete psychosocial screening instrument. It appears to be a useful instrument to screen for psychosocial problems after injury.