Background

Both anxiety and depression are common after a heart attack and heart surgery [1,2,3,4]. In addition, acute cardiac events are often experienced as traumatic, with people having to confront their mortality for the first time [5]. Survivors of acute events have to cope with having less control over their lives than they are used to. As Vilchinsky notes, these survivors are exposed to an ongoing internal somatic threat which is not exclusively related to past experience; ‘’rather, it is chronic, and in many ways it is anchored in fears and worries about the future, vis-a-vis treatment, illness progression, potential recurrence and even death’’ [6]. The range of emotions experienced post-event can include feelings of uncertainty [7,8,9] and fear of progression or recurrence of the condition [10,11,12], leading to feelings of vulnerability [13], hopelessness [5, 13], anger and resentment [5], along with sadness and grief for the life that they envisaged now being lost or changed, and loss of usual roles and responsibilities [5, 9].

The cardiac event often challenges people’s usual ways of coping. Coping with pain and having to pay more attention to what is happening physically [5] demands new coping skills. For some, the trauma associated with the cardiac event can trigger a resurfacing of past traumas or unresolved grief, with concomitant intrusive thoughts, nightmares [5] and sleep disorders such as insomnia [14].

We have argued elsewhere for the need for a nuanced understanding of the many challenging emotions, changes and experiences that follow an acute cardiac event which we believe can be conceptualised as ‘cardiac distress’. We have defined cardiac distress as ‘’a persistent negative emotional state rather than a transient state; involving multiple psychosocial domains; that challenges a patient’s capacity to cope with living with their heart condition, the treatment of the condition, and the resultant changes to daily living; and challenges the person’s sense of self and future orientation’’ [15, 16]. Cardiac distress spans multiple psychosocial domains, thereby incorporating patients’ responses to physical, affective, cognitive, behavioural and social symptoms and experiences related to their cardiac event and their recovery [15]. Although there are some measures of the psychological and emotional impacts of a cardiac event, none of the existing measures provides a comprehensive or detailed assessment of cardiac distress, largely due to a failure to adequately conceptualise cardiac distress [16]. Moreover, no currently existing measure enables a mental health professional to identify priority areas clearly enough to offer timely tailored psychosocial intervention for a distressed patient [17, 18].

To fill this gap in psychocardiology practice, this study aimed to develop a comprehensive measure of cardiac-related distress, using best practice instrument design, as described in a previously published paper [16]. The inventory development process described here builds on our earlier investigation of the prevalence and predictors of cardiac distress which relied on clinical knowledge to categorise items on an a priori basis [19]. The present process uses both clinical and empirical evidence to develop the new instrument, which is to be known as the Cardiac Distress Inventory (CDI).

The following description of the method adopted for creating the CDI is divided into two phases: item generation and item testing. The Results section provides full details on the outcome of application of the methods described.

Method

Phase 1: item generation

There were several steps in the item generation process:

  1. i.

    Initial generation of items by a multidisciplinary group of researchers and clinicians representing the disciplines of nursing, psychiatry, behavioural health, psychology, and cardiology.

  2. ii.

    Review of generic and condition-specific measures of distress to identify the elements comprising the construct of ‘distress’ as previously defined by the research team, in those measures and to identify items which could be used as is or adapted for the CDI.

  3. iii.

    Review of cardiac-specific measures incorporating elements of distress as defined by the present authors [15]. Review of items for appropriateness for a post-cardiac event population by the multidisciplinary investigator group.

  4. iv.

    Focus group testing with cardiac rehabilitation (CR) professionals from multiple disciplines.

  5. v.

    Consultation with, and feedback from, cardiac patients (key informants) on the structure and content of the CDI.

Phase 2: item testing

Study population

Inclusion and exclusion criteria

Eligible participants were those who had had an acute coronary event, namely acute myocardial infarction (AMI), percutaneous coronary intervention (PCI), coronary artery bypass graft surgery (CABGS), valve issues, heart rhythm disturbance, spontaneous coronary artery dissection (SCAD) or cardiac arrest in the previous 12 months. Patients who did not have adequate English language proficiency to read and understand the Patient Information and Consent Form (PICF) and questionnaire were excluded. The PICF and questionnaire were aimed at an 8th grade reading level.

Participant recruitment

Participants were recruited using three separate recruitment strategies.

  1. a)

    Australian hospital patients. Participants were recruited from two hospitals in Australia, one in metropolitan Melbourne (Monash Health) and one in regional Victoria (Barwon Health, Geelong).

At Barwon Health, most participants were recruited while they were inpatients at the University Hospital Geelong following their hospital admission for either AMI, CABG or PCI. At this contact, the Research Nurse provided eligible patients with a brief study outline and sought interest in participation. Interested patients were provided with the PICF; consent was obtained to re-contact participants via telephone 6 weeks later to complete the questionnaire. A small number of participants were recruited during their attendance at CR at Barwon Health.

At Monash Health, participants were recruited during their appointment at the Cardiothoracic Preadmission Clinic, prior to hospital admission for CABG, or in the Cardiac Care Unit (CCU) for those with AMI, PCI and other cardiac conditions. At this contact, the Research Nurse provided eligible patients with a brief study outline and sought interest in participation. Interested patients were provided with the PICF; consent was obtained to re-contact participants at their routine 6-8-week follow-up appointment.

The recruitment procedure changed part way through the study due to COVID-19 lockdowns which prevented patients from attending face-to-face appointments. Thus, instead of completing the questionnaire while waiting for their clinic appointments as done initially, participants were either directed to the website of the Australian Centre for Heart Health (ACHH) to use an online link to a Research Electronic Data Capture (REDCap) questionnaire or were mailed a hard copy of the questionnaire for completion at home and return in a reply-paid envelope to the ACHH.

  1. b)

    US hospital patients. For the Stanford site, participants were recruited using study flyers that were posted in waiting rooms throughout the outpatient cardiology clinics at Stanford hospital. Study flyers were also offered to patients directly by cardiology providers and nursing staff in the clinics. In addition, the study flyer was posted online to the SCAD Alliance’s closed Facebook group, as well as the Hypertrophic Cardiomyopathy Association’s website.

  2. c)

    Social media recruitment. Participants were also recruited using direct engagement via an international social media recruitment drive through the ACHH Facebook page. Between 13 and 2021 and 23 November 2021, four Facebook recruitment posts were boosted, resulting in 12,743 people engaging with the posts through ‘liking’, sharing or commenting on them.

The study was promoted via email through the ACHH health professional networks.

Measures

The questionnaire took approximately 25 min to complete. No identifying information was collected as no participant follow-up was involved.

Demographic questionnaire:

Basic socio-demographic (age, sex, country of birth, marital status, living arrangement, employment status, educational level, close confidante, recent bereavement, financial strain, private health insurance), medical (other health conditions) and event-related information (event type, date of event, attendance at CR) was collected via self-report questionnaire, using standard questions used in previous ACHH studies.

Cardiac Distress Inventory item pool.: A pool of 74 items addressing various issues and concerns was generated, as outlined above. For each item, participants reported on whether or not they had experienced the issue/concern in the past 4 weeks by responding Yes or No. For endorsed items, participants then reported on the level of distress associated with the issue, using a response scale where 0 =‘no distress at all’, 1=’slight distress’, 2=’moderate distress’ and 3=’severe distress’.

Emotion Thermometers.: The Emotion Thermometers are single-response measures of distress (DT), anxiety (AnxT), depression, (DepT) and anger (AngT). They consist of a thermometer with numerals displayed vertically from 0 to 10. Patients rate their distress ‘over the last week’, with 0 indicating ‘no distress’ and 10 indicating ‘high distress’. A total score from all four mood thermometers (ETsum) indicates overall emotional problems. These thermometers, based on the National Comprehensive Cancer Center (NCCN) cancer distress thermometer (CDT) [20], have been shown to be a clinically sensitive measure of distress in patients with mixed cardiovascular conditions [21]. The Emotion Thermometers have good internal consistency and diagnostic capabilities in cancer patients [22] (Beck et al. 2016).

Patient Health Questionnaire-4:23 (PHQ-4): The PHQ-4 is a validated brief screener (4-items) for anxiety and depression, which combines the Patient Health Questionnaire-2 (PHQ-2) and the Generalized Anxiety Disorder-2 (GAD-2) [23]. Total scores range from 0 to 12, with 0 indicating ‘no distress’ and 12 indicating ‘severe distress’. The PHQ-4 has good reliability with pre-operative surgical patients [24] and has good prognostic value with CVD patients [25].

Kessler Psychological Distress Scale-6:26 (K6): The K6 is a brief measure of psychological distress which has been validated in an Australian general population [26]. The K6 is both an effective screening measure and an indicator of distress severity. Scores range from 6 to 30, with lower scores indicating higher levels of distress. The scale has excellent internal consistency reliability (α = .89 [27]) and very good discrimination between individuals with serious mental illness and without serious mental illness [28].

COVID-19 concern:

Participants were asked to rate their concern or anxiety about the COVID-19 situation over the past week using the same response format as the Emotion Thermometers. This question was asked to assist with ascertaining the degree to which participant-reported distress was attributable to the experience of living through the COVID-19 pandemic and associated lockdowns and other restrictions, or their cardiac condition and associated difficulties.

Data analysis

In developing the CDI scales, methods advocated as best practices in scale development were used [29, 30]. The general approach was to extract the optimal number of scales using exploratory factor analysis (EFA), confirm dimensionality and reduce and optimize the number of items using Rasch analysis, and finally examine preliminary evidence for criterion validity using concurrent measures. Rasch analysis is often used after EFA [31,32,33] since it has many processes that augment factor analysis. Perhaps the main advantage of Rasch analysis is its ability to put psychological constructs such as depression or distress, on a mathematical ruler analogous to how we would measure height or weight for example [34]. Unlike EFA which is more concerned with model fit, the Rasch model accounts for the difficulty level (or extent of endorsement) of individual items and transforms responses based on ordinal scales into the ruler – an interval scale via logits [35, 36]. Importantly, the Rasch model permits analysis of spread and redundancy across a wide range of person ability (or latent construct) scores within each factor through an item-person map [35, 37]. If we rely totally on EFA factor loadings it is possible that high inter-item correlations between items due to inclusion of similar worded items may artificially contribute to high factor loadings [38], which may be at a similar location on our theoretical ruler which is not desirable because some of these items may be redundant. Unlike EFA, the Rasch approach is able to give the researcher detailed information about how participants who have high levels of the underlying latent construct (e.g. distress) endorse different items, than those who have lower levels of the construct. Finally, since a factor solution is useful only if It can be interpreted in a meaningful way [39], an iterative process that considered both the empirical data and the clinical/theoretical interpretation of scales was used throughout.

Exploratory factor analysis

EFA was used to investigate the number of latent constructs measured by the CDI items and to match items to factors (scales). The analysis began with all 74 items with parallel analysis [40] used to inform an optimal number of factors. To avoid underfactoring (too few factors extracted) [41] we took the approach recommended by Brown (2015) [42] who stated that “factor intercorrelations above .80 or .85 may imply poor discriminant validity and suggest that a more parsimonious solution could be obtained” (p. 28). For the initial EFA, principal axis factoring (PAF) was used with a direct oblimin rotation to allow factors to be correlated. The suitability of the data for EFA was assessed using the Kaiser–Meyer–Olkin measure of sampling adequacy and the Bartlett test of sphericity. Criteria for suitability are Kaiser–Meyer–Olkin 0.8 and a p-value for Bartlett’s χ2 of less than 0.01. A scree plot was also inspected. Interpretation of EFA solutions involved the assessment of the meaning of the set of variables with high loadings on each factor, with loadings > 0.32 considered substantive [43]. Model fit and clinical/theoretical relevance of the factor loadings were examined using several non-orthogonal rotation techniques such as oblimin, promax and simplimax. Ultimately, a cluster-based oblique rotation method, which is thought to provide a more interpretable result than those of widely known rotation techniques [44] and provided the best model fit and clinical relevance with our data, was chosen.

Model fit was assessed by multiple tests and fit indices. Chi-square should be non-significant (p > 0.05), but it is not always a reliable indicator since it is sensitive to sample size and non-normally distributed data. The Standardized Root Mean Square Residual (SRMR) is also reported, as well as the Root Mean Square Error of Approximation (RMSEA), Bentler’s Comparative Fit Index (CFI), and the Tucker-Lewis Index (TLI). Hu and Bentler [45] suggest that values below 0.08 for SRMR and 0.06 for RMSEA are considered a good fit, while CFI and TLI should be 0.90 or above.

Rasch data analysis

Rasch analysis was conducted using WINSTEPS Software Version 5.2.2 [46]. The rating scale model (RSM) in Rasch was applied to estimate the parameters [47]. The RSM assumes that the distance between thresholds of adjacent options is the same across rating scales. Separate Rasch analyses were conducted for each factor identified by EFA. Rasch analysis is an iterative process where item or person estimate cycles are repeated until essential criteria for all Rasch parameters are met. At each iteration change to model fit was checked and item deletion was rejected if model fit did not improve. In retaining or deleting items clinical and theoretical importance of items within each factor was also considered. A sample size between 250 and 500 is regarded as a good size for Rasch analysis of a well-targeted scale [48]. Thus, the current sample size of 405 in this study could provide accurate and stable person and item estimates.

Central to Rasch measurement theory is the assumption of unidimensionality which assumes that all items within a given factor (subscale) measure the same underlying latent construct. Unidimensionality was assessed using point measure correlations, item fit statistics, the Wright Unidimensionality Index, and PCA of Rasch residuals as described below.

Point measure correlations

Point measure correlations were examined to assess the relationship between real observations and the predicted Rasch measures. Correlations in the positive direction indicate that observations agree with the unidimensional Rasch model.

Principal component analysis of Rasch residuals

The principal component analysis (PCA) of Rasch residuals (PCAR) was examined to assess unidimensionality of the Rasch measure. The PCAR aims to first extract the primary (unidimensional) Rasch dimension and then to examine if the remaining residuals contribute to a meaningful secondary dimension or are random noise. Unidimensionality is considered valid when the Rasch dimension explains at least 40% variance of the observed data and the eigenvalue of the first residual contrast is not greater than 2.0. If this value is > 2.0 the first contrast of the PCAR can be further examined to identify which items load on a potential secondary dimension.

Item fit statistics

Fit of individual items was assessed with infit (information-weighted) and outfit (outlier sensitive) mean square statistics (optimal range: 0.5–1.5) and standardized z scores (> 2.0 representing a statistically significant misfit at 0.05 alpha level). As a general rule, it is recommended to begin fit analysis by looking at Outfit before Infit and mean-square (MNSQ) before z-standardized (ZSTD) mean. The expected value for MNSQ is approximately 1.0, and values between 0.5 and 1.5 are considered productive for measurement [37]. If the MNSQ value is beyond this range, ZSTD must be checked – ZSTD values of 2.0 or more indicate statistically significant model misfit [37].

Person and item reliability, and separation indices

The item and person separation indices were calculated in order to give an estimate of the spread of items or individuals along the continuum of ability (endorsement). The person reliability index represents the reproducibility of the rater observations whereas the item reliability indicates the consistency of items. The separation indices are interpreted as acceptable if values are ≥ 2.0 [49]. The reliability indices were interpreted with values ≥ 0.5 regarded as adequate, ≥ 0.80 as good, and ≥ 0.90 as high reliability [50]. An item separation index greater than 3.0 coupled with reliability greater than 0.90 is an indication that the hierarchical structure of items according to level of latent trait will be stable in a new sample [35].

Wright’s Unidimensionality Index

To assess how well the observed data fit the Rasch model, Wright’s Unidimensionality Index was calculated. This is the person separation index using real standard errors divided by the person separation index using model standard errors. A value of ≥ 0.9 is indicative of unidimensionality and ≤ 0.5 suggests multidimensionality [51].

Local independence

The assumption of local independence is met if responses to each item in within each subscale are mutually independent of the responses to another item. If there is a significant correlation among the items after accounting for the latent construct, then the items are locally dependent or there is a secondary dimension of measurement influencing the correlation [35]. Residual correlations > 0.30 were considered a concern [37].

Monotonicity

This assumption was tested by evaluating the rating scale for each item based on Linacre’s three essential criteria [37]. First, to ensure stability of estimates, the number of observations for each category of the rating scale was examined to confirm that there were at least 10 observations. Second, the mean ratings for each rating were examined to verify if lower ratings were associated with lower mean person ability and higher ratings with higher mean person ability. Ideally, average category measures should advance monotonically up the rating scale.

Differential item functioning (DIF)

DIF was used to analyze the extent to which items function differently across sub-groups such as gender or age group (split at 60 years).

Concurrent validity and discriminant validity

The associations between CDI scale scores and well-established scales such as the Kessler (K-6) or the Emotion Thermometer were examined. Stronger significant association in Pearson correlation coefficients suggests support for concurrent validity [29]. Conversely, lower correlations between the CDI and established scales measuring different concepts (e.g., the COVID-19 Stress Thermometer) further supported discriminant validity [29].

Discriminant accuracy in prediction of emotion Thermometer distress

Receiver operating characteristic (ROC) analysis was used to identify the optimal CDI scale cut-off score for distinguishing whether respondents experienced clinically significant distress as defined by the established cut- off thresholds for ETsum (the sum of all four mood thermometers). The area under the curve (AUC) was used to estimate the overall discriminative accuracy of the CDI scale cut- off score relative to the established cutoff scores of ETsum (a score > 14 indicates moderate and > 20 indicates severe emotional problems). Using qualitative guidelines for interpreting AUC values, namely AUC ≤ 0.70 as acceptable discrimination, AUC ≤ 0.80 as good discrimination and AUC ≤ 0.90 as excellent discrimination.

Results

Phase 1: item generation

To identify potentially relevant items from existing measures that could be adapted and incorporated into the Cardiac Distress Inventory item pool, multi database Boolean searches in EBSCO Discovery Services (include databases such as CSA Illumina, PsycInfo, PubMed, JSTOR etc.) were conducted using the search terms ‘emotional distress’, ‘psychological distress’, ‘psychosocial distress’ ‘distress measur*’, ‘cardiac’, ‘chronic illness (inc. diabetes, cancer)’. The database search was limited to English-language scholarly articles of peer-reviewed journals published from January 1990 to 2018, when this component of the study was completed. This resulted in the identification of 39 measures (generic, condition-specific and cardiac-related) used to assess elements of distress. Figure 1 provides a description of the measures and number of items used as is or adapted for inclusion in the CDI item pool.

Fig. 1
figure 1

Measures reviewed for potential inclusion of items into the CDI item pool

Full size image

The included items from the measures listed in Fig. 1 were reviewed by the full research team to confirm their suitability from a multi-disciplinary standpoint. Where no items were selected from a particular measure, this does not mean that there were no relevant items, but rather that at the time they were reviewed, similar relevant items may already have been selected.

Inspection of Fig. 1 shows that items from the Cardiac Anxiety Questionnaire and the Cardiac Depression Scale comprised only 15% of the total item pool. The Diabetes Distress Scale, on which the CDI was modelled, contributed 13.5% of the CDI test items adapted for cardiac relevance.

Focus group testing of items

Two focus groups, comprising 27 people, were convened to test the face validity of the item pool. Focus group participants were asked to consider each of the suggested items in light of the definition of cardiac distress adopted for the study. They were asked to select those items which they believed best reflected the aspects of cardiac distress they observed in their clinical practice. The sessions were facilitated by a member of the team of investigators (AJ). One group was drawn from experienced cardiac health professionals attending an ACHH intensive course on CR (n = 14), while the other comprised members of the National Executive of the Australian Cardiovascular Health and Rehabilitation Association (ACRA) (n = 13). Two-thirds of the participants were cardiac nurses, with the remainder comprising a physiotherapist, a dietician, a psychologist, three exercise physiologists, an allied health professional, and a health promotion professional. Participants had a mean age of 41.6 years, a mean of 19.1 years of practice in their disciplines, and a mean of 9.8 years of cardiac practice. Fifty-seven per cent of the focus group participants were CR co-ordinators, seeing a mean of 128 CR patients each per annum. Item endorsement was high, and a small number of suggestions were made to add items in areas deemed not to be covered.

Conclusion of phase 1

The processes conducted in Phase 1 resulted in a pool of 144 items that were then reviewed by the full multidisciplinary development team individually and in groups to remove duplicates, ambiguities, and to refine the item pool to reflect the conceptualisation of cardiac distress which had previously been adopted. The resulting pool was then shared with patient representatives (n = 6) from Heartbeat Victoria and Heart Support Australia. This process resulted in 74 items being selected for testing. These items were grouped into seven key conceptual domains, determined a priori by the project team, namely symptoms, self-perception, concerns about the future, negative affect, self-management, social functioning, and role functioning.

Phase 2: item testing

Results

Four hundred and five people were recruited for the study: 231 from hospitals and 112 from social media. A further 62 people were recruited by other means, for example, by directly enrolling from the ACHH website, or through being introduced to the study by family or friends. Characteristics of participants are presented in Table 1. Briefly, just over half the participants were male, and almost two-thirds were aged 60 and over. Approximately half were not in the paid workforce or unemployed, and around three-quarters reported some degree of financial strain. Close to a quarter reported a history of anxiety or depression. Over two-thirds had their cardiac event within the past 3 months and fewer than half had attended a CR program.

Table 1 Characteristics of participants
Full size table

Exploratory factor analysis

Initial EFA of the 74 CDI items revealed three items that were not loading on any factor and had questionable clinical relevance: item 62 ‘Not accepting that this has happened to me’ which could be regarded as a coping strategy (i.e. denial) rather than something people are likely to find distressing; item 61 ‘Getting lost in familiar places’ did not load onto any factor and has little clinical relevance; and item 28 ‘Having difficulty meeting my everyday expenses’ did not load highly onto any factor and conceptually did not belong in any of the factors due to it not being a consequence of cardiac distress itself.

Results of the final EFA for the seven-factor solution for the remaining 71 CDI items are shown in Supplementary Table 1. Parallel analysis resulted in 7 factors varying in size from 4 items through to 14 items. The overall Kaiser-Meyer-Olkin measure of sampling adequacy was 0.938 indicating excellent suitability of items for EFA. The 7-factor solution had good overall fit (RMSEA = 0.046 (95% CI 0.043–0.048); TLI = 0.944). The correlations between factors extracted are presented in Table 2. Correlations ranged from 0.231 to 0.64 with the majority above 0.32 and Bartlett’s test of sphericity (χ2 = 14888.46, df = 2485, p < 0.001) confirming the non-orthogonality of the factors. All correlation coefficients were positive thus reflecting unidimensionality and shared variance in the latent construct, cardiac distress.

Table 2 Pearson correlation coefficients between factors extracted in the exploratory factor analysis
Full size table

Rasch analysis

Since all responses to items were of the same format (0,1,2,3,4), all Rasch analyses were conducted using the Rating Scale Model (RSM) in preference to the partial credit model. Seven separate Rasch RSM analyses were run for each factor identified by the EFA. Initial analyses were conducted with 0 to 4 soring including missing data. Issues with lack of monotonicity of items and disordered category thresholds were identified with these settings. Thus, analyses were re-run with cases that had missing data on the CDI removed (n = 20, leaving n = 384 for full Rasch analysis) and collapsed categories 0 and 1 (no binary endorsement and yes to binary endorsement but no level of distress) into 0 in order to run all analyses on a 0 to 3 rating score (2, 3, 4 recoded to 1,2,3). This collapse of categories affected only between 1 and 4 per cent of respondents who had indicated binary endorsement of an item but also indicated no level of distress caused. The 0 to 3 scoring system resulted in superior model fit, improved category thresholds, and no disorder of items in the Rasch model.

Table 3 Cardiac Distress Inventory subscales and item fit; items sorted by item endorsement within the subscale (higher to lower)
Full size table

The initial Rasch analyses identified two problems with dimensionality (Eigen values of the first contrast in PCAR above 2.0) for Factors 1 and 3. Examination of the factor loadings identified two items in Factor 1 (item 3 Being afraid of dying and item 19 Thinking about dying) and two items in Factor 3 (item 58 Not knowing how my family will cope if something should happen to me and item 68 Not knowing what will happen to other people if I die) which affected dimensionality in both scales. The project team decided to remove these four items into a new distinct subscale (Death concern) and ran separate Rasch analyses on this subscale.

After consideration of item fit, separation of items on the Wright Map, dimensionality and DIF, the following changes to the factors identified in the EFA (Supplementary Table 1) as a result of the Rasch analyses were made: Factor 1 – drop items 18, 5, 20, 59. Move items 19 and 3 to new Factor 8 (Death concern); Factor 2 – Drop items 13, 15, 64, 33, 57; Factor 3 – Drop item 54. Move items 58 and 68 to new Factor 8; Factor 4 – Drop items 35, 4; Factor 5 – Retain all items; Factor 6 – Drop items 7, 40; Factor 7 – Drop items 73, 16; Factor 8 (new) – include items 19, 3, 58 and 68.

In total, Rasch analyses resulted in a further 16 items being dropped from the 71 items used in the EFA, resulting in a 55-item CDI consisting of 8 sub-scales (Table 3). Regarding item goodness-of-fit statistics, all item infit and outfit statistics fell within the recommended fit criteria. All items had positive point measure correlations within each scale with Wright’s Unidimensionality Index ranging from 0.90 to 0.97, thus indicating excellent unidemensionality of the latent construct within each scale. The newly constructed scale (Death concern) had acceptable item fit statistics. In addition to the Rasch statistics, all scales demonstrated high to excellent internal reliability with Cronbach alpha ranging from 0.73 to 0.87. Item-total statistics and mean scale scores for each scale are presented in Supplementary Table 2. In general, this analysis corroborated the Rasch analysis. Items that had lower fit according to the Rasch analysis also had larger potential gains to reliability if deleted. Since there were no major potential gains in reliability if an item was deleted however, we chose not to delete any items from any of the scales on this basis. The final 55 item CDI and scoring instructions is presented in the supplementary material.

Criterion validity

Evidence of concurrent validity of the total CDI score was apparent with the highest correlation obtained with Distress Thermometer ratings and relatively lower correlations for anger and COVID-19 concerns (see Table 4). With regard to the K6, higher scores on the CDI total score were associated with higher frequencies of everything being an effort, feeling hopeless, being physically tired, being restless, nervous, and depressed (see Fig. 2). On all K6 items there were significant post-hoc comparisons between most levels of each item on mean CDI total score (e.g., for everything being an effort: all vs. a little, all vs. none, a little vs. most, some vs. none, some vs. a little, were all p < 0.001, a little vs. none p < 0.05).

Fig. 2
figure 2

Relationship between Kessler K-6 items and CDI total score (Mean, 95% CI)

Most items take the form ?In the past 4 weeks, about how often did you feel…?

Full size image
Table 4 Pearson correlation coefficients between total CDI score and five stress thermometer ratings
Full size table

Discriminative accuracy of the total CDI score in prediction of emotional distress

The CDI total score provided good overall discriminative accuracy relative to the established cutoff scores of the Emotion Thermometers (ETsum score > 14 indicates moderate emotional problems) with an AUC of 0.88 (95% CI 0.84–0.91) (see Fig. 3). The optimal cut-off score to indicate moderate emotional distress as indicated by the highest Youden Index value (0.593) was a CDI total score of 25.0 which provided a sensitivity of 78% and a specificity of 81%. A CDI total score of 37 was an optimal cut-off score for the prediction of severe emotional distress (ETsum score > 20) and provided 71% sensitivity and 86% specificity (AUC = 0.87 95% CI 0.83–0.91).

Fig. 3
figure 3

Diagnostic characteristics of the CDI total score in predicting moderate emotional distress (Emotion Thermometer cutoff score of 14). AUC = 0.875.

Full size image

Discussion

This study has successfully developed a comprehensive measure of cardiac-related distress. The instrument is to be known as the Cardiac Distress Inventory (CDI). Exploratory factor analysis and Rasch analysis resulted in a 55-item CDI comprising eight subscales from the items tested. Validation of the CDI against the K-6 supports its criterion validity. Evidence of concurrent validity of the total CDI score was apparent with the highest correlation obtained with the Distress Thermometer ratings and a lower correlation for the more transient COVID-19 concerns. The CDI total score provided good overall discriminative accuracy relative to the established cutoff scores of the Emotion Thermometers.

The findings demonstrate that cardiac distress is a more complex phenomenon than simply the co-occurrence of anxiety and depression. The eight subscales of the CDI assess Fear and uncertainty, Disconnection and hopelessness, Changes to roles and relationships, Overwhelm and depletion, Cognitive challenges, Physical challenges, Health system challenges, and Death concerns. Items from anxiety or depression measures comprised only a small proportion of the full item pool. We believe that these subscales comprehensively assess the broad construct of cardiac distress, thereby enhancing its clinical utility.

The instrument has broad utility for use in both research and clinical practice. In research, the full CDI can be used to provide an overall measure of distress, while individual subscales can be used to provide assessment of specific areas of distress. In clinical practice, the full inventory can be administered to assist practitioners to identify specific areas of concern for their patients or, where indicated, individual subscales could be used to assess or monitor specific areas of distress. The CDI is likely to have great utility in assisting practitioners to target and tailor therapeutic interventions to relevant areas of need for individual patients.

A particular strength of the CDI is the robust developmental process employed in this study that acknowledged both empirical evidence and clinical knowledge [39]. This has resulted in a comprehensive and fine-grained instrument. However, our search for validated instruments from which to derive items focussed largely on English-language instruments. Additionally, as Legare and colleagues have noted, studies reporting the development of instruments are generally not well-indexed in electronic databases [88] thus potentially limiting the development group’s knowledge of relevant measures.

The CDI is likely to be relevant for a range of language and cultural groups, although not in its current form. Further testing with other cultural/language groups is currently underway to develop culturally relevant versions of the instrument. This will enable cross-cultural comparison of the prevalence, severity and components of cardiac distress and will extend our understanding of the universal applicability of this construct.

Study limitations and recommendations

The present study has a number of limitations which should be acknowledged. First, the investigation was limited by its cross-sectional research design, which did not allow assessment of the test-retest reliability and sensitivity to change of the CDI. As a result, it is premature to comment on the suitability of the CDI for use in longitudinal studies and randomised controlled trials. In order to address this, future longitudinal studies are needed to assess test-retest reliability and sensitivity to change of the CDI. Second, the study sample was restricted to those who had had an acute cardiac event, therefore its utility with those with a chronic condition such as heart failure, is unclear. A longitudinal study of heart failure patients to explore the relevance of the CDI for people experiencing distress related to a chronic condition rather than an acute cardiac event is warranted. Finally, assessment of the discriminant validity of the CDI was outside the scope of the present study. To address this, future studies could compare distress levels for specific cardiac conditions and could investigate the impact of co-morbidities on both distress prevalence and severity.

Conclusion

In our original conceptual paper, we called for more nuanced understanding of cardiac distress and for the development of a measure that reflected its complexity. We have addressed this by developing the CDI. We look forward to the use of this 55-item version in both research studies and as a clinical assessment tool to guide counselling practice. A short-form version of the CDI will be developed for use as a screening tool in settings such as primary care, cardiac rehabilitation, and counselling services.