Background

Cancer can impact all aspects of patients’ lives, including physical, emotional, and practical consequences [1]. Many patients in the community experience significant levels of distress, with estimates ranging from 25 to 50% [2]. The term distress is not limited to mental health issues, rather it is defined as, “a multifactorial unpleasant experience of a psychological (i.e., cognitive, behavioral, emotional), social, spiritual, and/or physical nature that may interfere with one’s ability to cope effectively with cancer, its physical symptoms, and its treatment” [3]. Given the prevalence of distress in the cancer experience, as well as its impact on physical health and cancer outcomes [4, 5], screening for distress has become an integral part of caring for the whole patient in cancer care [1]. Notably, distress screening is now mandated for cancer centers to achieve continuing accreditation by the American College of Surgeons Commission on Cancer as well as to meet requirements for American Society of Clinical Oncology Quality Oncology Practice Initiative (QOPI®) standards, Oncology Care Model (OCM) quality measures, and National Comprehensive Cancer Network (NCCN) distress screening guidelines [3, 6,7,8].

To meet the demand for integrated and effective distress screening in cancer care settings, there is a well-established need for reliable and valid methods of screening and referring cancer patients for distress management [1, 9, 10]. There is also increased focus on the importance of understanding the feasibility of, and barriers to, implementing distress screening programs in real-world practice settings [10,11,12]. Developing distress screening tools that are both psychometrically sound and feasible for patients and providers is particularly important for uptake in community-based cancer care settings, in which the need for assessment and demand for resources may be substantial relative to the number of trained support staff.

In response to this critical need, Cancer Support Community (CSC) developed CancerSupportSource® (CSS), a reliable, valid, multidimensional distress screening program with the capacity to identify those at risk for clinically significant levels of anxiety and depression. CSS is web-based and, in full implementation, includes a follow-up and referral program to assist community-based cancer centers in meeting distress screening accreditation standards linked to patient-centered cancer care [13].

Development and comprehensive psychometric validation of CSS has been previously described [14]. CSS assesses distress over five key domains (emotional well-being, symptom burden and impact, body image and healthy lifestyle, health care team communication, and relationships and intimacy) and also includes an item assessing tobacco and substance use. The five-factor model is replicable and the risk subscales demonstrated high sensitivity and adequate specificity [14].

CSS is currently implemented across CSC’s network of community-based cancer support affiliates as well as in oncology practices and hospital cancer centers nationwide. In community-based care settings, there is a need for flexibility in assessment while still maintaining instrument fidelity. The creation of shorter scales is a well-established practice when an abbreviated measure is needed due to variety of practical considerations, including being used for a different purpose (e.g., in clinical trials versus longer clinical evaluation) [15], or in different settings (e.g., fast-paced medical clinic versus explanatory research) [16]. Availability of a concise, abbreviated version of CSS may enhance opportunities for distress screening in care settings where time and patient burden considerations are significant.

The aims of the current study were to (1) develop a shortened version of CSS and (2) to examine the psychometric properties of the shortened scale.

Methods and results

An abbreviated version of CSS was guided by best-practice guidelines, including using internal, external, and judgmental item characteristics to inform item retention decisions [17]. Development and validation were completed in two phases with two separate samples of participants: (1) scale reduction and (2) corroboration of psychometric properties and dimensionality.

Phase I: scale reduction

Participants

Participants for scale-reduction decisions were the same as in the CSS validation study [14]. Data were collected through the Cancer Support Community’s Cancer Experience Registry (CER), an online, community-based research initiative examining the social and emotional impact of cancer. Participants in the validation study included 1436 cancer patients and survivors (Table 1) who participated in the CER survey from March 2013 to December 2016, were 18 years or older, lived in the USA, completed at least 22 of the 25 distress items, and also completed at least one comparative validation measure.

Table 1 Descriptive characteristics of the sample
Full size table

Ethical and Independent Review Services (E&I, Independence, MO) served as the IRB of record. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the CER.

Measures

Socio-demographics and clinical history: participants provided demographic and clinical background information (age, gender, race, ethnicity, education, employment status, household income, self-reported cancer diagnosis, stage at diagnosis, time since diagnosis, and types of treatments received.

CancerSupportSource: cancer-related distress was assessed using the 25-item version of CSS (CSS-25). Patients rated their level of concern (0 not at all, 1 slightly, 2 moderately, 3 seriously, and 4 very seriously) for each item; request for follow-up services was not assessed for the current study.

PROMIS: participant self-reported symptoms and functioning were examined using the Patient-Reported Outcomes Measurement Information System-29 (PROMIS-29 v2.0) [18]. Five domains assess symptoms with higher scores corresponding to worse symptomology (depression, anxiety, pain interference, fatigue, and sleep disturbance) and two domains assess function with lower scores corresponding to worse functioning (physical function, ability to participate in social roles and activities). Participants rate each item with reference to the past seven days; function scales have no timeframe specified. Scale scores are converted to standardized T scores (mean = 50, SD = 10); normative reference groups are the U.S. general population, except sleep disturbance, where comparisons are to a mix of the U.S. population and people with chronic illness.

Analysis

Data analysis was conducted using IBM SPSS Statistics 24.0 [19] and R 3.4.0 [20], with GPArotation [21] and psych [22] R packages. For each item in CSS-25, external item quality, internal item quality, and judgmental item quality were assessed independently by authors; these three indices were then reconciled in a series of consensus meetings to reduce the scale to a shortened version:

External item quality: external item quality was assessed by examining Pearson correlations between CSS-25 items and PROMIS-29 subscales.

Internal item quality: internal item quality was assessed by evaluating item discrimination indices, inter-scale and inter-factor correlations, factor loadings and structure, and item communalities from an exploratory factor analysis of CSS-25. The exploratory factor analysis, which was previously published in the validation of CSS-25 [14], was conducted with direct oblique rotation and principal axis factoring (PAF) extraction.

Judgmental item quality: judgmental item quality involved the ranking and prioritization of all CSS-25 items by CSS-25 developers, accounting for theoretical and practical implications, with particular attention to content validity.

Results

Participant socio-demographics: participants were predominantly female (72%), White (90%), and completed a college degree (65%). The average age was 58 years (range = 19 to 87), and average time since initial cancer diagnosis was 4.6 years (range = < 1 to 52). The most commonly represented diagnoses included breast cancer (35%) and hematologic cancers (35%; Table 1).

External item quality: scores for each item were correlated with standardized T scores for each PROMIS-29 subscale (range: 0.49 to 0.70). Stronger correlations were exhibited between items measuring similar constructs as PROMIS subscales; e.g., the two items that make up the depression risk screening subscale and the two items that make up the anxiety risk screening subscale were highly correlated with the PROMIS depression and anxiety subscales, respectively. Correlations for each subscale were sorted from high to low within each CSS-25 factor.

Internal item quality: the first CSS-25 factor includes 8 items assessing emotional concerns; the depression risk screening items and the anxiety risk screening items were the highest loading items within this factor. Corrected item-factor correlations ranged from 0.60 to 0.82. The highest loading items within the 8-item symptom burden and impact domain included items assessing pain/physical discomfort, functional ability, and fatigue; corrected item-factor correlations ranged from 0.51 to 0.81. Within the 4-item domain assessing body and healthy lifestyle concerns, the items “Exercising and being physically active” and “Recent weight change (gain or loss)” had the highest loadings; corrected item-factor correlations ranged from 0.51 to 0.72. The exercise item and an item assessing body image concerns had the highest discrimination indices. The CSS-25 factors for health care team communication and relationship concerns are both comprised of 2 items each. These two 2-item factors, which continue to exhibit adequate factor loadings and item discrimination in the current EFA, are unmodified in the CSS-15+ scale. (Table 2).

Table 2 CancerSupportSource®: exploratory factor analysis of 25 items
Full size table

Judgmental item quality: three authors involved in the development of CSS-25 ranked each item from 1 (most relevant to short form) to 25 (least relevant to short form). An average rank was calculated for each item. The two items that make up the depression risk screening subscale and the two items that make up the anxiety risk screening subscale were the highest ranked items. Lowest ranked items included “Finding meaning and purpose in life,” “Feeling irritable,” and “Eating and nutrition”.

Scale reduction decisions: Criteria from all three indices were taken into consideration and used to sort items. Item reduction decisions were made, with preference given to items where all three quality indices were high. Scale refinement resulted in a 15-item short form of CSS plus a tobacco and substance use screener item (CSS-15+). At least two items from each of the five CSS-25 domains were retained to preserve multidimensionality, including five items from emotional well-being, four items from symptom burden and impact, two items from body and healthy lifestyle, two items from health care team communication, and two items from relationships and intimacy. All items from the anxiety and depression risk screening subscale items were retained (depression risk sensitivity = 91.4%, specificity = 79.5%; anxiety risk sensitivity = 91.8%, specificity = 70.9%) as previously described [14]. Additionally, the item about tobacco and substance use was retained due to clinical significance for risk assessment (Table 2).

Phase II: corroboration of psychometric properties and dimensionality

Participants

Participant eligibility for Phase II was the same as Phase I of the study (see above) except for the date range: Phase II inclusion was limited to those 957 cancer patients and survivors who participated in the CER between January and December of 2017, were 18 years or older, lived in the USA, and had complete data for CSS-15+ and PROMIS items (used for validation analyses).

Measures

Procedures and measures were identical to Phase I of the study, with the exception that cancer-related distress was analyzed using the 15-item version of CSS plus the additional item assessing tobacco and substance use (CSS-15+), and 221 participants completed CSS again 30 - 90 min later to examine test-retest reliability.

Analysis

Data analysis was again conducted using IBM SPSS Statistics 24.0 [19] and R 3.6.2 [20], with lavaan [21] and psych [22] R packages. To confirm dimensionality of the shortened scale, a confirmatory factor analysis was conducted using maximum likelihood factor extraction, fixing factor loadings for the first indicator in each factor to 1.0. Both absolute fit indices and relative fit indices were used to measure goodness of fit [23]. Internal consistency reliability was evaluated using Cronbach’s alpha. Test–retest reliability was measured with intraclass correlation coefficients. Nonparametric receiver operating characteristic (ROC) curve analyses were used to determine scoring thresholds for the 2-item CSS depression and anxiety risk scales, using PROMIS depression (T ≥ 60) and anxiety (T ≥ 62) scales as criterion scores [24, 25] as these correspond to conventional cutoffs for clinical risk significance using the PHQ-9 and GAD-7 legacy instruments [26,27,28]. Convergent validity was evaluated through Pearson correlations with PROMIS subscales. Discriminant validity was examined through the known-groups validation method, using Cohen’s d to estimate effect sizes between groups with the square root of the distress score.

Results

Participant socio-demographics: participants were predominantly female (67%), White (83%), and completed a college degree (56%). The average age was 58 years (range = 21–88), and average time since initial cancer diagnosis was 4.3 years (range = <1 to 49 years). The most commonly represented diagnoses included breast cancer (31%) and prostate cancers (16%; Table 1).

Confirmatory factor analysis: in confirmatory factor analysis, the model explained 59% of the variance and demonstrated good fit (RMSEA = 0.075, 90% CI = 0.069–0.082; SRMR = 0.033; CFI = 0.951; χ2(80) = 516.36, p < 0.001).

Internal consistency and test-retest reliability: Cronbach’s alpha for the full CSS-15+ scale was 0.94 (Table 3). Correlation between CSS-15+ and CSS-25 was 0.99, p < 0.001. The factors demonstrated moderate to large inter-correlations but were not redundant. Full scale test–retest reliability was .90, while individual factor ICCs were ≥ .79 (emotional well-being = .88; symptom burden and impact =.91; relationships and intimacy = .85; body image and healthy lifestyle = .79; health care team communication = .79).

Table 3 CSS-15+ scale and factor descriptive characteristics, inter-correlations, and internal reliability values (Cronbach’s α)
Full size table

Receiver operating characteristic curve analysis: using a PROMIS depression score of ≥ 60 to indicate risk for clinical levels of depression, a score of ≥ 3 on the 2-item CSS depression risk scale yielded a sensitivity and specificity of 81.5% and 84.8% (AUC = 0.888; Table 4). Using a PROMIS anxiety score of ≥ 62 to indicate risk for possible clinical levels of anxiety, a score of ≥ 3 on the 2-item CSS anxiety scale yielded a sensitivity and specificity of 82.9% and 74.8% (AUC = 0.865). Based on a cutoff score of 3 for each CSS risk scale, 30.2% of participants were at risk for clinically significant levels of depression and 40.3% were at risk for clinically significant levels of anxiety.

Table 4 Calculations of sensitivity and specificity for CancerSupportSource 2-item depression and 2-item anxiety risk scales
Full size table

Convergent validity: CSS15+ total distress was moderately to strongly associated with all PROMIS subscales in the expected direction (rs = ± 0.50 to 0.74, ps < 0.001). Individual factors were moderately to strongly correlated with PROMIS subscales of similar concepts (Table 5). The CSS depression risk scale was strongly correlated with PROMIS depression (r = 0.78, p < 0.001) and the CSS anxiety risk scale with PROMIS anxiety (r = 0.71, p < 0.001).

Table 5 Pearson correlations between CSS-15+ and PROMIS validation measures
Full size table

Discriminant validity: several group comparisons supported known-groups validity. The CSS-15+ total distress score was significantly (t = 3.29, p < 0.01) higher among those in active cancer treatment (n = 443) than among those who were not (n = 434, Cohen’s d = 0.22) and among those who were within 5 years of their cancer diagnosis (n = 629) than among those who were beyond 5 years (n = 242; t = 3.86, p < 0.001; Cohen’s d = 0.29), consistent with a small to medium value of d. Female participants reported more distress than male participants (Cohen’s d = 0.25; t = 3.64, p < 0.001). The square root of the total distress score was inversely associated with age (r = − 0.21, p < 0.001, n = 906) and time since diagnosis (r = − 0.11, p < 0.001, n = 871).

Discussion

CancerSupportSource-15 (CSS-15+) is a reliable, valid, multidimensional distress screening tool that offers the benefits of the original 25-item CSS with a significant reduction in number of items. Notably, the total distress scores in CSS-15+ and the full-scale 25-item version of CSS approached nearly exact correspondence (r = 0.99), and internal consistency of the total distress score was similarly excellent in both tools. Despite its reduced length, CSS-15+ retains the same depression and anxiety risk scales as the 25-item CSS, and these scales demonstrated similarly high sensitivity coupled with adequate specificity [14]. Given the American Society of Clinical Oncology guidelines call for screening both depression and anxiety, retaining these clinical risk scales remains a valuable attribute of CSS-15+ [29].

Like the 25-item CSS, when implemented in oncology practice, CSS-15+ fulfills the American College of Surgeons Commission on Cancer patient-centered standards for distress screening and other quality and merit–incentive program standards (e.g., QOPI, OCM, and NCCN) [3, 6,7,8]. This shorter form will allow providers in busy community practices to quickly and efficiently identify patients who have clinically indicated levels of distress and/or specific unmet needs and, in full implementation, connect them with tailored resources, educational materials, supportive services, and referrals. Typical administration time for CSS-15+, including respondents indicating desired support resources for each item, ranges from 5 to 8 min. Given the recent focus on understanding barriers and facilitators of implementing screening practices [30], shorter length scales have the potential benefit of being a facilitator by increasing feasibility and acceptability to providers in busy settings [31]. Future research will examine CSS and CSS-15+ implementation in real-world clinical and community settings to determine how these tools can be optimally utilized to improve psychosocial and health outcomes [30].

One strength of CSS-15+ is its continued inclusion of two brief subscales that identify those individuals at risk for depression and/or anxiety. These items demonstrated good internal and external validity, and previously established cutoff scores [14] were reaffirmed in demonstrating both high sensitivity and adequate specificity. In screening for depression and anxiety, it is important to have high sensitivity to avoid missing those who may benefit from help. If patients meet one or both cut-off scores, we would recommend more detailed assessment of depression and anxiety be conducted to assist in the most appropriate triage. Additionally, CSS-15+ allows for flexible administration of the tobacco and substance use screening item, with its inclusion based on needs of the clinical setting. For consistency in research applications of the tool, we recommend calculating total distress based on the core 15 scale items.

Other strengths of this study include participation by a broad sample of survivors across diverse cancer care settings, diagnoses, and geographic regions. The study was well-powered [32], and decisions about item retention and removal were guided by systematic consideration of internal, external, and judgmental characteristics of items [17]. Limitations include self-selected samples of participants who are predominantly female, White, and well-educated, which may limit the generalizability of its findings with a more diverse socio-economic population. This sample is not representative of all cancer patients and survivors across the US, and there was greater representation of breast cancer patients, though it is representative of those who currently seek social and emotional support for cancer with a community-based cancer support organization. Given these programs were designed to detect distress and unmet needs and connect patients and survivors to free tailored resources and support services, the tool has potential for increasing behavioral health utilization among groups who historically have had less access to care [30]. We also used a validated measure of quality of life, PROMIS-29, to support examination of scale multidimensionality and convergent validity across the CSS-15+ factors; measurement of convergent validity for the relationships and intimacy factor was in part limited by a lack of PROMIS items focusing on partnered relationships.

Psychosocial distress screening is the necessary first step to comprehensively assessing and addressing cancer patient distress and unmet needs [33]. The multifactorial definition of psychosocial distress requires a multidimensional screening process that systematically examines patients’ concerns across key areas of the cancer experience, including emotional well-being, symptom burden and impact, body image and healthy lifestyle, health care team communication, and relationships and intimacy. Future work will include increased attention to the psychometric support for CSS and CSS-15+ within diverse populations across the cancer continuum, prioritizing how distress and unmet needs may vary based on individual trajectories of cancer care and survivorship. Additional research will also evaluate the facilitation of practical use of distress screening among health and supportive care teams, including rescreening, as well as the impact of distress screening on quality- and cost-related outcomes. Together, these efforts will support the provision of comprehensive, high quality, patient-centric cancer care.