Background

Apart from physical and psychological health problems, patients with cancer usually have sleep disturbances [1] regardless of cancer types (e.g., breast cancer, lung cancer, head and neck cancer, and gynecologic cancer). Indeed, the prevalence of disturbed sleep in patients with cancer (30–93.1%) [2, 3] is higher than what it is reported in general population (9–33%) [4, 5]. Moreover, patients with advanced cancer seem to have even a poorer sleep [6]: It is reported that 50–75% of patients with advanced cancer have different levels of sleep disturbances [7, 8].

Patients with advanced cancer are at high risk of developing insomnia, an ailment that jeopardizes the health of an individual. According to the Diagnostic and Statistical Manual of Mental Disorders, 5th ed (DSM-5), insomnia includes a dissatisfaction with sleep quality or quantity accompanied with complaints of difficulty in initiating or maintaining sleep, or early morning awakening for at least 3 months despite adequate opportunity for sleep, and daytime functioning impairments [9]. Given the evidence of the association between sleep disturbances and poor clinical outcomes (e.g., low overall survival, rapid disease progress, and poor treatment response) [10], insomnia problems in patients with advanced cancer should be seriously treated.

Screening insomnia among patients with advanced cancer using a practical (e.g., can be quickly assessed in a busy clinical setting) and valid (e.g., has strong psychometric properties) tool should be routinely performed. Two tools, Athens Insomnia Scale (AIS) and Insomnia Severity Index (ISI), are frequently used in patients with advanced cancer [11]. However, both instruments were designed using older definition of insomnia. More specifically, the AIS adopts the tenth International Statistical Classification of Diseases and Related Health Problems (ICD-10) and ISI uses the DSM-IV version [11, 12]. A newly developed insomnia screening tool named the Sleep Condition Indicator (SCI) has been designed using DSM-5 insomnia criteria [13]. SCI assists healthcare providers in identifying people at risk of insomnia using the latest insomnia criteria.

Two other considerations in the use of SCI are (1) whether such subjective measures can accurately assess and quantify the insomnia and (2) whether the SCI has good psychometric properties using modern test theory. More specifically, the external criteria that examine the criterion-related validity of the SCI in the current literature are clinical insomnia diagnosis or subjective measures (e.g., self-reported depression); and most psychometric testing applied classical test theory [13,14,15,16,17,18]. Using objective sleep measure (i.e., actigraphy) would add psychometric information regarding the SCI on current literature and would further verify the psychometric properties of SCI. Moreover, adopting another test theory (i.e., modern test theory such as Rasch model) can provide additional psychometric information for healthcare providers to judge the SCI in different aspects.

The present study aimed to examine the SCI psychometric properties in a comprehensive and thorough manner using a large sample of Iranian patients with advanced cancer. More specifically, the two major approaches in psychometric testing (classical test theory and Rasch models in the modern test theory) were used. Apart from using different test theories, sensitivity and specificity of SCI were performed to obtain the optimal cutoff for screening insomnia (with a gold standard of DSM-5 diagnosis). A series of external criteria, including both subjective and objective measures related to sleep, were used to capture the concurrent validity of the SCI. According to the literature examining the psychometric properties of the SCI [13,14,15,16,17,18,19], it was hypothesized that (1) the present study’s findings would reveal satisfactory properties for the Persian SCI in both classical test theory and Rasch analyses; (2) SCI would have good sensitivity and specificity to screen/identify patients with advanced cancer who are at risk of insomnia; and (3) SCI would associate with all external criteria regardless of subjective or objective measures.

Methods

Participants and recruitment procedure

Using the convenience sampling method, participants of the study were recruited from five Iranian oncology centers in Tehran (N = 2), Qazvin (N = 1) and Tabriz (N = 2) from August 2017 to May 2019. Selection criteria were (1) histologically verified malignant disease with cancer stage III or IV; (2) were 18 years of age or older; (3) understood and communicated in the Persian language; and (4) provided signed informed consent. Patients were excluded if they (1) were pregnant or lactating; (2) had scheduled surgery within the next week; (3) had cognitive problems as assessed by the Mini-Mental State Examination (≤ 23; Folstein et al. 1975); and (4) had substance abuse problems. The study was approved by the ethics committee of the Qazvin University of Medical Sciences (IR.QUMS.REC.1398.082).

The eligibility of the patients was determined by two trained research assistants (nurses). The research assistants asked patients to wear an Actigraph device on the non-dominant wrist in the hospitals (Ambulatory Monitoring, Inc. USA). The Actigraphy was recorded in the patient’s home for 3 consecutive days to monitor daily activity and nocturnal sleep.

Instrumentation

Sleep condition indicator (SCI)

The SCI includes eight items, which are rated on a five-point Likert scale (from 0 to 4) and are designed according to the DSM-5 workgroup draft criteria [13]. A total score can be obtained after summing up all item scores with a range between 0 and 32, and a lower score indicates higher possibility of insomnia. Two domains were found in the eight-item SCI (Sleep pattern includes 5 items and Daytime impact includes 3 items). The SCI has good psychometrics in the internal consistency (Cronbach’s α = 0.86) and convergent validity (r = − 0.79 with ISI) [19].

Insomnia diagnosis (using DSM-5)

Three trained psychiatrists who did not know the patients’ conditions for insomnia before the study interviewed the patients to determine the presence of a DSM-5 insomnia diagnosis. The interviews with the patients were conducted face-to-face in clinic rooms. The DSM-5 insomnia diagnosis among the psychiatrists had acceptable inter-rater reliability as indicated by intraclass correlation coefficient (ICC; 0.81) in our pilot testing (N = 18).

Insomnia Severity Index (ISI)

The ISI includes seven items rated on a five-point Likert scale (from 0 to 4) and assess the severity and effects of insomnia. A total score can be obtained after summing up all item scores with a range between 0 and 28. Using the Persian ISI total score, a cutoff score of 9 was suggested to identify insomnia in patients with advanced cancer [11]. The Persian ISI has good psychometrics for the internal consistency (McDonald’s ω = 0.79), test–retest reliability (ICC = 0.79), and convergent validity (r = 0.64 with Athens Insomnia Scale) [11].

Pittsburgh Sleep Quality Inventory (PSQI)

The PSQI includes 19 items rated on a four-point Likert scale (from 0 to 3) and it assesses sleep quality and disturbances. A total score can be obtained after summing up all item scores with a range between 0 and 57. A higher score indicates poor sleep quality. The Persian PSQI has a good internal consistency (α = 0.77) [20].

Epworth Sleepiness Scale (ESS)

The ESS includes eight items rated on a four-point Likert scale (from 0 to 3) and it assesses the severity of daytime sleepiness. A total score can be obtained after summing up all item scores with a range between 0 and 24. A higher score indicates greater sleepiness. The Persian ESS has a good internal consistency (α = 0.79) and test–retest reliability (ICC = 0.84) [21].

Objectively measured sleep (actigraphy)

Slightly more than half of the participants (n = 491; 57.2%) were willing to wear a wrist actigraphy at home (Ambulatory Monitoring, Inc. USA) to collect objective sleep measures (e.g., sleep and wake pattern; activity cycle). After inputting the raw data recording in 1-min epochs from the wrist actigraphy to the actigraphy software, the sleep patterns were digitized and analyzed using the zero-crossing mode by Action-W software, version 2.6.9905. Subsequently, objective sleep parameters were estimated using a Sadeh algorithm [22, 23].

Hospital Anxiety and Depression Scale (HADS)

The HADS includes 14 items rated on a four-point Likert scale (from 0 to 3). It assesses two types of psychological distress: anxiety (7 items) and depression (7 items). Two subscale scores can be obtained after summing up the item scores in the same subscale with a range between 0 and 21. A higher score indicates greater level of distress. The Persian HADS has a good internal consistency (α = 0.83) [24].

General Health Questionnaire (GHQ)

The GHQ-12 includes 12 items rated on a four-point Likert scale (from 0 to 3). It assesses health and psychiatric wellbeing. A total score can be obtained after summing up all item scores with a range between 0 and 36. A higher score indicates poor health. The Persian GHQ has good psychometrics for the internal consistency (α = 0.87) and convergent validity (r = 0.56 with quality of life measure) [25].

Edmonton Symptom Assessment System (ESAS)

The ESAS includes 10 items rated on a 0-10 scale. It assesses the objective and subjective symptoms of patients with cancer across all stages. A total score can be obtained after summing up all item scores with a range between 0 and 100; a higher score indicating a worse condition in symptoms [26]. The Persian ESAS has good psychometrics for the internal consistency (α = 0.88) and test–retest reliability (r = 0.86) [27].

Karnofsky Performance Scale (KPS)

The KPS is a commonly used instrument to quantify the functional status of cancer survivors. Using 11 descriptors, the KPS score ranges between 0 (indicating dead) and 100 (indicating normal, no complaints, no evidence of disease). The construct validity of the KPS is supported by the significant relationships with survival length [28].

Data analysis

Psychometric testing using classical test theory

In the item level of psychometric testing, acceptance (a higher value is preferred), corrected item-total correlation (a value > 0.4 is preferred) [29], factor loading derived from a two-factor confirmatory factor analysis (CFA; a value > 0.4 is preferred), and test–retest reliability in a 2-week interval using ICC (a value > 0.7 is preferred) [29, 30] were generated. In the scale level of psychometric testing, internal consistency was assessed using Cronbach’s α (a value > 0.7 is preferred); item coherence was assessed using both average variance extracted (a value > 0.5 is preferred) and composite reliably (a value > 0.6 is preferred); distribution of the SCI total score was assessed using ceiling and floor effects (a value < 20% is preferred); and construct validity of the SCI was assessed using CFA. The CFA results were generated using mean- and variance-adjusted weighted least squares (WLSMV) estimator. In addition, fit indices of comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR) were used. Superficially, both CFI and TLI > 0.9 together with both RMSEA and SRMR < 0.08 indicate satisfactory construct validity [30].

Psychometric testing using Rasch analysis

The Rasch analysis was conducted using a partial credit model. In the item level of psychometric testing, infit mean square (MnSq), outfit MnSq (a value between 0.5 and 1.5 is preferred), and differential item functioning (DIF; a DIF contrast < 0.5 is preferred) were computed. The infit and outfit MnSq indicate the item fit; the DIF detects whether any SCI item has substantial DIF across sex or insomnia condition (i.e., participants in different groups interpret item contents differently). In the scale level of psychometric testing, the reproducibility of the participants and items was assessed using a separation person and item reliability, respectively (a value > 0.7 is preferred); the spread of the participants and items was assessed using separation person and item index, respectively (a value > 2 is preferred) [31]. The Rasch analysis software program WINSTEPS®, version 4.1.0 was used to perform the Rasch analysis.

Optimal cutoff for the SCI in identifying insomniacs

Receiver Operating Characteristic (ROC) curve together with Youden’s index were used to determine the optimal cutoff for SCI total score. Specifically, DSM-5 diagnosis and ISI cutoffs were used as the standard to decide the optimal cutoff. Apart from sensitivity and specificity, the area under curve (AUC) was computed, where an AUC > 0.7 was anticipated [32].

Psychometric testing using external criteria

Several Pearson correlations were conducted to understand the associations between the two SCI domain scores and total score with different external criteria (ISI, PSQI, ESS, HADS, GHQ, and KPS). In addition, actigraph data were used to evaluate to what extent the SCI cutoff can effectively differentiate the insomniacs and non-insomniacs among patients with advanced cancer. More specifically independent t tests were used to compare the differences in actigraph data between insomniacs and non-insomniacs.

Results

Participant characteristics

On average, the 859 patients with advanced cancer were aged 61.3 (SD 10.7) years. More than half of them were males (n = 511; 59.5%) and the majority of the patients had cancer stage IV (n = 519; 60.4%). The patients, on average, had received 8.9 years of education (SD 4.1), scored 55.2 (SD 11.1) in the KPS, and were 30.9 months after diagnosis (SD 22.4; Table 1).

Table 1 Characteristics of the participants (n = 859)
Full size table

Item-level psychometric properties

The SCI demonstrated promising Item-level psychometric properties (Online Appendix A). Acceptances were high (81–99%); CFA-derived factor loadings were strong (0.76–0.89); corrected item-total correlations were promising (0.59–0.82); test–retest reliability was satisfactory (0.80–0.93); infit (0.63–1.31) and outfit MnSq (0.61–1.23) were in the anticipation; and no substantial DIF was found across sex (DIF contrast = -0.09 to 0.19) or insomnia condition (DIF contrast = − 0.33 to 0.26).

Scale-level psychometric properties

The SCI demonstrated promising scale-level psychometric properties (Online Appendix B). The ceiling (5.4%) and floor effects (0.8%) were low; internal consistency (ω = 0.91) was satisfactory; average variance extracted (0.80) and composite reliability (0.97) were promising; test–retest reliability was acceptable (ICC = 0.88); and separation reliability (0.81–0.99) and separation indices (2.19 and 8.94) were in anticipation. Moreover, using the same SCI factorial structure (i.e., 5 items on Sleep pattern and 3 on Daytime impact), CFA fit indices were excellent (CFI = 0.991, TLI = 0.987, RMSEA = 0.049, and SRMR = 0.019).

ROC results

The optimal cutoff score for SCI in distinguishing a patient with advanced cancer having insomnia was 20 with 83% sensitivity and 87% specificity [AUC = 0.92] according to DSM-5 diagnosis; 17 with 97% sensitivity and 98% specificity [AUC = 0.98] according to ISI cutoff (Online Appendix C).

Psychometric properties with external criteria

SCI domain and total scores were significantly correlated to all the external criteria (all p values < 0.001; Online Appendix D). In addition, actigraph data (total sleep time, sleep efficiency, bedtime, wake time, sleep onset latency, and wake after sleep onset) were significantly different between insomniacs and non-insomniacs using the optimal cutoff determined in the ROC findings with DSM-5 diagnosis (Table 2).

Table 2 Comparing sleep between good and poor sleepers determined using the Sleep Condition Indicator (SCI) (N = 491)
Full size table

Discussion

To the best of the authors’ knowledge, this is the first study to examine the psychometric properties of the SCI among patients with advanced cancer. Empirical evidence found in the present study supports the use of SCI for healthcare providers in identifying insomnia among patients with an advanced cancer diagnosis. The use of an actigraph to capture objective sleep provides new information about the psychometric properties of the SCI. Moreover, the two-factor structure of the SCI was supported by the CFA and the unidimensional structure of each subscale in the SCI was supported by the Rasch analysis. Previous studies agree on the two-factor structure of the SCI [13, 14, 18] and the present study’s results further corroborate the prior findings. However, Hellström et al. [16] recently reported a unidimensional structure for the Swedish SCI and justified the reasons due to different sample characteristics. Hellström et al. [16] recommended future studies to examine the SCI structure for different populations, because the prior findings on SCI structure are based on classical test theory, which is strongly influenced by sample characteristics. Given that the previous studies on SCI properties did not examine the factorial structure [13,14,15,16,17,18] for patients with advanced cancer (samples in prior research are usually general population or community-dwelling adults), future studies are warranted in investigating the SCI structures in patients with cancer to corroborate the present findings.

In addition to the factorial structures, the present study revealed other psychometric properties of the SCI comparable to previous studies’ findings [13,14,15,16,17,18]. More specifically, all the studies (including this one) demonstrated good internal consistency (α = 0.71 to 0.91), satisfactory test–retest reliability in both item- and scale- levels (r or ICC = 0.44–0.93), supported criterion-related validity, and adequate ROC findings (AUC = 0.89–0.98). However, the cutoff identifying insomnia proposed by the present study is somewhat different from the suggested cutoffs in other studies (Wong et al. [18] proposed a cutoff at 21; Bayard et al. [14] and Espie et al. [13] proposed a cutoff at 16). The differences in proposed cutoffs are possibly due to the sample characteristics mentioned above. As a result, future studies on patients with advanced cancer are needed to corroborate the cutoff proposed by the present study. After verifying the cutoff for the SCI, healthcare providers can quickly screen at-risk insomnia among patients with advanced cancer.

Strength and limitations

The study had two major strengths. First, given that prior psychometric studies on SCI only focused on classical test theory, the present study extended the findings of SCI properties to modern test theory (i.e., the Rasch models). With different approaches in assessing the psychometric properties of the SCI, potential users can obtain comprehensive information and use the information they are familiar with or interested in. For example, most healthcare providers are familiar with classical test theory and they are confident in interpreting the results using classical test theory [33]. On the other hand, researchers may be interested in the Rasch results, because this model conveys psychometric information with several well-known advantages, including reliability estimations separately for item and respondents, converting ordinal scale (e.g., Likert type scale) to interval scale (i.e., using the unit of logit), and measurement invariance (i.e., DIF) at the item level [31].

The second strength of this study is that it used both subjective and objective external criteria to demonstrate the SCI validity. The major benefit of using objectively measured sleep (i.e., actigraphy) is to eliminate recall and social desirability biases. The validity of actigraphy has been well established [34, 35]. However, because SCI measures insomnia differently than the actigraphy data (i.e., insomnia-related complains vs. activity level), we also used other subjective sleep measures (e.g., ISI) as additional external criteria to compensate the limitation of using actigraphy as a single external criterion.

There are some limitations to the present study. First, the sample characteristics restrict the generalizability of the present study’s findings to other populations. Indeed, all the participants were patients with advanced cancer who were recruited in Iran using a convenience sampling method. Therefore, the results cannot be generalized to other ethnic groups, other disease groups, or early stage of cancer, although the psychometric properties of the SCI have been reported for other populations [13,14,15,16,17,18]. Second, given that participants with cognitive problems were excluded from the study, using SCI to assess insomnia might be questionable because of frequently observed cognitive impairment (e.g., chemo brain) among patients with cancer [36]. Third, some types of cancer accounted for a small portion of the sample (e.g., 39 patients with skin cancer; detailed information in Table 1). Therefore, the generalizability of the findings might not apply to all cancer types, such as myeloma and skin cancer.

Clinical implications

This study validates the Persian SCI for use among patients with advanced cancer in Iran. By using a large sample, strong and robust psychometric testing has been performed and psychometric testing based on different theories (classical and modern test theory) demonstrated the promising properties of the SCI in screening insomnia. Also, objective sleep measure and DSM-5 insomnia diagnosis were strongly associated to the SCI score. Given the high prevalence of insomnia among patients with advanced cancer and busy clinical setting to thoroughly examine their sleep problems, using this simple assessment tool to target the individuals who are at risk of insomnia can help healthcare professionals to provide personalized care efficiently.