Participants
A total of 481 persons participated in the current study, consisting of three different samples: one clinical sample, and two non-clinical samples. All participants gave informed consent to voluntarily participate and anonymity was guaranteed. Demographic characteristics are presented in Table 1.
Table 1 Demographic characteristics The first sample (clinical sample) consisted of 232 patients who participated in a MBCT course for patients with recurrent depressive disorder provided at the department of psychiatry at the Radboud University Medical Center, Nijmegen, The Netherlands. All trainers teaching MBCT at the Radboud University Medical Center were experienced in working with depressed patients and completed a 2-year mindfulness teacher training. The second sample (non-clinical sample 1) consisted of 127 persons participating in a MBSR course for the general public at the Radboud University Medical Center, Nijmegen, The Netherlands. The third sample (non-clinical sample 2) consisted of 122 participants who had followed an MBCT/MBSR course in Flanders in the past. This sample was recruited using snowball sampling via e-mail. An e-mail with detailed information was sent to mindfulness trainers in Flanders, requesting circulation of this invitational e-mail to former MBCT/MBSR attendees.
Procedures
The study was divided in three stages. First, the German version of the CHIME was translated into Dutch according to a standardized procedure (Guillemin et al. 1993). Second, the model fit, validity, and sensitivity to change of the translated version were tested. Third, a short form of the CHIME was developed and its psychometric properties were evaluated.
Translation Procedure
The procedure of the translation consisted of four steps. First, the German version of the CHIME was translated into Dutch by two independent native Dutch speakers (forward translation). One of the translators had a background in psychiatry and was a mindfulness trainer (HvR); the other translator had a background in human resource management and no experiences with mindfulness. These two translations were combined in one version by consensus.
Second, two native German speakers, living in the Netherlands and Belgium, independently re-translated this Dutch version into German (back translation). One of the back-translators had a background in management science and was an experienced mindfulness trainer; the other back-translator had a background in psychology and mindfulness research (MCM). The back-translators were blind to the original German version.
Third, the back-translations were compared to the original version and a final Dutch version was created after several consensus meetings. During this phase, the translation was also evaluated by two native Flemish speakers (FR and KVdG) to ensure that the questionnaire would be suitable within both the Dutch and Flemish language area. Additionally, eight students and members of the Department of Psychiatry of the Radboudumc with a wide range of mindfulness experiences were asked to fill in the questionnaire and to report any difficulties with the phrasing of the items. Based on these evaluations, the final Dutch version of the CHIME was developed.
Data Collection
The clinical sample completed questionnaires before the MBCT on mindfulness skills, depressive symptoms, worry, self-compassion, and other outcomes not presented in this paper (e.g., quality of life). One hundred forty-nine (64.2%) completed the same measures after the course. These questionnaires were administered as routine outcome monitoring (ROM) and were used for research purposes in anonymized form.
The non-clinical sample 1 completed an online questionnaire on mindfulness skills, worry, burnout, and other questions not presented in this paper (e.g., information about working hours) before and after the MBSR course. Measures were administered to answer several research questions; the measures that are used for this research are described below. Ninety-three (73.2%) participants completed the measures after the MBSR. The non-clinical sample 2 received an e-mail with the address of a website where participants could conclude the questionnaires.
Measures
Comprehensive Inventory of Mindfulness Experiences
The CHIME is a 37-item questionnaire with eight subscales (Bergomi et al. 2013a, b, 2014): awareness of internal experiences (inner awareness), awareness of external experiences (outer awareness), acting with awareness (acting with awareness), accepting and non-judgmental attitude (acceptance), nonreactive decentering (decentering/nonreactivity), openness to experiences (openness), awareness of thought’s relativity (relativity), and insightful understanding (insight). The CHIME does not cover the aspect “labeling, describing” as research has shown that this aspect is related to mindfulness skills rather than being a part of mindfulness skills (Bergomi et al. 2013b). Items are scored on a 6-point Likert scale, ranging from 1 (almost never) to 6 (almost always). The CHIME was administered in all samples.
Five Facet Mindfulness Questionnaire Short Form
In the clinical and non-clinical sample 1, mindfulness skills were also assessed with the Dutch version of the 24-item Five Facet Mindfulness Questionnaire Short Form (FFMQ-SF) (Bohlmeijer et al. 2011). The FFMQ-SF consists of five subscales: observing, describing, acting with awareness, non-judging of inner experience, and nonreactivity to inner experience. The FFMQ-SF is positively related to well-being and openness to experience and inversely related to measures of psychological symptoms, experiential avoidance, and neuroticism (Bohlmeijer et al. 2011). In the current study, Cronbach’s alpha was α = .86 in the clinical sample and α = .74 in non-clinical sample 1.
Self-Compassion Scale
The Self-Compassion Scale (SCS) is a 26-item questionnaire measuring self-compassion. The Dutch version of the SCS has good psychometric properties and high scores on the SCS are related to psychological well-being (Neff and Vonk 2009). A Dutch 12-item short form of the SCS was developed by Raes et al. (2011), which has good psychometric properties, high internal consistency, and a nearly perfect correlation with the long form of the SCS. The 26-item version was used in the clinical sample, whereas the 12-item version was used in non-clinical sample 1. In the current study, Cronbach’s alpha was α = .93 in the clinical sample and α = .86 in non-clinical sample 1.
Penn State Worry Questionnaire
The Penn State Worry Questionnaire (PSWQ) was developed to measure aspects of clinically significant worry (Meyer et al. 1990). The questionnaire was administered in the clinical and non-clinical sample 1. It measures the tendency, intensity, and uncontrollability of worry and consists of 16 items rated on a 5-point Likert scale, with values ranging from 1 = not at all typical of me to 5 = very typical of me (i.e., “I am always worrying about something,” or “I have been a worrier all my life”). The Dutch version of the PSWQ has a high internal consistency in clinical as well as in non-clinical samples (Kerkhof et al. 2000; Van Van Rijsoort et al. 1997). Cronbach’s alpha value was α = .99 in the clinical sample and α = .95 in non-clinical sample 1.
Beck Depression Inventory-II
The 21-item Beck Depression Inventory-II (BDI-II) is widely used to assess the severity of depressive symptomatology (Beck et al. 1996; Dutch version: Van der Does 2002). Each item is a list of four statements about a symptom of depression, arranged in order of severity. Items are rated on a 4-point scale ranging from 0 (not at all) to 3 (extreme form of each symptom), which corresponds to each statement. Items are summed to give a single total score, which ranges from 0 to 63; a score of 0–13 is considered to reflect minimal depression, 14–19 mild depression, 20–29 moderate depression, and 30–63 severe depression. The questionnaire was administered in the clinical sample and the Cronbach’s alpha was α = .91.
Maslach Burnout Inventory
The emotional exhaustion subscale of the validated Dutch version of the Maslach Burnout Inventory (MBI) was used to measure burnout in non-clinical sample 1 (Maslach and Jackson 1986; Schaufeli and Van Dierendonck 2000). Emotional exhaustion is seen as the core component of burnout and contained of 8 items. Items are scored on a 7-point Likert scale (0 = never, 6 = every day). Cronbach’s alpha was α = .89 in non-clinical sample 1.
Intervention
The MBCT and MBSR programs were based on the original programs by Kabat-Zinn (1990) and Segal et al. (2013) and consisted of eight weekly 2.5-h group sessions, one silent day, and daily 45-min home practice. Mindfulness exercises, such as body scan, gentle yoga, sitting, and walking meditation, were practiced. Participants received teachings on stress and/or depression and were invited to share their experiences. In total, 96.5% of the whole sample attended a MBCT/MBSR course provided by a certified mindfulness trainer, of the remaining 3.5%, no information regarding the trainer was available.
Data Analyses
The data of all three samples were used to investigate the factor structure of the CHIME. For descriptive statistics, internal consistency, sensitivity to change analyses, and correlations, SPSS (version 22) was used, whereas SPSS AMOS was used to perform confirmatory factor analyses (CFAs) and to test measurement invariance. To compute a corrected correlation to evaluate the CHIME-SF, Levy’s formula (Levy 1967) was used as incorporated in the Shortform Version 1.1 software developed by Barrett (2005).
Factor Structure
Based on the original validation study, the model fit of different CFA models was evaluated (Bergomi et al. 2014). For these analyses, data of all three samples were included (n = 481). Of the clinical sample and non-clinical sample 1, the first measurement (i.e., before attending MBSR/MBCT) was used. First, the model fit of a 1-factor model with all items loading on one factor was tested to evaluate whether the items of the CHIME measure one unidimensional construct of mindfulness. Secondly, a correlational and a hierarchical 8-factor model were examined. In the correlational 8-factor model, it is assumed that the items load on eight subscales, which in turn are correlated. This model showed the best fit in the original validation study (Bergomi et al. 2014). In the hierarchical 8-factor model, it is assumed that the items load on subscales, which in turn load on an overall mindfulness factor. Third, based on the original validation study, we examined the model fit of a correlational and hierarchical 7 + 2-factor model. In the correlational 7 + 2-factor model, it is assumed that the factors “inner awareness” and “outer awareness” load on an “awareness” factor, which in turn is correlated with the remaining six subscales. In the hierarchical 7 + 2-factor model, it is assumed that the factors “inner awareness” and “outer awareness” load on an “awareness” factor, which together with the remaining six factors loads on one general mindfulness factor.
In all models, items were restricted to load on one factor only, error terms were not allowed to correlate and the loading of one item per factor was fixed to 1. The overall model chi-square statistic (χ2), where smaller values indicate a better fit, was investigated. In addition, the following indices were used to evaluate the models: root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI). In line with the original validation study (Bergomi et al. 2014), the following cut-offs were used to evaluate the fit of the models: RMSEA ≤ 0.08, SRMR ≤ 0.10, and CFI ≥ 0.90. In addition, we examined the ratio of the chi-square statistic to its degree of freedom (χ2/df), for which values below a value of 5 were considered as an acceptable model fit (Bohlmeijer et al. 2011; Watkins 1989).
Because we combined data of clinical and non-clinical participants for the main analyses, we also tested measurement invariance between clinical (= 232) and non-clinical groups (non-clinical sample 1 and non-clinical sample 2, n = 249). We tested configural and metric invariance. Configural invariance means the global factor structure, i.e., the number of factors and number of factor loadings for each factor are the same for clinical and non-clinical groups (Byrne 2010). To test metric invariance, we constrain factor loadings to be the same in both groups. Metric invariance therefore indicates that the items contribute to the latent constructs to a similar degree across groups. Models were compared by changes in CFI (≤ − 0.005) and changes in RMSEA (≤ 0.010) as proposed by Chen (2007).
Internal Consistency and Intercorrelation of the Subscales
Internal consistency of the CHIME subscales was evaluated with Cronbach’s alpha. A higher score of Cronbach’s alpha indicates greater internal consistency (Cicchetti 1994; Tavakol and Dennick 2011). Pearson correlations between the subscales were calculated. It was assumed that subscales would be small (< r = .30) to moderately (r = .30–.50) correlated but not strongly (> r = .50), as it is hypothesized that the subscales measure related but unique aspects of mindfulness.
Convergent and Discriminant Validity of the CHIME
Pearson correlations between the CHIME subscales and other measures collected before the MBCT/MBSR were computed for the clinical and non-clinical sample 1. It was hypothesized that the subscales of the CHIME should show moderate or strong positive correlations with another measure of mindfulness skills (FFMQ). It was hypothesized that especially subscales representing similar factors would show moderate to strong correlations. We therefore hypothesized a moderate to strong positive correlation between the subscales “decentering/nonreactivity” (CHIME) and “nonreactivity of inner experience” of the FFMQ, and moderate to strong positive correlations between “acting with awareness” (CHIME) and the subscale “acting with awareness” of the FFMQ (convergent validity). Additionally, we hypothesized that the subscales of the CHIME would show moderate to high positive correlations with a measure of self-compassion (SCS). In contrast, we hypothesized small to moderate negative correlations between subscales of the CHIME and questionnaires measuring depression, worry, or burnout (discriminant validity).
Sensitivity to Change
To assess the ability of the CHIME to detect changes in mindfulness skills, paired-sample t tests on subscales of the CHIME before and after following an 8-week MBCT course (clinical sample) or an 8-week MBSR course (non-clinical sample 1) were conducted. A Cohen’s d effect size based on the baseline standard deviation was computed. Effect sizes between .20 and .50 were considered small, effect sizes between .50 and .80 moderate, and effect sizes greater than .80 large (Cohen 1988).
Development of the Short Form
The steps taken to develop the short form were based on the guidelines by Marsh et al. (2005). Criteria for the development of the CHIME short form (CHIME-SF) included that (1) the CHIME-SF should retain the content coverage of the CHIME; (2) each subscale should include three items; (3) the reliability of each subscale of the CHIME-SF should be acceptable; (4) the CHIME-SF should show a similar factor structure as the CHIME with an acceptable or good model fit; and (5) the CHIME-SF should be comparable to the CHIME in terms of sensitivity to change and discriminant and convergent validity. For each subscale, three items were selected that best reflected the underlying construct, based on the standardized factor loadings in the correlated 8-factor model, minimal cross-loadings as evidenced by the CFA modification indices, and low error correlations with other items. Additionally, the content of all items was subjectively evaluated in order to maintain the coverage of the original subscale to avoid sacrificing important items by relying exclusively on statistical measures (Smith et al. 2000). By using the same evaluation criteria as for the CHIME, the model fit of the CHIME-SF of a 1-factor model and a hierarchical and correlational 8-factor model were evaluated and compared to the CHIME. Configural and metric invariance of the correlational 8-factor model across the clinical and non-clinical group was tested. To compare the CHIME and the CHIME-SF, Pearson correlations were computed. Because both versions were based on a single administration, corrected correlations (rc) were computed to control for spuriously inflated correlations due to shared measurement error (Levy 1967). Corrected correlations were computed by using the Shortform Version 1.1 developed by Barrett (2005). Additionally, convergent and discriminant validity and sensitivity to change of the CHIME-SF were investigated.