Validation of the Dutch Comprehensive Inventory of Mindfulness Experiences (CHIME) and Development of a Short Form (CHIME-SF)

Abstract

Objectives

In recent years, there has been a growing interest in the assessment of mindfulness skills. The present study describes the validation of the Dutch Comprehensive Inventory of Mindfulness Experiences (CHIME) and the development of a short form.

Methods

The original scale was translated into Dutch. The psychometric properties of the CHIME were examined, including its factor structure, internal consistency, and convergent and discriminant validity in a group of 481 participants consisting of clinical and non-clinical samples. In addition, a 24-item short form (CHIME-SF) was developed and its psychometric properties were assessed in the same sample.

Results

Confirmatory factor analyses showed an acceptable model fit of an 8-factor structure of the CHIME with the following subscales: awareness of internal experiences, awareness of external experiences, acting with awareness, accepting and non-judgmental attitude, nonreactive decentering, openness to experiences, awareness of thought’s relativity, and insightful understanding (χ² = 1534.90, df = 601, χ²/df = 2.55, CFI = 0.90, RMSEA = 0.057 [90% CI 0.053, 0.060], SRMR = 0.06). The CHIME-SF showed a good model fit (χ² = 486.13, df = 224, χ²/df = 0.17, CFI = 0.95, RMSEA = 0.049 [90% CI 0.043, 0.055], SRMR = 0.05.) and the subscales of the CHIME and CHIME-SF showed high corrected correlations (r_c = .69–.88). The CHIME and the CHIME-SF proved to be sensitive to change.

Conclusions

The results indicate that the Dutch version of the CHIME and CHIME-SF are valid instruments to assess mindfulness skills in clinical and non-clinical populations. The psychometric properties of the CHIME-SF should be further evaluated in an independent sample.

Mindfulness is defined as bringing one’s attention to the moment-to-moment experience in a non-judgmental and accepting way (Kabat-Zinn 1990) and has been receiving increasing interest within clinical and non-clinical settings. During the last three decades, several mindfulness-based interventions, such as mindfulness-based stress reduction (MBSR) (Kabat-Zinn 1990) and mindfulness-based cognitive therapy (MBCT) (Segal et al. 2013) have been developed and their efficacy has been established in a large number of studies (Khoury et al. 2013; Kuyken et al. 2016).

Consequently, the assessment of mindfulness has increasingly received attention and several self-reporting questionnaires have been developed. These include the Mindfulness Attention Awareness Scale (Brown and Ryan 2003), the Freiburg Mindfulness Inventory (Walach et al. 2006), the Kentucky Inventory of Mindfulness Skills (Baer et al. 2004), the Southampton Mindfulness Questionnaire (Chadwick et al. 2008), the Cognitive and Affective Mindfulness Scale (Feldman et al. 2007), the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al. 2006), the Philadelphia Mindfulness Scale (Cardaciotto et al. 2008), and the Toronto Mindfulness Scale (Lau et al. 2006). Bergomi et al. (2013b) examined these eight validated mindfulness questionnaires in detail and identified nine distinguishable aspects of mindfulness skills covered in these instruments. These aspects include (1) observing, attending to experiences; (2) acting with awareness; (3) non-judgment, acceptance of experiences; (4) self-acceptance; (5) willingness and readiness to expose oneself to experiences, non-avoidance; (6) nonreactivity to experience; (7) non-identification with own experiences; (8) insightful understanding; and (9) labeling, describing. Bergomi et al. (2013a) concluded that none of the available instruments covered all aspects, as each questionnaire was based on a slightly different definition of mindfulness and emphasized different aspects of mindfulness skills. In addition, several studies imply that the comprehension of mindfulness scales is influenced by meditation experience (Baer et al. 2006; Baer et al. 2008; Curtiss and Klemanski 2014). In a recent study, Gu et al. (2016) examined the factor structure of the FFMQ in participants taking part in MBCT. Based on the results, they propose to exclude the subscale “observing” when comparing scores before and after mindfulness interventions, as scores are likely to be inflated by a better understanding of the construct. Taken together, none of the available mindfulness questionnaires covers the whole range of the mindfulness construct and is equally applicable for samples varying in meditation experience.

Taking these limitations into account, Bergomi and colleagues (Bergomi et al. 2013b; Bergomi et al. 2014) developed the Comprehensive Inventory of Mindfulness Experiences (CHIME). The CHIME is based on a thorough literature review and comparing of the existing mindfulness questionnaires. The CHIME not only covers a broad range of the mindfulness aspects but the developers also paid special attention to make the scale more comprehensible for community samples irrespective of meditation experience. Psychometric properties of the CHIME were evaluated as good in samples consisting of community members and participants of MBSR (Bergomi et al. 2014). Importantly, a recent study investigated whether scores of the CHIME are dependent on a better understanding or response shift due to a mindfulness intervention. The results showed that scores of the CHIME were only minimally influenced by a better or changed understanding of the concept of mindfulness (Krägeloh et al. 2018). Therefore, the CHIME has potential to be a useful instrument in mindfulness research. However, as the CHIME was validated in German, it is not yet available in Dutch. Furthermore, even though the sensitivity to change of the CHIME has already been demonstrated in a non-clinical sample of MBSR participants (Bergomi et al. 2014), it has not yet been tested in a clinical sample receiving MBCT within a mental health care institution.

The purpose of this study was to translate the CHIME questionnaire into Dutch and to evaluate its psychometric properties. The factor structure, internal consistency, and validity of the Dutch CHIME were investigated in a mixed sample consisting of one clinical and two non-clinical samples. Furthermore, the sensitivity to change of the CHIME before and after a mindfulness intervention (MBCT/MBSR) was evaluated in the clinical and one of the non-clinical samples. The second aim of this study was to develop a short form of the CHIME, with similar content validity and psychometric properties to the full version. A shorter version of the CHIME is particularly relevant for research settings where time constraints make the use of the long form less feasible or the questionnaire is administered on multiple occasions.

Method

Participants

A total of 481 persons participated in the current study, consisting of three different samples: one clinical sample, and two non-clinical samples. All participants gave informed consent to voluntarily participate and anonymity was guaranteed. Demographic characteristics are presented in Table 1.

Table 1 Demographic characteristics

The first sample (clinical sample) consisted of 232 patients who participated in a MBCT course for patients with recurrent depressive disorder provided at the department of psychiatry at the Radboud University Medical Center, Nijmegen, The Netherlands. All trainers teaching MBCT at the Radboud University Medical Center were experienced in working with depressed patients and completed a 2-year mindfulness teacher training. The second sample (non-clinical sample 1) consisted of 127 persons participating in a MBSR course for the general public at the Radboud University Medical Center, Nijmegen, The Netherlands. The third sample (non-clinical sample 2) consisted of 122 participants who had followed an MBCT/MBSR course in Flanders in the past. This sample was recruited using snowball sampling via e-mail. An e-mail with detailed information was sent to mindfulness trainers in Flanders, requesting circulation of this invitational e-mail to former MBCT/MBSR attendees.

Procedures

The study was divided in three stages. First, the German version of the CHIME was translated into Dutch according to a standardized procedure (Guillemin et al. 1993). Second, the model fit, validity, and sensitivity to change of the translated version were tested. Third, a short form of the CHIME was developed and its psychometric properties were evaluated.

Translation Procedure

The procedure of the translation consisted of four steps. First, the German version of the CHIME was translated into Dutch by two independent native Dutch speakers (forward translation). One of the translators had a background in psychiatry and was a mindfulness trainer (HvR); the other translator had a background in human resource management and no experiences with mindfulness. These two translations were combined in one version by consensus.

Second, two native German speakers, living in the Netherlands and Belgium, independently re-translated this Dutch version into German (back translation). One of the back-translators had a background in management science and was an experienced mindfulness trainer; the other back-translator had a background in psychology and mindfulness research (MCM). The back-translators were blind to the original German version.

Third, the back-translations were compared to the original version and a final Dutch version was created after several consensus meetings. During this phase, the translation was also evaluated by two native Flemish speakers (FR and KVdG) to ensure that the questionnaire would be suitable within both the Dutch and Flemish language area. Additionally, eight students and members of the Department of Psychiatry of the Radboudumc with a wide range of mindfulness experiences were asked to fill in the questionnaire and to report any difficulties with the phrasing of the items. Based on these evaluations, the final Dutch version of the CHIME was developed.

Data Collection

The clinical sample completed questionnaires before the MBCT on mindfulness skills, depressive symptoms, worry, self-compassion, and other outcomes not presented in this paper (e.g., quality of life). One hundred forty-nine (64.2%) completed the same measures after the course. These questionnaires were administered as routine outcome monitoring (ROM) and were used for research purposes in anonymized form.

The non-clinical sample 1 completed an online questionnaire on mindfulness skills, worry, burnout, and other questions not presented in this paper (e.g., information about working hours) before and after the MBSR course. Measures were administered to answer several research questions; the measures that are used for this research are described below. Ninety-three (73.2%) participants completed the measures after the MBSR. The non-clinical sample 2 received an e-mail with the address of a website where participants could conclude the questionnaires.

Measures

Comprehensive Inventory of Mindfulness Experiences

The CHIME is a 37-item questionnaire with eight subscales (Bergomi et al. 2013a, b, 2014): awareness of internal experiences (inner awareness), awareness of external experiences (outer awareness), acting with awareness (acting with awareness), accepting and non-judgmental attitude (acceptance), nonreactive decentering (decentering/nonreactivity), openness to experiences (openness), awareness of thought’s relativity (relativity), and insightful understanding (insight). The CHIME does not cover the aspect “labeling, describing” as research has shown that this aspect is related to mindfulness skills rather than being a part of mindfulness skills (Bergomi et al. 2013b). Items are scored on a 6-point Likert scale, ranging from 1 (almost never) to 6 (almost always). The CHIME was administered in all samples.

Five Facet Mindfulness Questionnaire Short Form

In the clinical and non-clinical sample 1, mindfulness skills were also assessed with the Dutch version of the 24-item Five Facet Mindfulness Questionnaire Short Form (FFMQ-SF) (Bohlmeijer et al. 2011). The FFMQ-SF consists of five subscales: observing, describing, acting with awareness, non-judging of inner experience, and nonreactivity to inner experience. The FFMQ-SF is positively related to well-being and openness to experience and inversely related to measures of psychological symptoms, experiential avoidance, and neuroticism (Bohlmeijer et al. 2011). In the current study, Cronbach’s alpha was α = .86 in the clinical sample and α = .74 in non-clinical sample 1.

Self-Compassion Scale

The Self-Compassion Scale (SCS) is a 26-item questionnaire measuring self-compassion. The Dutch version of the SCS has good psychometric properties and high scores on the SCS are related to psychological well-being (Neff and Vonk 2009). A Dutch 12-item short form of the SCS was developed by Raes et al. (2011), which has good psychometric properties, high internal consistency, and a nearly perfect correlation with the long form of the SCS. The 26-item version was used in the clinical sample, whereas the 12-item version was used in non-clinical sample 1. In the current study, Cronbach’s alpha was α = .93 in the clinical sample and α = .86 in non-clinical sample 1.

Penn State Worry Questionnaire

The Penn State Worry Questionnaire (PSWQ) was developed to measure aspects of clinically significant worry (Meyer et al. 1990). The questionnaire was administered in the clinical and non-clinical sample 1. It measures the tendency, intensity, and uncontrollability of worry and consists of 16 items rated on a 5-point Likert scale, with values ranging from 1 = not at all typical of me to 5 = very typical of me (i.e., “I am always worrying about something,” or “I have been a worrier all my life”). The Dutch version of the PSWQ has a high internal consistency in clinical as well as in non-clinical samples (Kerkhof et al. 2000; Van Van Rijsoort et al. 1997). Cronbach’s alpha value was α = .99 in the clinical sample and α = .95 in non-clinical sample 1.

Beck Depression Inventory-II

The 21-item Beck Depression Inventory-II (BDI-II) is widely used to assess the severity of depressive symptomatology (Beck et al. 1996; Dutch version: Van der Does 2002). Each item is a list of four statements about a symptom of depression, arranged in order of severity. Items are rated on a 4-point scale ranging from 0 (not at all) to 3 (extreme form of each symptom), which corresponds to each statement. Items are summed to give a single total score, which ranges from 0 to 63; a score of 0–13 is considered to reflect minimal depression, 14–19 mild depression, 20–29 moderate depression, and 30–63 severe depression. The questionnaire was administered in the clinical sample and the Cronbach’s alpha was α = .91.

Maslach Burnout Inventory

The emotional exhaustion subscale of the validated Dutch version of the Maslach Burnout Inventory (MBI) was used to measure burnout in non-clinical sample 1 (Maslach and Jackson 1986; Schaufeli and Van Dierendonck 2000). Emotional exhaustion is seen as the core component of burnout and contained of 8 items. Items are scored on a 7-point Likert scale (0 = never, 6 = every day). Cronbach’s alpha was α = .89 in non-clinical sample 1.

Intervention

The MBCT and MBSR programs were based on the original programs by Kabat-Zinn (1990) and Segal et al. (2013) and consisted of eight weekly 2.5-h group sessions, one silent day, and daily 45-min home practice. Mindfulness exercises, such as body scan, gentle yoga, sitting, and walking meditation, were practiced. Participants received teachings on stress and/or depression and were invited to share their experiences. In total, 96.5% of the whole sample attended a MBCT/MBSR course provided by a certified mindfulness trainer, of the remaining 3.5%, no information regarding the trainer was available.

Data Analyses

The data of all three samples were used to investigate the factor structure of the CHIME. For descriptive statistics, internal consistency, sensitivity to change analyses, and correlations, SPSS (version 22) was used, whereas SPSS AMOS was used to perform confirmatory factor analyses (CFAs) and to test measurement invariance. To compute a corrected correlation to evaluate the CHIME-SF, Levy’s formula (Levy 1967) was used as incorporated in the Shortform Version 1.1 software developed by Barrett (2005).

Factor Structure

Based on the original validation study, the model fit of different CFA models was evaluated (Bergomi et al. 2014). For these analyses, data of all three samples were included (n = 481). Of the clinical sample and non-clinical sample 1, the first measurement (i.e., before attending MBSR/MBCT) was used. First, the model fit of a 1-factor model with all items loading on one factor was tested to evaluate whether the items of the CHIME measure one unidimensional construct of mindfulness. Secondly, a correlational and a hierarchical 8-factor model were examined. In the correlational 8-factor model, it is assumed that the items load on eight subscales, which in turn are correlated. This model showed the best fit in the original validation study (Bergomi et al. 2014). In the hierarchical 8-factor model, it is assumed that the items load on subscales, which in turn load on an overall mindfulness factor. Third, based on the original validation study, we examined the model fit of a correlational and hierarchical 7 + 2-factor model. In the correlational 7 + 2-factor model, it is assumed that the factors “inner awareness” and “outer awareness” load on an “awareness” factor, which in turn is correlated with the remaining six subscales. In the hierarchical 7 + 2-factor model, it is assumed that the factors “inner awareness” and “outer awareness” load on an “awareness” factor, which together with the remaining six factors loads on one general mindfulness factor.

In all models, items were restricted to load on one factor only, error terms were not allowed to correlate and the loading of one item per factor was fixed to 1. The overall model chi-square statistic (χ²), where smaller values indicate a better fit, was investigated. In addition, the following indices were used to evaluate the models: root mean square error of approximation (RMSEA), the standardized root mean square residual (SRMR), the comparative fit index (CFI). In line with the original validation study (Bergomi et al. 2014), the following cut-offs were used to evaluate the fit of the models: RMSEA ≤ 0.08, SRMR ≤ 0.10, and CFI ≥ 0.90. In addition, we examined the ratio of the chi-square statistic to its degree of freedom (χ²/df), for which values below a value of 5 were considered as an acceptable model fit (Bohlmeijer et al. 2011; Watkins 1989).

Because we combined data of clinical and non-clinical participants for the main analyses, we also tested measurement invariance between clinical (= 232) and non-clinical groups (non-clinical sample 1 and non-clinical sample 2, n = 249). We tested configural and metric invariance. Configural invariance means the global factor structure, i.e., the number of factors and number of factor loadings for each factor are the same for clinical and non-clinical groups (Byrne 2010). To test metric invariance, we constrain factor loadings to be the same in both groups. Metric invariance therefore indicates that the items contribute to the latent constructs to a similar degree across groups. Models were compared by changes in CFI (≤ − 0.005) and changes in RMSEA (≤ 0.010) as proposed by Chen (2007).

Internal Consistency and Intercorrelation of the Subscales

Internal consistency of the CHIME subscales was evaluated with Cronbach’s alpha. A higher score of Cronbach’s alpha indicates greater internal consistency (Cicchetti 1994; Tavakol and Dennick 2011). Pearson correlations between the subscales were calculated. It was assumed that subscales would be small (< r = .30) to moderately (r = .30–.50) correlated but not strongly (> r = .50), as it is hypothesized that the subscales measure related but unique aspects of mindfulness.

Convergent and Discriminant Validity of the CHIME

Pearson correlations between the CHIME subscales and other measures collected before the MBCT/MBSR were computed for the clinical and non-clinical sample 1. It was hypothesized that the subscales of the CHIME should show moderate or strong positive correlations with another measure of mindfulness skills (FFMQ). It was hypothesized that especially subscales representing similar factors would show moderate to strong correlations. We therefore hypothesized a moderate to strong positive correlation between the subscales “decentering/nonreactivity” (CHIME) and “nonreactivity of inner experience” of the FFMQ, and moderate to strong positive correlations between “acting with awareness” (CHIME) and the subscale “acting with awareness” of the FFMQ (convergent validity). Additionally, we hypothesized that the subscales of the CHIME would show moderate to high positive correlations with a measure of self-compassion (SCS). In contrast, we hypothesized small to moderate negative correlations between subscales of the CHIME and questionnaires measuring depression, worry, or burnout (discriminant validity).

Sensitivity to Change

To assess the ability of the CHIME to detect changes in mindfulness skills, paired-sample t tests on subscales of the CHIME before and after following an 8-week MBCT course (clinical sample) or an 8-week MBSR course (non-clinical sample 1) were conducted. A Cohen’s d effect size based on the baseline standard deviation was computed. Effect sizes between .20 and .50 were considered small, effect sizes between .50 and .80 moderate, and effect sizes greater than .80 large (Cohen 1988).

Development of the Short Form

The steps taken to develop the short form were based on the guidelines by Marsh et al. (2005). Criteria for the development of the CHIME short form (CHIME-SF) included that (1) the CHIME-SF should retain the content coverage of the CHIME; (2) each subscale should include three items; (3) the reliability of each subscale of the CHIME-SF should be acceptable; (4) the CHIME-SF should show a similar factor structure as the CHIME with an acceptable or good model fit; and (5) the CHIME-SF should be comparable to the CHIME in terms of sensitivity to change and discriminant and convergent validity. For each subscale, three items were selected that best reflected the underlying construct, based on the standardized factor loadings in the correlated 8-factor model, minimal cross-loadings as evidenced by the CFA modification indices, and low error correlations with other items. Additionally, the content of all items was subjectively evaluated in order to maintain the coverage of the original subscale to avoid sacrificing important items by relying exclusively on statistical measures (Smith et al. 2000). By using the same evaluation criteria as for the CHIME, the model fit of the CHIME-SF of a 1-factor model and a hierarchical and correlational 8-factor model were evaluated and compared to the CHIME. Configural and metric invariance of the correlational 8-factor model across the clinical and non-clinical group was tested. To compare the CHIME and the CHIME-SF, Pearson correlations were computed. Because both versions were based on a single administration, corrected correlations (r_c) were computed to control for spuriously inflated correlations due to shared measurement error (Levy 1967). Corrected correlations were computed by using the Shortform Version 1.1 developed by Barrett (2005). Additionally, convergent and discriminant validity and sensitivity to change of the CHIME-SF were investigated.

Results

Factor Structure of the CHIME

Table 2 presents the fit indices for the CFA model tested for the CHIME. The single-factor model of the CHIME showed a poor fit to the data, indicating that all items of the CHIME as a group do not measure a unidimensional construct of mindfulness. Both the hierarchical and correlational 8-factor models showed a satisfactory fit; however, in line with the original validation study (Bergomi et al. 2014), the correlational model performed slightly better than the hierarchical. However, one item (item 3) had a low factor loading (.29), indicating that this item contributed minimally to the subscale “insight,” see Table S1 in the Appendix. In accordance with the original validation study, we also examined correlational and hierarchical 7 + 2-factor models, these showed similar model fits to the 8-factor models.

Table 2 CFA fit indices for the models tested for the CHIME and the CHIME-SF (N = 481)

These findings show that the CHIME measures eight distinct but related aspects of mindfulness skills, of which two factors (inner awareness and outer awareness) can be combined as an overall factor measuring awareness. The eight distinct factors can be considered as facets of an overall mindfulness factor.

To test whether the factor structure of the CHIME is stable across clinical and non-clinical groups, configural and metric invariance was investigated. First, the model fit of the correlational 8-factor model was tested separately for the clinical sample and non-clinical group. In both samples, the model had a satisfactory fit (clinical sample: χ² = 1037.90, df = 601, χ²/df = 1.73, CFI = 0.89, RMSEA = 0.056 [90% CI 0.05, 0.062], SRMR = 0.07; non-clinical sample: χ² = 1346.54, df = 601, χ²/df = 2.24, CFI = 0.84, RMSEA = 0.071 [90% CI 0.066, 0.076], SRMR = 0.08). When testing the correlational 8-factor model in a multiple group analysis, a satisfactory model fit was found: χ² = 2384.44, df = 1202, χ²/df = 1.98, CFI = 0.86, RMSEA = 0.045 [90% CI 0.043, 0.048]. This shows that the global factor structure is the same for the clinical and non-clinical group (configural invariance). Next, we constrained the factor loadings to be the same across both groups (metric invariance), which revealed a satisfactory model fit, χ² = 2416.93, df = 1231, χ²/df = 1.96, CFI = 0.861, RMSEA = 0.045 [90% CI 0.042, 0.047]. Model comparisons of the configural model and metric model indicate noninvariance (∆CFI < 0.001, ∆RMSEA < 0.001). This indicates that in the correlational 8-factor model, each item contributes to the latent constructs to a similar degree across the clinical and non-clinical group.

Internal Consistency and Intercorrelations of the Subscales

Correlations between the eight subscales of the CHIME are displayed in Table 3. All subscales were significantly related with Pearson correlations ranging from .17 to .69, indicating that the subscales measure related but distinct facets of mindfulness skills. The reliability of the subscales measured with Cronbach’s alpha and highly acceptable, ranging from α = .71 for acting with awareness to α = .89 for acceptance (see Table 3).

Table 3 Internal consistency and correlations of subscales of the CHIME

Convergent and Discriminant Validity of the CHIME

Pearson correlations between the CHIME and convergent and discriminant measures are displayed in Table 4. In general, the correlations met the predictions about their direction and magnitude. However, the correlations were stronger in the clinical sample compared with the non-clinical sample. As expected, the subscale “decentering/nonreactivity” was strongly, positively correlated with the subscale “reacting” of the FFMQ (clinical sample r = .77; non-clinical sample 1 r = .72). The subscale “acting with awareness” was moderately, positively correlated with the subscale “acting with awareness” of the FFMQ (clinical sample r = .49; non-clinical sample 1 r = .38). In the clinical sample, all subscales of the CHIME showed significant positive correlations with a measure of self-compassion (SCS). All subscales except from “inner awareness” showed significant negative correlations with measures of depression (BDI-II) and worry (PSWQ). That the subscale “inner awareness” was not significantly correlated with measures of depression (BDI-II) and worry (PSWQ) indicates that this subscale measures a construct distinct to negative mental health. The same results were found in non-clinical sample 1, except that in addition, “openness” was not significantly correlated with measures of self-compassion (SCS), depression (BDI-II), or worry (PSWQ) and that “relativity” showed no correlation with depression (BDI-II) and worry (PSWQ).

Table 4 Pearson correlations between the CHIME and other constructs

Sensitivity to Change

The scores on all subscales of the CHIME increased significantly over the course of an MBCT/MBSR training in both the clinical and non-clinical sample 1 (see Table 5). Effect sizes (Cohen’s d) were small to moderate, ranging from .33 (openness) to .70 (insight) in the clinical sample and .20 (acting with awareness) to .66 (decentering/nonreactivity) in the non-clinical sample.

Table 5 Sensitivity to change of the CHIME and CHIME-SF

Evaluation of the CHIME-SF

Based on statistical and content-related considerations described above, 24 items were selected for the CHIME-SF. The items of the CHIME-SF can be found in the Appendix. As with the CHIME, a 1-factor model assuming that all items load on one unidimensional construct showed a poor fit with the data, whereas both 8-factor models showed a good model fit, with the correlational 8-factor model performing slightly better than the hierarchical model (see Table 2). To test configural invariance, the model fit of the correlational 8-factor model was tested separately for the clinical and non-clinical group. After that, the model fit in a multiple group analysis was tested. In both separate analyses (clinical group: χ² = 349.16, df = 224, χ²/df = 1.55, CFI = 0.94, RMSEA = 0.049 [90% CI 0.039, 0.059], SRMR = 0.06; non-clinical group: χ² = 455.99, df = 224, χ²/df = 2.04, CFI = 0.91, RMSEA = [90% CI 0.056, 0.073], SRMR = 0.06) and the multiple group analysis (χ² = 805.14, df = 448, χ²/df = 1.79, CFI = 0.93, RMSEA = 0.04 [90% CI 0.036, 0.045]), the model fit was satisfactory. This shows that the global factor structure of the CHIME-SF is the same for the clinical and non-clinical group (configural invariance). Next, we constrain the factor loadings to be the same across both groups (metric invariance), which revealed a satisfactory model fit, χ² = 817.14, df = 464, χ²/df = 1.76, CFI = 0.93, RMSEA = 0.04 [90% CI 0.035, 0.044]. Model comparisons of the configural and metric invariance models indicate noninvariance (∆CFI = 0.001, ∆RMSEA < 0.01). This indicates configural and metric invariance of the CHIME-SF across clinical and non-clinical groups.

The uncorrected and corrected correlations between the CHIME and the CHIME-SF were high: for “inner awareness” the correlation was r = .91 (r_c = .74), for “outer awareness” r = .98 (r_c = .88), for “acting with awareness” r = .95 (r_c = .72), for “acceptance” r = .97 (r_c = .86), for “decentering/nonreactivity” r = .94 (r_c = .86), for “openness” r = .97 (r_c = .69), for “relativity” r = .95 (r_c = .75), for “insight” r = .93 (r_c = .82). This shows that the CHIME-SF and the CHIME measure highly related constructs.

When investigating the sensitivity to change, the CHIME-SF performed very similar to the CHIME, with significant increases on all subscales and comparable effect sizes (Table 5).

The internal consistency of the CHIME-SF ranged from α = .65 for “openness” to α = .88 for “decentering/nonreactivity,” see Table 6. In order to investigate whether the CHIME-SF and CHIME represent the same content, correlations with other measures were compared. The correlations of the CHIME-SF with other measures (Table 7) were very comparable to those of the CHIME (Table 4). This indicates that the CHIME and CHIME-SF show comparable discriminant and convergent validity (Table 7).

Table 6 Correlations of subscales of the CHIME-SF

Table 7 Pearson correlations between the CHIME-SF and other constructs

Discussion

This study had three aims. First, the CHIME was translated into Dutch according to a standardized procedure. Secondly, the psychometric properties and sensitivity to change of the Dutch CHIME were evaluated. The third aim was to develop a short form of the CHIME with similar psychometric properties and validity.

The CHIME was translated according to a standardized procedure and the model fit was evaluated in a large mixed sample (N = 481) consisting of one clinical sample and two non-clinical samples. The CFA showed an acceptable model fit for the correlated 8-factor structure of the CHIME. This result is in line with the results of the original validation study of the German CHIME (Bergomi et al. 2014) and confirms that the Dutch version of the CHIME measures eight distinct but related aspects of mindfulness skills. However, one item (item 3) had a low factor loading (< .40), indicating that this item contributed minimally to the subscale “insight.” Nevertheless, internal consistency of this subscale (Cronbach’s α = .79) is considered sufficient for research purposes. Future research should evaluate whether this item is also problematic in other populations and if adaptation of the CHIME is necessary. We found configural and metric invariance across the clinical and non-clinical participants, which indicates that the global factor structure of the CHIME is stable across these groups. In line with the original validation study (Bergomi et al. 2014), the CHIME’s construct validity was confirmed by the correlations with discriminant and convergent constructs. The correlations with a distinct mindfulness skills questionnaire, self-compassion, measures of psychopathology (depressive symptoms, burnout), and worry met the predictions about their direction and magnitude. Sensitivity to change analysis showed significant effects of mindfulness-based interventions on the CHIME with small to moderate effect sizes. The largest changes were found for the subscales acceptance, decentering/nonreactivity, and insight, which is again consistent with the original validation study (Bergomi et al. 2014). These results were found in both the clinical and a non-clinical sample, indicating that the Dutch CHIME is an adequate measure to assess change in mindfulness skills in patients as well as in community samples.

Overall, the results indicate that the Dutch version of the CHIME is a valid measure to assess mindfulness skills. The CHIME covers aspects of mindfulness that are not included in the FFMQ, reflected in the subscales openness to experiences, awareness of thought’s relativity, and insightful understanding. Therefore, the CHIME could be particularly useful in research on the differential effects of mindfulness facets and their association with related measures.

Evaluation of the Short Form

The evaluation of the CHIME-SF indicated that the short form we developed was sufficiently reliable and valid. CFA showed a good model fit for the correlated 8-factor structure which was even better compared to the long version CHIME. This may be explained by the fact that items with low factor loadings (including item 3) were eliminated. Furthermore, the internal consistency, convergent and divergent validity, and sensitivity to change of short form also remained similar to the full-length form. In addition, high corrected correlations were found between the short form and the full-length form. These results indicate that the content validity and the psychometric properties were sufficiently preserved in the short form. Previous research has shown that short forms of questionnaires can have as much predictive value as the full version of a questionnaire (Thalmayer et al. 2011). The CHIME-SF seems to be a useful instrument to assess mindfulness skills in research designs including a large number of other instruments or repeated measures. However, until the validity of the CHIME-SF is further evaluated in an independent sample and because full versions in general cover the assessed concept more broadly, the full CHIME should be preferred if time allows.

Limitations and Suggestions for Future Research

A limitation of this study is that the full and the short form were assessed in the same samples. To compare the forms, we used corrected correlations; however, the factor structure of the CHIME-SF should be confirmed in an independent sample. Second, the test retest reliability of the Dutch CHIME should be assessed to make sure that the increase in scores after a mindfulness intervention is not due to measurement error. Third, although we found indications for configural and metric invariance of the CHIME and CHIME-SF across clinical and non-clinical participants, further studies should investigate scalar and residual invariance before mean scores of patients and community samples can be compared. Fourth, it would be valuable to further investigate the sensitivity to change and whether changes in the CHIME are correlated with changes in psychiatric symptoms or well-being. Although the current sample was rather large, response rate after the MBCT/MBSR was not optimal which may have resulted in selection bias with an overrepresentation of participants who benefited from the mindfulness intervention. Additionally, our samples included relatively few men. Finally, translating the CHIME into other languages would increase the accessibility of the questionnaire.