The field of meditation research is continually accumulating solid empirical evidence related to the psychological effects of meditative practices and the neurophysiological changes that occur when entering a “meditative state” (Brandmeyer & Delorme, 2018; Matko et al., 2021; Nash & Newberg, 2013; Schoenberg & Vago, 2019). However, the psychometric investigation of meditative states has mainly been confined to the measurement of the so-called mindful trait, whereas the development of instruments measuring the “mindful state” has received far less attention (Bergomi et al., 2012; Sauer et al., 2013). In addition, concerns have been raised on the validity of these measures for populations with different levels of previous meditation experience, in particular for beginners (Grossman, 2019; Ireland et al., 2019; Kiken et al., 2015). As a result, the few available state instruments have been narrowed to evaluate the state qualities of mindfulness, leaving out relevant features of the complex and rich meditative states (Grossman, 2011; Przyrembel & Singer, 2018; Vago & Zeidan, 2016; Wahbeh et al., 2018).

There are several sources that have studied meditative states, which may be categorized into five main categories. A first—and also the oldest—source is found in the great amount of knowledge held by the ancient traditions from which meditation originated. A second one is the qualitative research investigating meditation states, mostly from phenomenological approaches. A third source is the research on meditation-related theoretical frameworks; a fourth is the neurophysiological characterization of meditation states and, lastly, a fifth source can be found in a review of the existing state mindfulness instruments.

A first source of information can thus be found in the knowledge held by the traditions where meditation originated, mostly Buddhism and Hinduism (Awasthi, 2012; Loizzo, 2014; Tomasino et al., 2014; Wahbeh et al., 2018). These traditions place great value on the experiences that occur during meditative practice, even considering them as an indicator of the meditator’s progress in their long and deep process of “transformation across ethical, perceptual, emotional, and cognitive domains” (Davis & Vago, 2013). This vision is not being taken into account in current mainstream mindfulness research, which might increase the risk of distorting and trivializing meditative practices (Awasthi, 2012; Grossman, 2011; Lindahl et al., 2014; Loizzo, 2014; Lutz et al., 2015; Petitmengin et al., 2019).

It is therefore necessary to recall and clarify the foundation on which these practices are based: in their original contexts, the final goal of meditation is to reach deep states that “transcend” the method itself, which have been referred to—among other terms—as transcendent states (Wahbeh et al., 2018), states of mental tranquility (Vago & Zeidan, 2016), or non-dual awareness states (Hanley et al., 2018; Josipovic, 2014). These deep meditative states are beginning to be studied in experimental neuroscientific frameworks, such as in the experiences of “selflessness” (Dor-Ziderman et al., 2013), “mindful self” (Xiao et al., 2017), “unified compassionate awareness” (Schoenberg et al., 2018), non-dual experiences (Berman & Stevens, 2015; Josipovic, 2014), “awakening” (Britton et al., 2014), “enlightenment” (Davis & Vago, 2013), or “dissolution of body boundaries” (Ataria et al., 2015).

In fact, it has been recently confirmed that a majority of meditators have referred to having mystical and extraordinary experiences, such as, e.g., “the experience of the fusion of your personal self into a larger whole” (Vieten et al., 2018), along with unpleasant meditation-related experiences, such as anxiety or distorted emotions (Schlosser et al., 2019). Traditional sources warn of the need to research these experiences as they might be bewildering for the meditator when not expected (Lindahl et al., 2014).

A second approach is the one provided by qualitative studies. Each of these qualitative approaches has contributed with a different—and to some extent overlapping—categorization of themes arising in meditative practice. A selection of relevant works is briefly outlined below (for extended reviews, see, e.g., Przyrembel and Singer (2018) or Vieten et al. (2018)). A relevant contribution was carried out by Travis and Pearson (2000), who found three common themes: (1) loss of space, time, and body sense; (2) a feeling of peacefulness; and (3) a perception of “the unbounded.” Another interesting study is that of Dor-Ziderman et al. (2013) who addressed the study from a “selflessness” mode and found three categories: (1) the absence of “property/possession,” in which one experiences what is happening without the sense of “this is mine,” but rather as an “observing witness”; (2) an “altered experience,” in a somatic/sensory sense, e.g., “as if I was observing myself from behind my back”; and (3) “less things happening, less judgment, less effort, more quietness, more relaxed.” Another study worth highlighting is the one carried out by Garrison et al., (2013a, b), where two categories of experiences were defined. In the first one, two sub-categories are found (1.a.): “undistracted awareness,” associated to concentration and the observation of the sensorial experience; and (1.b.): “effortless doing,” related to “not doing” and “contentment,” in which descriptions connected to an equanimity state appeared. The second category included (2.a.) “distracted awareness,” associated with the inability to do the exercise, or with the experience of a “wandering” mind, and (2.b.) “controlling,” related to the “effort” of actively trying to change the experience, and “displeasure”: wanting the exercise to end, feeling discomfort, frustration, or anger. Lastly, we would like to highlight the recent work of Przyrembel and Singer (2018) who found different patterns of experiences during diverse kinds of meditation practices, including differential cognitive, affective, perceptive, and somatic states.

A third source of information can be found in meditation-related theoretical frameworks, a field in which research has still not been able to agree on a validated theoretical model. One of the most comprehensive works on this topic is that of Hölzel et al. (2011), which suggests four mechanisms of action of meditation practice: (1) attention regulation (Hölzel et al., 2011; Tang & Posner, 2009, 2013), and more specifically the study of the so-called mind wandering episodes during meditation (Braboszcz & Delorme, 2011; Brandmeyer & Delorme, 2018; Hasenkamp et al., 2012; Vago & Zeidan, 2016); (2) body awareness, on which neuroscience is reporting important findings, like the fact that meditation produces greater interoceptive attention and enhanced regulation of somatic mind wandering (Berkovich-Ohana et al., 2013; Farb et al., 2013; Kerr et al., 2013); (3) emotion regulation: a growing body of literature suggests that meditation practices promote emotion regulation through a wide variety of mechanisms (Chiesa et al., 2013; Garland et al., 2015; Lutz et al., 2008); and (4) change in the perspective of the self, a “selfless processing” mode (Hadash et al., 2016; Shiah, 2016), in the sense that meditative experiences lead to a more “selfless” perception (Dambrun & Ricard, 2011). Other noteworthy theoretical contributions worth highlighting are the conceptual frameworks suggested in Nash and Newberg (2013) and Vago and Silbersweig (2012), and the more recent reviews in Matko et al. (2021), van Dam et al. (2017), and Vieten et al. (2018).

In the fourth place, the study of meditative states is currently being actively addressed by neuroscience, particularly by the emerging field of neurophenomenology (Khachouf et al., 2013; Lutz, 2002; Lutz & Thompson, 2003; Thompson, 2006). Neurophenomenology tackles the joint analysis of (1) the information related to the subjective experience during meditation and (2) its neurologic correlates (usually through electroencephalogram (EEG), functional magnetic resonance imaging, or magnetoencephalography). In this field, we are witnessing important developments in the mapping of meditative states phenomenology, which incorporates both psychometric and neurophysiological coordinates (Schoenberg & Vago, 2019; Tang et al., 2015; Thomas & Cohen, 2014). As it is claimed “future research would benefit from having closer measurements of neurophysiological changes as they directly relate to first-person reports on phenomenology of experiences, such as clarity in the context of meditation and throughout daily life” (Vago & Zeidan, 2016, p. 109).

In particular, there is ample evidence of the effects of meditation on EEG activity (Fell et al., 2010; Lee et al., 2018; Lomas et al., 2015; Schoenberg & Vago, 2019). Among the range of EEG frequency bands (from alpha to gamma), the role of the gamma band has been recently highlighted, with reports evidencing increased activity in this band during meditation, mostly in parieto-occipital regions (Berkovich-Ohana et al., 2012; Berman & Stevens, 2015; Braboszcz et al., 2017; Cahn et al., 2010). Neuroscience researchers suggest that this gamma increase is related to enhanced attention and sensory awareness, characteristic of meditative states (Braboszcz et al., 2017; Cahn et al., 2010; Fell et al., 2010).

The meditation-related psychometric instruments are the fifth and last source. There is very little availability of validated state psychometric measures on meditative states, compared to that of trait measures. Among them, the most used and cited is the Toronto Mindfulness Scale (TMS; Lau et al., 2006), followed far behind by the State Mindfulness Scale (SMS; Tanay & Bernstein, 2013) and the Effects of Meditation Scale (Reavley & Pallant, 2009) in its “During Meditation” version (EOM-DM). In June 2021, the TMS had been cited 567 times on the Scopus ® database, compared with the 147 citations of the SMS and the 23 of the EOM-DM (see Table 1).

Table 1 Comparative study among meditative state instruments regarding sources used

The TMS is a questionnaire with 13 items with a bifactorial structure: a “Curiosity” factor, with items such as “I was curious about what I might learn about myself by just taking notice of what my attention gets drawn to,” and another of “Decentering,” with items such as “I was more interested in just watching my experiences as they arose, than in figuring out what they could mean.” The validity of the TMS has been widely assessed, with uneven results, at least regarding the Curiosity subfactor, on which it has been suggested that further review is needed for its use in the state mindfulness evaluation (Frewen et al., 2011; Ireland et al., 2019). On the other hand, the SMS has 21 items and is built on two factors which measure the subjective levels of present-moment attention and the awareness of the (1) bodily sensations and (2) mental events (Tanay & Bernstein, 2013). Finally, the EOM-DM (Reavley & Pallant, 2009), with 29 items and a five-factor structure (cognitive effects, emotional effects, mystical experiences, relaxation, and physical discomfort), presents as its main limitation the fact that it was validated exclusively with a sample of individuals with previous experience in meditation.

Other efforts have been carried out that are not exactly state instruments, to assess the experiences that appear during meditation practice. This is the case of the Meditation Breath Attention Scores, a performance variable designed as a self-report state measure of focused attention during the practice of mindfulness of the breath (Frewen et al., 2016). Other examples are self-report instruments measuring different aspects of the meditation experience, such as vividness, drowsiness, success, mental effort, and perceived efficacy (Dor-Ziderman et al., 2013; Garrison et al., 2013a, b), or a list of experiences during meditation (Frewen et al., 2011).

In order to provide a global view of the existing state instruments found in the literature review, Table 1 includes an assessment of the main instruments and if their design integrates (yes/no) each of the five sources of information identified in the introduction, along with additional information, such as their year of publication and their number of citations. As may be noted, currently, there is seemingly no instrument that feeds from and includes all the knowledge available from the five different sources of information.

The present research proposes to address the psychometric research on the meditative state from a novel multidisciplinary approach. It constitutes the first attempt—to the best of our knowledge—to integrate the findings of the abovementioned five sources in the design and validation process of a new state instrument: the Meditative State Scale (MSS). The design of the MSS aims at remaining faithful to the traditions where meditation originated, paying attention to the qualitative aspects of meditation, consistent with contemporary cognitive and behavioral research, and with a special focus on its associations with neurophenomenological research. The research in this paper is structured into two studies, as described as follows.

Study 1

In study 1, the MSS is designed and its factor structure is explored. In addition, measurement invariance of the MSS across groups of naïve and experienced meditators is tested. It is hypothesized that the MSS can show measurement invariance across both groups; in particular, configural, metric, and scalar invariance are expected. In addition, the research searched for correlations between MSS scores and the two subfactors (i.e., Curiosity and Decentering) of the most cited existing state measure, the TMS. The construct validity of the MSS was further explored by evaluating correlations with a set of related constructs.

Accordingly, the research selected reliable psychometric instruments, predicting positive correlations would be found with measures of trait mindfulness (Bravo et al., 2018), emotional regulation (Hill & Updegraff, 2012), self-acceptance (Pepping et al., 2013), body awareness (Cox et al., 2020), openness to experience (Kiken et al., 2015), and negative correlations with rumination (Eisenlohr-Moul et al., 2016). Furthermore, it was also predicted that MSS scores would be significantly higher in the group with meditation experience than in the meditation-naïve group.

Method

Participants

The sample comprised a total of 241 participants (27.38% male). The mean age was 30.98 years (SD = 12.63). Regarding previous meditation experience, the sample comprised “beginners”—which had never meditated before (n = 122)—and “experienced” participants (n = 119), which had previous experience with meditation practice. The majority (87.6%) had a college education.

Procedure

After the revision of the five sources of information mentioned in the introduction, we concluded that the initial version of the MSS would be designed from four tentative dimensions. The first one—cognitive (C)—mainly refers to keeping attention and reducing mental contents and mind wandering during meditative practice, such as “Thoughts disappeared little by little.” The affective (A) dimension is related to the appearance of feelings during the meditative practice, such as “I felt the exercise was boring.” The somatic (S) dimension is related to body awareness aspects, with items such as “It was difficult to maintain my posture.” The last dimension is the transcendent (T) one, which included different transcendent states that might occur during meditative practice, such as “I felt I expanded into an infinite wholeness.” The information on which of the five sources are taken into consideration in each of the four tentative dimensions of the MSS is included in the last column of Table 1.

We conducted a study where a deductive phase was followed by an inductive one given the importance of content validity with regard to the formulation of the items. After a close reading of the scientific literature, we started with the operationalization of the first draft of the items to be included in the instrument. The design followed an iterative process involving repeated feedback provided by experts on the subject (academics and meditators of different meditative techniques). The draft was then administered in two experimental tests: one with the laboratory staff from our institution (Comillas Pontifical University) and another with a group of psychotherapists from a center where practices related to meditation are included in the intervention protocol. With the appropriate modifications, a revised version of the instrument was elaborated, which consisted of 43 items, balanced in the number of items included in each of the four tentative dimensions already described.

All participants gave informed consent to participate in the study, and received no monetary compensation. The study received ethics approval by the Ethics Committee of Comillas Pontifical University (Ethics Reference No. 2021/77). The questionnaire was designed in two formats: on paper and online. The paper version was administered to university students and members of a meditation center. The online version was sent to associations and groups related to meditation. Experienced meditators (n = 119) came from meditation-related groups (n = 102) and from university students (n = 17). All beginners (n = 122) were university students. In total, 67% (n = 163) of the participants filled the paper version of the questionnaire and the rest (33%, n = 78) completed it online.

All the participants were asked to fill the questionnaire after a meditation session. The basic instructions of the meditation practice across sessions were similar and belonged to those frequently included in contemporary “focused attention” meditation techniques (Travis & Shear, 2010): a sitting and static position should be assumed, with eyes closed, and an attentional focus on the breath and the body sensations, while maintaining a non-judgmental awareness of subjective experience.

Measures

MSS consists of 43 items (12 reverse-scored), rated on a 5-point Likert scale ranging from 1 (not at all) to 5 (very much). The items are presented randomly and in a balanced amount regarding the four dimensions described in the introduction. The psychometric properties of the instrument on reliability and validity are described in the “Results” section.

Toronto Mindfulness Scale (TMS; Lau et al., 2006) is an instrument consisting of 13 items in a 5-point Likert scale that evaluates the “mindful state,” understood as the state in which a person approaches their experience in each instant, allowing for thoughts, emotions, and sensations to appear. The TMS has a bifactorial structure: Curiosity (T-CUR), with one example item being “I was curious about what I might learn about myself by taking notice of how I react to certain thoughts, feelings or sensations” and Decentering (T-DEC) (e.g. “I experienced my thoughts more as events in my mind than as a necessarily accurate reflection of the way things ‘really’ are”). For its use in the present research, the English version was translated into Spanish by several members of the research team. Then, a reverse translation was produced by a native English speaker. Finally, both texts were compared and a final version was agreed on after analyzing the discrepancies between them. In this study, Cronbach’s alpha (α) results and MacDonald’s omega (ω) were computed. For T-CUR, the corresponding results are as follows: Cronbach’s α = 0.88 and McDonald’s ω = 0.88. For T-DEC, an alpha value of α = 0.73 and an omega value of ω = 0.75 were obtained.

Mindful Attention Awareness Scale (MAAS; Brown & Ryan, 2003) is a scale that globally evaluates the dispositional ability of an individual to stay focused and conscious of the present moment experience in day-to-day life. An example item is “I find it difficult to stay focused on what’s happening in the present.” This instrument was selected with the goal of measuring the meditative trait and its potential relation with the MSS. We used the Spanish adaptation (Soler et al., 2012) of the original version by Brown and Ryan (2003). The MAAS is a unifactorial 15-item questionnaire with items in a 6-point Likert scale, where high scores reveal low dispositions. In this study, Cronbach’s alpha result is α = 0.87, and MacDonald’s omega is ω = 0.88.

We used the Body Awareness subscale (BAW) of the Scale of Body Connection (SBC; Price & Thompson, 2007), Spanish version (Quezada-Berumen et al., 2014) of the original scale (Price & Thompson, 2007). The SBC is an instrument with a bifactorial structure, consisting of a “body awareness” and a “bodily dissociation” dimension. The BAW subscale—with an example item being “I notice how my body changes when I am angry” —includes 12 items with a 5-point Likert scale. Resulting values are Cronbach’s α = 0.82 and McDonald’s ω = 0.83.

We used the “Openness to Experience” subscale of the NEO-FFI Personality Inventory (NEO-FFI; Costa & McCrae, 1992), Spanish adaptation by Manga et al. (2004). The “openness to experience” subscale has 12 items with a 5-point Likert scale, with one example item being “I see myself as someone who is curious about many different things.” Cronbach’s α = 0.80 and McDonald’s ω = 0.81.

We used “Self-acceptance” subscale of Carol Ryff’s (1989) Psychological Wellbeing scale (Ryff, 1989) following the adaptation by Díaz et al. (2006). Ryff suggested a multidimensional model of psychological well-being which included six dimensions. In this research, we used the self-acceptance subscale, which has 4 items with a 5-point Likert scale, one example item being “In general, I feel confident and positive about myself.” Its alpha coefficient is 0.88 and its omega coefficient is 0.89.

“Non-acceptance of emotional responses” subscale of the Difficulties in Emotion Regulation Scale (Gratz & Roemer, 2004) is an instrument with 36 items with a 5-point Likert scale that evaluates different aspects of the process of emotional regulation where difficulties may arise. The non-acceptance subscale includes items such as, e.g., “When I’m upset, I feel guilty for feeling that way.” We used the Spanish adaptation by Hervás and Jódar (2008), based on the original version by Gratz and Roemer (2004). The applied subscale has 6 items (Cronbach’s α = 0.89 and McDonald’s ω = 0.89).

Ruminative Responses Scale (RRS; Nolen-Hoeksema, 1991) is a 22-item scale with a 4-point Likert scale which consists of two subfactors: brooding and reflection. Brooding relates to a tendency toward self-criticism and a negative evaluation of one’s current situation (e.g., “when I am sad I think: What am I doing to deserve this?”). Reflection includes a cognitive appraisal of the situation, in an attempt to understand one’s mood (e.g., “when I am sad, I write down what I am thinking and analyze it”). We used the adaptation by Torres and Hervás (2008) from the original version by Nolen-Hoeksema (1991). Alpha values from our data are 0.91 for the global scale, 0.81 for the brooding subscale and 0.72 for the reflection subscale. Corresponding omega values are 0.92, 0.81, and 0.73.

Data Analyses

With the aim of investigating the factorial structure of the MSS, we first performed sequential exploratory factor analyses (EFAs), starting with the pool of 43 items. The primary goal of an EFA is to reduce a large set of items to items that best exemplify one of the theoretical dimensions without loading high on the others. In this stage, it is also important that the critical analyses related to the content validity of each item as a guideline to decide which items to keep and which items to remove (Henson & Roberts, 2006). An extraction method of maximum likelihood with Oblimin rotation was used. The number of factors to be extracted was determined in each EFA by a parallel analysis. The following criteria was used to accept a final factorial solution with a set of items: loadings greater than 0.4 in one factor, and lower than 0.3 in the rest, contribution of each item to the reliability of its factor, and the statistical significance of the saturations. The EFA was repeated until an acceptable solution was found, eliminating an item in each new analysis, repeating the parallel analysis to set the number of selected factors in the next EFA.

The factorial solution found in EFA was then tested, firstly only in the group of experienced meditators (n = 119) using confirmatory factor analysis (CFA). Then, an invariance testing across groups of experienced meditators and beginners was performed, comparing the models on configural, metric, scalar, and residual invariance. Chosen fit measures were χ2 statistic, chi-square ratio (χ2/df), comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error approximation (RMSEA), and standardized root mean square residual (SRMR). Cutoff criteria for χ2 is a nonsignificant value (p > 0.05); for χ2/df, ratio is ≤ 3; for CFI and TLI, ≥ 0.95; for RMSEA, ≤ 0.05; and for SRMR, < 0.80 (Hu & Bentler, 1999). We used Δχ2, ΔCFI, ΔRMSEA, and ΔSRMR as goodness of fit indexes of the different levels of measurement invariance. Following the recommendations of Chen (2007), we assume no invariance when p < 0.05 for Δχ2, ΔCFI ≥  − 0.005, ΔRMSEA ≥ 0.010, and ΔSRMR ≥ 0.025.

We calculated Cronbach’s alpha (α) results and MacDonald’s omega (ω) (McNeish, 2018) to estimate reliability and we carried out correlation analyses to explore convergent validity. We used the statistical software M-Plus, v.6., to carry out the CFA and the invariance testing, the JASP software (http://jasp-stats.org/) to calculate McDonald’s omega, and SPSS v.20 for the rest of the analyses performed.

Results

Internal Consistency and Factorial Structure

We carried out a reliability analysis of the designed 43-item questionnaire. The resulting Cronbach’s alpha was 0.95 and McDonald’s omega was 0.95. The parallel analysis initially showed a 4-factor solution, with several items with saturations in several factors (see Supplementary Materials). In recursive EFA, to retain or eliminate items, we applied saturation criteria (over 0.40 on its own factor and under 0.30 on the rest), we evaluated Cronbach’s alpha (depending on the increment if the element is deleted), and we checked whether the parallel analysis still converged to 4 factors. At one point (see in Supplementary Materials all the saturations’ tables from each step), the parallel analysis converged to three factors. In this stage, the items were subjected to a critical review based on the scientific literature and our conceptual model to decide to what extent the retained items reflected the multidimensionality of the construct and whether crucial aspects had been overlooked. After the content analysis for critical aspects of “mindful state,” the parallel analysis and the saturation criteria, a final 12-item version of the MSS emerged, encompassing three dimensions with four items each (see Table 2), with appropriate internal consistency indexes (Cronbach’s alpha was 0.92, 0.89, and 0.88 and McDonald’s omega was 0.92, 0.89, and 0.88, for factors 1, 2, and 3).

Table 2 Exploratory factor analysis (EFA): item factor loadings

According to the content of the factors, we decided to name them as follows: factor 1—“transcendence” (items 6, 10, 37, 43); factor 2—“difficulties” (items 9, 23, 32, 33); factor 3—“mental quietening” (items 1, 12, 30, 38). Factor 1 (“transcendence”) comprises 4 positive scored items from the transcendent dimension of the theoretical design, while, interestingly, in factor 2 (“difficulties”) two items originating from the theoretical affective dimension and two from the somatic dimension converge in a balanced way, the four of them being reverse-scored. Finally, factor 3 consists of 4 positive scored items originating from the cognitive dimension.

The existence of an overarching latent variable was also explored by computing item loadings for a forced one-factor solution, and item-to-total correlations (see Table 2). In addition, the corresponding reliability analysis of the total scale was also carried out, resulting in α = 0.91 and ω = 0.91 values. These results support the usage of the total MSS score for the assessment of meditative states, along with that of its three subscales.

Confirmatory Factor Analysis and Measurement Invariance

According with the CFA, the 12-item, 3-dimension model (see Fig. 1) adjusted correctly in the subsample of experienced meditators: (χ2(51) = 74.80, p = 0.012, RMSEA = 0.063 (90% CI = 0.028, 0.092), CFI = 0.968, TLI = 0.958, SRMR = 0.068). Model 1 specified configural invariance forcing the same factor structure simultaneously estimated in experienced meditators and beginners, with no between-group constraints on parameter estimates. The tests of model 1, which tested for configural invariance, suggest a well-fitting model (χ2(102) = 141.0, p = 0.006, RMSEA = 0.056 [90% CI = 0.031, 0.078], CFI = 0.977, TLI = 0.970, SRMR = 0.064). Model 1 is used as the base model to compare the metric invariance (Model 2). Table 3 shows the comparison between invariance models. The factor loadings are forced to be the same among beginners and experienced meditators in model 2, that also showed adequate adjustment indexes (χ2(111) = 149.31, p = 0.009, RMSEA = 0.054 [90% CI = 0.028, 0.075], CFI = 0.977, TLI = 0.973, SRMR = 0.075). Model 2 does not result in a deterioration of fit compared with the configural model (∆χ2 = 8.28, p = 0.506). However, the scalar invariance model (model 3, item intercepts are forced to be equal) did not show such a good adjustment (χ2(120) = 171.32, p = 0.002, RMSEA = 0.060 [90% CI = 0.038, 0.079], CFI = 0.969, TLI = 0.966, SRMR = 0.091), presenting a deterioration of fit (∆χ2 = 22.01, p = 0.009). The residual invariance (model 4) did not reach the desired adjustment (χ2(130) = 254.21, p < 0.001, RMSEA = 0.089 [90% CI = 0.073, 0.105], CFI = 0.926, TLI = 0.925, SRMR = 0.216), with a significantly worse adjustment than the model of scalar invariance (∆χ2 = 82.89, p < 0.001).

Fig. 1
figure 1

Confirmatory factor analysis, expert group

Table 3 Invariance models between beginners and experienced meditators

Convergent and Discriminant Validity

First, we analyzed the association between the scores obtained in the three subfactors of the MSS scale and the two subfactors of the TMS, performing a correlation analysis (Pearson bivariate) with the whole sample (n = 241). As shown in Table 4, almost all cases presented significant correlations, the highest being among factors 2 (difficulties) and 3 (mental quietening) in relation to factor 2 (decentering) of the TMS scale.

Table 4 Correlations between MSS and TMS subfactor scores with trait variables (n = 241)

However, we should highlight that the subfactor 1, “Curiosity,” of the TMS (TMS-CUR) has no significant correlations with factors 2 and 3 of the MSS, which means that curiosity during the meditative practice—at least in the way the TMS measures it—has no clear correlation with mental quietening or the reduction of difficulties (an example of an item of the “curiosity” factor is “I was curious about what I might learn about myself by taking notice of how I react to certain thoughts, feelings or sensations”). On the other hand, the TMS “decentering” subfactor, in which a distance is kept from the content of the experience, does seem to be related to significantly higher scores in the three subfactors of the MSS (an example of an item in the “decentering” subfactor is: “I approached each experience by trying to accept it, no matter whether it was pleasant or unpleasant”).

Next, we analyzed the correlations among MSS and TMS subfactor scores with the rest of the variables. The results (Pearson bivariate correlations) are included in Table 4. In general, as shown by the values in Table 4, the correlations obtained confirm the convergent validity of the MSS. However, the results obtained by the TMS are not as satisfactory, mainly due to the “curiosity” subfactor, which shows uneven correlations and, in some cases, the opposite of the expected result (e.g., a positive correlation in obtained for the Brooding subfactor of the RRS).

In particular, and starting with the correlations with MAAS scores, the three subfactors of the MSS obtain significant negative correlations, as expected, while TMS subfactor scores only obtain a significant correlation with the “decentering” subfactor. The same applies to the “self-acceptance” subfactor of the Ryff scale, where the expected correlations are obtained, except, again, for the “curiosity” subfactor of the TMS. On the other hand, both in the case of “body awareness,” measured with the BAW, and in “openness to the experience” of the NEO-FFI, the three subfactors of the MSS and the TMS obtain statistically significant positive correlations, as predicted.

As for the correlations obtained for the “reflection” subfactor of the RRS, they resulted in a positive sign for some of the subfactors of both instruments. However, they did not correlate with subfactors 2 and 3 of the MSS, probably because of the reflexive aspects of the concept which we will comment on in the “Discussion” section. On the other hand, our results showed a positive correlation with the “curiosity” subfactor of the TMS. Regarding the DERS “non-acceptance” subfactor, factors 2 and 3 showed negative correlations, while no significant correlations were found with the two subfactors of the TMS, as was initially expected.

Differences Based on the Existence of Previous Meditation Experience

In order to test the discriminating power of the MSS and compare it with that of the TMS, we evaluated whether there were statistically significant differences in the scores of the MSS and TMS subfactors between beginners (n = 122) and experienced (n = 119) groups. The results of the statistical analysis (ANOVA), together with the analysis of the effect size (Cohen’s d), are shown in Table 5.

Table 5 Effect of previous meditation experience on MSS and TMS factor scores

In the case of the MSS, we found statistically significant differences between groups for the three subfactors: the effect sizes ranging from 0.41 (for F1: transcendence) to 1.21 (for F3: mental quietening). In the case of the TMS, no significant differences were found for the “curiosity” subfactor (T-CUR); however, they were present for the “decentering” subfactor (T-DEC).

Discussion

In study 1, we reduced the number of items through EFA and content analyses in order to keep those items with the strongest psychometric properties and alignment with the proposed underlying structure. In addition, we tested the stability of the structure through confirmatory analyses and measurement invariance, and finally we tested construct validity. According to the results obtained, we can conclude that the MSS and its three subscales show good levels of reliability, factorial validity, configural and metric measurement invariance across groups of experienced meditators and beginners, convergent and divergent validity, and discriminating power. Study 2, below, provides additional evidence of the validity of the MSS, through its application in an experiment with a neurophenomenological approach.

Study 2

In study 2, an experiment was designed to explore the link between reported experiential features of meditative states—measured by the MSS—and their neural correlates—in terms of their EEG activity. More specifically, and following suggestions from recent research, study 2 focused in the EEG gamma frequency activity (Berkovich-Ohana et al., 2012; Berman & Stevens, 2015; Braboszcz et al., 2017; Cahn et al., 2010). To our knowledge, our experiment is one of the first studies linking a self-report measure of the meditative state—the MSS—with its associated EEG brain activity patterns. Hence, we cannot rely on earlier findings to define specific research hypothesis on the relation between the MSS and their brain activity correlates during meditation practice. However, there is evidence suggesting that gamma activity in parieto-occipital areas is mostly related to enhanced attentive processes, with increased sensory awareness (Berkovich-Ohana et al., 2012; Braboszcz et al., 2017; Cahn et al., 2010). As these attentive processes can be concomitant with different meditative “state qualities,” we may expect positive correlations with all the three MSS subscales, but mostly with F3, “mental quietening,” as this subscale is the one which has more relation to the engagement in attentional processes

Method

Participants

Twelve participants participated in the experiment (8 females and 4 males; mean age = 34.92 years old; SD = 11.90). Individuals with any current formally diagnosed psychiatric disorder or under psychiatric medication were excluded. All participants took part voluntarily, with no monetary compensation. Before starting the experiment, they were informed of the characteristics of the study and had the possibility to ask questions about the experiment before giving their informed consent. With the goal of having a balanced sample of participants regarding their meditative practice, the sample included 4 participants in each of the following groups: (1) a group of high experienced meditators (with more than 8 years of practice), (2) another with intermediate experience (between 1 and 8 years of experience) and (3) a group of beginners (less than a year of experience), following similar neurophenomenological studies (Berman & Stevens, 2015; Lutz et al., 2004, 2008). The experiment was approved by the Ethics Committee of Comillas Pontifical University (Ethics Reference No 2021/77).

Procedure

All participants were invited to come to the Psychology Lab on the day the experiment was to take place. They were informed about the characteristics of the experiment. After giving informed consent and filling a basic short form with demographic data, they were taken to the experimental room of the Psychology Lab, where EEG data collection took place. Study 2 received ethics approval by the Ethics Committee of Comillas Pontifical University. EEG data recordings began with a standard 5-min recording of baseline EEG activity. To that end—and following the indications of previous studies (Braboszcz et al., 2017; Hauswald et al., 2015) —the participants were instructed to let their minds wander during these 5 min, with their eyes closed. Then, the participants started a 20-min meditation session. They were instructed to meditate in a sitting position, with their eyes closed, paying attention to the breathing sensation in the abdomen, an instruction included in the “focused-attention” meditation techniques (Travis & Shear, 2010), and frequent in Zazen meditation practice (Austin, 2013). Immediately after the meditation session, the participants were asked to complete the MSS questionnaire.

EEG activity was recorded using a 64-electrode cap placed on the scalp in an extended 10–20 configuration (ActiCHamp, Revision 2, Brain Products GmbH, Munich, Germany), supported by BrainVision PyCorder software (Version 1.10, Brain Products GmbH, Munich, Germany). Electrooculographic data were recorded with two electrodes located in a horizontal and vertical direction to record eye movement. All electrodes were referenced to the mastoids (right and left). Electrode impedances were kept below 10 kΩ while recording. EEG data were sampled at 500 Hz and the amplifier frequency bands were set between 0.1 and 50 Hz. Ocular artifacts were corrected with an independent component analysis.

Data were then filtered into different frequency bandwidths; as justified earlier, for the purpose of our study, we selected the gamma bandwidth (30.5–50 Hz). After that, we followed the standard procedure and computed mean spatially aggregated power values for 12 cortical regions: 8 lateral clusters (right and left)—anterior frontal, frontal central, central parietal, and parieto-occipital—and 4 medial clusters: anterior frontal, frontal central (MFC), central parietal (MCP), and parieto-occipital. The distribution of the 64 electrodes and the 12 cortical regions is represented in Fig. 2. Differences in gamma-band power between baseline control condition and meditative state were computed for each of the 12 cortical regions as a difference in corresponding mean power values.

Fig. 2
figure 2

Location of the electrodes and the 12 cortical regions

Measures

MSS is the 12-item version validated in study 1. The participants also filled a short form with some basic demographic data and information on their previous meditative practice.

Data Analyses

Given the size of the sample, we used Spearman’s correlation to explore the correlations between MSS scores and differences in the gamma-band power (de Winter et al., 2016). SPSS software (v.20) was used.

Results

Table 6 shows resulting Spearman correlations between MSS subscale scores and corresponding changes in gamma power between the meditative and the baseline conditions, for each of the 12 cortical regions. Results show that the subfactor “mental quietening” (F3) positively correlates with changes in gamma power in central parietal and parieto-occipital left regions, as well as MFC and MCP regions. These results indicate that greater gamma power changes in these regions occur among participants who reported experiencing higher “mental quietening” (F3) during the meditation session, while lower gamma power changes were recorded in those with a lower score in this subscale.

Table 6 Correlations between changes in gamma power and MSS subscale scores

Discussion

Study 2 findings provide additional evidence supporting the validity of the MSS. As we had initially hypothesized, we found MSS scores—in particular “mental quietening” (F3) scores—to be positively correlated with changes in gamma-band EEG activity in specific cortical regions.

General Discussion

The present research was intended to design and validate a novel psychometric instrument, the MSS, aimed to evaluate meditative states from an inclusive approach that respects the foundational pillars in which meditation practices were grounded. To our knowledge, the MSS constitutes the first attempt to validate a meditation-related psychometric instrument that integrates the knowledge from a wide range of disciplines that have studied the meditative experience, i.e., traditional meditation sources, qualitative research, contemporary psychological science, neuroscientific evidence, and psychometric research.

Study 1 found evidence of the reliability and psychometric validity of the MSS, which supports the conviction that the MSS allows the evaluation of the meditative state with enough guarantees for its use in meditation research. The exploratory factorial analysis showed a structure consisting of three subfactors: F1, “transcendence”; F2, “difficulties”; and F3, “mental quietening.” The appearance of a transcendent factor is consistent with recent research showing that the majority of meditators experience mystical/extraordinary experiences while meditating (Vieten et al., 2018). Regarding the “difficulties” subscale, it integrates items of both the affective (e.g., “I wanted the exercise to end”) and somatic (e.g., “It was difficult to maintain my posture”) hypothesized dimensions, and is aligned with evidence suggesting that meditative practice does not only entail “positive” experiences, but also the scarcely investigated unpleasant meditation-related experiences (Schlosser et al., 2019). Lastly, it was highly expected to find a “mental quietening” subscale, which mainly refers to the largely analyzed attentional aspects of meditation practice, in which mind wandering is reduced and mental tranquility arises (Vago & Zeidan, 2016).

The MSS factorial structure revealed measurement invariance (on a configural and metric levels). Measurement invariance was not obtained on a scalar or residual level, which is not unusual in state measures and in the differential behavior of several trait dimensions (Baer et al., 2008; de Bruin et al., 2012; Lecuona et al., 2019). Additionally, we confirmed the validity of MSS in relation to the TMS, concluding that the MSS includes information that is not captured by the TMS, and that it has better psychometric properties. In this sense, one of the most remarkable results—because the opposite was expected—is the absence of a significant positive correlation between the “mental quietening” subfactor of the MSS and the “curiosity” subfactor of the TMS. It is possible that “curiosity”—at least the way it is measured by the TMS—is not an attitude that favors the reduction of thoughts or their quietening, but rather it is related to an increase in mental activity.

Another interesting discussion regards the correlations obtained between both TMS and MSS subfactors and the “reflection” subscale of the RRS. Our findings show that the RRS “reflection” subscale positively correlates with certain subscales of both the MSS and the TMS. This is a surprising result—as negative correlations were expected—given that this “reflection” in the ruminative scale is related to the cognitive ability to analyze what is happening, which is usually put aside during the meditation practice. It is possible, however, that the source of confusion is the way the items in the “reflection” subfactor of the RRS are worded (e.g., “You analyze your personality to try to understand why you are depressed”), which may have “constructive” connotations, closer to the attitudes toward present-moment awareness that are nurtured through meditation. These attitudes, sometimes referred to as “mindfulness qualities” or “mindfulness skills,” include aspects such as acceptance, kindness, openness, non-judging, patience, non-striving, and disidentification from present-moment experience (Hadash, 2019; Shapiro et al., 2006). An attitude of “reflection,” understood as “collaborating” with these aforementioned mindful qualities, could help to let some thoughts “pass through,” instead of keeping “ruminating” them.

A second unexpected result relates to the negative correlations of the “curiosity” subfactor of the TMS with the “brooding” subfactor of the RSS and with both F2 (“difficulties”) and F3 (“mental quietening”) subfactors of the MSS. This result is consistent with prior findings that suggest that the TMS curiosity items may not adequately reflect the qualities of the attention that should be present when meditating (Frewen et al., 2011, 2016; Ireland et al., 2019). In fact, it has been warned that “attending without bringing the ‘heart qualities’ -acceptance, kindness and openness- into the practice may result in practice that is condemning or judgmental of inner experience” (Shapiro et al., 2006, p. 377). A similar line of thought is proposed in a recent exploration of the construct validity of the TMS, which has suggested that this scale could be “described as a measure of a mindful orientation toward experience rather than an overall measure of state mindfulness” (Ireland et al., 2019, p. 13).

Additional evidence supporting the validity of the MSS is provided by study 2 findings. These results are aligned with emergent neuroscientific evidence highlighting the important role of gamma activity in deep meditative states (Berkovich-Ohana et al., 2012; Braboszcz et al., 2017; Schoenberg et al., 2018). More specifically, gamma power increases in parietal and occipital regions have been associated with enhanced attentional abilities, along with a greater implication in “not engaging” with distracting stimuli (Cahn et al., 2010), aspects that are related to the “mental quietening” subscale.

Limitations and Further Research

The present research presents some limitations. First, regarding the sample in study 1, it is pertinent to mention the low percentage of males, even though there is no empiric evidence that sex is a variable influencing reached meditative states (Braboszcz et al., 2017; Lau et al., 2006; Tanay & Bernstein, 2013). Study 1 findings may also be affected by common method bias (Podsakoff et al., 2012)—a phenomenon that affects any research using self-reported measures. Its presence and the extent of its influence on the results should be explored in future research in which the MSS is administered. In addition, study 1 findings should also be interpreted considering the potential confounding factor brought by the two different administration formats (on paper and online). Although all the participants in study 1 followed similar meditation instructions, these two administration formats may bring a potential bias that should be addressed in future studies. These future study extensions should also explore the validity of the MSS when related to other types of meditation practices, including dynamic ones, such as walking meditation, or yoga, which are part of many mindfulness-based programs.

Another future research direction could be that investigating to what extent trajectories of change in MSS scores vary depending on the initial meditation experience level. This research path is of special interest for the case of beginners, and could be followed by studies administering the MSS at different stages of a mindfulness-based intervention program. This would allow to overcome a last limitation of study 1 findings: that regarding the need for the evaluation of the test–retest reliability of the scale.

Regarding study 2, a first limitation is related to the sample size, although our size is comparable to that of similar neuroimaging studies (see meta-analysis: Lomas et al., 2015). A further limitation of this study refers to the fact that data were collected for a single type of meditative exercise (breath focus), even though the different meditation practices share a great part of their phenomenological content (Berkovich-Ohana et al., 2013; Schoenberg & Vago, 2019). To address these limitations, future research could attempt to replicate study 2 findings through comparative designs using other type of meditative practices and larger samples, while further exploring potential associations between factor 1 and factor 2 MSS scores and their neurological correlates. This would also contribute to further understanding the role that previous meditation experience plays in the observed neurophysiological changes.

Another promising line of research could be that of using the MSS to achieve a better understanding of the associations between state and trait meditation measures and how they develop while engaging a meditative practice. This includes investigating the conditions under which the trajectories of change in the experienced meditative states may lead to the documented effects of meditation in certain psychological health outcomes (Bravo et al., 2018; Kiken et al., 2015).

The present findings set the stage for future applications of the MSS that may help overcome important challenges of the meditation research field (Thomas & Cohen, 2014; Van Dam et al., 2017; Vieten et al., 2018). Reliable psychometric instruments such as the MSS hold the potential to systematize the meditative experience in a way that can guide investigations where the first-person perspective of experience and the third-person perspective of neuroscience are integrated, as it is being recently claimed (Frewen et al., 2016; Petitmengin et al., 2019; Schoenberg & Vago, 2019; Vago & Zeidan, 2016). This integration of perspectives can promote an enhanced understanding of the complex mechanisms involved in meditation practice, leading to a better-supported and more rigorous use of meditation in research, educational, and clinical contexts. We hope that in this challenge—and hand in hand with contemplative traditions—experience and science may “enlighten” one another.