Response Shift After a Mindfulness-Based Intervention: Measurement Invariance Testing of the Comprehensive Inventory of Mindfulness Experiences

Abstract

Mindfulness-based interventions (MBIs) have been linked to positive outcomes for a range of psychological and physical health conditions, prompting the need for a high degree of validity and reliability in the measurement of mindfulness. While a number of mindfulness self-report instruments are available with demonstrated psychometric robustness, limited empirical data are available on the extent to which ratings at different time points may be affected by changes in standards of reference that may occur as a result from having completed an MBI. The present study investigated the presence of response shift in a sample of 181 MBI course participants who completed the 37-item Comprehensive Inventory of Mindfulness Experiences (CHIME) during the first and final week of the course. Measurement invariance testing using confirmatory factor analysis investigated invariance of the factor structure (configural invariance), factor loadings (metric invariance), and intercepts (scalar invariance) across the two measurement time points. Lack of scalar invariance indicated evidence of response shift for 4 or possibly 7 of the 37 items. The relatively minor amount of response shift is encouraging for the field of mindfulness measurement, particularly since it has generally been hypothesized that mindfulness is particularly prone to this phenomenon. Further studies using other instruments and techniques to investigate response shift are recommended.

This is a preview of subscription content, log in to check access.

References

  1. Baer, R. A., Smith, G. T., & Allen, K. B. (2004). Assessment of mindfulness by self-report—the Kentucky Inventory of Mindfulness Skills. Assessment, 11(3), 191–206. doi:10.1177/1073191104268029.

    Article  PubMed  Google Scholar 

  2. Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13(1), 27–45. doi:10.1177/1073191105283504.

    Article  PubMed  Google Scholar 

  3. Belzer, F., Schmidt, S., Lucius-Hoene, G., Schneider, J. F., Orellana-Rios, C. L., & Sauer, S. (2013). Challenging the construct validity of mindfulness assessment—a cognitive interview study of the Freiburg Mindfulness Inventory. Mindfulness, 4(1), 33–44. doi:10.1007/s12671-012-0165-7.

    Article  Google Scholar 

  4. Bergomi, C., Tschacher, W., & Kupper, Z. (2013). Measuring mindfulness: first steps towards the development of a comprehensive mindfulness scale. Mindfulness, 4(1), 18–32. doi:10.1007/s12671-012-0102-9.

    Article  Google Scholar 

  5. Bergomi, C., Tschacher, W., & Kupper, Z. (2014). Konstruktion und erste Validierung eines Fragebogens zur umfassenden Erfassung von Achtsamkeit [Construction and initial validation of a questionnaire for the comprehensive investigation of mindfulness]. Diagnostica, 60(3), 111–125. doi:10.1026/0012-1924/a000109.

    Article  Google Scholar 

  6. Billington R., & Krägeloh, C. U. (2015). Quality of life and higher education. In M. A. Henning, C. U. Krägeloh, & G. Wong-Toi (Eds.), Student motivation and quality of life in higher education (pp. 28–36). Oxon, United Kingdom: Routledge.

  7. Brown, K. W., & Ryan, R. M. (2003). The benefits of being present: mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology, 84(4), 822–848. doi:10.1037/0022-3514.84.4.822.

    Article  PubMed  Google Scholar 

  8. Bryant, F. B., & Satorra, A. (2012). Principles and practice of scaled difference chi-square testing. Structural Equation Modeling, 9(3), 372–398. doi:10.1080/10705511.2012.687671.

    Article  Google Scholar 

  9. Buchheld, N., Grossman, P., & Walach, H. (2001). Measuring mindfulness in insight meditation (Vipassana) and meditation-based psychotherapy: the development of the Freiburg Mindfulness Inventory (FMI). Journal of Meditation and Meditation Research, 1(1), 11–34.

    Google Scholar 

  10. Camfield, L., & Skevington, S. M. (2008). On subjective well-being and quality of life. Journal of Health Psychology, 13(6), 764–775. doi:10.1177/1359105308093860.

    Article  PubMed  Google Scholar 

  11. Chiesa, A., Anselmi, R., & Serretti, A. (2014). Psychological mechanisms of mindfulness-based interventions: what do we know? Holistic Nursing Practice, 28(2), 124–148. doi:10.1097/HNP.0000000000000017.

    Article  PubMed  Google Scholar 

  12. Dimidjian, S., & Linehan, M. M. (2003). Defining an agenda for future research on the clinical application of mindfulness practice. Clinical Psychology: Science and Practice, 10, 166–171. doi:10.1093/clipsy/bpg019.

    Google Scholar 

  13. Frewen, P. A., Unholzer, F., Logie-Hagan, K. R.-J., & MacKinley, J. D. (2014). Meditation breath attention scores (MBAS): test-retest reliability and sensitivity to repeated practice. Mindfulness, 5(2), 161–169. doi:10.1007/s12671-012-0161-y.

    Article  Google Scholar 

  14. Grossman, P. (2008). On measuring mindfulness in psychosomatic and psychological research. Journal of Psychosomatic Research, 64(4), 405–408. doi:10.1016/j.jpsychores.2008.02.001.

    Article  PubMed  Google Scholar 

  15. Grossman, P. (2011). Defining mindfulness by how poorly I think I pay attention during everyday awareness and other intractable problems for psychology’s (re)invention of mindfulness: comment on Brown et al. (2011). Psychological Assessment, 23(4), 1034–1040. doi:10.1037/a0022713.

    Article  PubMed  Google Scholar 

  16. Hofmann, S. G., Sawyer, A. T., Witt, A. A., & Oh, D. (2010). The effect of mindfulness-based therapy on anxiety and depression: a meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 169–183. doi:10.1037/a0018555.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Hu, L., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: sensitivity to underparameterized model misspecification. Psychological Methods, 3(4), 424–453. doi:10.1037/1082-989X.3.4.424.

    Article  Google Scholar 

  18. Jöreskog, K. G. (1990). New developments in LISREL: analysis of ordinal variables using polychoric correlations and weighted least squares. Quality & Quantity, 24(4), 387–404. doi:10.1007/BF00152012.

    Article  Google Scholar 

  19. Jöreskog, K. G., & Sörbom, D. (1993). LISREL 8: structural equation modelling with the SIMPLIS command language. Hillsdale: Erlbaum Associates.

    Google Scholar 

  20. Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degress of freedom. Sociological Methods & Research, 44(3), 486–507. doi:10.1177/0049124114543236.

    Article  Google Scholar 

  21. Leigh, J., Bowen, S., & Marlatt, G. A. (2005). Spirituality, mindfulness and substance abuse. Addictive Behaviors, 30(7), 1335–1341. doi:10.1016/j.addbeh.2005.01.010.

    Article  PubMed  Google Scholar 

  22. Levinson, D. B., Stoll, E. L., Kindy, S. D., Merry, H. L., & Davidson, R. J. (2014). A mind you can count on: validating breath counting as a behavioral measure of mindfulness. Frontiers in Psychology, 5, 1202. doi:10.3389/fpsyg.2014.01202.

  23. Medvedev, O. N., Siegert, R. J., Feng, X. J., Billington, D. R., Jang, J. Y., & Krägeloh, C. U. (2016a). Measuring trait mindfulness: how to improve the precision of the Mindful Attention Awareness Scale using a Rasch model. Mindfulness, 7(2), 384–395. doi:10.1007/s12671-015-0454-z.

    Article  Google Scholar 

  24. Medvedev, O. N., Siegert, R. J., Kersten, P., & Krägeloh, C. U. (2016b). Rasch analysis of the Kentucky Inventory of Mindfulness Skills. Mindfulness, 7(2), 466–478. doi:10.1007/s12671-015-0475-7.

    Article  Google Scholar 

  25. Medvedev, O. N., Siegert, R. J., Kersten, P., & Krägeloh, C. U. (2017a). Improving the precision of the Five Facet Mindfulness Questionnaire using a Rasch approach. Mindfulness. doi:10.1007/s12671-016-0676-8.

  26. Medvedev, O. N., Krägeloh, C. U., Narayanan, A., & Siegert, R. J. (2017b). Measuring mindfulness: applying generalizability theory to distinguish between state and trait. Mindfulness. doi:10.1007/s12671-017-0679-0.

  27. Oort, F. J. (2005). Using structural equation modeling to detect response shifts and true change. Quality of Life Research, 14(3), 587–598. doi:10.1007/s11136-004-0830-y.

    Article  PubMed  Google Scholar 

  28. Park, T., Reilly-Spong, M., & Gross, C. R. (2013). Mindfulness: a systematic review of instruments to measure an emergent patient-reported outcome (PRO). Quality of Life Research, 22(10), 2639–2659. doi:10.1007/s11136-013-0395-8.

    Article  PubMed  Google Scholar 

  29. Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: the state of the art and future directions for psychological research. Developmental Review, 41, 71–90. doi:10.1016/j.dr.2016.06.004.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Raju, N. S., Lafitte, L. J., & Byrne, B. M. (2002). Measurement equivalence: a comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 517–529. doi:10.1037//0021-9010.87.3.517.

    Article  PubMed  Google Scholar 

  31. Reiner, K., Tibi, L., & Lipsitz, J. D. (2013). Do mindfulness-based interventions reduce pain intensity? A critical review of the literature. Pain Medicine, 14(2), 230–242. doi:10.1111/pme.12006.

    Article  PubMed  Google Scholar 

  32. Sauer, S., Walach, H., Offenbächer, M., Lynch, S., & Kohls, N. (2011). Measuring mindfulness: a Rasch analysis of the Freiburg Mindfulness Inventory. Religions, 2(4), 693–706. doi:10.3390/rel2040693.

    Article  Google Scholar 

  33. Sauer, S., Walach, H., Schmidt, S., Hinterberger, T., Lynch, S., Büssing, A., & Kohls, N. (2013a). Assessment of mindfulness: review on the state of the art. Mindfulness, 4(1), 3–17. doi:10.1007/s12671-012-0122-5.

    Article  Google Scholar 

  34. Sauer, S., Ziegler, M., Danay, E., Ives, J., & Kohls, N. (2013b). Specific objectivity of mindfulness—a Rasch analysis of the Freiburg Mindfulness Inventory. Mindfulness, 4(1), 45–54. doi:10.1007/s12671-012-0145-y.

    Article  Google Scholar 

  35. Scholderer, J., Grunert, K. G., & Brunsø, K. (2005). A procedure for eliminating additive bias from cross-cultural survey data. Journal of Business Research, 58(1), 72–78. doi:10.1016/S0148-2963(02)00475-7.

    Article  Google Scholar 

  36. Schwartz, C. E. (2010). Applications of response shift theory and methods to participation measurement: a brief history of a young field. Archives of Physical Medicine and Rehabilitation, 91(9 Suppl), S38–S43. doi:10.1016/j.apmr.2009.11.029.

    Article  PubMed  Google Scholar 

  37. Schwartz, C. E., & Sprangers, M. A. G. (1999). Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science & Medicine, 48(11), 1531–1548. doi:10.1016/S0277-9536(99)00047-7.

    Article  Google Scholar 

  38. Shennan, C., Payne, S., & Fenlon, D. (2011). What is the evidence for the use of mindfulness-based interventions in cancer care? A review. Psycho-Oncology, 20(7), 681–697. doi:10.1002/pon.1819.

    Article  PubMed  Google Scholar 

  39. Van Dam, N. T., Earlywine, M., & Borders, A. (2010). Measuring mindfulness? An Item Response Theory analysis of the Mindful Attention Awareness Scale. Personality and Individual Differences, 49(7), 805–810. doi:10.1016/j.paid.2010.07.020.

    Article  Google Scholar 

  40. Visted, E., Vøllestad, J., Nielsen, M. B., & Nielsen, G. H. (2015). The impact of group-based mindfulness training on self-reported mindfulness: a systematic review and meta-analysis. Mindfulness, 6(3), 501–522. doi:10.1007/s12671-014-0283-5.

    Article  Google Scholar 

  41. Walach, H., Buchheld, N., Buttenmüller, V., Kleinknecht, N., & Schmidt, S. (2006). Measuring mindfulness—the Freiburg Mindfulness Inventory (FMI). Personality and Individual Differences, 40(8), 1543–1555. doi:10.1016/j.paid.2005.11.025.

    Article  Google Scholar 

  42. Wu, W., & West, S. G. (2010). Sensitivity of fit indices to mispecification in growth curve models. Multivariate Behavioral Research, 45(3), 420–452. doi:10.1080/00273171.2010.483378.

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Christian U. Krägeloh.

Ethics declarations

The study was conducted in compliance with the guidelines of the university ethics committee of one of the authors.

Conflict of Interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Krägeloh, C.U., Bergomi, C., Siegert, R.J. et al. Response Shift After a Mindfulness-Based Intervention: Measurement Invariance Testing of the Comprehensive Inventory of Mindfulness Experiences. Mindfulness 9, 212–220 (2018). https://doi.org/10.1007/s12671-017-0764-4

Download citation

Keywords

  • Mindfulness
  • Comprehensive Inventory of Mindfulness Experiences (CHIME)
  • Response shift
  • Confirmatory factor analysis
  • Measurement invariance
  • Mindfulness-based interventions (MBIs)