Skip to main content
SpringerLink
Log in

What Can We Learn from Randomized Clinical Trials About the Construct Validity of Self-Report Measures of Mindfulness? A Meta-Analysis

  • REVIEW
  • Published:
Mindfulness Aims and scope Submit manuscript

Abstract

Because they provide data on responsiveness to experimental manipulation, clinical trials involving mindfulness-based interventions are a source of evidence for the construct validity of self-report measures of mindfulness. Within-group and between-group changes in mindfulness were examined from randomized clinical trials comparing mindfulness interventions to other bona fide treatment comparison conditions or waitlist control conditions. We also examined changes in clinical outcomes and the magnitude of these changes relative to changes in mindfulness. We included 69 published studies representing 55 unique samples (n = 4743). Self-report mindfulness measures showed relatively larger gains in mindfulness intervention conditions vis-à-vis waitlist comparison conditions at both post-treatment (effect size [ES] = 0.52, 95% CI [0.40, 0.64]) and follow-up (ES = 0.52 [0.20, 0.84]), although the effect at follow-up diminished to non-significance in a trim-and-fill analysis intended to account for publication bias (ES = 0.35 [− 0.03, 0.72]). Measures of mindfulness also showed relatively larger gains in mindfulness intervention conditions vis-à-vis bona fide comparison conditions, but only at post-treatment (ES = 0.25 [0.11, 0.38], 0.10 [− 0.08, 0.28], at post-treatment and follow-up, respectively). All three conditions (mindfulness, bona fide, waitlist) showed relatively larger improvements on measures of clinical outcomes than measures of mindfulness, with the exception of waitlist conditions for which this effect was no longer significant at follow-up. Taken together, findings provide partial support for the unique responsiveness of mindfulness self-report measures to interventions that include promotion of mindfulness meditation practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Agency for Healthcare Research and Quality. (2014). Methods guide for effectiveness and comparative effectiveness reviews. Rockville, MD: Agency for Healthcare Research and Quality.

    Google Scholar 

  • American Psychological Association. (2017). Research-supported psychological treatments. Retrieved from: https://www.div12.org/psychological-treatments/

  • Atkins, D., Eccles, M., Flottorp, S., Guyatt, G. H., Henry, D., Hill, S., et al. (2004). Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches the GRADE Working Group. BMC Health Services Research, 4(38). https://doi.org/10.1186/1472-6963-4-38.

  • Baer, R. A., Smith, G. T., & Allen, K. B. (2004). Assessment of mindfulness by self-report: the Kentucky Inventory of Mindfulness Skills. Assessment, 11, 191–206.

    Article  PubMed  Google Scholar 

  • Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13, 27–45.

    Article  Google Scholar 

  • Baer, R., Smith, G., Lykins, E., Button, D., Krietemeyer, J., Sauer, S., et al. (2008). Construct validity of the Five Facet Mindfulness Questionnaire in meditating and nonmeditating samples. Assessment, 15, 329–342. https://doi.org/10.1177/1073191107313003.

  • Barnett, A. G., Van Der Pols, J. C., & Dobson, A. J. (2004). Regression to the mean: what it is and how to deal with it. International Journal of Epidemiology, 34(1), 215–220.

    Article  PubMed  Google Scholar 

  • Baron, R. & Kenny, D. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182.

  • Baskin, T. W., Tierney, S. C., Minami, T., & Wampold, B. E. (2003). Establishing specific in psychotherapy: a meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology, 71(6), 973–979. https://doi.org/10.1037/0022-006X.71.6.973.

    Article  PubMed  Google Scholar 

  • Becker, B. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical and Statistical Psychology, 41, 257–278.

    Article  Google Scholar 

  • Black, D. S. (2012). Mindfulness research guide: a new paradigm for managing empirical health information. Mindfulness, 1(3), 174–176. https://doi.org/10.1007/s12671-010-0019-0.

    Article  Google Scholar 

  • Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. New York: Wiley.

    Book  Google Scholar 

  • Brown, K., & Ryan, R. (2003). The benefits of being present: mindfulness and its role in psychological well-being. Journal of Personality and Social Psychology, 84(4), 822–848.

    Article  Google Scholar 

  • Brown, K. W., Weinstein, N., & Creswell, J. D. (2013). Trait mindfulness modulates neuroendocrine and affective responses to social evaluative threat. Psychoneuroendocrinology, 37, 2037–2041. https://doi.org/10.1016/j.psyneuen.2012.04.003.

    Article  Google Scholar 

  • Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.

    Article  Google Scholar 

  • Creswell, J. D., Way, B. M., Eisenberger, N. I., & Lieberman, M. D. (2007). Neural correlates of dispositional mindfulness during affect labeling. Psychosomatic Medicine, 69, 560–565.

    Article  PubMed  Google Scholar 

  • Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Mason, Ohio: Cengage Learning.

    Google Scholar 

  • Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302.

    Article  Google Scholar 

  • Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A., ... & Meyer, D. E. (2018). Mind the hype: a critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science, 13(1), 36–61. https://doi.org/10.1177/1745691617709589.

  • Davidson, R. J., & Kaszniak, A. W. (2015). Conceptual and methodological issues in research on mindfulness and meditation. American Psychologist, 70(7), 581–592. https://doi.org/10.1037/a0039512.

    Article  PubMed  Google Scholar 

  • DeCoster, J., Sparks, E. A., Sparks, J. C., Sparks, G. G., & Sparks, C. W. (2015). Opportunistic biases: their origins, effects, and an integrated solution. American Psychologist, 70(6), 499–514. https://doi.org/10.1037/a0039191.

    Article  PubMed  Google Scholar 

  • Del Re, A. C., Hoyt, W. T. (2010). MAd: meta-analysis with mean differences. R package version 0.8, http://CRAN.R-project.org/package=MAd

  • Garland, E. L., Boettiger, C. A., Gaylord, S., Chanon, V. W., & Howard, M. O. (2011). Mindfulness is inversely associated with alcohol attentional bias among recovering alcohol-dependent adults. Cognitive Therapy and Research, 36, 441–450.

    Article  PubMed  PubMed Central  Google Scholar 

  • Giluk, T. L. (2009). Mindfulness, big five personality, and affect: a meta-analysis. Personality and Individual Differences, 47, 805–811.

    Article  Google Scholar 

  • Goldberg, S. B., Davis, J. M., & Hoyt, W. T. (2013). The role of therapeutic alliance in mindfulness interventions: therapeutic alliance in mindfulness training for smokers. Journal of Clinical Psychology, 69(9), 936–950. https://doi.org/10.1002/jclp.21973.

    Article  PubMed  Google Scholar 

  • Goldberg, S. B., Wielgosz, J., Dahl, C., Schuyler, B., MacCoon, D. S., Rosenkranz, M., et al. (2016). Does the Five Facet Mindfulness Questionnaire measure what we think it does? Construct validity evidence from an active controlled randomized clinical trial. Psychological Assessment, 28(8), 1009–1014. https://doi.org/10.1037/pas0000233.

  • Goldberg, S. B., Tucker, R. P., Greene, P. A., Simpson, T. L., Kearney, D. J., & Davidson, R. J. (2017). Is mindfulness research methodology improving over time? A systematic review. PLoS One, 12(10), e0187298. https://doi.org/10.1371/journal.pone.0187298.

  • Goldberg, S. B., Tucker, R. P., Greene, P. A., Davidson, R. J., Wampold, B. E., Kearney, D. J., & Simpson, T. L. (2018). Mindfulness-based interventions for psychiatric disorders: a meta-analysis. Clinical Psychology Review, 59, 52–60. https://doi.org/10.1016/j.cpr.2017.10.011.

    Article  PubMed  Google Scholar 

  • Goyal, M., Singh, S., Sibinga, E. M., Gould, N. F., Rowland-Seymour, A., Sharma, R., et al. (2014). Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Internal Medicine, 174(3), 357–368. https://doi.org/10.1001/jamainternmed.2013.13018.

  • Grossman, P. (2008). On measuring mindfulness in psychosomatic and psychological research. Journal of Psychosomatic Research, 64, 405–408.

    Article  PubMed  Google Scholar 

  • Gu, J., Strauss, C., Bond, R., & Cavanagh, K. (2015). How do mindfulness-based cognitive therapy and mindfulness-based stress reduction improve mental health and wellbeing? A systematic review and meta-analysis of mediation studies. Clinical Psychology Review, 37, 1–12. https://doi.org/10.1016/j.cpr.2015.01.006.

    Article  PubMed  Google Scholar 

  • Hayes, S. C., Strosahl, K., & Wilson, K. (1999). Acceptance and commitment therapy: an experiential approach to behavior change. New York: Guilford Press.

    Google Scholar 

  • Hoyt, W. T., & Del Re, A. C. (2018). Effect size calculation in meta-analyses of psychotherapy outcome research. Psychotherapy Research, 28, 379–388. https://doi.org/10.1080/10503307.2017.1405171.

    Article  PubMed  Google Scholar 

  • Hoyt, W., Warbasse, R., & Chu, E. (2006). Construct validation in counseling psychology research. The Counseling Psychologist, 34(6), 769–805.

    Article  Google Scholar 

  • Jadad, A. R., Moore, A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., et al. (1996). Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Controlled Clinical Trials, 17, 1–12.

    Article  PubMed  Google Scholar 

  • Kabat-Zinn, J. (1990). Full catastrophe living: using the wisdom of your body and mind to face stress, pain, and illness. New York: Delta.

    Google Scholar 

  • Kenny, D. A., Kashy, D. A., & Bolger, N. (1998). Data analysis in social psychology. In D. T. Gilbert, S. T. Fiske, & G. Lindzey (Eds.), The handbook of social psychology (4th ed., pp. 233–265). New York: Oxford University Press.

  • Levinson, D. B., Stoll, E. L., Kindy, S. D., Merry, H. L., & Davidson, R. J. (2014). A mind you can count on: validating breath counting as a behavioral measure of mindfulness. Frontiers in Psychology, 5(1202), 1–10. https://doi.org/10.3389/fpsyg.2014.01202.

    Article  Google Scholar 

  • Liebherz, S., Schmidt, N., & Rabung, S. (2016). How to assess the quality of psychotherapy outcome studies: a systematic review of quality assessment criteria. Psychotherapy Research, 26(5), 573–589. https://doi.org/10.1080/10503307.2015.1044763.

    Article  PubMed  Google Scholar 

  • Linehan, M. M. (1993). Cognitive-behavioral treatment of borderline personality disorder. New York: Guilford Press.

    Google Scholar 

  • Lutz, A., Jha, A. P., Dunne, J. D., & Saron, C. D. (2015). Investigating the phenomenological matrix of mindfulness-related practices from a neurocognitive perspective. American Psychologist, 70(7), 632–658. https://doi.org/10.1037/a0039585.

    Article  PubMed  Google Scholar 

  • MacCoon, D. G., Imel, Z. E., Rosenkranz, M. A., Sheftel, J. G., Weng, H. Y., Sullivan, J. C., et al. (2012). The validation of an active control intervention for Mindfulness Based Stress Reduction (MBSR). Behaviour Research and Therapy, 50, 3–12.

  • MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. Mahwah, NJ: Erlbaum.

  • Quaglia, J. T., Braun, S. E., Freeman, S. P., McDaniel, M. A., & Brown, K. W. (2016). Meta-analytic evidence for effects of mindfulness training on dimensions of self-reported dispositional mindfulness. Psychological Assessment, 28(7), 803–818. https://doi.org/10.1037/pas0000268.

    Article  PubMed  Google Scholar 

  • Segal, Z., Williams, J. W., & Teasdale, J. (2002). Mindfulness-based cognitive therapy for depression: a new approach to preventing relapse. New York: Guilford Press.

    Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

    Google Scholar 

  • Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: new procedures and recommendations. Psychological Methods, 7(4), 422–445.

    Article  PubMed  Google Scholar 

  • Tracey, T. J. G. (2016). A note on socially desirable responding. Journal of Counseling Psychology, 63(2), 224–232. https://doi.org/10.1037/cou0000135.

    Article  PubMed  Google Scholar 

  • Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–49.

    Article  Google Scholar 

  • Wampold, B., & Imel, Z. E. (2015). The great psychotherapy debate: the evidence for what makes psychotherapy work (2nd ed.). New York: Routledge.

    Book  Google Scholar 

  • Zoogman, S., Goldberg, S. B., Hoyt, W. T., & Miller, L. (2015). Mindfulness interventions with youth: a meta-analysis. Mindfulness, 6, 290–302. https://doi.org/10.1007/s12671-013-0260-4.

    Article  Google Scholar 

Download references

Funding

This work was funded by the National Center for Complementary and Alternative Medicine (Grant P01AT004952) and the Mind & Life Institute (Francisco J. Varela Award). NCCAM and Mind and Life Institute were not directly involved in study design; collection, analysis, or interpretation of data; in writing of the report; or in the decision to submit the article for publication. Any views, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect those of the Mind & Life Institute.

Author information

Authors and Affiliations

  1. Department of Counseling Psychology, University of Wisconsin – Madison, 335 Education Building, 1000 Bascom Mall, Madison, WI, 53706, USA

    Simon B. Goldberg & William T. Hoyt

  2. Center for Healthy Minds, University of Wisconsin – Madison, Madison, WI, USA

    Simon B. Goldberg & Richard J. Davidson

  3. Department of Psychology, Louisiana State University, Baton Rouge, LA, USA

    Raymond P. Tucker

  4. VA Puget Sound Health Care System, Seattle, WA, USA

    Preston A. Greene, Tracy L. Simpson & David J. Kearney

  5. Center for Excellence in Substance Abuse Treatment and Education, VA Puget Sound Health Care System, Seattle, WA, USA

    Tracy L. Simpson

  6. Department of Psychology, University of Wisconsin – Madison, Madison, WI, USA

    Richard J. Davidson

Authors
  1. Simon B. Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond P. Tucker
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Preston A. Greene
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tracy L. Simpson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. William T. Hoyt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David J. Kearney
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Richard J. Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Simon B. Goldberg.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interest

RD is the founder, president, and serves on the board of directors for the non-profit organization, Healthy Minds Innovations, Inc. In addition, RD serves on the board of directors for the Mind and Life Institute. The remaining authors (SG, RT, PG, TS, WH, DK) declare that they have no conflict of interest.

Electronic supplementary material

ESM 1

(DOCX 518 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goldberg, S.B., Tucker, R.P., Greene, P.A. et al. What Can We Learn from Randomized Clinical Trials About the Construct Validity of Self-Report Measures of Mindfulness? A Meta-Analysis. Mindfulness 10, 775–785 (2019). https://doi.org/10.1007/s12671-018-1032-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12671-018-1032-y

Keywords

Search

Navigation