, Volume 10, Issue 4, pp 724–736 | Cite as

The (Lack of) Replication of Self-Reported Mindfulness as a Mechanism of Change in Mindfulness-Based Relapse Prevention for Substance Use Disorders

  • Yu-Yu Hsiao
  • Davood Tofighi
  • Eric S. Kruger
  • M. Lee Van Horn
  • David P. MacKinnon
  • Katie WitkiewitzEmail author


The development and evaluation of mindfulness-based interventions for a variety of psychological and medical disorders have grown exponentially over the past 20 years. Yet, calls for increasing the rigor of mindfulness research and recognition of the difficulties of conducting research on the topic of mindfulness have also increased. One of the major difficulties is the measurement of mindfulness, with varying definitions across studies and ambiguity with respect to the meaning of mindfulness. There is also concern about the reproducibility of findings given few attempts at replication. The current secondary analysis addressed the issue of reproducibility and robustness of the construct of self-reported mindfulness across two separate randomized clinical trials of mindfulness-based relapse prevention (MBRP), as an aftercare treatment for substance use disorder. Specifically, we tested the robustness of our previously published findings, which identified a latent construct of mindfulness as a significant mediator of the effect of MBRP on reducing craving following treatment. First, we attempted to replicate the findings in a separate randomized clinical trial of MBRP. Second, we conducted sensitivity analyses to test the assumption of the no-omitted confounder bias in a mediation model. The effect of MBRP on self-reported mindfulness and overall mediation effect failed to replicate in a new sample. The effect of self-reported mindfulness in predicting craving following treatment did replicate and was robust to the no-omitted confounder bias. The results of this work shine a light on the difficulties in the measurement of mindfulness and the importance of examining the robustness of findings.


Mindfulness Craving Substance use disorder Replicability Reproducibility Mediation Sensitivity analyses Mindfulness-based relapse prevention 


Author Contributions

YYH conducted the data analyses and wrote the results. DT collaborated with the design and execution of the study and conceptualizing the analyses. ESK, MLVH, and DPM collaborated with the analyses and editing of the final manuscript. KW designed and executed the study and wrote the introduction and discussion sections of the manuscript. All authors contributed to the final editing of the manuscript.

Funding Information

The current study was funded by NIAAA R01 AA025539 (Witkiewitz and Tofighi, MPIs) and NIDA R37DA09757 (MacKinnon, PI).

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

All procedures performed in our study were approved by the institutional review board at the University of Washington.

Informed Consent

Informed consent was obtained from all participants in the present study.

Supplementary material

12671_2018_1023_MOESM1_ESM.docx (13 kb)
ESM 1 (DOCX 12 kb)


  1. Alsubaie, M., Abbott, R., Dunn, B., Dickens, C., Keil, T. F., Henley, W., & Kuyken, W. (2017). Mechanisms of action in mindfulness-based cognitive therapy (MBCT) and mindfulness-based stress reduction (MBSR) in people with physical and/or psychological conditions: a systematic review. Clinical Psychology Review, 55, 74–91. Scholar
  2. Anheyer, D., Haller, H., Barth, J., Lauche, R., Dobos, G., & Cramer, H. (2017). Mindfulness-based stress reduction for treating low back pain. Annals of Internal Medicine, 166, 799–807. Scholar
  3. Baer, R. A., Smith, G. T., Hopkins, J., Krietemeyer, J., & Toney, L. (2006). Using self-report assessment methods to explore facets of mindfulness. Assessment, 13, 27–45. Scholar
  4. Bowen, S., Chawla, N., Collins, S. E., Witkiewitz, K., Hsu, S., Grow, J., et al. (2009). Mindfulness-based relapse prevention for substance use disorders: a pilot efficacy trial. Substance Abuse, 30, 295–305. Scholar
  5. Bowen, S., Chawla, N., & Marlatt, G. A. (2011a). Mindfulness-based relapse prevention for addictive behaviors: a clinician’s guide. New York, NY: Guilford Press.Google Scholar
  6. Bowen, S., Witkiewitz, K., Chawla, N., & Grow, J. (2011b). Integrating mindfulness meditation and cognitive behavioral traditions for the long-term treatment of addictive behaviors. Journal of Clinical Outcomes Management, 18, 473–479.Google Scholar
  7. Bowen, S., Witkiewitz, K., Clifasefi, S. L., Grow, J., Chawla, N., Hsu, S. H., et al. (2014). Relative efficacy of mindfulness-based relapse prevention, standard relapse prevention, and treatment as usual for substance use disorders. JAMA Psychiatry, 71, 547–556. Scholar
  8. Bravo, A. J., Pearson, M. R., Wilson, A. D., & Witkiewitz, K. (2018). When traits match states: examining the associations between self-report trait and state mindfulness following a state mindfulness induction. Mindfulness, 9, 199–211. Scholar
  9. Brown, T.A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York: Guilford Press.Google Scholar
  10. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Beverly Hills, CA: Sage.Google Scholar
  11. Carrière, K., Khoury, B., Günak, M. M., & Knäuper, B. (2017). Mindfulness-based interventions for weight loss: a systematic review and meta-analysis. Obesity Reviews, 19, 164–177. Scholar
  12. Chawla, N. (2010). Experiential avoidance and substance use relapse. ProQuest Dissertation Publishing & Theses Global.Google Scholar
  13. Chawla, N., Collin, S., Bowen, S., Hsu, S., Grow, J., Douglass, A., & Marlatt, G. A. (2010). The mindfulness-based relapse prevention adherence and competence scale: development, interrater reliability, and validity. Psychotherapy Research, 20, 388–397. Scholar
  14. Chen, F. F., Sousa, K. H., & West, S. G. (2005). Teacher’s corner: testing measurement invariance of second-order factor models. Structural Equation Modeling: A Multidisciplinary Journal, 12, 471–492. Scholar
  15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale: Erlbaum.Google Scholar
  16. Costa, J., Marôco, J., Pinto-Gouveia, J., & Galhardo, A. (2014). Validation of the psychometric properties of Acceptance and Action Questionnaire-II in clinical and nonclinical groups of Portuguese population. International Journal of Psychology and Psychological Therapy, 14, 353–364.Google Scholar
  17. Cox, M. G., Kisbu-Sakarya, Y., Miočević, M., & MacKinnon, D. P. (2013). Sensitivity plots for confounder bias in the single mediator model. Evaluation Review, 37, 405–431. Scholar
  18. Davidson, R. J., & Kaszniak, A. W. (2015). Conceptual and methodological issues in research on mindfulness and meditation. American Psychologist, 70, 581–592. Scholar
  19. Dimidjian, S., & Segal, Z. V. (2015). Prospects for a clinical science of mindfulness-based intervention. American Psychologist, 70, 593–620. Scholar
  20. Flannery, B. A., Volpicelli, J. R., & Pettinati, H. M. (1999). Psychometric properties of the Penn Alcohol Craving Scale. Alcoholism: Clinical and Experimental Research, 23, 1289–1295.CrossRefGoogle Scholar
  21. Fritz, M. S., Kenny, D. A., & MacKinnon, D. P. (2016). The combined effects of measurement error and omitting confounders in the single-mediator model. Multivariate Behavioral Research, 51, 681–697. Scholar
  22. Garland, E. L. (2016). Restructuring reward processing with mindfulness-oriented recovery enhancement: novel therapeutic mechanisms to remediate hedonic dysregulation in addiction, stress, and pain. Annals of the New York Academy of Sciences, 1373, 25–37. Scholar
  23. Gethin, R. (2011). On some definitions of mindfulness. Contemporary Buddhism, 12, 263–279.CrossRefGoogle Scholar
  24. Goodman, S. N. (1992). A comment on replication, p-values and evidence. Statistics in Medicine, 11, 875–879.CrossRefGoogle Scholar
  25. Grossman, P. (2011). Defining mindfulness by how poorly I think I pay attention during everyday awareness and other intractable problems for psychology’s (re) invention of mindfulness: comment on Brown et al. (2011). Psychological Assessment, 23, 1034-40-6.
  26. Grossman, P., & Van Dam, N. T. (2011). Mindfulness, by any other name …: trials and tribulations of sati in western psychology and science. Contemporary Buddhism, 12, 219–239. Scholar
  27. Grossman, P., Niemann, L., Schmidt, S., & Walach, H. (2004). Mindfulness-based stress reduction and health benefits: a meta-analysis. Journal of Psychosomatic Research, 57, 35–43. Scholar
  28. Gu, J., Strauss, C., Bond, R., & Cavanagh, K. (2015). How do mindfulness-based cognitive therapy and mindfulness-based stress reduction improve mental health and wellbeing? A systematic review and meta-analysis of mediation studies. Clincal Psychology Review, 37, 1–12. Scholar
  29. Gu, J., Strauss, C., Crane, C., Barnhofer, T., Karl, A., Cavanagh, K., & Kuyken, W. (2016). Examining the factor structure of the 39-item and 15-item versions of the Five Facet Mindfulness Questionnaire before and after mindfulness-based cognitive therapy for people with recurrent depression. Psychological Assessment, 28, 791–802. Scholar
  30. Hayes, S. C., Strosahl, K., Wilson, K., Bissett, R., Pistorello, J., Toarmino, D., et al. (2004). Measuring experiential avoidance: a preliminary test of a working model. The Psychological Record, 54, 553–578.CrossRefGoogle Scholar
  31. Holland, P.W. (1988). Causal inference, path analysis, and recursive structural equations models. Sociological Methodology, 18, 449–484. doi:
  32. Hölzel, B. K., Lazar, S. W., Gard, T., Schuman-Olivier, Z., Vago, D. R., & Ott, U. (2011). How does mindfulness meditation work? Proposing mechanisms of action from a conceptual and neural perspective. Perspective on Psychological Science: A Journal of the Association for Psychological Science, 6, 537–559.CrossRefGoogle Scholar
  33. Hopwood, T. L., & Schutte, N. S. (2017). A meta-analytic investigation of the impact of mindfulness-based interventions on post traumatic stress. Clinical Psychology Review, 57, 12–20. Scholar
  34. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. Scholar
  35. Hunter-Reel, D., McCrady, B., & Hildebrandt, T. (2009). Emphasizing interpersonal factors: an extension of the Witkiewitz and Marlatt relapse model. Addiction, 104, 1281–1290. Scholar
  36. Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309–334.CrossRefGoogle Scholar
  37. Judd, C. M., & Kenny, D. A. (1981). Process analysis: estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619. Scholar
  38. Kabat-Zinn, J. (2011). Some reflections on the origins of MBSR, skillful means, and the trouble with maps. Contemporary Buddhism, 12, 281–306. Scholar
  39. Kabat-Zinn, J. (2013). Full catastrophe living: using the wisdom of the body and mind to face stress, pain, and illness. New York: Bantam.Google Scholar
  40. Kabat-Zinn, J., Lipworth, L., & Burney, R. (1985). The clinical use of mindfulness meditation for the self-regulation of chronic pain. Journal of Behavioral Medicine, 8, 163–190.CrossRefGoogle Scholar
  41. Kazdin, A. E., & Nock, M. K. (2003). Delineating mechanisms of change in child and adolescent therapy: methodological issues and research recommendations. Journal of Child Psychology and Psychiatry, 44, 1116–1129. Scholar
  42. Khoury, B., Lecomte, T., Fortin, G., Masse, M., Therien, P., Bouchard, V., et al. (2013). Mindfulness-based therapy: a comprehensive meta-analysis. Clinical Psychology Review, 33, 763–771. Scholar
  43. Kline, R. B. (2016). Principle and practice of structural equation modeling (4th ed.). New York, NY, US: Guilford Press.Google Scholar
  44. Kuyken, W., Warren, F. C., Taylor, R. S., Whalley, B., Crane, C., Bondolfi, G., et al. (2016). Efficacy of mindfulness-based cognitive therapy in prevention of depressive relapse. JAMA Psychiatry, 73, 565. Scholar
  45. Langer, E. F. (1989). Mindfulness. Cambridge: De Capo Press.Google Scholar
  46. Levinson, D. B., Stoll, E. L., Kindy, S. D., Merry, H. L., & Davidson, R. J. (2014). A mind you can count on: validating breath counting as a behavioral measure of mindfulness. Frontiers in Psychology, 5, 1202. Scholar
  47. Li, W., Howard, M. O., Garland, E. L., McGovern, P., & Lazar, M. (2017). Mindfulness treatment for substance misuse: a systematic review and meta-analysis. Journal of Substance Abuse Treatment, 75, 62–96. Scholar
  48. Lilienfeld, S. O. (2017). Psychology’s replication crisis and the grant culture: righting the ship. Perspectives on Psychological Science: A journal of the Association for Psychological Science, 12, 660–664. Scholar
  49. Louise, S., Fitzpatrick, M., Strauss, C., Rossell, S. L., & Thomas, N. (2017). Mindfulness- and acceptance-based interventions for psychosis: our current understanding and a meta-analysis. Schizophrenia Research, 192, 57–63. Scholar
  50. Lutz, A., Jha, A. P., Dunne, J. D., & Saron, C. D. (2015). Investigating the phenomenological matrix of mindfulness-related practices from a neurocognitive perspective. American Psychologist, 70, 632–658. Scholar
  51. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor & Francis Group.Google Scholar
  52. MacKinnon, D. P., & Pirlott, A. G. (2014). Statistical approaches for enhancing causal interpretation of the M to Y relation in mediation analysis. Personality and Social Psychology Review, 19, 30–43. Scholar
  53. MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83–104.CrossRefGoogle Scholar
  54. Manuel, J. A., Somohano, V. C., & Bowen, S. (2017). Mindfulness practice and its relationship to the Five-Facet Mindfulness Questionnaire. Mindfulness, 8, 361–367. Scholar
  55. Marsh, H. W., Wen, Z., & Hau, K.-T. (2004). Structural equation models of latent interactions: evaluation of alternative estimation strategies and indicator construction. Psychological Methods, 9, 275–300. Scholar
  56. McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64–82.CrossRefGoogle Scholar
  57. Medvedev, O. N., Siegert, R. J., Kersten, P., & Krägeloh, C. U. (2017). Improving the precision of the Five Facet Mindfulness Questionnaire using a Rasch approach. Mindfulness.
  58. Millsap, R. E., & Kwok, O.-M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93–115. Scholar
  59. Muthén, L. K., & Muthén, B. O. (1998-2017). Mplus user's guide (8th ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
  60. Ong, J. C., & Sholtes, D. (2010). A mindfulness-based approach to the treatment of insomnia. Journal of Clinical Psychology, 66, 1175–1184. Scholar
  61. Paterson, J., Medvedev, O. N., Sumich, A., Tautolo, E., Krägeloh, S. U., Sisk, R., McNamara, R., Berk, M., Narayanan, A., & Siegert, R. J. (2017). Distinguishing transient versus stable aspects of depression in New Zealand Pacific Island children using generalizability theory. Journal of Affective Disorders, 227, 698–704.CrossRefGoogle Scholar
  62. Piet, J., Würtzen, H., & Zachariae, R. (2012). The effect of mindfulness-based therapy on symptoms of anxiety and depression in adult cancer patients and survivors: a systematic review and meta-analysis. Journal of Consulting and Clinical Psychology, 80, 1007–1020. Scholar
  63. Segal, Z. V., Williams, J. M. G., & Teasdale, J. D. (2002). Mindfulness-based cognitive therapy for depression: a new approach to preventing relapse. New York, NY, US: Guilford Press.Google Scholar
  64. Tang, Y.-Y., & Leve, L. D. (2016). A translational neuroscience perspective on mindfulness meditation as a prevention strategy. Translational Behavioral Medicine, 6, 63–72. Scholar
  65. Tapper, K. (2017). Mindfulness and craving: effects and mechanisms. Clinical Psychology Review, 59, 101–117. Scholar
  66. Taylor, D. J., & Muller, K. E. (1996). Bias in linear model power and sample size calculation due to estimating noncentrality. Communications in Statistics: Theory and Methods, 25, 1595–1610. Scholar
  67. Tofighi, D., & Kelley, K. (2016). Assessing omitted confounder bias in multilevel mediation models. Multivariate Behavioral Research, 51, 86–105. Scholar
  68. Tofighi, D., & MacKinnon, D. P. (2011). RMediation: an R package for mediation analysis confidence intervals. Behavioral Research Methods, 43, 692–700. Scholar
  69. Tofighi, D., Hsiao, Y. Y., Kruger, E. S., MacKinnon, D. P., Van Horn, M. L., & Witkiewitz, K. (in press). Sensitivity analysis in latent growth curve mediation models. Structural Equation Modeling: A Multidisciplinary Journal.Google Scholar
  70. Valente, M. J., Pelham, W. E., Smyth, H., & MacKinnon, D. P. (2017). Confounding in statistical mediation analysis: what it is and how to address it. Journal of Counseling Psychology, 64, 659–671. Scholar
  71. Valeri, L., & Vanderweele, T. J. (2013). Mediation analysis allowing for exposure-mediator interactions and causal interpretation: theoretical assumptions and implementation with SAS and SPSS macros. Psychological Methods, 18, 137–150. Scholar
  72. Van Dam, N. T., Earleywine, M., & Danoff-Burg, S. (2009). Differential item function across meditators and non-meditators on the Five Facet Mindfulness Questionnaire. Personality and Individual Differences, 47, 516–521. Scholar
  73. Van Dam, N.T., van Vugt, M.K., Vago, D.R., Schmalzl, L., Saron, C.D., Olendzki, A., … Meyer, D.E. (2018). Mind the hype: a critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 13, 36–61.
  74. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. Scholar
  75. van der Velden, A. M., Kuyken, W., Wattar, U., Crane, C., Pallesen, K. J., Dahlgaard, J., et al. (2015). A systematic review of mechanisms of change in mindfulness-based cognitive therapy in the treatment of recurrent major depressive disorder. Clinical Psychological Review, 37, 26–39. Scholar
  76. VanderWeele, T. J. (2010). Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology, 21, 540–551. Scholar
  77. Witkiewitz, K., & Bowen, S. (2010). Depression, craving, and substance use following a randomized trial of mindfulness-based relapse prevention. Journal of Consulting and Clinical Psychology, 78, 362–374. Scholar
  78. Witkiewitz, K., Bowen, S., Douglas, H., & Hsu, S. H. (2013). Mindfulness-based relapse prevention for substance craving. Addictive Behaviors, 38, 1563–1571. Scholar
  79. Witkiewitz, K., Bowen, S., Harrop, E. N., Douglas, H., Enkema, M., & Sedgwick, C. (2014a). Mindfulness-based treatment to prevent addictive behavior relapse: theoretical models and hypothesized mechanisms of change. Substance Use & Misuse, 49, 513–524. Scholar
  80. Witkiewitz, K., Warner, K., Sully, B., Barricks, A., Stauffer, C., Thompson, B. L., & Luoma, J. B. (2014b). Randomized trial comparing mindfulness-based relapse prevention with relapse prevention for women offenders at a residential addiction treatment center. Substance Use & Misuse, 49, 536–546. Scholar
  81. Young, K. S., van der Velden, A. M., Craske, M. G., Pallesen, K. J., Fjorback, L., Roepstorff, A., & Parsons, C. E. (2018). The impact of mindfulness-based interventions on brain activity: a systematic review of functional magnetic resonance imaging studies. Neuroscience and Biobehavioral Reviews, 84, 424–433. Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of New MexicoAlbuquerqueUSA
  2. 2.Department of Individual, Family and Community EducationUniversity of New MexicoAlbuquerqueUSA
  3. 3.Department of PsychologyArizona State UniversityTempeUSA

Personalised recommendations