Abstract
Measures of clinical significance offer important information about psychological interventions that cannot be garnered from tests of the statistical significance of the change from pretest to posttest. For example, post-intervention comparisons to a nonclinical group often offer valuable information about the practical value of the change that occurred. This study explored the manner in which researchers conduct clinical significance analyses in an effort to summarize the effectiveness of an intervention at the group level. The focus was on the use of the original Jacobson and Truax (Journal of Consulting and Clinical Psychology, 59, 12–19, 1991) method and the normative comparisons method due to Kendall et al. (Journal of Consulting and Clinical Psychology, 67, 285–299, 1999). The results highlight that although the Jacobson and Truax method is routinely adopted for summarizing group-level clinical significance, advanced strategies for summarizing the results are very infrequently applied. Further, the Kendall et al. method, which provides valuable and distinct information regarding how the treated group is performing relative to a normal comparison group, is rarely adopted and even when it is it is often not conducted appropriately. Recommendations are provided for conducting group-level clinical significance analyses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Atkins, D. C., Bedics, J. D., McGlinchey, J. B., & Beauchaine, T. P. (2005). Brief report: Assessing clinical significance: Does it matter which method we use? Journal of Consulting and Clinical Psychology, 73, 982–989.
Bauer, S., Lambert, M. J., & Nielsen, S. L. (2004). Clinical significance methods: A comparison of statistical techniques. Journal of Personality Assessment, 82, 60–70.
Cella, D., Bullinger, M., Scott, C., & Barofsky, I. (2002). Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Proceedings of the Mayo Clinic, 77, 384–392.
Cribbie, R. A., & Arpin-Cribbie, C. A. (2009). Evaluating clinical significance through equivalence testing: Extending the normative comparisons approach. Psychotherapy Research, 19, 677–686.
Golinski, C., & Cribbie, R. A. (2009). The expanding role of quantitative methodologists in advancing psychology. Canadian Psychologist, 50, 83–90.
Gruman, J., Cribbie, R. A., & Arpin-Cribbie, C. A. (2007). The effects of heteroscedasticity on tests of equivalence. Journal of Modern Applied Statistical Methods, 6, 133–140.
Hageman, W. J., & Arrindell, W. A. (1999). Establishing clinically significant change: Increment of precision and the distinction between individual and group level of analysis. Behaviour Research and Therapy, 37, 1169–1193.
Hsu, L. M. (1999). Caveats concerning comparisons of change rates obtained with five methods of identifying significant client changes: Comment on Speer and Greenbaum (1995). Journal of Consulting and Clinical Psychology, 67, 594–598.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.
Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336–352.
Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332–339.
Kendall, P. C., Marrs-Garcia, A., Nath, S. R., & Sheldrick, R. C. (1999). Normative comparisons for the evaluation of clinical significance. Journal of Consulting and Clinical Psychology, 67, 285–299.
Koh, A., & Cribbie, R. A. (2013). Robust tests of equivalence for k independent groups. British Journal of Mathematical and Statistical Psychology, 66, 426–434.
Maassen, G. H. (2000). Principles of defining reliable change indices. Journal of Clinical and Experimental Neuropsychology, 22, 622–632.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156–166.
Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology Review, 21, 421–446.
Rogers, J. L., Howard, K. I., & Vessey, J. T. (1993). Using significance tests to evaluate equivalence between two experimental groups. Psychological Bulletin, 113, 553–565.
Ronk, F. R., Hooke, G. R., & Page, A. C. (2012). How consistent are clinical significance classifications: When calculation methods and outcome measures differ? Clinical Psychology: Science and Practice, 19, 167–179.
Schuirmann, D. J. (1987). A comparison of the two-sided tests procedure and the power approach for assigning equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics, 15, 657–680.
Seaman, M. A., & Serlin, R. C. (1998). Equivalence confidence intervals for two-group comparisons of means. Psychological Methods, 3, 403–411.
Speer, D. C. (1992). Clinically significant change: Jacobson and Truax (1991) revisited. Journal of Consulting and Clinical Psychology, 60, 402–408.
Speer, D. C., & Greenbaum, P. E. (1995). Five methods for computing significant individual client change and improvement rates: Support for an individual growth curve approach. Journal of Consulting and Clinical Psychology, 63, 1044–1048.
van Wieringen, K., & Cribbie, R. A. (2014). Robust normative comparison tests for evaluating clinical significance. British Journal of Mathematical and Statistical Psychology, 67, 213–230.
Westlake, W. J. (1981). Response to T.B.L. Kirkwood: Bioequivalence testing — a need to rethink. Biometrics, 37, 589–594.
Wilkinson, L., & APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cribbie, R.A., Arpin-Cribbie, C.A., Vendittelli, R. et al. Group Level Clinical Significance: An Analysis of Current Practice. Curr Psychol 34, 672–680 (2015). https://doi.org/10.1007/s12144-014-9280-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12144-014-9280-z