Abstract
Evidence-based practice depends in part on knowledge derived from relevant research. For any given topic, there are likely to be many, potentially relevant studies; a careful appraisal and synthesis of the results of these studies is needed to understand the state of the empirical evidence. Meta-analysis is widely used to combine results of quantitative studies; yet this method is unfamiliar to many people and, as a result, meta-analyses are often uncritically accepted. In this article, we argue that meta-analysis is only one component of a good research synthesis. We critique a recent metaanalysis on the effectiveness of long-term psychodynamic psychotherapy, showing that this metaanalysis failed to meet current standards for the conduct and reporting of systematic research reviews and meta-analyses. We demonstrate the use of AMSTAR, a straightforward tool for assessing the quality of systematic reviews and meta-analyses.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beck, A. T., & Bhar, S. (2009). Analyzing the effectiveness of long-term psychodynamic psychotherapy. JAMA, 301, 931.
Becker, B. J. (2005). Failsafe N or file-drawer number. In H. R. Rothstein, A. J. Sutton, & M. Bornstein (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustment (pp. 111–125). West Sussex, England: Wiley.
Bhar, S. S., Thombs, B. D., Pignotti, M., Bassel, M., Jewett, L., Coyne, J. C., et al. (2010). Is longer-term psychodynamic psychotherapy more effective than shorter-term therapies? Review and critique of the evidence. Psychotherapy and Psychosomatics, 79, 208–216.
Bushman, B. J., & Wells, G. L. (2001). Narrative impressions of literature: The availability bias and the corrective properties of meta-analytic approaches. Personal and Social Psychology Bulletin, 27, 1123–1130.
Chalmers, I., Enkin, M., & Keirse, M. J. N. C. (1993). Preparing and updating systematic reviews of randomized controlled trials of health care. Milbank Quarterly, 71, 411–437.
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8.
Glass, R. M. (2008). Psychodynamic psychotherapy and research evidence: Bambi survives Godzilla? JAMA, 300, 1587–1589.
Higgins, J. P. T., & Green, S. (Eds.). (2008). Cochrane handbook for systematic reviews of interventions. Chichester, UK: Wiley. Available at: http://www.cochrane.dk/cochrane/handbook/hbook.htm.
Hopewell, S., McDonald, S., Clarke, M., & Egger, M. (2007). Grey literature in meta-analyses of randomized trials of health care interventions. The Cochrane Database of Systematic Reviews, 2007(Issue 2), Art. No.: MR000010. doi:10.1002/14651858.MR000010.pub3.
Jüni, P., Altman, D. G., & Egger, M. (2001). Assessing the quality of controlled clinical trials. British Medical Journal, 323(7303), 42–46.
Kriston, L., Holzel, L., & Harter, M. (2009). Analyzing the effectiveness of long-term psychodynamic psychotherapy. JAMA, 301, 930–931.
Leichsenring, F., & Rabung, S. (2008). The effectiveness of psychodynamic psychotherapy: A meta-analysis. JAMA, 300, 1551–1565.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage Publications.
Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta-analysis. New York: Oxford University Press.
Moher, D., Cook, D. J., Eastwood, S., Olkin, I., Rennie, D., Stroup, D. F., et al. (1999). Improving the quality of reports of meta-analyses of randomised controlled trials: The QUOROM statement. The Lancet, 354, 1896–1900.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6, e1000097. doi:10.1371/journal.pmed.1000097.
Roepke, S. (2009). Analyzing the effectiveness of long-term psychodynamic psychotherapy. JAMA, 301, 931–932.
Rothstein, H., Sutton, A. J., & Bornstein, M. (2005). Publication bias in meta-analysis: Prevention, assessment, and adjustments. Chichester, UK: Wiley.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., et al. (2007). Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7. Available at: http://www.biomedcentral.com/1471-2288/7/10.
Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, G. D., Rennie, D., Moher, D., et al. (2000). Meta-analysis of observational studies in epidemiology: A proposal for reporting. JAMA, 283, 2008–2012.
Thombs, B. D., Bassel, M., & Jewett, L. R. (2009). Analyzing the effectiveness of long-term psychodynamic psychotherapy. JAMA, 301, 930.
Wells, K., & Littell, J. H. (2009). Study quality assessment in systematic reviews of research on intervention effects. Research on Social Work Practice, 19, 52–62.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Littell, J.H., Shlonsky, A. Making Sense of Meta-Analysis: A Critique of “Effectiveness of Long-Term Psychodynamic Psychotherapy”. Clin Soc Work J 39, 340–346 (2011). https://doi.org/10.1007/s10615-010-0308-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10615-010-0308-z