Abstract
Two recent meta-analyses have generated evidence for child and adolescent psychotherapy effects. However, critics note that such meta-analyses often include studies with methodological shortcomings which might invalidate their results. In the present study, we explored whether the results of the most extensive child/adolescent meta-analysis might have been influenced by such methodological variables, focusing on internal validity and external validity factors. Together, these factors accounted for two-thirds as much variance as the substantive factors (e.g., type of therapy, age) in the original metaanalysis. This suggests that relative to these therapy and child-characteristic variables, methodological factors have a substantial, though smaller, impact on metaanalysis results. In general, increased experimental rigor was related to larger effect sizes; this argues against the hypothesis that methodologically weak studies have led to an overestimate of therapy effects. No significant interactive relations were found between validity factors and predictors of outcome; this suggests that the relations noted in previous metaanalyses between outcome and various variables were notdistorted by the validity factors tested here.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Appelbaum, M. I., & Cramer, E. M. (1974). Some problems in the nonorthogonal analysis of variance.Psychological Bulletin, 81, 335–343.
Campbell, D. T., & Stanley, J. C. (1963).Experimental and quasiexperimental designs for research. Boston: Houghton-Mifflin.
Casey, R. J., & Berman, J. S. (1985). The outcome of psychotherapy with children.Psychological Bulletin, 98, 388–400.
Cohen, L. H. (1977).Statistical power analysis for the behavioral sciences (rev. ed.). New York: Academic Press.
Fleiss, J. L. (1981).Statistical methods for rates and proportions (2nd. ed.). New York: Wiley.
Gould, M. S., Shaffer, D., & Kaplan, D. (1985). The characteristics of dropouts from a child psychiatry clinic.Journal of the American Academy of Child Psychiatry, 24, 316–328.
Hays, W. L. (1981).Statistics (3rd ed.). New York: Holt, Rinehart, & Winston.
Hedges, L. V. (1982). Estimation of effect size from a series of independent estimates.Psychological Bulletin, 92, 490–499.
Howell, D. C. (1982).Statistical methods for psychology. Boston: Duxbury Press.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982).Meta-analysis: Cumulating research findings across studies. Beverly Hills: Sage Publications.
Kazdin, A. E. (1978a). Evaluating the generality of findings in analogue therapy research.Journal of Consulting and Clinical Psychology, 46, 673–686.
Kazdin, A. E. (1978b).Research design in clinical psychology. New York: Harper & Row.
Kraemer, H. C., & Andrews, G. (1982). A nonparametric technique for meta-analysis effect size calculation.Psychological Bulletin, 91, 404–412.
Lambert, J. M., Shapiro, D. A., & Bergin, A. E. (1986). The effectiveness of psychotherapy. In S. L. Garfleld & A. E. Bergin (Eds.),Handbook of psychotherapy and behavior change (3rd ed.). New York: Wiley.
McAdoo, W. G., & Roeske, N. A. (1973). A comparison of defectors and continuers in a child guidance clinic.Journal of Consulting and Clinical Psychology, 40, 328–334.
Mintz, J. (1983). Integrating research evidence: A commentary on meta-analysis.Journal of Consulting and Clinical Psychology, 46, 71–75.
Rachman, S., & Wilson, G. T. (1980).The effects of psychological therapy. Oxford: Pergamon Press.
Shapiro, D. A. (1985). Recent applications of meta-analysis in clinical research.Clinical Psychology Review, 5, 13–34.
Shapiro, D. A., & Shapiro, D. (1982). Meta-analysis of comparative therapy outcome studies: A replication and refinement.Psychological Bulletin, 92, 581–604.
Shapiro, D. A., & Shapiro, D. (1983). Comparative therapy outcome research: Methodological implications of meta-analysis.Journal of Consulting and Clinical Psychology, 51, 42–53.
Smith, M. L., Glass, G. V., & Miller, T. I. (1980).The benefits of psychotherapy. Baltimore: Johns Hopkins University Press.
Strube, M. J., & Hartmann, D. P. (1983). Meta-analysis: Techniques, applications, and functions.Journal of Consulting and Clinical Psychology, 51, 14–27.
Weisz, J. R., & Weiss, B. (1989a). Assessing the effects of clinic-based psychotherapy with children and adolescents.Journal of Consulting and Clinical Psychology, 57, 741–746.
Weisz, J. R., & Weiss, B. (1989b). Cognitive mediators of the outcome of psychotherapy with children. In B. B. Lahey & A. E. Kazdin (Eds.),Advances in clinical child psychology (Vol. 12, pp. 27–51). New York: Academic Press.
Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987a). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians.Journal of Consulting and Clinical Psychology, 55, 542–549.
Weisz, J. R., Weiss, B., & Langmeyer, D. (1987b). Giving up on child psychotherapy: Who drops out?Journal of Consulting and Clinical Psychology, 55, 916–918.
Wilson, G. T., & Rachman, S. J. (1983). Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities.Journal of Consulting and Clinical Psychology, 51, 54–64.
Author information
Authors and Affiliations
Additional information
The research described here was supported in part by grants from the North Carolina Department of Human Resources, Division of Mental Health, Mental Retardation, and Substance Abuse Services (41626), and from the National Institute of Mental Health (1 RO3 MH38450). The authors gratefully acknowledge the assistance of Todd Morton, who provided reliability coding.
Rights and permissions
About this article
Cite this article
Weiss, B., Weisz, J.R. The impact of methodological factors on child psychotherapy outcome research: A meta-analysis for researchers. J Abnorm Child Psychol 18, 639–670 (1990). https://doi.org/10.1007/BF01342752
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01342752