What Works Clearinghouse Standards and Generalization of Single-Case Design Evidence
- 666 Downloads
A recent review of existing rubrics designed to help researchers evaluate the internal and external validity of single-case design (SCD) studies found that the various options yield consistent results when examining causal arguments. The authors of the review, however, noted considerable differences across the rubrics when addressing the generalization of findings. One critical finding is that the What Works Clearinghouse (WWC) review process does not capture details needed for report readers to evaluate generalization. This conclusion is reasonable if considering only the WWC’s SCD design standards. It is important to note that these standards are not used in isolation, and thus generalization details cannot be fully understood without also considering the review protocols and a tool called the WWC SCD review guide. Our purpose in this commentary is to clarify how the WWC review procedures gather information on generalization criteria and to describe a threshold for judging how much evidence is available. It is important to clarify how the system works so that the SCD research community understands the standards, which in turn might facilitate use of future WWC reports and possibly influence both the conduct and the reporting of SCD studies.
KeywordsSingle-case design Generalization Internal validity External validity
- Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change (2nd ed.). Boston, MA: Pearson.Google Scholar
- Bowman-Perrott, L., Davis, H., Vannest, K. J., Williams, L., Greenwood, C. R., & Parker, R. (2013). Academic benefits of peer tutoring: A meta-analytic review of single-case research. School Psychology Review, 42(1), 39–59.Google Scholar
- Hitchcock, J. H., Horner, R. H., Kratochwill, T. R., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2014). The What Works Clearinghouse single-case design pilot standards: Who will guard the guards? Remedial and Special Education Advance online publication. doi:10.1177/0741932513518979. contributors are listed by alphabetical order.Google Scholar
- Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.Google Scholar
- Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D., & Shadish, W. R. M. (2010). Single case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
- Kratochwill, T. R., & Levin, J. R. (Eds.). (2014). Single-case intervention research: Methodological and statistical advances. Washington, DC: American Psychological Association.Google Scholar
- Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W.H., & Shavelson, R.J. (2007). Estimating casual effects using experimental and nonexperimental designs (report from the Governing Board of the American Educational Research Association Grants Program). Washington, DC: American Educational Research Association.Google Scholar
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.Google Scholar
- What Works Clearinghouse. (2013). Procedures and standards handbook (Version 3.0). Retrieved from http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=19.
- What Works Clearinghouse (2014) WWC intervention report: Repeated Reading. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_repeatedreading_051314.pdf.