When is the Story in the Subgroups?
- 476 Downloads
This paper examines strategies for interpreting and reporting estimates of intervention effects for subgroups of a study sample. The paper considers: why and how subgroup findings are important for applied research, alternative ways to define subgroups, different research questions that motivate subgroup analyses, the importance of pre-specifying subgroups before analyses are conducted, the importance of using existing theory and prior research to distinguish between subgroups for whom study findings are confirmatory (hypothesis testing) as opposed to exploratory (hypothesis generating), and the conditions under which study findings should be considered confirmatory. Each issue is illustrated by selected empirical examples.
KeywordsSubgroup analysis Intervention effects Reporting intervention research
- Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, 57, 289–300.Google Scholar
- Bloom, D. (2009). The Joyce Foundation’s transitional jobs reentry demonstration: Testing strategies to help former prisoners find and keep jobs and stay out of prison. Chicago, IL: The Joyce Foundation.Google Scholar
- Bloom, D., Redcross, C., Zweig, J., & Azurdia, G. (2007). Transitional jobs for ex-prisoners: Early impacts from a random assignment evaluation of the Center for Employment Opportunities (CEO) prisoner reentry program. New York: MDRC.Google Scholar
- Bloom, H. S., Thompson, S. L., & Unterman, R. (2010). Transforming the high school experience: How New York city’s new small schools are boosting student achievement and graduation rates. New York: MDRC.Google Scholar
- Council, N. R. (2007). Crime, parole, desistance from crime, and community integration. Committee on Community Supervision and Desistance from Crime. Washington, DC: The National Academies Press.Google Scholar
- Kim, S., Leblanc, A., & Michalopoulos, C. (2009). Working toward Wellness: Early results from a telephonic care management program for medicaid recipients with depression. New York: MDRC.Google Scholar
- Kim, S., Leblanc, A., Simon, G., Walter, J., & Michalopoulos, C. (2010). Working toward Wellness: Eighteen-month results from a telephonic care management program for medicaid recipients with depression. New York: MDRC.Google Scholar
- Michalopoulos, C., & Schwartz, C. (2000). What works best for whom: Impacts of 20 welfare-to-work programs by subgroup. Washington, DC: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation and Administration for Children and Families, and U.S. Department of Education.Google Scholar
- Redcross, C., Bloom, D., Azurdia, G., Zweig, J., & Pindus, N. (2007). Transitional jobs for ex-prisoners: Implementation, two-year impacts, and costs of the Center for Employment Opportunities (CEO) prisoner reentry program. New York: MDRC.Google Scholar
- Schochet, P. M. (2008). Technical methods report: Guidelines for multiple testing in impact evaluations. Washington, DC: U.S. Department of Education, Institute of Education Sciences.Google Scholar