Research design issues for evaluating complex multicomponent interventions in neighborhoods and communities
Major advances in population health will not occur unless we translate existing knowledge into effective multicomponent interventions, implement and maintain these in communities, and develop rigorous translational research and evaluation methods to ensure continual improvement and sustainability. We discuss challenges and offer approaches to evaluation that are key for translational research stages 3 to 5 to advance optimized adoption, implementation, and maintenance of effective and replicable multicomponent strategies. The major challenges we discuss concern (a) multiple contexts of evaluation/research, (b) complexity of packages of interventions, and (c) phases of evaluation/research questions. We suggest multiple alternative research designs that maintain rigor but accommodate these challenges and highlight the need for measurement systems. Longitudinal data collection and a standardized continuous measurement system are fundamental to the evaluation and refinement of complex multicomponent interventions. To be useful to T3–T5 translational research efforts in neighborhoods and communities, such a system would include assessments of the reach, implementation, effects on immediate outcomes, and effects of the comprehensive intervention package on more distal health outcomes.
KeywordsHealth Well-being Evaluation design Complex intervention Translational research
A grant from the National Institute on Drug Abuse (DA028946) supported the authors’ work on this manuscript. The funders had no role in the manuscript, and the views expressed are solely those of the authors. We thank Christine Cody and Bethany Livingston for their excellent editorial assistance.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no competing interests.
Adherence to ethical standards
No human subject research was involved.
- 1.National Research Council, Institute of Medicine. Preventing mental, emotional, and behavioral disorders among young people: progress and possibilities. Washington, DC: The National Academies Press, 2009.Google Scholar
- 3.Fishbein DH, Stahl M, Ridenour TA, Sussman S. Translating emerging research into prevention of high-risk behaviors: state-of-the-science, barriers, and innovations. Translational behavioral medicine 2015Google Scholar
- 4.Holmes BJ, Finegood DT, Riley BL, Best A. Systems thinking in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and implementation research in health. New York: Oxford University Press; 2012: 175-91.Google Scholar
- 6.Komro KA, Tobler AL, Delisle AL, O’Mara RJ, Wagenaar AC. Beyond the clinic: improving child health through evidence-based community development. BMC Pediatr. 2013; 13(172): 1471-2431.Google Scholar
- 8.Patton MQ. Utilization-focused evaluation. 4th ed: Guilford Press, 2008Google Scholar
- 9.Flay BR, Biglan A, Boruch RF, et al. Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prevention Science 2005;6(3):151–75 doi: DOI 10.1007/s11121-005-5553-y
- 12.Brownson RC, Colditz GA, Proctor EK. Design and analysis in dissemination and implementation research: Oxford University Press, 2012Google Scholar
- 14.Haskins R, Baron J. Building the connection between policy and evidence: The Obama evidence-based initiatives. Nesta: London, England; 2011.Google Scholar
- 17.Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. New York: Houghton Mifflin Company; 2002.Google Scholar
- 22.Wagenaar AC, Komro K. Natural experiments: research design elements for optimal causal inference without randomization. In: Wagenaar AC, Burris S, eds. Public health law research: theory and methods. San Francisco: Jossey-Bass; 2013.Google Scholar
- 23.Landsverk J, Brown CH, Chamberlain P, et al. Design and analysis in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, eds. Dissemination and implementation research in health. New York: Oxford University Press; 2012.Google Scholar