Translational Behavioral Medicine

, Volume 4, Issue 3, pp 238–251 | Cite as

Evaluating individual intervention components: making decisions based on the results of a factorial screening experiment

  • Linda M Collins
  • Jessica B Trail
  • Kari C Kugler
  • Timothy B Baker
  • Megan E Piper
  • Robin J Mermelstein
Original Research


The multiphase optimization strategy (MOST) is a framework for not only evaluating but also optimizing behavioral interventions. A tool critical for MOST is the screening experiment, which enables efficient gathering of information for deciding which components to include in an optimized intervention. This article outlines a procedure for making decisions based on data from a factorial screening experiment. The decision making procedure is illustrated with artificial data generated to resemble empirical data. The illustration suggests that this approach is useful for selecting intervention components and settings based on the results of a factorial screening experiment. It is important to develop methods for making decisions based on factorial screening experiments. The approach demonstrated here is potentially useful, but has limited generalizability. Future research should develop additional decision making procedures for a variety of situations.


Comparative effectiveness Multiphase optimization strategy Factorial experiments Behavioral interventions 



This project was supported by Award Number P50CA143188-3 from the National Cancer Institute and by Award Number P50DA010075-15 from the National Institute on Drug Abuse. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute, the National Institute on Drug Abuse, or the National Institutes of Health. This work has benefitted from discussions with John Dziak and other colleagues at The Methodology Center. The authors thank Amanda Applegate for editorial assistance.


  1. 1.
    Collins LM, Baker TB, Mermelstein RJ, et al. The multiphase optimization strategy for engineering effective tobacco use interventions. Ann Behav Med. 2011; 41(2): 208-226.PubMedCrossRefPubMedCentralGoogle Scholar
  2. 2.
    Clapham C, Nicholson J. The Concise Oxford Dictionary of Mathematics. 4th ed. New York: Oxford University Press; 2009.CrossRefGoogle Scholar
  3. 3.
    Collins LM, Chakraborty B, Murphy SA, Strecher V. Comparison of a phased experimental approach and a single randomized clinical trial for developing multicomponent behavioral interventions. Clin Trials. 2009; 6(1): 5-15.PubMedCrossRefPubMedCentralGoogle Scholar
  4. 4.
    Baker TB, Mermelstein R, Collins LM, et al. New methods for tobacco dependence treatment research. Ann Behav Med. 2011; 41(2): 192-207.PubMedCrossRefPubMedCentralGoogle Scholar
  5. 5.
    Strecher VJ, McClure JB, Alexander GL, et al. Web-based smoking-cessation programs: results of a randomized trial. Am J Prev Med. 2008; 34(5): 373-381.PubMedCrossRefPubMedCentralGoogle Scholar
  6. 6.
    Caldwell LL, Smith EA, Collins LM, et al. Translational research in South Africa: evaluating implementation quality using a factorial design. Child Youth Care For. 2012; 41(2): 119-136.CrossRefGoogle Scholar
  7. 7.
    Smith EA, Palen L, Caldwell LL, et al. Substance use and sexual risk prevention in Cape Town, South Africa: an evaluation of the HealthWise program. Prev Sci. 2008; 9: 311-321.PubMedCrossRefGoogle Scholar
  8. 8.
    Collins LM, Dziak JJ, Li R. Design of experiments with multiple independent variables: a resource management perspective on complete and reduced factorial designs. Psychol Methods. 2009; 14(3): 202-224.PubMedCrossRefPubMedCentralGoogle Scholar
  9. 9.
    Wu CFJ, Hamada M. Experiments: Planning, Analysis, and Parameter Design Optimization. New York: Wiley; 2011.Google Scholar
  10. 10.
    Chakraborty B, Collins LM, Strecher VJ, Murphy SA. Developing multicomponent interventions using fractional factorial designs. Stat Med. 2009; 28(21): 2687-2708.PubMedCrossRefPubMedCentralGoogle Scholar
  11. 11.
    Kirk RE. Experimental Design: Procedures for the Behavioral Sciences. 4th ed. Los Angeles: Sage; 2013.Google Scholar
  12. 12.
    Kugler KC, Trail JB, Dziak JJ, Collins LM. Effect Coding Versus Dummy Coding in Analysis of Data from Factorial Experiments. [Technical Report No. 12–120]. University Park: The Methodology Center, Penn State: The Methodology Center, Penn State; 2012.Google Scholar
  13. 13.
    Fisher JO. The Design of Experiments. New York: Hafner; 1971.Google Scholar
  14. 14.
    Collins LM, Murphy SA, Bierman KL. A conceptual framework for adaptive preventive interventions. Prev Sci. 2004; 5(3): 185-196.PubMedCrossRefPubMedCentralGoogle Scholar
  15. 15.
    Kirk RE. Effect magnitude: a different focus. J Stat Plan Infer. 2007; 137: 1634-1646.CrossRefGoogle Scholar

Copyright information

© Society of Behavioral Medicine 2013

Authors and Affiliations

  • Linda M Collins
    • 1
    • 2
  • Jessica B Trail
    • 1
    • 3
  • Kari C Kugler
    • 1
  • Timothy B Baker
    • 4
  • Megan E Piper
    • 4
  • Robin J Mermelstein
    • 5
  1. 1.The Methodology CenterPennsylvania State UniversityState CollegeUSA
  2. 2.Department of Human Development and Family StudiesPennsylvania State UniversityState CollegeUSA
  3. 3.Department of StatisticsPennsylvania State UniversityState CollegeUSA
  4. 4.Center for Tobacco Research and InterventionUniversity of WisconsinMadisonUSA
  5. 5.Department of Psychology and Institute for Health Research and PolicyUniversity of IllinoisChicagoUSA

Personalised recommendations