Abstract
Preventive interventions are often designed and tested with the immediate program period in mind, and little thought that the intervention sample might be followed up for years or even decades beyond the initial trial. However, depending on the type of intervention and the nature of the outcomes, long-term follow-up may well be appropriate. The advantages of long-term follow-up of preventive interventions are discussed and include the capacity to examine program effects across multiple later life outcomes, the ability to examine the etiological processes involved in the development of the outcomes of interest, and the ability to provide more concrete estimates of the relative benefits and costs of an intervention. In addition, researchers have identified potential methodological risks of long-term follow-up such as inflation of type 1 error through post hoc selection of outcomes, selection bias, and problems stemming from attrition over time. The present paper presents a set of seven recommendations for the design or evaluation of studies for potential long-term follow-up organized under four areas: Intervention Logic Model, Developmental Theory and Measurement Issues; Design for Retention; Dealing with Missing Data; and Unique Considerations for Intervention Studies. These recommendations include conceptual considerations in the design of a study, pragmatic concerns in the design and implementation of the data collection for long-term follow-up, as well as criteria to be considered for the evaluation of an existing intervention for potential for long-term follow-up. Concrete examples from existing intervention studies that have been followed up over the long term are provided.
Similar content being viewed by others
References
Aos, S., Lieb, R., Mayfield, J., Miller, M., & Pennucci, A. (2004). Benefits and costs of prevention and early intervention programs for youth. Olympia: Washington State Institute for Public Policy.
Bailey, J. A., Hill, K. G., Hawkins, J. D., Catalano, R. F., & Abbott, R. D. (2008). Men’s and women’s patterns of substance use around pregnancy. Birth: Issues in Perinatal Care, 35, 50–59.
Beldavs, Z., Forgatch, M. S., Patterson, G. R., & DeGarmo, G. (2006). Reducing the detrimental effects of divorce: enhancing the parental competence of single mothers. In N. Heinrichs, K. Haalweg, & M. Döpfner (Eds.), Strengthening families: evidence-based approaches to support child mental health (pp. 143–185). Munster: Psychotherapie.
Beskow, L. M., Dame, L., & Costello, E. J. (2008). Certificates of confidentiality and compelled disclosure of data. Science, 322, 1054–1055.
Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbott, R. D. (2007). Effects of Communities That Care on prevention services systems: findings from the community youth development study at 1.5 years. Prevention Science, 8, 180–191.
Catalano, R. F., & Hawkins, J. D. (1996). The social development model: a theory of antisocial behavior. In J. D. Hawkins (Ed.), Delinquency and crime: current theories (pp. 149–197). New York: Cambridge University Press.
Chen, H.-T. (1990). Theory-driven evaluations. Thousand Oaks: Sage.
Cohen, P. (1991). A source of bias in longitudinal investigations of change. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 18–30). Washington: American Psychological Association.
Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 137–148). Washington: American Psychological Association.
Cotter, R. B., Burke, J. D., Loeber, R., & Navratil, J. L. (2002). Innovative retention methods in longitudinal research: a case study of the developmental trends study. Journal of Child and Family Studies, 11, 485–498.
Cotter, R. B., Burke, J. D., Stouthamer-Loeber, M., & Loeber, R. (2005). Contacting participants for follow-up: How much effort is required to retain participants in longitudinal studies? Evaluation and Program Planning, 28, 15–21.
Elder, G. H., Jr. (1998). The life course as developmental theory. Child Development, 69, 1–12.
Epstein, M., Hill, K. G., Bailey, J. A., & Hawkins, J. D. (2013). The effect of general and drug-specific family environments on comorbid and drug-specific problem behavior: a longitudinal examination. Developmental Psychology, 49, 1151–1164.
Farrington, D. P. (2006). Key longitudinal-experimental studies in criminology. Journal of Experimental Criminology, 2, 121–141.
Farrington, D. P., Gallagher, B., Morley, L., St. Ledger, R. J., & West, D. J. (1990). Minimizing attrition in longitudinal research: methods of tracing and securing cooperation in a 24-year follow-up study. In D. Magnusson & L. Bergman (Eds.), Data quality in longitudinal research (pp. 122–147). Cambridge: Cambridge University Press.
Fernandez, C. V., Skedgel, C., & Weijer, C. (2004). Considerations and costs of disclosing study findings to research participants. Canadian Medical Association Journal, 170, 1417–1419.
Flay, B.R., Biglan, A., Boruch, R.F., Castro, F.G., Gottfredson, D., Kellam, S, Ji, P. (2005). Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175.
Forgatch, M. S., & DeGarmo, D. S. (2007). Accelerating recovery from poverty: prevention effects for recently separated mothers. Journal of Early and Intensive Behavioral Intervention, 4, 681–702.
Foster, E. M., & McCombs-Thornton, K. (2012). The economics of evidence-based practice in disorders of childhood and adolescence. In P. Sturmey & M. Hersen (Eds.), Handbook of evidence-based practice in clinical psychology, vol 1: child and adolescent disorders (pp. 103–127). Hoboken: Wiley.
Gest, S. D., & Davidson, A. J. (2011). A developmental perspective on risk, resilience, and prevention. In M. K. Underwood & L. H. Rosen (Eds.), Social development: relationships in infancy, childhood, and adolescence (pp. 427–454). New York: Guilford.
Gottfredson, D.C., Cook, T.D., Gardner, F.E.M., Gorman-Smith, D., Howe, G.W., Sandler, I. N., Zafft, K.M. (2015). Standards of evidence for efficacy, effectiveness, and scale up research in prevention science: next generation. Prevention Science, 16, 893–926. doi:10.1007/s11121-015-0555-x.
Graham, J. W. (2009). Missing data analysis: making it work in the real world. Annual Review of Psychology, 60, 549–576.
Grant, S., Mayo-Wilson, E., Hopewell, S., Macdonald, G., Moher, D., & Montgomery, P. (2013). Developing a reporting guideline for social and psychological intervention trials. Journal of Experimental Criminology, 9, 355–367.
Haggerty, K. P., Fleming, C. B., Catalano, R. F., Petrie, R. S., Rubin, R. J., & Grassley, M. H. (2008). Ten years later: locating and interviewing children of drug abusers. Evaluation and Program Planning, 31, 1–9.
Hansen, W.B., & Collins, L.M. (1994). Seven ways to increase power without increasing. In National Institute on Drug Abuse Research monograph series vol. 142 (pp. 184–195). Washington, DC: US Department of Health and Human Services.
Hansen, W. B., Tobler, N. S., & Graham, J. W. (1990). Attrition in substance abuse prevention research: a meta-analysis of 85 longitudinally followed cohorts. Evaluation Review, 14, 677–685.
Hawkins, J. D., Catalano, R. F., Morrison, D. M., O’Donnell, J., Abbott, R. D., & Day, L. E. (1992). The Seattle Social Development Project: effects of the first four years on protective factors and problem behaviors. In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: interventions from birth through adolescence (pp. 139–161). New York: Guilford.
Hawkins, J. D., Catalano, R. F., Kosterman, R., Abbott, R., & Hill, K. G. (1999). Preventing adolescent health-risk behaviors by strengthening protection during childhood. Archives of Pediatrics and Adolescent Medicine, 153, 226–234.
Hawkins, J. D., Guo, J., Hill, K. G., Battin-Pearson, S., & Abbott, R. D. (2001). Long-term effects of the Seattle Social Development intervention on school bonding trajectories. Applied Developmental Science: Special Issue: Prevention as Altering the Course of Development, 5, 225–236.
Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2005). Promoting positive adult functioning through social development intervention in childhood: long-term effects from the Seattle Social Development Project. Archives of Pediatrics and Adolescent Medicine, 159, 25–31.
Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., & Catalano, R. F. (2008a). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15–22.
Hawkins, J. D., Catalano, R. F., Arthur, M. W., Egan, E., Brown, E. C., Abbott, R. D., & Murray, D. M. (2008b). Testing Communities that Care: the rationale, design and behavioral baseline equivalence of the community youth development study. Prevention Science, 9, 178–190.
Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2008c). Effects of social development intervention in childhood 15 years later. Archives of Pediatrics & Adolescent Medicine, 162, 1133–1141.
Hill, K. G., Hawkins, J. D., Catalano, R. F., Abbott, R. D., & Guo, J. (2005). Family influences on the risk of daily smoking initiation. Journal of Adolescent Health, 37, 202–210.
Hill, K.G., Hawkins, J.D., Bailey, J.A., Catalano, R.F., Abbott, R.D., Shapiro, V.B. (2010). Person-environment interaction in the prediction of alcohol abuse and alcohol dependence in adulthood. Drug and Alcohol Dependence, [Epub ahead of print.] (March 16).
Hill, K. G., Bailey, J. A., Hawkins, J. D., Catalano, R. F., Kosterman, R., Oesterle, S., & Abbott, R. D. (2013). The onset of STI diagnosis through age 30: results from the Seattle Social Development Project intervention. Prevention Science. doi:10.1007/s11121-013-0382-x.
Holder, H. (2009). Prevention programs in the 21st century: what we do not discuss in public. Addiction, 105, 578–581.
Howard, W. J., Rhemtulla, M., & Little, T. D. (2015). Using Principal Components as Auxiliary Variables in Missing Data Estimation. Multivariate Behavioral Research, 50, 285–299. doi:10.1080/00273171.2014.999267.
Kuklinski, M. R., Briney, J. S., Hawkins, J. D., & Catalano, R. F. (2012). Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science, 13, 150–161.
Latimer, N. R. (2013). Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Medical Decision Making, 33, 743–754.
Lonczak, H. S., Abbott, R. D., Hawkins, J. D., Kosterman, R., & Catalano, R. F. (2002). Effects of the Seattle Social Development Project on sexual behavior, pregnancy, birth, and sexually transmitted disease outcomes by age 21 years. Archives of Pediatrics and Adolescent Medicine, 156, 438–447.
MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor & Francis/Lawrence Erlbaum Associates.
Magnusson, D., & Bergman, L. R. (Eds.). (1990). Data quality in longitudinal research. New York: Cambridge University Press.
Masten, A. S., & Cicchetti, D. (2010). Developmental cascades. Development and Psychopathology, 22, 491–495.
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Altman, D. G. (2010). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology, 63(8), e1–e37. doi:10.1016/j.jclinepi.2010.03.004
Nicolaas, G. (2011). Survey paradata: a review. Retrieved February 2015 from Economic and Social Research Council National Center for Research Methods http://eprints.ncrm.ac.uk/1719/
Oesterle, S., Hawkins, J. D., Hill, K. G., & Bailey, J. A. (2010). Men’s and women’s pathways to adulthood and their adolescent precursors. Journal of Marriage and Family, 72, 1436–1453.
Olweus, D., & Limber, S. P. (2010). Bullying in school: evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry, 80, 124–134.
Patterson, G. R., & Fleischman, M. J. (1979). Maintenance of treatment effects: some considerations concerning family systems and follow-up data. Behavior Therapy, 10, 168–185.
Patterson, G. R., Forgatch, M. S., & DeGarmo, D. S. (2010). Cascading effects following intervention. Development and Psychopathology, 22, 949–970.
Plotnick, R. D. (1994). Applying benefit-cost analysis to substance use prevention programs. International Journal of the Addictions, 29, 339–359.
Sanders, M.R., & Kirby, J.N. (2014). Surviving or thriving: quality assurance mechanisms to promote innovation in the development of evidence-based parenting interventions. Prevention Science, 1–11. doi:10.1007/s11121-014-0475-1
Sandler, I. N., Schoenfelder, E. N., Wolchik, S. A., & MacKinnon, D. P. (2011). Long-term impact of prevention programs to promote effective parenting: lasting effects but uncertain processes. Annual Review of Psychology, 62, 299–329.
Slade, E. P., & Becker, K. D. (2014). Understanding proximal–distal economic projections of the benefits of childhood preventive interventions. Prevention Science, 15, 807–817.
Stouthamer-Loeber, M., & van Kammen, W. B. (1995). Data collection and management: a practical guide. Thousand Oaks: Sage.
van Kammen, W. B., & Stouthamer-Loeber, M. (1998). Practical aspects of interview data collection and data management. In L. Bickman, D. J. Rog, L. Bickman, & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 375–397). Thousand Oaks: Sage.
Wolf, L. E., Dame, L. E., Patel, M. J., Williams, B. A., Austin, J. A., & Beskow, L. M. (2012). Certificates of confidentiality: legal counsels’ experiences with and perspectives on legal demands for research data. Journal of Empirical Research on Human Research Ethics, 7, 1–9.
Acknowledgments
This project was supported by the National Institute on Drug Abuse (NIDA; R01DA009679, R01DA024411-05-07) and grant 21548 from the Robert Wood Johnson Foundation. The content is solely the responsibility of the authors. The authors gratefully acknowledge our study participants for their continued contribution to the longitudinal studies. They also acknowledge the SDRG Survey Research Division for their hard work in maintaining high panel retention. Earlier versions of this manuscript were presented at an expert panel meeting on “Impact of Early Interventions on Trajectories of Violence” in October of 2010 organized by CDC as well as at the Society for Prevention Research meeting in 2011. The paper has been expanded to consider long-term follow-up of preventive interventions in general.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no competing interests.
Ethical Approval
Formal ethics review is not applicable for this theoretical/methodological paper; however, the work is consistent with COPE guidelines.
Informed Consent
Informed consent is not applicable for this work.
Rights and permissions
About this article
Cite this article
Hill, K.G., Woodward, D., Woelfel, T. et al. Planning for Long-Term Follow-Up: Strategies Learned from Longitudinal Studies. Prev Sci 17, 806–818 (2016). https://doi.org/10.1007/s11121-015-0610-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11121-015-0610-7