Prevention Science

, Volume 17, Issue 7, pp 806–818 | Cite as

Planning for Long-Term Follow-Up: Strategies Learned from Longitudinal Studies

  • Karl G. Hill
  • Danielle Woodward
  • Tiffany Woelfel
  • J. David Hawkins
  • Sara Green
Article

Abstract

Preventive interventions are often designed and tested with the immediate program period in mind, and little thought that the intervention sample might be followed up for years or even decades beyond the initial trial. However, depending on the type of intervention and the nature of the outcomes, long-term follow-up may well be appropriate. The advantages of long-term follow-up of preventive interventions are discussed and include the capacity to examine program effects across multiple later life outcomes, the ability to examine the etiological processes involved in the development of the outcomes of interest, and the ability to provide more concrete estimates of the relative benefits and costs of an intervention. In addition, researchers have identified potential methodological risks of long-term follow-up such as inflation of type 1 error through post hoc selection of outcomes, selection bias, and problems stemming from attrition over time. The present paper presents a set of seven recommendations for the design or evaluation of studies for potential long-term follow-up organized under four areas: Intervention Logic Model, Developmental Theory and Measurement Issues; Design for Retention; Dealing with Missing Data; and Unique Considerations for Intervention Studies. These recommendations include conceptual considerations in the design of a study, pragmatic concerns in the design and implementation of the data collection for long-term follow-up, as well as criteria to be considered for the evaluation of an existing intervention for potential for long-term follow-up. Concrete examples from existing intervention studies that have been followed up over the long term are provided.

Keywords

Intervention Assessment design Longitudinal Follow-up Retention Developmental 

References

  1. Aos, S., Lieb, R., Mayfield, J., Miller, M., & Pennucci, A. (2004). Benefits and costs of prevention and early intervention programs for youth. Olympia: Washington State Institute for Public Policy.Google Scholar
  2. Bailey, J. A., Hill, K. G., Hawkins, J. D., Catalano, R. F., & Abbott, R. D. (2008). Men’s and women’s patterns of substance use around pregnancy. Birth: Issues in Perinatal Care, 35, 50–59.CrossRefGoogle Scholar
  3. Beldavs, Z., Forgatch, M. S., Patterson, G. R., & DeGarmo, G. (2006). Reducing the detrimental effects of divorce: enhancing the parental competence of single mothers. In N. Heinrichs, K. Haalweg, & M. Döpfner (Eds.), Strengthening families: evidence-based approaches to support child mental health (pp. 143–185). Munster: Psychotherapie.Google Scholar
  4. Beskow, L. M., Dame, L., & Costello, E. J. (2008). Certificates of confidentiality and compelled disclosure of data. Science, 322, 1054–1055.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbott, R. D. (2007). Effects of Communities That Care on prevention services systems: findings from the community youth development study at 1.5 years. Prevention Science, 8, 180–191.CrossRefPubMedGoogle Scholar
  6. Catalano, R. F., & Hawkins, J. D. (1996). The social development model: a theory of antisocial behavior. In J. D. Hawkins (Ed.), Delinquency and crime: current theories (pp. 149–197). New York: Cambridge University Press.Google Scholar
  7. Chen, H.-T. (1990). Theory-driven evaluations. Thousand Oaks: Sage.Google Scholar
  8. Cohen, P. (1991). A source of bias in longitudinal investigations of change. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 18–30). Washington: American Psychological Association.CrossRefGoogle Scholar
  9. Collins, L. M. (1991). Measurement in longitudinal research. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change: recent advances, unanswered questions, future directions (pp. 137–148). Washington: American Psychological Association.CrossRefGoogle Scholar
  10. Cotter, R. B., Burke, J. D., Loeber, R., & Navratil, J. L. (2002). Innovative retention methods in longitudinal research: a case study of the developmental trends study. Journal of Child and Family Studies, 11, 485–498.CrossRefGoogle Scholar
  11. Cotter, R. B., Burke, J. D., Stouthamer-Loeber, M., & Loeber, R. (2005). Contacting participants for follow-up: How much effort is required to retain participants in longitudinal studies? Evaluation and Program Planning, 28, 15–21.CrossRefGoogle Scholar
  12. Elder, G. H., Jr. (1998). The life course as developmental theory. Child Development, 69, 1–12.CrossRefPubMedGoogle Scholar
  13. Epstein, M., Hill, K. G., Bailey, J. A., & Hawkins, J. D. (2013). The effect of general and drug-specific family environments on comorbid and drug-specific problem behavior: a longitudinal examination. Developmental Psychology, 49, 1151–1164.CrossRefPubMedGoogle Scholar
  14. Farrington, D. P. (2006). Key longitudinal-experimental studies in criminology. Journal of Experimental Criminology, 2, 121–141.CrossRefGoogle Scholar
  15. Farrington, D. P., Gallagher, B., Morley, L., St. Ledger, R. J., & West, D. J. (1990). Minimizing attrition in longitudinal research: methods of tracing and securing cooperation in a 24-year follow-up study. In D. Magnusson & L. Bergman (Eds.), Data quality in longitudinal research (pp. 122–147). Cambridge: Cambridge University Press.Google Scholar
  16. Fernandez, C. V., Skedgel, C., & Weijer, C. (2004). Considerations and costs of disclosing study findings to research participants. Canadian Medical Association Journal, 170, 1417–1419.CrossRefPubMedPubMedCentralGoogle Scholar
  17. Flay, B.R., Biglan, A., Boruch, R.F., Castro, F.G., Gottfredson, D., Kellam, S, Ji, P. (2005). Standards of evidence: criteria for efficacy, effectiveness and dissemination. Prevention Science, 6(3), 151–175.Google Scholar
  18. Forgatch, M. S., & DeGarmo, D. S. (2007). Accelerating recovery from poverty: prevention effects for recently separated mothers. Journal of Early and Intensive Behavioral Intervention, 4, 681–702.CrossRefGoogle Scholar
  19. Foster, E. M., & McCombs-Thornton, K. (2012). The economics of evidence-based practice in disorders of childhood and adolescence. In P. Sturmey & M. Hersen (Eds.), Handbook of evidence-based practice in clinical psychology, vol 1: child and adolescent disorders (pp. 103–127). Hoboken: Wiley.Google Scholar
  20. Gest, S. D., & Davidson, A. J. (2011). A developmental perspective on risk, resilience, and prevention. In M. K. Underwood & L. H. Rosen (Eds.), Social development: relationships in infancy, childhood, and adolescence (pp. 427–454). New York: Guilford.Google Scholar
  21. Gottfredson, D.C., Cook, T.D., Gardner, F.E.M., Gorman-Smith, D., Howe, G.W., Sandler, I. N., Zafft, K.M. (2015). Standards of evidence for efficacy, effectiveness, and scale up research in prevention science: next generation. Prevention Science, 16, 893–926. doi:10.1007/s11121-015-0555-x.
  22. Graham, J. W. (2009). Missing data analysis: making it work in the real world. Annual Review of Psychology, 60, 549–576.CrossRefPubMedGoogle Scholar
  23. Grant, S., Mayo-Wilson, E., Hopewell, S., Macdonald, G., Moher, D., & Montgomery, P. (2013). Developing a reporting guideline for social and psychological intervention trials. Journal of Experimental Criminology, 9, 355–367.CrossRefGoogle Scholar
  24. Haggerty, K. P., Fleming, C. B., Catalano, R. F., Petrie, R. S., Rubin, R. J., & Grassley, M. H. (2008). Ten years later: locating and interviewing children of drug abusers. Evaluation and Program Planning, 31, 1–9.CrossRefPubMedGoogle Scholar
  25. Hansen, W.B., & Collins, L.M. (1994). Seven ways to increase power without increasing. In National Institute on Drug Abuse Research monograph series vol. 142 (pp. 184–195). Washington, DC: US Department of Health and Human Services.Google Scholar
  26. Hansen, W. B., Tobler, N. S., & Graham, J. W. (1990). Attrition in substance abuse prevention research: a meta-analysis of 85 longitudinally followed cohorts. Evaluation Review, 14, 677–685.CrossRefGoogle Scholar
  27. Hawkins, J. D., Catalano, R. F., Morrison, D. M., O’Donnell, J., Abbott, R. D., & Day, L. E. (1992). The Seattle Social Development Project: effects of the first four years on protective factors and problem behaviors. In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: interventions from birth through adolescence (pp. 139–161). New York: Guilford.Google Scholar
  28. Hawkins, J. D., Catalano, R. F., Kosterman, R., Abbott, R., & Hill, K. G. (1999). Preventing adolescent health-risk behaviors by strengthening protection during childhood. Archives of Pediatrics and Adolescent Medicine, 153, 226–234.CrossRefPubMedGoogle Scholar
  29. Hawkins, J. D., Guo, J., Hill, K. G., Battin-Pearson, S., & Abbott, R. D. (2001). Long-term effects of the Seattle Social Development intervention on school bonding trajectories. Applied Developmental Science: Special Issue: Prevention as Altering the Course of Development, 5, 225–236.Google Scholar
  30. Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2005). Promoting positive adult functioning through social development intervention in childhood: long-term effects from the Seattle Social Development Project. Archives of Pediatrics and Adolescent Medicine, 159, 25–31.CrossRefPubMedGoogle Scholar
  31. Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbott, R. D., & Catalano, R. F. (2008a). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15–22.CrossRefPubMedGoogle Scholar
  32. Hawkins, J. D., Catalano, R. F., Arthur, M. W., Egan, E., Brown, E. C., Abbott, R. D., & Murray, D. M. (2008b). Testing Communities that Care: the rationale, design and behavioral baseline equivalence of the community youth development study. Prevention Science, 9, 178–190.CrossRefPubMedPubMedCentralGoogle Scholar
  33. Hawkins, J. D., Kosterman, R., Catalano, R. F., Hill, K. G., & Abbott, R. D. (2008c). Effects of social development intervention in childhood 15 years later. Archives of Pediatrics & Adolescent Medicine, 162, 1133–1141.CrossRefGoogle Scholar
  34. Hill, K. G., Hawkins, J. D., Catalano, R. F., Abbott, R. D., & Guo, J. (2005). Family influences on the risk of daily smoking initiation. Journal of Adolescent Health, 37, 202–210.CrossRefPubMedGoogle Scholar
  35. Hill, K.G., Hawkins, J.D., Bailey, J.A., Catalano, R.F., Abbott, R.D., Shapiro, V.B. (2010). Person-environment interaction in the prediction of alcohol abuse and alcohol dependence in adulthood. Drug and Alcohol Dependence, [Epub ahead of print.] (March 16).Google Scholar
  36. Hill, K. G., Bailey, J. A., Hawkins, J. D., Catalano, R. F., Kosterman, R., Oesterle, S., & Abbott, R. D. (2013). The onset of STI diagnosis through age 30: results from the Seattle Social Development Project intervention. Prevention Science. doi:10.1007/s11121-013-0382-x.PubMedPubMedCentralGoogle Scholar
  37. Holder, H. (2009). Prevention programs in the 21st century: what we do not discuss in public. Addiction, 105, 578–581.CrossRefPubMedGoogle Scholar
  38. Howard, W. J., Rhemtulla, M., & Little, T. D. (2015). Using Principal Components as Auxiliary Variables in Missing Data Estimation. Multivariate Behavioral Research, 50, 285–299. doi:10.1080/00273171.2014.999267.
  39. Kuklinski, M. R., Briney, J. S., Hawkins, J. D., & Catalano, R. F. (2012). Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science, 13, 150–161.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Latimer, N. R. (2013). Survival analysis for economic evaluations alongside clinical trials—extrapolation with patient-level data: inconsistencies, limitations, and a practical guide. Medical Decision Making, 33, 743–754.CrossRefPubMedGoogle Scholar
  41. Lonczak, H. S., Abbott, R. D., Hawkins, J. D., Kosterman, R., & Catalano, R. F. (2002). Effects of the Seattle Social Development Project on sexual behavior, pregnancy, birth, and sexually transmitted disease outcomes by age 21 years. Archives of Pediatrics and Adolescent Medicine, 156, 438–447.CrossRefPubMedGoogle Scholar
  42. MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York: Taylor & Francis/Lawrence Erlbaum Associates.Google Scholar
  43. Magnusson, D., & Bergman, L. R. (Eds.). (1990). Data quality in longitudinal research. New York: Cambridge University Press.Google Scholar
  44. Masten, A. S., & Cicchetti, D. (2010). Developmental cascades. Development and Psychopathology, 22, 491–495.CrossRefPubMedGoogle Scholar
  45. Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Altman, D. G. (2010). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology, 63(8), e1–e37. doi:10.1016/j.jclinepi.2010.03.004
  46. Nicolaas, G. (2011). Survey paradata: a review. Retrieved February 2015 from Economic and Social Research Council National Center for Research Methods http://eprints.ncrm.ac.uk/1719/
  47. Oesterle, S., Hawkins, J. D., Hill, K. G., & Bailey, J. A. (2010). Men’s and women’s pathways to adulthood and their adolescent precursors. Journal of Marriage and Family, 72, 1436–1453.CrossRefPubMedPubMedCentralGoogle Scholar
  48. Olweus, D., & Limber, S. P. (2010). Bullying in school: evaluation and dissemination of the Olweus Bullying Prevention Program. American Journal of Orthopsychiatry, 80, 124–134.CrossRefPubMedGoogle Scholar
  49. Patterson, G. R., & Fleischman, M. J. (1979). Maintenance of treatment effects: some considerations concerning family systems and follow-up data. Behavior Therapy, 10, 168–185.CrossRefGoogle Scholar
  50. Patterson, G. R., Forgatch, M. S., & DeGarmo, D. S. (2010). Cascading effects following intervention. Development and Psychopathology, 22, 949–970.CrossRefPubMedPubMedCentralGoogle Scholar
  51. Plotnick, R. D. (1994). Applying benefit-cost analysis to substance use prevention programs. International Journal of the Addictions, 29, 339–359.CrossRefPubMedGoogle Scholar
  52. Sanders, M.R., & Kirby, J.N. (2014). Surviving or thriving: quality assurance mechanisms to promote innovation in the development of evidence-based parenting interventions. Prevention Science, 1–11. doi:10.1007/s11121-014-0475-1
  53. Sandler, I. N., Schoenfelder, E. N., Wolchik, S. A., & MacKinnon, D. P. (2011). Long-term impact of prevention programs to promote effective parenting: lasting effects but uncertain processes. Annual Review of Psychology, 62, 299–329.CrossRefPubMedPubMedCentralGoogle Scholar
  54. Slade, E. P., & Becker, K. D. (2014). Understanding proximal–distal economic projections of the benefits of childhood preventive interventions. Prevention Science, 15, 807–817.CrossRefPubMedPubMedCentralGoogle Scholar
  55. Stouthamer-Loeber, M., & van Kammen, W. B. (1995). Data collection and management: a practical guide. Thousand Oaks: Sage.CrossRefGoogle Scholar
  56. van Kammen, W. B., & Stouthamer-Loeber, M. (1998). Practical aspects of interview data collection and data management. In L. Bickman, D. J. Rog, L. Bickman, & D. J. Rog (Eds.), Handbook of applied social research methods (pp. 375–397). Thousand Oaks: Sage.Google Scholar
  57. Wolf, L. E., Dame, L. E., Patel, M. J., Williams, B. A., Austin, J. A., & Beskow, L. M. (2012). Certificates of confidentiality: legal counsels’ experiences with and perspectives on legal demands for research data. Journal of Empirical Research on Human Research Ethics, 7, 1–9.PubMedPubMedCentralGoogle Scholar

Copyright information

© Society for Prevention Research 2015

Authors and Affiliations

  • Karl G. Hill
    • 1
  • Danielle Woodward
    • 1
  • Tiffany Woelfel
    • 1
  • J. David Hawkins
    • 1
  • Sara Green
    • 1
  1. 1.Social Development Research Group, School of Social WorkUniversity of WashingtonSeattleUSA

Personalised recommendations