Skip to main content

Advertisement

Log in

Can We Trust Positive Findings of Intervention Research? The Role of Conflict of Interest

  • Published:
Prevention Science Aims and scope Submit manuscript

Abstract

In recent years, there has been increased attention to the issue of conflict of interest within prevention research. The aims of this paper are to discuss these developments and to relate them to discussions of conflict of interest in the broader scientific literature. Although there has been concern expressed about the extent to which conflicts of interest can be defined and measured, empirical research suggests that financial conflicts can be easily identified and assessed in meta-analyses focused on their effects on research quality. Research evidence also shows that conflict of interest is associated with use of flexible data analysis practices and the reporting of chance positive findings, both within prevention research and related disciplines such as public health and psychology. However, the overwhelming majority of published studies report positive results, and there are a number of other influences within academia (such as pressure to publish) that account for this and for the use of flexible data analysis practices. Accordingly, introducing measures to improve research quality in general, rather than just focusing on problems specific to research in which there is a clearly identifiable conflict of interest, may prove more effective and less controversial. Most such efforts focus on introducing greater transparency into research design, practice, and reporting. These both curtail employment of flexible data analysis practices and make their use transparent to investigators seeking to assess their effects on research quality. Also, requiring detailed disclosures of conflicts be reported by all investigators (not just senior authors) would improve current disclosure practices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Australian New Zealand Clinical Trial Registry. (2015). ANZCTR. Retrieved from http://www.anzctr.org.au

  • Berkeley Initiative for Transparency in the Social Sciences. (2015). BITSS Catalyst Program: Pilot 2015–2018. Retrieved from: http://www.bitss.org/wp-content/uploads/2015/12/bitss_catalyst_program_cn.pdf

  • Bierman, K. L., Nix, R. L., Maples, J. J., Murphy, S. A., & Conduct Problems Prevention Research Group. (2006). Examining clinical judgment in an adaptive intervention design: The Fast Track Program. Journal of Consulting and Clinical Psychology, 74, 468–481.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bierman, K.L., Coie, J., Dodge, K., Greenberg, M., Lochman, J., McMahon, R., Pinderhughes, E., & Conduct Problems Prevention Research Group. (2013). School outcomes of aggressive-disruptive children: Prediction from kindergarten risk factors and impact of the Fast Track Prevention Program. Aggressive Behavior, 39: doi:10.1002/ab.21467

  • BioMed Central. (2015). ISRCTN Registry. Retrieved from http://www.isrctn.com

  • Bosch, X., Pericas, J. M., Hernandez, C., & Doti, P. (2013). Financial, nonfinanacial and editors’ conflicts of interest in high-impact biomedical journals. European Journal of Clinical Investigation, 43, 660–667.

    Article  PubMed  Google Scholar 

  • Campbell, D. T. (1984). Can we be scientific in applied social science? In R. F. Connor, D. G. Altman, & C. Jackson (Eds.), Evaluation Studies: Review Annual (Vol. 9, pp. 26–48). Beverly Hills, CA: Sage.

    Google Scholar 

  • Center for Open Science. (n.d.) The $1,000,000 Preregistration Challenge: Eligible Journals. Retrieved from https://cos.io/prereg/

  • Committee on Publication Ethics. (2011). Code of Conduct and Best Practices Guidelines for Journal Editors. Retrieved from http://publicationethics.org/files/Code_of_conduct_for_journal_editors_Mar11.pdf.

  • Conduct Problems Prevention Research Group. (2007). Fast Track randomized controlled trial to prevent externalizing psychiatric disorders: Findings from grades 3 to 9. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 1250–1262.

    Article  PubMed Central  Google Scholar 

  • Conduct Problems Prevention Research Group. (2010a). Fast Track intervention effects on youth arrests and delinquency. Journal of Experimental Criminology, 6, 131–157.

  • Conduct Problems Prevention Research Group. (2010b). The difficulty of maintaining positive intervention effects: A look at disruptive behavior, deviant peer relations, and social skills during the middle school years. Journal of Early Adolescence, 30, 593–624.

  • Conduct Problems Prevention Research Group. (2010c). The effects of a multi-year universal social-emotional learning program: The role of student and school characteristics. Journal of Consulting and Clinical Psychology, 78,156-168.

  • Conduct Problems Prevention Research Group. (2011). The effects of the Fast Track preventive intervention on the development of conduct disorder across childhood. Child Development, 82, 331–345.

    Article  PubMed Central  Google Scholar 

  • Conduct Problems Prevention Research Group. (2013). Assessing findings from the fast track study. Journal of Experimental Criminology, 9, 119–126.

    Article  PubMed Central  Google Scholar 

  • Conduct Problems Prevention Research Group. (2014). Trajectories of risk for early sexual activity and early substance use in the Fast Track Prevention Program. Prevention Science, 15, S33–S46.

    Article  Google Scholar 

  • Conduct Problems Prevention Research Group. (2015). Impact of early intervention on psychopathology, crime, and well-being at age 25. American Journal of Psychiatry, 172, 59–70.

    Article  Google Scholar 

  • CONSORT. (2015). Endorsers: Journals and Organizations. Retrieved from: http://www.consort-statement.org/about-consort/endorsers.

    Google Scholar 

  • Dodge, K. A., Godwin, J., & the Conduct Problems Prevention Research Group. (2013). Social-information-processing patterns mediate the impact of preventive intervention on adolescent antisocial behavior. Psychological Science, 24, 456–465.

    Article  PubMed  PubMed Central  Google Scholar 

  • Eisner, M. (2009). No effect in independent prevention trials: Can we reject the cynical view? Journal of Experimental Criminology, 5, 163–183.

    Article  Google Scholar 

  • Eisner, M. (2014). The South Carolina Triple P System Population Trial to Prevent Child Maltreatment: Seven reasons to be skeptical about the study results (Violence Research Centre - Working Paper. Institute of Criminology, University of Cambridge).

  • Eisner, M., & Humphreys, D. (2011). Measuring conflict of interest in prevention and intervention research: A feasibility study. In T. Bliesener, A. Beelman, & M. Stemmler (Eds.), Antisocial behavior and crime: Contributions of developmental and evaluation research to prevention and intervention (pp. 165–180). Cambridge, MA: Hogrefe.

    Google Scholar 

  • Eisner, M. P., Humphreys, D. K., Wilson, P., & Gardner, F. (2015). Disclosure of financial conflicts of interest in interventions to improve child psychosocial health: A cross-sectional study. PloS ONE, 10, e0142803. doi:10.1371/journal.pone.0142803.

    Article  PubMed  PubMed Central  Google Scholar 

  • Fanelli, D. (2010a). “Positive” results increase down the hierarchy of science. PLoS ONE, 5, e10068. doi:10.1371/journal.pone.0010068.

  • Fanelli, D. (2010b). Do pressures to publish increase scientists’ bias? An empirical support for US States data. PLoS ONE, 5, e10271. doi:10.1371/journal.pone.0010271.

  • Fanelli, D. (2013). Redefine misconduct as distorted reporting. Nature, 494, 149.

    Article  CAS  PubMed  Google Scholar 

  • Fast Track Project. (2011). Publications. Retrieved from http://www.fasttrackproject.org/publications.php

  • Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7, 555–561.

    Article  PubMed  Google Scholar 

  • Fleming, P. S., Koletsi, D., Dwan, K., & Pamdis, N. (2015). Outcome discrepancies and selective reporting: Impacting the leading journals? PLoS One, 10, e0127495. doi:10.1371/journal.pone.0127495.

    Article  PubMed  PubMed Central  Google Scholar 

  • Foster, E. M. (2013). Reassessing findings from the Fast Track study: problems of method and analysis. Journal of Experimental Criminology, 9, 109–117.

    Article  Google Scholar 

  • Foster, E. M., Jones, D. E., & the Conduct Problems Prevention Research Group. (2006). Can a costly intervention be cost-effective? An analysis of violence prevention. Archives of General Psychiatry, 63, 1284–1291.

    Article  PubMed  PubMed Central  Google Scholar 

  • Goozner, M., Caplan, A., Moreno, J., Kramer, B. S., Babor, T. F., & Husser, W. C. (2009). A common standard for conflict of interest disclosure in addiction journals. Addiction, 104, 1779–1784.

    Article  PubMed  Google Scholar 

  • Gorman, D. M. (2014). Is Project Towards No Drug Abuse (Project TND) an evidence-based drug and violence prevention program? A review and reappraisal of the evaluation studies. Journal of Primary Prevention, 35, 217–232.

    Article  PubMed  Google Scholar 

  • Gorman, DM. (2015a). ‘Everything works’: The need to address confirmation bias in evaluations of drug misuse prevention interventions for adolescents. Addiction, 110, 1539–40

  • Gorman, DM. (2015b). Analytic flexibility in the evaluation of the Drug Education in Victoria Schools (DEVS) programme. International Journal of Drug Policy, 26, 719–720

  • Gorman, D. M., & Conde, E. (2007). Conflict of interest in the evaluation and dissemination of “model” school-based drug and violence prevention programs. Evaluation and Program Planning, 30, 422–429.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gorman, D. M., & Conde, E. (2010). The making of evidence-based practice: The case of Project ALERT. Children and Youth Services Review, 32, 214–222.

    Article  PubMed  PubMed Central  Google Scholar 

  • Holder, H. D. (2010). Prevention programs in the 21st century: What we do not discuss in public. Addiction, 105, 578–581.

    Article  PubMed  Google Scholar 

  • Institute of Medicine. (2002). Integrity in scientific research: Creating an environment that promotes responsible conduct. Washington, DC: National Academies Press.

    Google Scholar 

  • International Committee of Medical Journal Editors. (2015). Conflicts of interest. Retrieved from http://www.icmje.org/conflicts-of-interest/http://www.icmje.org/conflicts-of-interest/

  • Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani R. (2014a). Increasing value and reducing waste in research design, conduct, and analysis. Lancet, 383, 166–175.

  • Ioannidis, J. P. A., Munafo, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014b). Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends in Cognitive Sciences, 18, 235–241.

  • Jones, D. E., Godwin, J., Dodge, K. A., Bierman, K. L., Coie, J. D., Greenberg, M. T., & Pinderhughes, E. E. (2010). Impact of the Fast Track Prevention Program on health services use by conduct-problem youth. Pediatrics, 125, e130–e136.

    Article  PubMed  Google Scholar 

  • Kelling, G. L., Pate, T., Dieckman, D., & Brown, C. (1974). The Kansas City preventive patrol experiment: Summary report. Washington, DC: Police Foundation.

    Google Scholar 

  • Lipsey, M. (1995). What do we learn from 400 research studies on effectiveness of treatment with juvenile delinquents? In J. McGuiren (Ed.), What works? Reducing reoffending (pp. 63–78). New York: Wiley.

    Google Scholar 

  • MacCoun, R. (2005). Conflicts of interest in public policy. In D. A. Moore, D. M. Cain, G. Loewenstein, & M. Bazerman (Eds.), Conflicts of interest: Challenges and solutions in business, law, medicine, and public policy (pp. 233–262). London: Cambridge University Press.

    Chapter  Google Scholar 

  • Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., & Van der Laan, M. (2014). Promoting transparency in social science research. Science, 343, 30–31.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Nix, R. L., Bierman, K. L., McMahon, R. J., & the Conduct Problems Prevention Research Group. (2009). How attendance and quality of participation affect treatment response to parent management training. Journal of Consulting and Clinical Psychology, 77, 429–438.

    Article  PubMed  PubMed Central  Google Scholar 

  • Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., & Yarkoni, T. (2015). Promoting an open research culture. Science, 348, 1422–1425.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia II. Restructuring incentives and practices to promote truth over publishability. Psychological Science, 7, 615–631.

    Google Scholar 

  • Okulicz-Kozaryn, K., & Foxcroft, D. R. (2012). Effectiveness of the Strengthening Families Programme 10–14 in Poland for the prevention of alcohol and drug misuse: Protocol for a randomized controlled trial. BMC Public Health, 12, 319. doi:10.1186/1471-2458-12-319.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pasalich, D. S., Witkiewitz, K., McMahon, R. J., Pinderhughes, E. E., & the Conduct Problems Prevention Research Group. (2016). Indirect effects of the Fast Track intervention on conduct disorder symptoms and callous-unemotional traits: Distinct pathways involving discipline and warmth. Journal of Abnormal Child Psychology, 44, 587–597.

    Article  PubMed  PubMed Central  Google Scholar 

  • Popper, K. (1974). Replies to my critics. In P. A. Schilpp (Ed.), The Philosophy of Karl Popper (pp. 961–1197). La Salle, Il: Open Court.

    Google Scholar 

  • Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 18, 18. doi:10.1186/1741-7015-8-18.

    Article  Google Scholar 

  • Sherman, L. W. (2015). A tipping point for “totally evidenced policing”: Ten ideas for building an evidence-based police agency. International Criminal Justice Review, 25, 11–29.

    Article  Google Scholar 

  • Sherman, L. W., & Strang, H. (2009). Testing for analysts’ bias in crime prevention experiments: Can we accept Eisner’s one-tailed test? Journal of Experimental Criminology, 5, 185–200.

    Article  Google Scholar 

  • Slough, N. M., McMahon, R. J., & the Conduct Problems Prevention Research Group. (2008). Preventing serious conduct problems in school-age youth: The Fast Track Program. Cognitive and Behavioral Practice, 15, 3–17.

    Article  PubMed  PubMed Central  Google Scholar 

  • Society for Prevention Research. (2015). Conflict of interest policy. Retrieved from http://www.preventionresearch.org/about-spr/conflict-of-interest-policy/

  • Sterman, S. (2000). Business dynamics: Systems thinking and modeling for a complex world. New York: McGraw Hill.

    Google Scholar 

  • Sussman, S. (2009). Conflict of interest disclosure statement. Society for Prevention Research 17th Annual Meeting: Program Book, 121. Retrieved from http://www.preventionresearch.org/wp-content/uploads/2011/07/SPR2009program.pdf

  • Tobin, M. J. (2003). CoI and AJRCCM: Restating policy and a new form to upload. American Journal of Respiratory and Critical Care Medicine, 167, 1161–1164.

    Article  PubMed  Google Scholar 

  • US National Institutes of Health. (2015). ClinicalTrials.gov. Retrieved from https://clinicaltrials.gov.

  • Welsh, B. C., Braga, A. A., & Hollis-Peel, E. (2012). Can “disciplined passion” overcome the cynical view? An empirical enquiry of evaluator influence on police crime prevention program outcomes. Journal of Experimental Criminology, 8, 415–431.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dennis M. Gorman.

Ethics declarations

Conflict of Interest

The author declares that he has no conflict of interest.

Funding

None

Ethical Approval

The article does not contain any studies with human participants or animals performed by the author.

Informed Consent

Non-applicable, as no human subjects.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gorman, D.M. Can We Trust Positive Findings of Intervention Research? The Role of Conflict of Interest. Prev Sci 19, 295–305 (2018). https://doi.org/10.1007/s11121-016-0648-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11121-016-0648-1

Keywords

Navigation