Prevention Science

, Volume 19, Issue 3, pp 295–305 | Cite as

Can We Trust Positive Findings of Intervention Research? The Role of Conflict of Interest

Article

Abstract

In recent years, there has been increased attention to the issue of conflict of interest within prevention research. The aims of this paper are to discuss these developments and to relate them to discussions of conflict of interest in the broader scientific literature. Although there has been concern expressed about the extent to which conflicts of interest can be defined and measured, empirical research suggests that financial conflicts can be easily identified and assessed in meta-analyses focused on their effects on research quality. Research evidence also shows that conflict of interest is associated with use of flexible data analysis practices and the reporting of chance positive findings, both within prevention research and related disciplines such as public health and psychology. However, the overwhelming majority of published studies report positive results, and there are a number of other influences within academia (such as pressure to publish) that account for this and for the use of flexible data analysis practices. Accordingly, introducing measures to improve research quality in general, rather than just focusing on problems specific to research in which there is a clearly identifiable conflict of interest, may prove more effective and less controversial. Most such efforts focus on introducing greater transparency into research design, practice, and reporting. These both curtail employment of flexible data analysis practices and make their use transparent to investigators seeking to assess their effects on research quality. Also, requiring detailed disclosures of conflicts be reported by all investigators (not just senior authors) would improve current disclosure practices.

Keywords

Prevention Conflict of interest Research quality Research transparency 

Notes

Compliance with Ethical Standards

Conflict of Interest

The author declares that he has no conflict of interest.

Funding

None

Ethical Approval

The article does not contain any studies with human participants or animals performed by the author.

Informed Consent

Non-applicable, as no human subjects.

References

  1. Australian New Zealand Clinical Trial Registry. (2015). ANZCTR. Retrieved from http://www.anzctr.org.au
  2. Berkeley Initiative for Transparency in the Social Sciences. (2015). BITSS Catalyst Program: Pilot 2015–2018. Retrieved from: http://www.bitss.org/wp-content/uploads/2015/12/bitss_catalyst_program_cn.pdf
  3. Bierman, K. L., Nix, R. L., Maples, J. J., Murphy, S. A., & Conduct Problems Prevention Research Group. (2006). Examining clinical judgment in an adaptive intervention design: The Fast Track Program. Journal of Consulting and Clinical Psychology, 74, 468–481.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bierman, K.L., Coie, J., Dodge, K., Greenberg, M., Lochman, J., McMahon, R., Pinderhughes, E., & Conduct Problems Prevention Research Group. (2013). School outcomes of aggressive-disruptive children: Prediction from kindergarten risk factors and impact of the Fast Track Prevention Program. Aggressive Behavior, 39: doi:10.1002/ab.21467Google Scholar
  5. BioMed Central. (2015). ISRCTN Registry. Retrieved from http://www.isrctn.com
  6. Bosch, X., Pericas, J. M., Hernandez, C., & Doti, P. (2013). Financial, nonfinanacial and editors’ conflicts of interest in high-impact biomedical journals. European Journal of Clinical Investigation, 43, 660–667.CrossRefPubMedGoogle Scholar
  7. Campbell, D. T. (1984). Can we be scientific in applied social science? In R. F. Connor, D. G. Altman, & C. Jackson (Eds.), Evaluation Studies: Review Annual (Vol. 9, pp. 26–48). Beverly Hills, CA: Sage.Google Scholar
  8. Center for Open Science. (n.d.) The $1,000,000 Preregistration Challenge: Eligible Journals. Retrieved from https://cos.io/prereg/
  9. Committee on Publication Ethics. (2011). Code of Conduct and Best Practices Guidelines for Journal Editors. Retrieved from http://publicationethics.org/files/Code_of_conduct_for_journal_editors_Mar11.pdf.
  10. Conduct Problems Prevention Research Group. (2007). Fast Track randomized controlled trial to prevent externalizing psychiatric disorders: Findings from grades 3 to 9. Journal of the American Academy of Child and Adolescent Psychiatry, 46, 1250–1262.CrossRefPubMedCentralGoogle Scholar
  11. Conduct Problems Prevention Research Group. (2010a). Fast Track intervention effects on youth arrests and delinquency. Journal of Experimental Criminology, 6, 131–157.Google Scholar
  12. Conduct Problems Prevention Research Group. (2010b). The difficulty of maintaining positive intervention effects: A look at disruptive behavior, deviant peer relations, and social skills during the middle school years. Journal of Early Adolescence, 30, 593–624.Google Scholar
  13. Conduct Problems Prevention Research Group. (2010c). The effects of a multi-year universal social-emotional learning program: The role of student and school characteristics. Journal of Consulting and Clinical Psychology, 78,156-168.Google Scholar
  14. Conduct Problems Prevention Research Group. (2011). The effects of the Fast Track preventive intervention on the development of conduct disorder across childhood. Child Development, 82, 331–345.CrossRefPubMedCentralGoogle Scholar
  15. Conduct Problems Prevention Research Group. (2013). Assessing findings from the fast track study. Journal of Experimental Criminology, 9, 119–126.CrossRefPubMedCentralGoogle Scholar
  16. Conduct Problems Prevention Research Group. (2014). Trajectories of risk for early sexual activity and early substance use in the Fast Track Prevention Program. Prevention Science, 15, S33–S46.CrossRefGoogle Scholar
  17. Conduct Problems Prevention Research Group. (2015). Impact of early intervention on psychopathology, crime, and well-being at age 25. American Journal of Psychiatry, 172, 59–70.CrossRefGoogle Scholar
  18. CONSORT. (2015). Endorsers: Journals and Organizations. Retrieved from: http://www.consort-statement.org/about-consort/endorsers.Google Scholar
  19. Dodge, K. A., Godwin, J., & the Conduct Problems Prevention Research Group. (2013). Social-information-processing patterns mediate the impact of preventive intervention on adolescent antisocial behavior. Psychological Science, 24, 456–465.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Eisner, M. (2009). No effect in independent prevention trials: Can we reject the cynical view? Journal of Experimental Criminology, 5, 163–183.CrossRefGoogle Scholar
  21. Eisner, M. (2014). The South Carolina Triple P System Population Trial to Prevent Child Maltreatment: Seven reasons to be skeptical about the study results (Violence Research Centre - Working Paper. Institute of Criminology, University of Cambridge).Google Scholar
  22. Eisner, M., & Humphreys, D. (2011). Measuring conflict of interest in prevention and intervention research: A feasibility study. In T. Bliesener, A. Beelman, & M. Stemmler (Eds.), Antisocial behavior and crime: Contributions of developmental and evaluation research to prevention and intervention (pp. 165–180). Cambridge, MA: Hogrefe.Google Scholar
  23. Eisner, M. P., Humphreys, D. K., Wilson, P., & Gardner, F. (2015). Disclosure of financial conflicts of interest in interventions to improve child psychosocial health: A cross-sectional study. PloS ONE, 10, e0142803. doi: 10.1371/journal.pone.0142803.CrossRefPubMedPubMedCentralGoogle Scholar
  24. Fanelli, D. (2010a). “Positive” results increase down the hierarchy of science. PLoS ONE, 5, e10068. doi: 10.1371/journal.pone.0010068.
  25. Fanelli, D. (2010b). Do pressures to publish increase scientists’ bias? An empirical support for US States data. PLoS ONE, 5, e10271. doi: 10.1371/journal.pone.0010271.
  26. Fanelli, D. (2013). Redefine misconduct as distorted reporting. Nature, 494, 149.CrossRefPubMedGoogle Scholar
  27. Fast Track Project. (2011). Publications. Retrieved from http://www.fasttrackproject.org/publications.php
  28. Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7, 555–561.CrossRefPubMedGoogle Scholar
  29. Fleming, P. S., Koletsi, D., Dwan, K., & Pamdis, N. (2015). Outcome discrepancies and selective reporting: Impacting the leading journals? PLoS One, 10, e0127495. doi: 10.1371/journal.pone.0127495.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Foster, E. M. (2013). Reassessing findings from the Fast Track study: problems of method and analysis. Journal of Experimental Criminology, 9, 109–117.CrossRefGoogle Scholar
  31. Foster, E. M., Jones, D. E., & the Conduct Problems Prevention Research Group. (2006). Can a costly intervention be cost-effective? An analysis of violence prevention. Archives of General Psychiatry, 63, 1284–1291.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Goozner, M., Caplan, A., Moreno, J., Kramer, B. S., Babor, T. F., & Husser, W. C. (2009). A common standard for conflict of interest disclosure in addiction journals. Addiction, 104, 1779–1784.CrossRefPubMedGoogle Scholar
  33. Gorman, D. M. (2014). Is Project Towards No Drug Abuse (Project TND) an evidence-based drug and violence prevention program? A review and reappraisal of the evaluation studies. Journal of Primary Prevention, 35, 217–232.CrossRefPubMedGoogle Scholar
  34. Gorman, DM. (2015a). ‘Everything works’: The need to address confirmation bias in evaluations of drug misuse prevention interventions for adolescents. Addiction, 110, 1539–40Google Scholar
  35. Gorman, DM. (2015b). Analytic flexibility in the evaluation of the Drug Education in Victoria Schools (DEVS) programme. International Journal of Drug Policy, 26, 719–720Google Scholar
  36. Gorman, D. M., & Conde, E. (2007). Conflict of interest in the evaluation and dissemination of “model” school-based drug and violence prevention programs. Evaluation and Program Planning, 30, 422–429.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Gorman, D. M., & Conde, E. (2010). The making of evidence-based practice: The case of Project ALERT. Children and Youth Services Review, 32, 214–222.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Holder, H. D. (2010). Prevention programs in the 21st century: What we do not discuss in public. Addiction, 105, 578–581.CrossRefPubMedGoogle Scholar
  39. Institute of Medicine. (2002). Integrity in scientific research: Creating an environment that promotes responsible conduct. Washington, DC: National Academies Press.Google Scholar
  40. International Committee of Medical Journal Editors. (2015). Conflicts of interest. Retrieved from http://www.icmje.org/conflicts-of-interest/http://www.icmje.org/conflicts-of-interest/
  41. Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani R. (2014a). Increasing value and reducing waste in research design, conduct, and analysis. Lancet, 383, 166–175.Google Scholar
  42. Ioannidis, J. P. A., Munafo, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014b). Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention. Trends in Cognitive Sciences, 18, 235–241.Google Scholar
  43. Jones, D. E., Godwin, J., Dodge, K. A., Bierman, K. L., Coie, J. D., Greenberg, M. T., & Pinderhughes, E. E. (2010). Impact of the Fast Track Prevention Program on health services use by conduct-problem youth. Pediatrics, 125, e130–e136.CrossRefPubMedGoogle Scholar
  44. Kelling, G. L., Pate, T., Dieckman, D., & Brown, C. (1974). The Kansas City preventive patrol experiment: Summary report. Washington, DC: Police Foundation.Google Scholar
  45. Lipsey, M. (1995). What do we learn from 400 research studies on effectiveness of treatment with juvenile delinquents? In J. McGuiren (Ed.), What works? Reducing reoffending (pp. 63–78). New York: Wiley.Google Scholar
  46. MacCoun, R. (2005). Conflicts of interest in public policy. In D. A. Moore, D. M. Cain, G. Loewenstein, & M. Bazerman (Eds.), Conflicts of interest: Challenges and solutions in business, law, medicine, and public policy (pp. 233–262). London: Cambridge University Press.CrossRefGoogle Scholar
  47. Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K. M., Gerber, A., & Van der Laan, M. (2014). Promoting transparency in social science research. Science, 343, 30–31.CrossRefPubMedPubMedCentralGoogle Scholar
  48. Nix, R. L., Bierman, K. L., McMahon, R. J., & the Conduct Problems Prevention Research Group. (2009). How attendance and quality of participation affect treatment response to parent management training. Journal of Consulting and Clinical Psychology, 77, 429–438.CrossRefPubMedPubMedCentralGoogle Scholar
  49. Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., & Yarkoni, T. (2015). Promoting an open research culture. Science, 348, 1422–1425.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific Utopia II. Restructuring incentives and practices to promote truth over publishability. Psychological Science, 7, 615–631.Google Scholar
  51. Okulicz-Kozaryn, K., & Foxcroft, D. R. (2012). Effectiveness of the Strengthening Families Programme 10–14 in Poland for the prevention of alcohol and drug misuse: Protocol for a randomized controlled trial. BMC Public Health, 12, 319. doi: 10.1186/1471-2458-12-319.CrossRefPubMedPubMedCentralGoogle Scholar
  52. Pasalich, D. S., Witkiewitz, K., McMahon, R. J., Pinderhughes, E. E., & the Conduct Problems Prevention Research Group. (2016). Indirect effects of the Fast Track intervention on conduct disorder symptoms and callous-unemotional traits: Distinct pathways involving discipline and warmth. Journal of Abnormal Child Psychology, 44, 587–597.CrossRefPubMedPubMedCentralGoogle Scholar
  53. Popper, K. (1974). Replies to my critics. In P. A. Schilpp (Ed.), The Philosophy of Karl Popper (pp. 961–1197). La Salle, Il: Open Court.Google Scholar
  54. Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 Statement: Updated guidelines for reporting parallel group randomised trials. BMC Medicine, 18, 18. doi: 10.1186/1741-7015-8-18.CrossRefGoogle Scholar
  55. Sherman, L. W. (2015). A tipping point for “totally evidenced policing”: Ten ideas for building an evidence-based police agency. International Criminal Justice Review, 25, 11–29.CrossRefGoogle Scholar
  56. Sherman, L. W., & Strang, H. (2009). Testing for analysts’ bias in crime prevention experiments: Can we accept Eisner’s one-tailed test? Journal of Experimental Criminology, 5, 185–200.CrossRefGoogle Scholar
  57. Slough, N. M., McMahon, R. J., & the Conduct Problems Prevention Research Group. (2008). Preventing serious conduct problems in school-age youth: The Fast Track Program. Cognitive and Behavioral Practice, 15, 3–17.CrossRefPubMedPubMedCentralGoogle Scholar
  58. Society for Prevention Research. (2015). Conflict of interest policy. Retrieved from http://www.preventionresearch.org/about-spr/conflict-of-interest-policy/
  59. Sterman, S. (2000). Business dynamics: Systems thinking and modeling for a complex world. New York: McGraw Hill.Google Scholar
  60. Sussman, S. (2009). Conflict of interest disclosure statement. Society for Prevention Research 17th Annual Meeting: Program Book, 121. Retrieved from http://www.preventionresearch.org/wp-content/uploads/2011/07/SPR2009program.pdf
  61. Tobin, M. J. (2003). CoI and AJRCCM: Restating policy and a new form to upload. American Journal of Respiratory and Critical Care Medicine, 167, 1161–1164.CrossRefPubMedGoogle Scholar
  62. US National Institutes of Health. (2015). ClinicalTrials.gov. Retrieved from https://clinicaltrials.gov.
  63. Welsh, B. C., Braga, A. A., & Hollis-Peel, E. (2012). Can “disciplined passion” overcome the cynical view? An empirical enquiry of evaluator influence on police crime prevention program outcomes. Journal of Experimental Criminology, 8, 415–431.CrossRefGoogle Scholar

Copyright information

© Society for Prevention Research 2016

Authors and Affiliations

  1. 1.Department of Epidemiology and BiostatisticsSchool of Public Health, Texas A&M UniversityCollege StationUSA

Personalised recommendations