Skip to main content

Advertisement

SpringerLink
Log in

Registry versus publication: discrepancy of primary outcomes and possible outcome reporting bias in child and adolescent mental health

  • Original Contribution
  • Published:
European Child & Adolescent Psychiatry Aims and scope Submit manuscript

Abstract

Outcome reporting bias is one of the fundamental forms of publication bias. It implies publishing only outcomes that have positive results. The aim of this observational study was to explore primary outcome discrepancies between registry of clinical trials and their corresponding publications, since these can indicate outcome reporting bias in child mental health. Data were extracted from completed interventional clinical trials from ClinicalTrial.gov registry and its Archive site. Trials were registered under “Behaviours and Mental Disorders” category, and conducted on underage participants (0–17 years). Their primary outcomes were compared to those published in publication which had a corresponding NCT number stated in the text. Sixteen percent of trials did not have the minimum information on primary outcome stated in the registry—neither the measure used nor the measurement time points; 38.9% of trials had the minimum information stated to describe primary outcome, while only 3.3% of trials had all the necessary elements stated in the registry. Most of the publication in our sample had positive results (66.4%). Half of the trials registered before completion had non-matching primary outcomes in the registry and publication; 85.4% of trials with non-matching outcomes indicated possible outcome reporting bias for some of the primary outcome. Middle-sized trials and industry-funded trials were related with higher quality of primary outcome registration. Industry funding was related with positive findings in publication. Non-industry funding proved to be the only significant predictor of discrepancy between registered and published primary outcomes, and possible outcome reporting bias. Journal impact factor was not related with any of the outcome measures. The main limitation of the study is that it primarily offers an insight into discrepancy of registered and published outcomes. The methodology does not imply an access to results of unpublished outcomes — therefore, it was not possible to determine the presence of the bias with sufficient certainty in large number of trials. Further research should be done with improved methodology and additional data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Code availability

Not applicable.

References

  1. Catalogue of Bias Collaboration. https://catalogofbias.org/about/. Accessed 27 Oct 2020

  2. Dal-Re R, Bobes J, Cuijpers P (2017) Why prudence is needed when interpreting articles reporting clinical trial results in mental health. Trials 18(1):143. https://doi.org/10.1186/s13063-017-1899-2

    Article  PubMed  PubMed Central  Google Scholar 

  3. Editors ICoMJ (2019) Recommendations for the conduct, reporting, editing, and publication of scholarly work in Medical Journals

  4. Milette K, Roseman M, Thombs BD (2011) Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: a systematic review. J Psychosom Res 70(3):205–217. https://doi.org/10.1016/j.jpsychores.2010.09.015

    Article  PubMed  Google Scholar 

  5. Raghav KP, Mahajan S, Yao JC, Hobbs BP, Berry DA, Pentz RD, Tam A, Hong WK, Ellis LM, Abbruzzese J, Overman MJ (2015) From protocols to publications: a study in selective reporting of outcomes in randomized trials in oncology. J Clin Oncol Off J Am Soc Clin Oncol 33(31):3583–3590. https://doi.org/10.1200/JCO.2015.62.4148

    Article  Google Scholar 

  6. Li XQ, Yang GL, Tao KM, Zhang HQ, Zhou QH, Ling CQ (2013) Comparison of registered and published primary outcomes in randomized controlled trials of gastroenterology and hepatology. Scand J Gastroenterol 48(12):1474–1483. https://doi.org/10.3109/00365521.2013.845909

    Article  PubMed  Google Scholar 

  7. Smith SM, Wang AT, Pereira A, Chang RD, McKeown A, Greene K, Rowbotham MC, Burke LB, Coplan P, Gilron I, Hertz SH, Katz NP, Lin AH, McDermott MP, Papadopoulos EJ, Rappaport BA, Sweeney M, Turk DC, Dworkin RH (2013) Discrepancies between registered and published primary outcome specifications in analgesic trials: ACTTION systematic review and recommendations. Pain 154(12):2769–2774. https://doi.org/10.1016/j.pain.2013.08.011

    Article  PubMed  Google Scholar 

  8. Howard B, Scott JT, Blubaugh M, Roepke B, Scheckel C, Vassar M (2017) Systematic review: outcome reporting bias is a problem in high impact factor neurology journals. PLoS ONE 12(7):e0180986. https://doi.org/10.1371/journal.pone.0180986

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Roest AM, de Jonge P, Williams CD, de Vries YA, Schoevers RA, Turner EH (2015) Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. JAMA Psychiatry 72(5):500–510. https://doi.org/10.1001/jamapsychiatry.2015.15

    Article  PubMed  Google Scholar 

  10. Trinquart L, Abbe A, Ravaud P (2012) Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials. PLoS ONE 7(4):e35219. https://doi.org/10.1371/journal.pone.0035219

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Cuijpers P, Smit F, Bohlmeijer E, Hollon SD, Andersson G (2010) Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. Br J Psychiatry J Ment Sci 196(3):173–178. https://doi.org/10.1192/bjp.bp.109.066001

    Article  Google Scholar 

  12. Driessen E, Hollon SD, Bockting CL, Cuijpers P, Turner EH (2015) Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of health-funded trials. PLoS ONE 10(9):e0137864. https://doi.org/10.1371/journal.pone.0137864

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358(3):252–260. https://doi.org/10.1056/NEJMsa065779

    Article  CAS  PubMed  Google Scholar 

  14. Turner EH, Knoepflmacher D, Shapley L (2012) Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Med 9(3):e1001189. https://doi.org/10.1371/journal.pmed.1001189

    Article  PubMed  PubMed Central  Google Scholar 

  15. Association WM (2013) WMA Declaration of Helsinki—ethical principles for medical research involving human subjects. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. Accessed 15 July 2020

  16. Thomas ET HC (2017) Catalogue of Bias Collaboration. Outcome reporting bias. In: Catalogue Of Biases 2017. www.catalogueofbiases.org/outcomereportingbias. Accessed 27 Oct 2020

  17. Bradley HA, Rucklidge JJ, Mulder RT (2017) A systematic review of trial registration and selective outcome reporting in psychotherapy randomized controlled trials. Acta Psychiatr Scand 135(1):65–77. https://doi.org/10.1111/acps.12647

    Article  CAS  PubMed  Google Scholar 

  18. Shinohara K, Tajika A, Imai H, Takeshima N, Hayasaka Y, Furukawa TA (2015) Protocol registration and selective outcome reporting in recent psychiatry trials: new antidepressants and cognitive behavioural therapies. Acta Psychiatr Scand 132(6):489–498. https://doi.org/10.1111/acps.12502

    Article  CAS  PubMed  Google Scholar 

  19. Knuppel H, Metz C, Meerpohl JJ, Strech D (2013) How psychiatry journals support the unbiased translation of clinical research. A cross-sectional study of editorial policies. PLoS ONE 8(10):e75995. https://doi.org/10.1371/journal.pone.0075995

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Mathieu S, Chan AW, Ravaud P (2013) Use of trial register information during the peer review process. PLoS ONE 8(4):e59910. https://doi.org/10.1371/journal.pone.0059910

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. van Lent M, IntHout J, Out HJ (2014) Differences between information in registries and articles did not influence publication acceptance. J Clin Epidemiol 68(2015):1059–1067

    PubMed  Google Scholar 

  22. Vrljicak Davidovic N, Tokalic R, Burilovic E, Pejdo S, Marusic A, Singh S, Franic T (2019) Low dissemination rates, non-transparency of trial premature cessation and late registration in child mental health: observational study of registered interventional trials. Eur Child Adolesc Psychiatry. https://doi.org/10.1007/s00787-019-01392-8

    Article  PubMed  Google Scholar 

  23. Hannink G, Gooszen HG, Rovers MM (2013) Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg 257(5):818–823. https://doi.org/10.1097/SLA.0b013e3182864fa3

    Article  PubMed  Google Scholar 

  24. van Lent M, Overbeke J, Out HJ (2013) Recommendations for a uniform assessment of publication bias related to funding source. BMC Med Res Methodol 13:120. https://doi.org/10.1186/1471-2288-13-120

    Article  PubMed  PubMed Central  Google Scholar 

  25. Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, Hrobjartsson A, Mann H, Dickersin K, Berlin JA, Dore CJ, Parulekar WR, Summerskill WS, Groves T, Schulz KF, Sox HC, Rockhold FW, Rennie D, Moher D (2013) SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med 158(3):200–207. https://doi.org/10.7326/0003-4819-158-3-201302050-00583

    Article  PubMed  PubMed Central  Google Scholar 

  26. Reyes MM, Panza KE, Martin A, Bloch MH (2011) Time-lag bias in trials of pediatric antidepressants: a systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry 50(1):63–72. https://doi.org/10.1016/j.jaac.2010.10.008

    Article  PubMed  Google Scholar 

  27. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P (2009) Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 302(9):977–984. https://doi.org/10.1001/jama.2009.1242

    Article  CAS  PubMed  Google Scholar 

  28. Pica N, Bourgeois F (2016) Discontinuation and nonpublication of randomized clinical trials conducted in children. Pediatrics. https://doi.org/10.1542/peds.2016-0223

    Article  PubMed  PubMed Central  Google Scholar 

  29. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF (2013) Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 347:f6104. https://doi.org/10.1136/bmj.f6104

    Article  PubMed  PubMed Central  Google Scholar 

  30. Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias G (2013) Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS ONE 8(7):e66844. https://doi.org/10.1371/journal.pone.0066844

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Sune P, Sune JM, Montoro JB (2013) Positive outcomes influence the rate and time to publication, but not the impact factor of publications of clinical trial results. PLoS ONE 8(1):e54583. https://doi.org/10.1371/journal.pone.0054583

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. van Lent M, Overbeke J, Out HJ (2014) Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals. PLoS ONE 9(8):e104846. https://doi.org/10.1371/journal.pone.0104846

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Walker KF, Stevenson G, Thornton JG (2014) Discrepancies between registration and publication of randomised controlled trials: an observational study. JRSM Open 5(5):2042533313517688. https://doi.org/10.1177/2042533313517688

    Article  PubMed  PubMed Central  Google Scholar 

  34. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3(8):e3081. https://doi.org/10.1371/journal.pone.0003081

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Bourgeois FT, Murthy S, Mandl KD (2010) Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med 153(3):158–166. https://doi.org/10.7326/0003-4819-153-3-201008030-00006

    Article  PubMed  PubMed Central  Google Scholar 

  36. Smyth RM, Jacoby A, Altman DG, Gamble C, Williamson PR (2015) The natural history of conducting and reporting clinical trials: interviews with trialists. Trials 16:16. https://doi.org/10.1186/s13063-014-0536-6

    Article  PubMed  PubMed Central  Google Scholar 

  37. Anand V, Scales DC, Parshuram CS, Kavanagh BP (2014) Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med 40(5):700–722. https://doi.org/10.1007/s00134-014-3250-7

    Article  PubMed  Google Scholar 

  38. Cybulski L, Mayo-Wilson E, Grant S (2016) Improving transparency and reproducibility through registration: the status of intervention trials published in clinical psychology journals. J Consult Clin Psychol 84(9):753–767. https://doi.org/10.1037/ccp0000115

    Article  PubMed  Google Scholar 

  39. Viergever RF, Karam G, Reis A, Ghersi D (2014) The quality of registration of clinical trials: still a problem. PLoS ONE 9(1):e84727. https://doi.org/10.1371/journal.pone.0084727

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Perlmutter AS, Tran VT, Dechartres A, Ravaud P (2017) Statistical controversies in clinical research: comparison of primary outcomes in protocols, public clinical-trial registries and publications: the example of oncology trials. Ann Oncol Off J Eur Soc Med Oncol 28(4):688–695. https://doi.org/10.1093/annonc/mdw682

    Article  CAS  Google Scholar 

  41. Grant S, Mayo-Wilson E, Montgomery P, Macdonald G, Michie S, Hopewell S, Moher D, Group obotC-SPI (2018) CONSORT-SPI 2018 Explanation and elaboration: guidance for reporting social and psychological intervention trials. Trials 19(1):406. https://doi.org/10.1186/s13063-018-2735-z

    Article  Google Scholar 

  42. Ramagopalan S, Skingsley AP, Handunnetthi L, Klingel M, Magnus D, Pakpoor J, Goldacre B (2014) Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study. F1000Research 3:77. https://doi.org/10.12688/f1000research.3784.1

    Article  PubMed  PubMed Central  Google Scholar 

  43. Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR (2011) Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database System Rev. https://doi.org/10.1002/14651858.MR000031.pub2

    Article  Google Scholar 

  44. Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF (2015) Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med 13:282. https://doi.org/10.1186/s12916-015-0520-3

    Article  PubMed  PubMed Central  Google Scholar 

  45. Zhang S, Liang F, Li W (2017) Comparison between publicly accessible publications, registries, and protocols of phase III trials indicated persistence of selective outcome reporting. J Clin Epidemiol 91:87–94. https://doi.org/10.1016/j.jclinepi.2017.07.010

    Article  PubMed  Google Scholar 

  46. Dal-Re R, Bracken MB, Ioannidis JP (2015) Call to improve transparency of trials of non-regulated interventions. BMJ 350:h1323. https://doi.org/10.1136/bmj.h1323

    Article  PubMed  Google Scholar 

  47. Shepshelovich D, Goldvaser H, Wang L, Abdul Razak AR, Bedard PL (2017) Comparison of reporting phase I trial results in ClinicalTrials.gov and matched publications. Investig New Drugs 35(6):827–833. https://doi.org/10.1007/s10637-017-0510-8

    Article  CAS  Google Scholar 

  48. Vanclay JK (2012) Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics. https://doi.org/10.1007/s11192-011-0561-0

    Article  Google Scholar 

  49. Smyth RM, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR (2011) Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ 342:c7153. https://doi.org/10.1136/bmj.c7153

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Wayant C, Scheckel C, Hicks C, Nissen T, Leduc L, Som M, Vassar M (2017) Evidence of selective reporting bias in hematology journals: a systematic review. PLoS ONE 12(6):e0178379. https://doi.org/10.1371/journal.pone.0178379

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Ioannidis JP, Caplan AL, Dal-Re R (2017) Outcome reporting bias in clinical trials: why monitoring matters. BMJ 356:j408. https://doi.org/10.1136/bmj.j408

    Article  PubMed  Google Scholar 

  52. Goldacre B, Drysdale H, Powell-Smith A, Dale A, Milosevic I, Slade E, Hartley P, Marston C, Mahtani K, Heneghan C (2016) The COMPare Trials Project. www.COMPare-trials.org. Accessed 15 July 2020

  53. Chan AW, Krleza-Jeric K, Schmid I, Altman DG (2004) Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ Can Med Assoc J J Assoc Med Can 171(7):735–740. https://doi.org/10.1503/cmaj.1041086

    Article  Google Scholar 

  54. Schulz KF, Altman DG, Moher D, Group C (2010) CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 63(8):834–840. https://doi.org/10.1016/j.jclinepi.2010.02.005

    Article  Google Scholar 

  55. Gopal AD, Wallach JD, Aminawung JA, Gonsalves G, Dal-Re R, Miller JE, Ross JS (2018) Adherence to the International Committee of Medical Journal Editors’ (ICMJE) prospective registration policy and implications for outcome integrity: a cross-sectional analysis of trials published in high-impact specialty society journals. Trials 19(1):448. https://doi.org/10.1186/s13063-018-2825-y

    Article  PubMed  PubMed Central  Google Scholar 

  56. Meerpohl JJ, Schell LK, Bassler D, Gallus S, Kleijnen J, Kulig M, La Vecchia C, Marusic A, Ravaud P, Reis A, Schmucker C, Strech D, Urrutia G, Wager E, Antes G, consortium Op, (2015) Evidence-informed recommendations to reduce dissemination bias in clinical research: conclusions from the OPEN (Overcome failure to Publish nEgative fiNdings) project based on an international consensus meeting. BMJ Open 5(5):e006666. https://doi.org/10.1136/bmjopen-2014-006666

    Article  PubMed  PubMed Central  Google Scholar 

  57. AllTrials (2013) The AllTrials campaign. www.alltrials.net. Accessed 15 July 2020

  58. Bauchner H, Golub RM, Fontanarosa PB (2019) Reporting and interpretation of randomized clinical trials. JAMA 322(8):732–735. https://doi.org/10.1001/jama.2019.12056

    Article  PubMed  Google Scholar 

  59. Dal-Re R, Caplan AL (2015) Journal editors impasse with outcome reporting bias. Eur J Clin Invest 45(9):895–898. https://doi.org/10.1111/eci.12484

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Authors received no funding for this research.

Funding

The authors did not receive funding for this research.

Author information

Authors and Affiliations

  1. School of Medicine, University of Split, Šoltanska 2, 21000, Split, Croatia

    Nikolina Vrljičak Davidović, Luka Komić, Ivana Mešin, Mihaela Kotarac, Donald Okmažić & Tomislav Franić

  2. Department of Psychiatry, Clinical Hospital Centre Split, Spinčićeva 1, 21000, Split, Croatia

    Tomislav Franić

Authors
  1. Nikolina Vrljičak Davidović
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luka Komić
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivana Mešin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mihaela Kotarac
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Donald Okmažić
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomislav Franić
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

NVD and TF made a study conception and design. Data collection were performed by DO, MK, IM, and LK, and analysis was performed by NVD. The first draft of the manuscript was written by NVD and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Nikolina Vrljičak Davidović.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Availability of data and materials (data transparency)

All data used are available on the Internet. The corresponding author will provide a database on request.

Ethical standards

The manuscript does not contain clinical studies or patient data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vrljičak Davidović, N., Komić, L., Mešin, I. et al. Registry versus publication: discrepancy of primary outcomes and possible outcome reporting bias in child and adolescent mental health. Eur Child Adolesc Psychiatry 31, 757–769 (2022). https://doi.org/10.1007/s00787-020-01710-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00787-020-01710-5

Keywords

Search

Navigation