Intensive Care Medicine

, Volume 40, Issue 3, pp 468–468 | Cite as

Research accomplishments that are too good to be true: reply to Ting

  • John P. A. Ioannidis


Clinical Research Biomedical Research Research Agenda Research Enterprise Safety Monitoring 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Dear Editor,

Ting makes three very interesting suggestions that may help improve the fidelity of biomedical research, decrease misconduct, and improve the consequences of the uptake of evidence in medical practice [1, 2]. I think that all his suggestions are definitely worthy of consideration and they may well be valuable. I only have a few comments.

First, data and safety monitoring boards (DSMBs) may indeed play a critical role in trial conduct and they may help ensure that data are properly collected and analyses are appropriately represented. However, only a tiny minority of clinical research is supervised by DSMBs. Even in trials with DSMBs, usually the DSMB members are not involved in the writing of the final manuscript. Perhaps, there can be some closer involvement of the DSMB in the preparation and publication of manuscripts. However, introducing DSMBs for each and all of the millions of biomedical studies may be very difficult, if not impossible.

Second, I fully agree with Ting about the importance of thorough declaration of conflicts of interest and the difficulty in appreciating subtle but influential conflicts. Going beyond declaration, it may be preferable to foster a research agenda where clinical research, including randomized trials, meta-analyses, and guidelines are performed primarily, if not entirely, by investigators without conflicts of interest [3]. This requires some brave re-engineering of the clinical research enterprise and its funding structure, but in the long term it may be worth it.

Third, I am also strongly in favor of post-publication review and transparent post-publication scrutiny. Several paradigms exist or are being developed, e.g., PubMed Commons, to serve this purpose. However, I suspect that post-publication review and commenting may become largely an exchange of opinions and biases, unless it is supplanted with access to raw data, protocols, and analyses codes. Such access would allow post-publication reviewers and replicators to make more meaningful contributions to scientific reproducibility and clinical implementation [4, 5].

We need to understand what the optimal process is to introduce these changes, since there is also the risk of performing all these changes and simply ending with a sterile, over-regulated, stifled research enterprise and vain, contrarian re-analyses that are done simply to show that the original authors are wrong or specific opinions are correct [6], without really advancing science or helping patients. Piloting of structural changes and experimental studies of different changes to the conduct of science would be useful in this regard.


  1. 1.
    Ioannidis JPA (2014) Research accomplishments that are too good to be true. Intensive Care Med 40:99–101. doi: 10.1007/s00134-013-3100-z PubMedCrossRefGoogle Scholar
  2. 2.
    Ting JY (2014) Research accomplishments that are too good to be true: comment. Intensive Care Med. doi: 10.1007/s00134-014-3207-2
  3. 3.
    Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP (2013) Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ 347:f5535PubMedCrossRefGoogle Scholar
  4. 4.
    Doshi P, Goodman SN, Ioannidis JPA (2013) Raw data from clinical trials: within reach? Trends Pharmacol Sci 34:645–647PubMedCrossRefGoogle Scholar
  5. 5.
    Doshi P, Jefferson T, Del Mar C (2012) The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med 9:e1001201PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Christakis D, Zimmerman FJ (2103) Rethinking reanalysis. JAMA 310:2499–2500CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg and ESICM 2014

Authors and Affiliations

  1. 1.Meta-Research Innovation Center at StanfordStanford Prevention Research CenterStanfordUSA

Personalised recommendations