Blum B. Information systems for patient care. New York: Springer; 1984.
Rowley J. The wisdom hierarchy: representations of the DIKW hierarchy. J Inf Sci. 2007;33:163–80.
Losee R. The science of information. San Diego, CA: Academic Press; 1990.
Stone J. Information theory – a tutorial introduction. Sebtel Press; 2015.
Shannon C, Weaver W. The mathematical theory of communication. Urbana, IL: University of Illinois Press; 1949.
Robison R. How big is the human genome? Medium. 2014 January 5, 2014.
Bar-Hillel Y, Carnap R. Semantic information. Br J Philos Sci. 1953;4:147–57.
Belis M, Guiasu S. A quantitative-qualitative measure of information in cybernetic systems. IEEE Trans Inf Theory. 1968;14:593–4.
Price D. Little science, big science. New York: Columbia University Press; 1963.
Pao M. Concepts of information retrieval. Englewood, CO: Libraries Unlimited; 1989.
Molyneux R. ACRL University Library Statistics. Chicago: Association of Research Libraries; 1989.
Jinha A. Article 50 million: an estimate of the number of scholarly articles in existence. Learned Publishing. 2010;23:258–63.
Ware M, Mabe M. The STM report. An overview of scientific and scholarly journal publishing. Oxford: International Association of Scientific, Technical and Medical Publishers2009 September, 2009.
Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326.
Durack D. The weight of medical knowledge. N Engl J Med. 1978;298:773–5.
Madlon-Kay D. The weight of medical knowledge: still gaining. N Engl J Med. 1989;321:908.
Larsen P, von Ins M. The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics. 2010;84:575–603.
Bornmann L, Mutz R. Growth rates of modern science: a bibliometric analysis based on the number of publications and cited references. J Am Soc Inf Sci Technol. 2015;66:2215–22.
King C. Multiauthor papers: onward and upward. Science Watch 2012 July, 2012.
Weber M. The effects of listing authors in alphabetical order: a review of the empirical evidence. Research Evaluation. 2018;27:238–45.
Ioannidis J, Klavans R, Boyack K. The scientists who publish a paper every five days. Nature. 2018;561:167–9.
Littenberg B. Technology assessment in medicine. Acad Med. 1992;67:424–8.
Arnett D, Blumenthal R, Albert M, Michos E, Buroker A, Williams K, et al. 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease. J Am Coll Cardiol. 2019;140:e596–646.
Poynard T, Munteanu M, Ratziu V, Benhamou Y, Martino VD, Taieb J, et al. Truth survival in clinical research: an evidence-based requiem? Ann Intern Med. 2002;136:888–95.
Shojania K, Sampson M, Ansari M, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147:224–33.
Beller E, Chen J, Wang U, Glasziou P. Are systematic reviews up-to-date at the time of publication? Syst Rev. 2013;2:36.
McLellan F. 1966 and all that – when is a literature search done? Lancet. 2001;358:646.
Antman E, Lau J, Kupelnick B, Mosteller F, Chalmers T. A comparison of results of meta-analyses of randomized controlled trials and recommendations of clinical experts: treatments for myocardial infarction. J Am Med Assoc. 1992;268:240–8.
Balas E, Boren S. Managing clinical knowledge for health care improvement. In: van Bemmel J, McCray A, editors. Yearbook of medical informatics. Stuttgart: Schattauer; 2000. p. 65–70.
Ziman J. Information, communication, knowledge. Nature. 1969;224:318–24.
Refinetti R. In defense of the least publishable unit. FASEB J. 1991;4:128–9.
Bornmann L, Daniel H. Multiple publication on a single research study: does it pay? The influence of number of research articles on total citation counts in biomedicine. J Am Soc Inf Sci Technol. 2007;58:1100–7.
Casadevall A, Fang F. Field science—the nature and utility of scientific fields. mBio. 2015;6(5):e01259–15.
Varga A. Shorter distances between papers over time are due to more cross-field references and increased citation rate to higher-impact papers. Proc Natl Acad Sci. 2019;116:22094–9.
Anonymous. Bibliometrics: an overview. Leeds, England: University of Leeds 2014 July, 2014.
Berger J, Baker C. Bibliometrics: an overview. RGUHS J Pharmaceut Sci. 2014;4(3):81–92.
Garfield E. “Science Citation Index” – a new dimension in indexing. Science. 1964;144:649–54.
Price D. Networks of scientific papers. Science. 1965;149:510–5.
Adam D. The counting house. Nature. 2002;415:726–8.
Ioannidis J, Boyack K, Small H, Sorensen A, Klavans R. Bibliometrics: is your most cited work your best? Nature. 2014;514:561–2.
Teplitskiy M, Duede E, Menietti M, Lakhani K. Citations systematically misrepresent the quality and impact of research articles: survey and experimental evidence from thousands of citers. arXivorg. 2020:arXiv:2002.10033.
Lawrence S. Free online availability substantially increases a paper’s impact. Nature. 2001;411:521.
Antelman K. Do open-access articles have a greater research impact? Coll Res Libr. 2004;65:372–82.
Eysenbach G. Citation advantage of open access articles. PLoS Biol. 2006;4(5):e157.
Piwowar H, Day R, Fridsma D. Sharing detailed research data is associated with increased citation rate. PLoS One. 2007;2(3).
Craig I, Plume A, McVeigh M, Pringle J, Amin M. Do open access articles have greater citation impact? A critical review of the literature. J Informet. 2007;1:239–48.
Aksnes D. Characteristics of highly cited papers. Res Eval. 2003;12:159–70.
Aksnes D. Citation rates and perceptions of scientific contribution. J Am Soc Inf Sci Technol. 2006;57:169–87.
Lozano G, Larivière V, Gingras Y. The weakening relationship between the impact factor and papers’ citations in the digital age. J Am Soc Inf Sci Technol. 2011;63:2140–5.
Acharya A, Verstak A, Suzuki H, S Henderson, Iakhiaev M, Chiung YuLin C et al. Rise of the rest: the growing impact of non-elite journals. arXivorg. 2014:arXiv:1410.2217.
Bornmann L, Ye A, Ye F. Identifying landmark publications in the long run using field-normalized citation data. J Doc. 2018;74:278–88.
Letchford A, Moat H, Preis T. The advantage of short paper titles. R Soc Open Sci. 2015;2(8):150266.
Letchford A, Preis T, Moat H. The advantage of simple paper abstracts. J Inform. 2016;10:1–8.
Greenberg S. How citation distortions create unfounded authority: analysis of a citation network. Br Med J. 2009;339:b2680.
Trinquart L, Johns D, Galea S. Why do we think we know what we know? A metaknowledge analysis of the salt controversy. Int J Epidemiol. 2016;45:251–60.
Vucetic S, Chanda A, Zhang S, Bai T, Maiti A. Peer assessment of CS doctoral programs shows strong correlation with faculty citations. Commun ACM. 2018;61(9):70–6.
Piwowar H, Priem J, Larivière V, Alperin J, Matthias L, Norlander B, et al. The state of OA: a large-scale analysis of the prevalence and impact of open access articles. PeerJ. 2018;6:e4375.
Larivière V, Gong K, Sugimoto C. Citations strength begins at home. Nature Index2018.
Leung P, Macdonald E, Stanbrook M, Dhalla IA, Juurlink D. A 1980 letter on the risk of opioid addiction. N Engl J Med. 2017;376:2194–5.
Porter J, Jick H. Addiction rare in patients treated with narcotics. N Engl J Med. 1980;302:123.
Koberlein B. The tale of a 1986 experiment that proved Einstein wrong. Forbes 2018 April 6, 2018.
Silvertooth E. Special relativity. Nature. 1986;322:590.
Silvertooth E. Experimental detection of the ether. Specul Sci Technol. 1986;10(1):3–7.
Dubin D. The most influential paper Gerard Salton never wrote. Libr Trends. 2004;52:748–64.
Pao M. An empirical examination of Lotka’s law. J Am Soc Inf Sci. 1986;37:26–33.
Bradford S. Documentation. Crosby Lockwood: London; 1948.
Urquhart J, Bunn R. A national loan policy for science serials. J Doc. 1959;15:21–5.
Trueswell R. Some behavioral patterns of library users: the 80/20 rule. Wilson Libr Bull. 1969;43:458–61.
Bates M. After the dot-bomb: getting Web information right this time. First Monday. 2002;7:7.
Self P, Filardo T, Lancaster F. Acquired immunodeficiency syndrome (AIDS) and the epidemic growth of its literature. Scientometrics. 1989;17:49–60.
Wilczynski N, Garg A, Haynes B, editors. A method for defining a journal subset for a clinical discipline using the bibliographies of systematic reviews. MEDINFO 2007 – Proceedings of the Twelfth World Congress on Health (Medical) Informatics; 2007; Brisbane: IOS Press.
Nash-Stewart C, Kruesi L, DelMar C. Does Bradford’s Law of Scattering predict the size of the literature in Cochrane Reviews? J Med Libr Assoc. 2012;100:135–8.
Venable G, Shepherd B, Loftis C, McClatchy S, Roberts M, Fillinger M, et al. Bradford’s law: identification of the core journals for neurosurgery and its subspecialties. J Neurosurg. 2016;124:569–79.
Garfield E. The impact factor. Current Contents. 1994;25:3–7.
West R. Impact factors need to be improved. Br Med J. 1996;313:1400.
Smith R. Commentary: the power of the unrelenting impact factor – is it a force for good or harm? Int J Epidemiol. 2006;35:1129–30.
Warraich H. Impact factor and the future of medical journals. Atlantica 2014 January 10, 2014.
Lee K, Schotland M, Bacchetti P, Bero L. Association of journal quality indicators with methodological quality of clinical research articles. J Am Med Assoc. 2002;287:2805–8.
Saha S, Saint S, Christakis D. Impact factor: a valid measure of journal quality? J Med Libr Assoc. 2003;91:42–6.
McKibbon K, Wilczynski N, Haynes R. What do evidence-based secondary journals tell us about the publication of clinically important articles in primary healthcare journals. BMC Med. 2004;2:33.
Garfield E. The history and meaning of the journal impact factor. J Am Med Assoc. 2006;295:90–3.
Dong P, Loh M, Mondry A. The “impact factor” revisited. Biomed Digi Libraries. 2005;2:7.
Browman H, Stergiou K. Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics Sci Environ Politics. 2008;8:1–3.
Simons K. The misused impact factor. Science. 2008;322:165.
Lawrence P. Lost in publication: how measurement harms science. Ethics Sci Environ Politics. 2008;8:9–11.
Radicchi F. In Science “there is no bad publicity”: Papers criticized in comments have high scientific impact. Scientific Reports. 2012;2:815.
Eyre-Walker A, Stoletzki N. The assessment of science: the relative merits of post-publication review, the impact factor, and the number of citations. PLoS Biol. 2013;11:e1001675.
Ingwersen P. The calculation of web impact factors. J Doc. 1998;54:236–43.
Noruzi A. The Web Impact Factor: a critical review. Electron Libr. 2006;24:490–500.
Thelwall M. What is this link doing here? Beginning a fine-grained process of identifying reasons for academic hyperlink creation. Inf Res. 2003;8:3.
Hirsch J. An index to quantify an individual's scientific research output. Proc Natl Acad Sci. 2005;102:16569–72.
Hersh W, Buckley C, Leone T, Hickam D, editors. OHSUMED: an interactive retrieval evaluation and new large test collection for research. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval; 1994; Dublin: Springer.
Kulkarni A, Aziz B, Shams I, Busse J. Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. J Am Med Assoc. 2009;302:1092–6.
Delgado-López-Cózar E, Robinson-García N, Torres-Salinas D. The Google Scholar experiment: how to index false papers and manipulate bibliometric indicators. J Am Soc Inf Sci Technol. 2014;65:446–54.
Egghe L. Theory and practise of the g-index. Scientometrics. 2006;69:131–52.
Callahan A, Winnenburg R, Shah N. U-Index, a dataset and an impact metric for informatics tools and databases. Scientific Data. 2018;5:180043.
Ioannidis J, Baas J, Klavans R, Boyack K. A standardized citation metrics author database annotated for scientific field. PLoS Biol. 2019;17(8):e3000384.
Hutchins B, Yuan X, Anderson J, Santangelo G. Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLoS Biol. 2016;14(9):e1002541.
Hutchins B, Baker K, Davis M, Diwersy M, Haque E, Harriman R, et al. The NIH Open Citation Collection: a public access, broad coverage resource. PLoS Biol. 2019;70(10):e3000385.
Lauer M, Roychowdhury D, Patel K, Walsh R, Pearson K. Marginal returns and levels of research grant support among scientists supported by the National Institutes of Health. bioRxiv. 2017.
Hall N. The Kardashian index: a measure of discrepant social media profile for scientists. Genome Biol. 2014;15:424.
Andrews J. An author co-citation analysis of medical informatics. J Med Libr Assoc. 2003;91:47–56.
Eggers S, Huang Z, Chen H, Yan L, Larson C, Rashid A, et al. Mapping medical informatics research. In: Chen H, Fuller S, Friedman C, Hersh W, editors. Medical informatics: knowledge management and data mining in biomedicine. New York, NY: Springer; 2005. p. 36–62.
Zerhouni E. The NIH Roadmap Science 2003;302:63–4, 72.
Trujillo C, Long T. Document co-citation analysis to enhance transdisciplinary research. Sci Adv. 2018;4(1):e1701130.
Chamberlain S. Consuming article-level metrics: observations and lessons. Inf Stand Quart. 2013;25(2):4–13.
Lin J, Fenner M. Altmetrics in evolution: defining & redefining the ontology of article-level metrics. Inf Stand Quart. 2013;25(2):20–6.
Yan K, Gerstein M. The spread of scientific information: insights from the web usage statistics in PLoS article-level metrics. PLoS One. 2011;6(5):e19917.
Warren H, Raison N, Dasgupta P. The rise of altmetrics. J Am Med Assoc. 2017;317:131–2.
Ioannidis J. Neglecting major health problems and broadcasting minor, uncertain issues in lifestyle science. J Am Med Assoc. 2019;322:2069–70.
Dawkins R. The selfish gene. New York: Oxford University Press; 1976.
Scheufele D, Krause N. Science audiences, misinformation, and fake news. Proc Natl Acad Sci. 2019;116:7662–9.
Haynes R. Of studies, syntheses, synopses, and systems: the “4S” evolution of services for finding current best evidence. ACP J Club. 2001;134:A11–A3.
Guyatt G, Rennie D, Meade M, Cook D, editors. Users’ guides to the medical literature: a manual for evidence-based clinical practice. 3rd ed. New York, NY: McGraw-Hill; 2014.
Guyatt G, Rennie D, Meade M, Cook D, editors. Users’ guides to the medical literature: essentials of evidence-based clinical practice. 3rd ed. New York, NY: McGraw-Hill; 2015.
Haynes R. Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. Evid Based Med. 2006;11:162–4.
DiCenso A, Bayley L, Haynes R. ACP Journal Club. Editorial: Accessing preappraised evidence: fine-tuning the 5S model into a 6S model. Annals of Internal Medicine. 2009;151(6):JC3–2, JC3.
Sackett D, Richardson W, Rosenberg W, Haynes R. Evidence-based medicine: how to practice and teach EBM. New York, NY: Churchill Livingstone; 2000.
Buckley D, Ansari M, Butler M, Soh C, Chang C. The refinement of topics for systematic reviews: lessons and recommendations from the Effective Health Care Program. J Clin Epidemiol. 2014;67:425–32.
Hersh W. “A world of knowledge at your fingertips”: the promise, reality, and future directions of on-line information retrieval. Acad Med. 1999;74:240–3.
Slawson D, Shaughnessy A. Teaching evidence-based medicine: should we be teaching information management instead? Acad Med. 2005;80:685–9.
Kuhn T. The structure of scientific revolutions. Chicago: University of Chicago Press; 1962.
Fouad Y, Aanei C. Revisiting the hallmarks of cancer. Am J Cancer Res. 2017;7:1016–36.
Bourne P. Ten simple rules for getting published. PLoS Comput Biol. 2005;1(5):e57.
McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356:1228–31.
Yarborough M, Nadon R, Karlin D. Point of View: four erroneous beliefs thwarting more trustworthy research. elife. 2019;8:e45261.
Anonymous. Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication: International Committee of Medical Journal Editors 2006.
Davidoff F, DeAngelis C, Drazen J, Hoey J, Hojgaard L, Hortin R, et al. Sponsorship, authorship, and accountability. Ann Intern Med. 2001;135:463–6.
Taichman D, Backus J, Baethge C, Bauchner H, Flanagin A, Florenzano F, et al. A disclosure form for work submitted to medical journals – a proposal from the International Committee of Medical Journal Editors. J Am Med Assoc. 2020;323:1050–1.
Taichman D, Sahni P, Pinborg A, Peiperl L, Laine C, James A, et al. Data sharing statements for clinical trials: a requirement of the International Committee of Medical Journal Editors. Ann Intern Med. 2017;167:63–5.
Kempner J, Perlis C, Merz J. Forbidden knowledge. Science. 2005;307:854.
Dzau V, Leshner A. Public health research on gun violence: long overdue. Ann Intern Med. 2018;168:876–7.
Fischetti M. Government attempts to silence science are revealed in detail – a tracker reveals more than 300 government attempts to suppress knowledge. Sci Am 2019 May, 2019.
Nost E. EPA Discontinues Updates to Climate Change Websites: Environmental Data & Governance Initiative Website Monitoring Report2018 October 31, 2018.
Salas R, Laden F, Jacobs W, Jha A. The U.S. Environmental Protection Agency’s proposed transparency rule threatens health. Ann Intern Med. 2019;170:197–8.
Anonymous. APLU, Other Higher Ed & Research Groups Send EPA Letter Expressing Concerns with Proposed Rule on Research-based Rulemaking. Association of Public and Land-grant Universities; 2019.
Dal-Ré R, Janiaud P, Ioannidis J. Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic? BMC Med. 2018;16(1):49.
Sox H, Lewis R. Pragmatic trials – practical answers to “real world” questions. J Am Med Assoc. 2016;316:1205–6.
Smith R. Why scientists should be held to a higher standard of honesty than the average person. Thebmjopinion 2014.
Moynihan R, Bero L, Hill S, Johansson M, Lexchin J, Macdonald H, et al. Pathways to independence: towards producing and using trustworthy evidence. Br Med J. 2019;367:l6576.
Jamieson K, McNutt M, Kiermer V, Sever R. Signaling the trustworthiness of science. Proc Natl Acad Sci. 2019;116:19231–6.
Prasad V, Vandross A, Toomey C, Cheung M, Rho J, Quinn S, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88:790–8.
Djulbegovic B, Kumar A, Glasziou P, Perera R, Reljic T, Dent L, et al. New treatments compared to established treatments in randomized trials. Cochrane Database Syst Rev. 2012;10:MR000024.
Prasad V, Cifu A. Ending medical reversal: improving outcomes, saving lives. Baltimore, MD: Johns Hopkins University Press; 2015.
Herrera-Perez D, Haslam A, Crain T, Gill J, Livingston C, Kaestner V, et al. A comprehensive review of randomized clinical trials in three medical journals reveals 396 medical reversals. elife. 2019;8:e45183.
Hooper M. Scholarly review, old and new. J Sch Publ. 2019;1:53–75.
Hargens L. Variation in journal peer review systems: possible causes and consequences. J Am Med Assoc. 1990;263:1348–52.
Purcell G, Donovan S, Davidoff F. Changes to manuscripts during the editorial process: characterizing the evolution of a clinical paper. J Am Med Assoc. 1998;280:227–8.
Garfunkel J, Lawson E, Hamrick H, Ulshen M. Effect of acceptance or rejection on the author’s evaluation of peer review of medical manuscripts. J Am Med Assoc. 1990;263:1376–8.
Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. J Am Med Assoc. 2002;287:2786–90.
Peters D, Ceci S. Peer-review practices of psychological journals: the fate of published articles, submitted again. Behav Brain Sci. 1982;5:187–255.
Ingelfinger F. Peer review in biomedical publication. Am J Med. 1974;56:686–92.
Garfunkel J, Ulshen M, Hamrick H, Lawson E. Problems identified by secondary review of accepted manuscripts. J Am Med Assoc. 1990;263:1369–71.
Emerson G, Warme W, Wolf F, Heckman J, Brand R, Leopold S. Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. Arch Intern Med. 2010;170:1934–9.
Siler K, Lee K, Bero L. Measuring the effectiveness of scientific gatekeeping. Proc Natl Acad Sci. 2015;112:360–5.
Tomkins A, Zhang M, Heavlina W. Reviewer bias in single- versus double-blind peer review. Proc Natl Acad Sci. 2017;114:12708–13.
Nicholson J. Nope! 8 Rejected Papers That Won the Nobel Prize. Authorea 2016.
Italie H. Not so fast: many Nobel winners endured initial rejections. AP News 2019 October 14, 2019.
Haug C. The downside of open-access publishing. N Engl J Med. 2013;368:791–3.
Beall J. Predatory journals exploit structural weaknesses in scholarly publishing. 4Open. 2018;1.
Grant A. The proliferation of questionable conferences. Phys Today. 2018;
Readings B. Caught in the net: notes from the electronic underground. Surfaces. 1994;4:9–10.
Young N, Ioannidis J, Al-Ubaydli O. Why current publication practices may distort science. PLoS Med. 2008;5(10):e201.
Schooler J. Unpublished results hide the decline effect. Nature. 2011;470:437.
Smith R. Classical peer review: an empty gun. Breast Cancer Res. 2010;12(Suppl 4):S13.
Smith R. A better way to publish science. BMJ Opinions 2015.
Stumpf W. “Peer” review. Science. 1980;207:822–3.
Li D, Agha L. Big names or big ideas: do peer-review panels select the best science proposals? Science. 2015;348:434–8.
Fang F, Bowen A, Casadevall A. NIH peer review percentile scores are poorly predictive of grant productivity. elife. 2016;2016(5):e13323.
Pier E, Brauer M, Filut A, Kaatz A, Raclaw J, Nathan M, et al. Low agreement among reviewers evaluating the same NIH grant applications. Proc Natl Acad Sci. 2018;115:2952–7.
Wang Y, Jones B, Wang D. Early-career setback and future career impact. Nat Commun. 2019;10:4331.
Smaldino P, Turner M, Andrés P, Kallens C. Open science and modified funding lotteries can impede the natural selection of bad science. OSF Preprints. 2019.
Smith S. The Jordan Rules. New York, NY: Pocket Books; 1994.
Kaplan D. How to fix peer review. The Scientist. 2005;19(11):10.
Bourne P, Korngreen A. Ten simple rules for reviewers. PLoS Comput Biol. 2006;2(9):e110.
Stossel T. Reviewer status and review quality: experience of the Journal of Clinical Investigation. N Engl J Med. 1985;312:658–9.
Evans A, McNutt R, Fletcher S, Fletcher R. The characteristics of peer reviewers who produce good quality reviews. J Gen Intern Med. 1993;8:422–8.
Nylenna M, Riis P, Karlsson Y. Multiple blinded reviews of the same two manuscripts: effect of referee characteristics and publication language. J Am Med Assoc. 1994;272:149–51.
Black N, van Rooyen S, Godlee F, Smith R, Evans S. What makes a good reviewer and a good review for a general medical journal? J Am Med Assoc. 1998;280:231–3.
Callaham M, Baxt W, Waeckerie J, Wears R. Reliability of editors’ subjective quality ratings of peer reviews of manuscripts. J Am Med Assoc. 1998;280:229–31.
Schroter S, Tite L, Hutchings A, Black N. Differences in review quality and recommendations for publication between peer reviewers suggested by authors or by editors. J Am Med Assoc. 2006;295:314–7.
Wager E, Parkin E, Tamber P. Are reviewers suggested by authors as good as those chosen by editors? Results of a rater-blinded, retrospective study. BMC Med. 2006;4:13.
Haug C. Peer-review fraud—hacking the scientific publication process. N Engl J Med. 2015;373:2393–5.
Normile D. China cracks down after investigation finds massive peer-review fraud. Sci News. 2017.
Garrow J, Butterfield M, Marshall J, Williamson A. The reported training and experience of editors in chief of specialist clinical medical journals. J Am Med Assoc. 1998;280:286–7.
Moher D, Galipeau J, Alam S, Barbour V, Bartolomeos K, Baskin P, et al. Core competencies for scientific editors of biomedical journals: consensus statement. BMC Med. 2017;15:167.
Anonymous. Transparent peer review one year on. Nature Communications. 2016;7:13626.
Cosgrove A, Cheifet B. Transparent peer review trial: the results. Genome Biol. 2018;19:206.
Polka J, Kiley R, Konforti B, Stern B, Vale R. Publish peer reviews. Nature. 2018;560:545–7.
Ingelfinger F. Annual discourse: swinging copy and sober science. N Engl J Med. 1969;281:526–32.
Angell M, Kassirer J. The Ingelfinger rule revisited. N Engl J Med. 1991;325:1371–3.
Calcagno V, Demoinet E, Gollner K, Guidi L, Ruths D, de Mazancourt C. Flows of research manuscripts among scientific journals reveal hidden submission patterns. Science. 2012;338:1065–9.
Chen J, Konstan J. Conference paper selectivity and impact. Commun ACM. 2010;53(6):79–83.
Niles M, Schimanski L, McKiernan E, Alperin J. Why we publish where we do: faculty publishing values and their relationship to review, promotion and tenure expectations. bioRxiv. 2019.
Chiarelli A, Johnson R, Pinfield S, Richens E. Accelerating scholarly communication: The transformative role of preprints Zenodo2019 September 24, 2019.
Sever R, Eisen M, Inglis J. Plan U: universal access to scientific and medical research. PLoS Biol. 2019;17(6):e3000273.
Abdill R, Blekhman R. Tracking the popularity and outcomes of all bioRxiv preprints. bioRxiv. 2018.
Maslove D. Medical preprints—a debate worth having. J Am Med Assoc. 2018;391:443–4.
Sheldon T. Preprints could promote confusion and distortion. Nature. 2018;559:445.
Oakden-Rayner L, Beam A, Palmer L. Medical journals should embrace preprints to address the reproducibility crisis. Int J Epidemiol. 2018;47:1363–5.
Chambers C. The registered reports revolution - lessons in cultural reform. Significance. 2019;16(4):23–7.
Ebrahim S, Sohani Z, Montoya L, Agarwal A, Thorlund K, Mills E, et al. Reanalyses of randomized clinical trial data. J Am Med Assoc. 2014;312:1024–32.
Ross J, Krumholz H. Ushering in a new era of open science through data sharing: the wall must come down. J Am Med Assoc. 2013;309:1355–6.
Mazor K, Richards A, Gallagher M, Arterburn D, Raebel M, Nowell W, et al. Stakeholders’ views on data sharing in multicenter studies. J Compar Effective Res. 2017;6:537–47.
Mello M, Lieou V, Goodman S. Clinical trial participants’ views of the risks and benefits of data sharing. N Engl J Med. 2018;378:2202–11.
Mello M, Francer J, Wilenzick M, Teden P, Bierer B, Barnes M. Preparing for responsible sharing of clinical trial data. N Engl J Med. 2013;369:1651–8.
Alsheikh-Ali A, Qureshi W, Al-Mallah M, Ioannidis J. Public availability of published research data in high-impact journals. PLoS One. 2011;6(9):e24357.
Longo D, Drazen J. Data sharing. N Engl J Med. 2016;374:276–7.
Berger B, Gaasterland T, Lengauer T, Orengo C, Gaeta B, Markel S, et al. ISCB’s initial reaction to The New England Journal of Medicine Editorial on data sharing. PLoS Comput Biol. 2016;12(3):e1004816.
Greene C, Garmire L, Gilbert J, Ritchie M, Hunter L. Celebrating parasites. Nat Genet. 2017;49:483–4.
Rosenbaum L. Bridging the data-sharing divide—seeing the devil in the details, not the other camp. N Engl J Med. 2017;376:2201–3.
Grossman R, Heath A, Ferretti V, Varmus H, Lowy D, Kibbe W, et al. Toward a shared vision for cancer genomic data. N Engl J Med. 2016;379:1109–12.
Collins F, Varmus H. A new initiative on precision medicine. N Engl J Med. 2015;372:793–5.
Kesselheim A, Avorn J. New “21st Century Cures” legislation: speed and ease vs science. J Am Med Assoc. 2017;317:581–2.
Fleurence R, Curtis L, Califf R, Platt R, Selby J, Brown J. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014;21:578–82.
Hripcsak G, Duke J, Shah N, Reich C, Huser V, Schuemie M, et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. 2015;216:574–8.
Dalerba P, Sahoo D, Paik S, Guo X, Yothers G, Song N, et al. CDX2 as a prognostic biomarker in stage II and stage III colon cancer. N Engl J Med. 2016;374:211–22.
Anonymous. Toward fairness in data sharing. N Engl J Med. 2016;375:405–7.
Merson L, Gaye O, Guerin P. Avoiding data dumpsters—toward equitable and useful data sharing. N Engl J Med. 2016;374:2414–5.
Rockhold F, Nisen P, Freeman A. Data sharing at a crossroads. N Engl J Med. 2016;375:1115–7.
Strom B, Buyse M, Hughes J, Knoppers B. Data sharing—is the juice worth the squeeze? N Engl J Med. 2016;375:1608–9.
Read K, Sheehan J, Huerta M, Knecht L, Mork J, Humphreys B. Sizing the problem of improving discovery and access to NIH-funded data: a preliminary study. PLoS One. 2015;10(7):e0132735.
Kush R, Goldman M. Fostering responsible data sharing through standards. N Engl J Med. 2014;370:2163–5.
Tenenbaum J, Avillach P, Benham-Hutchins M, Breitenstein M, Crowgey E, Hoffman M, et al. An informatics research agenda to support precision medicine: seven key areas. J Am Med Inform Assoc. 2016;23:791–5.
Khera R, Angraal S, Couch T, Welsh J, Nallamothu B, Girotra S, et al. Adherence to methodological standards in research using the National Inpatient Sample. J Am Med Assoc. 2017;318:2011–8.
Bierer B, Crosas M, Pierce H. Data authorship as an incentive to data sharing. N Engl J Med. 2017;376:1684–7.
Pierce H, Dev A, Statham E, Bierer B. Credit data generators for data reuse. Nature. 2019;570:30–2.
Lo B, DeMets D. Incentives for clinical trialists to share data. N Engl J Med. 2016;375:1112–5.
Shanahan D. Clinical trial data and articles linked for the first time. CrossRef Blog 2016.
Hersh W, Boone K, Totten A. Data from: Characteristics of the healthcare information technology workforce in the HITECH era: underestimated in size, still growing, and adapting to advanced uses. In: Repository DD, editor. 2018.
Federer L, Belter C, Joubert D, Livinski A, Lu Y, Snyders L, et al. Data sharing in PLOS ONE: an analysis of data availability statements. PLoS One. 2018;13(5):e0194768.
Naudet F, Sakarovitch C, Janiaud P, Cristea I, Fanelli D, Moher D, et al. Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine. Br Med J. 2018;360:k400.
Tannenbaum S, Ross J, Krumholz H, Desai N, Ritchie J, Lehman R, et al. Early experiences with journal data sharing policies: a survey of published clinical trial investigators. Ann Intern Med. 2018;169:586–8.
Fletcher R, Fletcher S. Clinical research in general medical journals: a 30-year perspective. N Engl J Med. 1979;301:180–3.
Ioannidis J. Contradicted and initially stronger effects in highly cited clinical research. J Am Med Assoc. 2005;294:218–28.
Ioannidis J. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
Montori V, Devereaux P, Adhikari N, Burns K, Eggert C, Briel M, et al. Randomized trials stopped early for benefit: a systematic review. J Am Med Assoc. 2005;294:2203–9.
Mueller P, Montori V, Bassler D, Koenig B, Guyatt G. Ethical issues in stopping randomized trials early because of apparent benefit. Ann Intern Med. 2007;146:878–81.
Allison D, Brown A, George B, Kaiser K. A tragedy of errors. Nature. 2016;530:27–9.
Glantz S. Biostatistics: how to detect, correct, and prevent errors in the medical literature. Circulation. 1980;61:1–7.
Moher D, Dulberg C, Wells G. Statistical power, sample size, and their reporting in randomized controlled trials. J Am Med Assoc. 1994;272:122–4.
Halpern S, Karlawish J, Berlin J. The continuing unethical conduct of underpowered clinical trials. J Am Med Assoc. 2002;288:358–62.
Georgescu C, Wren J. Algorithmic identification of discrepancies between published ratios and their reported confidence intervals and P-values. Bioinformatics. 2017;34:1758–66.
Boutron I, Dutton S, Ravaud P, Altman D. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. J Am Med Assoc. 2010;303:2058–64.
Ochodo E, de Haan M, Reitsma J, Hooft L, Bossuyt P, Leeflang M. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology. 2013;267:581–8.
Jellison S, Roberts W, Bowers A, Combs T, Beaman J, Wayant C, et al. Evaluation of spin in abstracts of papers in psychiatry and psychology journals. BMJ Evid Based Med. 2019:Epub ahead of print.
Khan M, Lateef N, Siddiqi T, Rehman K, Alnaimat S, Khan S, et al. Level and prevalence of spin in published cardiovascular randomized clinical trial reports with statistically nonsignificant primary outcomes – a systematic review. JAMA Netw Open. 2019;2(5):e192622.
Lerchenmueller M, Sorenson O, Jena A. Gender differences in how scientists present the importance of their research: observational study. Br Med J. 2019;367:l6573.
Chavalarias D, Wallach J, Li A, Ioannidis J. Evolution of reporting p values in the biomedical literature, 1990–2015. J Am Med Assoc. 2016;315:1141–8.
Nuzzo R. Scientific method: statistical errors. Nature. 2014;506:150–2.
Head M, Holman L, Lanfear R, Kahn A, Jennions M. The extent and consequences of p-hacking in science. PLoS Biol. 2015;13:e1002106.
Benjamin D, Berger J, Johannesson M, Nosek B, Wagenmakers E, Winship C, et al. Redefine statistical significance. Nat Hum Behav. 2018;2:6–10.
Ioannidis J. The proposal to lower p value thresholds to .005. J Am Med Assoc. 2018;319:1429–30.
Amrhein V, Greenland S, McShane B. Retire statistical significance. Nature. 2019;567:305–7.
Goodman S. Why is getting rid of p-values so hard? Musings on science and statistics. Am Stat. 2019;73:26–30.
Hoffrage U, Lindsey S, Hertwig R, Gigerenzer G. Communicating statistical information. Science. 2000;290:2261–2.
Abola M, Prasad V. The use of superlatives in cancer research. JAMA Oncol. 2016;2:139–41.
Krishnamurti T, Woloshin S, Schwartz L, Fischhoff B. A randomized trial testing US Food and Drug Administration “breakthrough” language. JAMA Intern Med. 2015;175:1856–8.
Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med. 2005;2(5):e138.
Smith R. Medical journals and pharmaceutical companies: uneasy bedfellows. Br Med J. 2003;326:1202–5.
Dickersin K, Rennie D. Registering clinical trials. J Am Med Assoc. 2003;290:516–23.
Silverstein F, Faich G, Goldstein J, Simon L, Pincus T, Whelton A, et al. Gastrointestinal toxicity with celecoxib vs nonsteroidal anti-inflammatory drugs for osteoarthritis and rheumatoid arthritis: the CLASS study: a randomized controlled trial. J Am Med Assoc. 2000;284:1247–55.
Silverstein F, Simon L, Faich G. Reporting of 6-month vs 12-month data in a clinical trial of celecoxib – in reply. J Am Med Assoc. 2001;286:2399–400.
Hrachovec J, Mora M. Reporting of 6-month vs 12-month data in a clinical trial of celecoxib. J Am Med Assoc. 2001;286:2398.
Wright J, Perry T, Bassett K, Chambers K. Reporting of 6-month vs 12-month data in a clinical trial of celecoxib. J Am Med Assoc. 2001;286:2398–9.
Jüni P, Rutjes A, Dieppe P. Are selective COX 2 inhibitors superior to traditional non steroidal anti-inflammatory drugs? Br Med J. 2002;324:1287–8.
Bombardier C, Laine L, Reicin A, Shapiro D, Burgos-Vargas R, Davis B, et al. Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med. 2000;343:1520–8.
Curfman G, Morrissey S, Drazen J. Expression of concern: Bombardier et al., “Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis”. N Engl J Med. 2005;353:2318–9.
Anonymous. Is GSK guilty of fraud? Lancet. 2004;363:1919.
DeAngelis C, Drazen J, Frizelle F, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. J Am Med Assoc. 2004;292:1363–4.
Zarin D, Tse T, Ide N. Trial Registration at ClinicalTrials.gov between May and October 2005. N Engl J Med. 2005;353:2779–87.
Haug C, Gotzsche P, Schroeder T. Registries and registration of clinical trials. N Engl J Med. 2005;353:2811–2.
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–9.
Macleod M, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis J, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383:101–4.
Alberts B, Kirschner M, Tilghman S, Varmus H. Rescuing US biomedical research from its systemic flaws. Proc Natl Acad Sci. 2014;111:5773–7.
Rzhetsky A, Foster J, Foster I, Evans J. Choosing experiments to accelerate collective discovery. Proc Natl Acad Sci. 2015;112:14569–74.
Smaldino P, McElreath R. The natural selection of bad science. R Soc Open Sci. 2016;3(9):160384.
Zarin D, Goodman S, Kimmelman J. Harms from uninformative clinical trials. J Am Med Assoc. 2019;322:813–4.
Wong C, Siah K, Lo A. Estimation of clinical trial success rates and related parameters. Biostatistics. 2019;20:273–86.
Ludwig D, Ebbeling C, Heymsfield S. Discrepancies in the registries of diet vs drug trials. JAMA Netw Open. 2019;2(11):e1915360.
Tatsioni A, Karassa F, Goodman S, Zarin D, Fanelli D, Ioannidis J. Lost evidence from registered large long-unpublished randomized controlled trials: a survey. Ann Intern Med. 2019;171:300–1.
Shepperd M, Guo Y, Li N, Arzoky M, Capiluppi A, Counsell S et al. The prevalence of errors in machine learning experiments. arXivorg. 2019:1909.04436.
Beam A, Manrai A, Ghassemi M. Challenges to the reproducibility of machine learning models in health care. J Am Med Assoc. 2020;323:305–6.
Arrowsmith J. Trial watch: Phase II failures: 2008–2010. Nat Rev Drug Discov. 2011;10:328–9.
Begley C, Ellis L. Raise standards for preclinical cancer research. Nature. 2012;483:531–3.
Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov. 2011;10:712.
Yong E. Replication studies: bad copy. Nature. 2012;485:298–300.
Anonymous. Estimating the reproducibility of psychological science. Science. 2015;349:aac4716.
LeNoury J, Nardo J, Healy D, Jureidini J, Raven M, Tufanaru C, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. Br Med J. 2015;351:h4320.
Ioannidis J. Acknowledging and overcoming nonreproducibility in basic and preclinical research. J Am Med Assoc. 2017;317:1019–20.
Coiera E, Ammenwerth E, Georgiou A, Magrabi F. Does health informatics have a replication crisis? J Am Med Inform Assoc. 2018;25:963–8.
Lin J, Crane M, Trotman A, Callan J, Chattopadhyaya I, Foley J, et al., editors. Toward reproducible baselines: the open-source IR reproducibility challenge. European Conference on Information Retrieval; 2016.
Baker M. Is there a reproducibility crisis? Nature. 2016;533:452–4.
Errington T, Iorns E, Gunn W, Tan F, Lomax J, Nosek B. Science forum: an open investigation of the reproducibility of cancer biology research. elife. 2014;3:e04333.
Nosek B, Errington T. Reproducibility in cancer biology: making sense of replications. elife. 2017;2017(6):e23383.
Anonymous. Reproducibility in cancer biology: the challenges of replication. eLife. 2017;2017(6):e23693.
Merali Z. Computational science: …Error. Nature. 2010;467:775–7.
Sainani K. Error! – What biomedical computing can learn from its mistakes. Biomed Comput Rev 2011 September 1, 2011.
Joppa L, McInerny G, Harper R, Salido L, Takeda K, O’Hara K, et al. Troubling trends in scientific software use. Science. 2013;340:814–5.
Eklund A, Nichols T, Knutsson H. Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates. Proc Natl Acad Sci. 2016;113:7900–5.
Baptista R, Kissinger J. Is reliance on an inaccurate genome sequence sabotaging your experiments? PLoS Pathog. 2019;15(9):e1007901.
Vasilevsky N, Brush M, Paddock H, Ponting L, Tripathy S, Larocca G, et al. On the reproducibility of science: unique identification of research resources in the biomedical literature. PeerJ. 2013;5(1):e148.
Baggerly K, Coombes K. What information should be required to support clinical “omics” publications? Clin Chem. 2011;57:688–90.
Peng R. Reproducible research in computational science. Science. 2011;334:1226–7.
Perkel J. By Jupyter, It all makes sense. Nature. 2018;563:145–6.
Maciocci G, Tsang E, Bentley N, Aufreiter M. Reproducible Document Stack: towards a scalable solution for reproducible articles. elife. 2019;
Somers J. The scientific paper is obsolete. The Atlantic. 2018 April 5, 2018.
Menke J, Roelandse M, B Ozyurt, Martone M, Bandrowski A. Rigor and Transparency Index, a new metric of quality for assessing biological and medical science methods. Biorxiv. 2020.
Anonymous. Reproducibility and Replicability in Science. Washington, DC: National Academies Press; 2019.
Pérignon C, Gadouche K, Hurlin C, Silberman R, Debonnel E. Certify reproducibility with confidential data. Science. 2019;6449:127–8.
Hanbury A, Müller H, Balog K, Brodt T, Cormack G, Eggel I, et al. Evaluation-as-a-service: overview and outlook. arXivorg. 2015:arXiv:1512.07454.
Roegiest A, Cormack G, editors. An architecture for privacy-preserving and replicable high-recall retrieval experiments. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval; 2016; Pisa, Italy.
DerSimonian R, Charette L, McPeek B, Mosteller F. Reporting on methods in clinical trials. N Engl J Med. 1982;306:1332–7.
Moher D, Schulz K, Altman D. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134:657–62.
Moher D, Pham B, Jones A, Cook D, Jadad A, Moher M, et al. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352:609–13.
Moher D, Jones A, Lepage L. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. J Am Med Assoc. 2001;285:2006–7.
Huwiler-Müntener K, Juni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. J Am Med Assoc. 2002;287:2801–4.
von Elm E, Altman D, Egger M, Pocock S, Gøtzsche P, Vandenbroucke J. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147:573–7.
Chan A, Hrobjartsson A, Haahr M, Gotzsche P, Altman D. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. J Am Med Assoc. 2004;291:2457–65.
de Vries Y, Roest A, Beijers L, Turner E, de Jonge P. Bias in the reporting of harms in clinical trials of second-generation antidepressants for depression and anxiety: a meta-analysis. Eur Neuropsychopharmacol. 2016;26:1752–9.
Derry S, Loke Y, Aronson J. Incomplete evidence: the inadequacy of databases in tracing published adverse drug reactions in clinical trials. BMC Med Res Methodol. 2001;1:7.
Fromme E, Eilers K, Mori M, Hsieh Y, Beer T. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. J Clin Oncol. 2004;22:3485–90.
Golder S, McIntosh H, Duffy S, Glanville J. Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE. Health Inf Libr J. 2006;23:3–12.
Fontanarosa P, Rennie D, DeAngelis C. Postmarketing surveillance – lack of vigilance, lack of trust. J Am Med Assoc. 2004;292:2647–50.
Devereaux P, Bhandari M, Clarke M, Montori V, Cook D, Yusuf S, et al. Need for expertise based randomised controlled trials. Br Med J. 2005;330:88.
Lilford R, Braunholtz D, Greenhalgh R, Edwards S. Trials and fast changing technologies: the case for tracker studies. Br Med J. 2000;320:43–6.
Politi M, Han P, Col N. Communicating the uncertainty of harms and benefits of medical interventions. Med Decis Mak. 2007;27:681–95.
Sedrakyan A, Shih C. Improving depiction of benefits and harms: analyses of studies of well-known therapeutics and review of high-impact medical journals. Med Care. 2007;45:S23–S8.
Sawaya G, Guirguis-Blake J, LeFevre M, Harris R, Petitti D. Update on the methods of the U.S. Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med. 2007;147:871–5.
Sheridan S, Pignone M, Lewis C. A randomized comparison of patients’ understanding of number needed to treat and other common risk reduction formats. J Gen Intern Med. 2003;18:884–92.
Berry D, Knapp P, Raynor T. Expressing medicine side effects: assessing the effectiveness of absolute risk, relative risk, and number needed to harm, and the provision of baseline risk information. Patient Educ Couns. 2006;63:89–96.
Gotzsche P, Olson O, editors. Misleading publications of major mammography screening trials in major medical journals. Fourth International Congress on Peer Review in Biomedical Publication; 2001; Barcelona: American Medical Association.
Clarke M, Alderson P, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals. J Am Med Assoc. 2002;287:2799–801.
Tatsioni A, Bonitsis N, Ioannidis I. Persistence of contradicted claims in the literature. J Am Med Assoc. 2007;298:2517–26.
Drazen J, VanDerWeyden M, Rosenberg S, Marusic A, Laine C, Kotzin S, et al. Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med. 2009;361:1896–7.
Lundh A, Sismondo S, Lexchin J, Busuioc O, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012;12:MR000033.
Dunn A, Arachi D, Hudgins J, Tsafnat G, Coiera E, Bourgeois F. Financial conflicts of interest and conclusions about neuraminidase inhibitors for influenza – an analysis of systematic reviews. Ann Intern Med. 2014;161:513–8.
Wayant C, Turner E, Meyer C, Sinnett P, Vassar M. Financial conflicts of interest among oncologist authors of reports of clinical drug trials. JAMA Oncol. 2018;4:1426–8.
Ornstein C, Thomas K. Top cancer researcher fails to disclose corporate financial ties in major research journals. New York, NY: ProPublica2018 September 8, 2018.
Smith R, Gøtzsche P, Groves T. Should journals stop publishing research funded by the drug industry? Br Med J. 2014;348:g171.
Barton D, Stossel T, Stell L. After 20 years, industry critics bury skeptics, despite empirical vacuum. Int J Clin Pract. 2014;68:666–73.
Battisti W, Wager E, Baltzer L, Bridges D, Cairns A, Carswell C, et al. Good publication practice for communicating company-sponsored medical research: GPP3. Ann Intern Med. 2015;163:461–4.
Tierney W, Meslin E, Kroenke K. Industry support of medical research: important opportunity or treacherous pitfall? J Gen Intern Med. 2016;23:544–52.
Dal-Ré R, Caplan A, Marusic A. Editors’ and authors’ individual conflicts of interest disclosure and journal transparency – a cross-sectional study of high-impact medical specialty journals. BMJ Open. 2019;9:e029796.
Kaestner V, Brown A, Tao D, Prasad V. Conflicts of interest in Twitter. Lancet Hematol. 2017;4:e408–e9.
Haynes R, Mulrow C, Huth E, Altman D, Gardner M. More informative abstracts revisited. Ann Intern Med. 1990;113:69–76.
Riesenberg L, Dontineni S, editors. Review of reference inaccuracies. Fourth International Congress on Peer Review in Biomedical Publication; 2001; Barcelona: American Medical Association.
Wager E, Middleton P, editors. Reference accuracy in peer-reviewed journals: a systematic review. Fourth International Congress on Peer Review in Biomedical Publication; 2001; Barcelona: American Medical Association.
Aronsky D, Ransom J, Robinson K. Accuracy of reference in five biomedical informatics journals. J Am Med Inform Assoc. 2005;12:225–8.
Crichlow R, Winbush N, Davies S. The accessibility and accuracy of Web references in five major medical journals. J Am Med Assoc. 2004;292:2723–4.
de Lacey G, Record C, Wade J. How accurate are quotations and references in medical journals? Br Med J. 1985;291:884–6.
Klein M, VandeSompel H, Sanderson R, Shankar H, Balakireva L, Zhou K, et al. Scholarly context not found: one in five articles suffers from reference rot. PLoS One. 2014;9(12):e115253.
Perkel J. The trouble with reference rot. Nature. 2015;521:111–2.
Ziemann M, Eren Y, El-Osta A. Gene name errors are widespread in the scientific literature. Genome Biol. 2016;17:177.
Byrne J, Labbé C, editors. Fact checking nucleotide sequences in life science publications: the Seek & Blastn Tool. Eighth International Congress on Peer Review and Scientific Publication; 2017; Chicago, IL.
Byrne J, Labbé C. Striking similarities between publications from China describing single gene knockdown experiments in human cancer cell lines. Scientometrics. 2017;110:1471–93.
Phillips N. Tool spots DNA errors in papers. Nature. 2017;551:422–3.
Haynes W, Tomczak A, Khatri P. Gene annotation bias impedes biomedical research. Sci Rep. 2018;8:1362.
Dickersin K. The existence of publication bias and risk factors for its occurrence. J Am Med Assoc. 1990;263:1385–9.
Dwan K, Gamble C, Williamson P, Kirkham J. Systematic review of the empirical evidence of study publication bias and outcome reporting bias – an updated review. PLoS One. 2013;8(7):e66844.
Sterling T. Publication decisions and their possible effects on inferences drawn from tests of significance - or vice versa. J Am Stat Assoc. 1959;54:30–4.
Rosenthal R. The “file drawer problem” and tolerance for null results. Psychol Bull. 1979;86:638–41.
Fanelli D. Negative results are disappearing from most disciplines and countries. Scientometrics. 2012;90:891–904.
Franco A, Malhotra N, Simonovits G. Publication bias in the social sciences: unlocking the file drawer. Science. 2014;345:1502–5.
Dickersin K, Min Y. Publication bias: a problem that won’t go away. Ann N Y Acad Sci. 1993;703:135–48.
Stern J, Simes R. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br Med J. 1997;315:640–5.
Callaham M, Wears R, Weber E, Barton C, Young G. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. J Am Med Assoc. 1998;280:254–7.
Scherer R, Langenberg P, editors. Full publication of results initially presented in abstracts: revisited. Fourth International Congress on Peer Review in Biomedical Publication; 2001; Barcelona: American Medical Association.
von Elm E, Costanza M, Walder B, Tramer M. More insight into the fate of biomedical meeting abstracts: a systematic review. BMC Med Res Methodol. 2003;3:12.
Crockett L, Okoli G, Neilson C, Rabbani R, Abou-Setta A, Klassen T. Publication of randomized clinical trials in pediatric research – a follow-up study. JAMA Netw Open. 2018;1(1):e180156.
Ioannidis J. Effect of statistical significance of results on the time to completion and publication of randomized efficacy trials. J Am Med Assoc. 1998;279:281–6.
Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350:326–9.
Turner E, Matthews A, Linardatos E, Tell R, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–60.
Turner E, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration Database. PLoS Med. 2012;9:3.
de Vries Y, Roest A, de Jonge P, Cuijpers P, Munafò M, Bastiaansen J. The cumulative effect of reporting and citation biases on the apparent efficacy of treatments: the case of depression. Psychol Med. 2018;48:2453–5.
Bohannon J. U.K. research charity will self-publish results from its grantees. Science Insider 2016.
Schwartz L, Woloshin S. Lost in transmission – FDA drug information that never reaches clinicians. N Engl J Med. 2009;361:1717–20.
Wieseler B, Kerekes M, Vervoelgyi V, McGauran N, Kaiser T. Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. Br Med J. 2012;344:d8141.
Turner E. How to access and process FDA drug approval packages for use in research. Br Med J. 2013;347:f5992.
Jefferson T, Jones M, Doshi P, DelMar C. Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and meta-analysis. Br Med J. 2009;339:b5106.
Doshi P, Jefferson T, DelMar C. The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med. 2012;9(4):e1001201.
Godlee F. Clinical trial data for all drugs in current use. Br Med J. 2012;345:e7304.
Godlee F. Goodbye PubMed, hello raw data. Br Med J. 2011;342:d212.
Abbasi K. The missing data that cost $20bn. Br Med J. 2014;348:g2695.
Strom B, Buyse M, Hughes J, Knoppers B. Data sharing, year 1 – access to data from industry-sponsored clinical trials. N Engl J Med. 2014;371:2052–4.
Zarin D, Tse T, Williams R, Carr S. Trial reporting in ClinicalTrials.gov – the final rule. N Engl J Med. 2016;375:1998–2004.
Piller C. FDA and NIH let clinical trial sponsors keep results secret and break the law. Sci News. 2020;
Bruckner T. Clinical trial transparency at US universities – Compliance with U.S. law and global best practices. Washington, DC: Transpari MED2019 March 25, 2019.
DeVito N, Bacon S, Goldacre B. Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. Lancet. 2020;395:361–9.
Zarin D, Fain K, Dobbins H, Tse T, Williams R. 10-year update on study results submitted to ClinicalTrials.gov. N Engl J Med. 2019;381:1966–74.
Chalmers I. Under-reporting scientific research is scientific misconduct. J Am Med Assoc. 1990;263:1405–8.
Wallach J, Krumholz H. Not reporting results of a clinical trial is academic misconduct. Ann Intern Med. 2019;171:293–4.
Friedman C, Wyatt J. Publication bias in medical informatics. J Am Med Inform Assoc. 2001;8:189–91.
Vawdrey D, Hripcsak G. Publication bias in clinical trials of electronic health records. J Biomed Inform. 2013;46:139–41.
Colaianni L. Retraction, comment, and errata policies of the U.S. National Library of Medicine. Lancet. 1992;340:536–7.
Hughes C. Academic medical libraries’ policies and procedures for notifying library users of retracted scientific publications. Med Ref Serv Q. 1998;17(2):37–40.
Friedman P. Correcting the literature following fraudulent publication. J Am Med Assoc. 1990;263:1416–9.
Kochen C, Budd J. The persistence of fraud in the literature: The Darsee case. J Am Soc Inf Sci. 1992;43:488–93.
Whitely W, Rennie D, Hafner A. The scientific community’s response to evidence of fraudulent publication: the Robert Slutsky case. J Am Med Assoc. 1994;272:170–3.
Garfield E, Welljams-Dorof A. The impact of fraudulent research on the scientific literature: the Stephen Breuning case. J Am Med Assoc. 1990;1990:1424–6.
Anonymous. The Top Retractions of 2019 The Scientist. 2019 December 16, 2019.
Anonymous. Meet the scientific sleuths: More than a dozen who’ve had an impact on the scientific literature. Retraction Watch 2018.
Marcus A, Oransky I. Meet the ‘data thugs’ out to expose shoddy and questionable research. Retraction Watch 2018.
Grey A, Avenell A, Klein A, Gunsalus C. Check for publication integrity before misconduct. Nature. 2020;577:167–9.
Brainard J, You J. Rethinking retractions. Science. 2018;362:390–3.
Anonymous. Retraction Notice. J Vibrat Control. 2014;20:1601–4.
Reich E. Cancer trial errors revealed. Nature. 2011;469:139–40.
Woodhead M. 80% of China’s clinical trial data are fraudulent, investigation finds. Br Med J. 2016;355:i5396.
Carlisle J. Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia. 2017;72:944–52.
Chawla D. Russian journals retract more than 800 papers after ‘bombshell’ investigation. Science. 2020;
Stigbrand T. Retraction note to multiple articles in Tumor Biology. Tumor Biol 2017.
Feldwisch-Drentrup H. Journal that holds record for retracted papers also has a problem with editorial board members Science News 2017.
Dansinger M. Dear plagiarist: a letter to a peer reviewer who stole and published our manuscript as his own. Ann Intern Med. 2017;166:143.
Finelli C, Crispino P, Gioia S, LaSala N, D'amico L, LaGrotta M, et al. Retraction: The improvement of large High-Density Lipoprotein (HDL) particle levels, and presumably HDL metabolism, depend on effects of low-carbohydrate diet and weight loss. EXCLI J. 2016;15:570.
Chaplain M, Kirschner D, Iwasa Y. JTB editorial malpractice: a case report. J Theor Biol. 2020;488:110171.
McHugh U, Yentis S. An analysis of retractions of papers authored by Scott Reuben. Joachim Boldt and Yoshitaka Fujii Anaesthesia. 2018;74:17–21.
Marcus A, Oransky I. How the Biggest Fabricator in Science Got Caught. Nautilus 2015.
Marcus A. Anesthesiologist joins the 100-retraction club. Retraction Watch 2020.
Milne G. This science vigilante calls out bogus results in prestigious journals. Medium. 2019 November;5:2019.
Bik E, Casadevall A, Fang F. The prevalence of inappropriate image duplication in biomedical research publications. mBio. 2016;7(3):e00809–16.
Bik E, Fang F, Kullas A, Davis R, Casadevall A. Analysis and correction of inappropriate image duplication: the Molecular and Cellular Biology experience. Mol Cel Biol. 2018;38:e00309–18.
Bauchner H, Fontanarosa P, Flanagin A, Thornton J. Scientific misconduct and medical journals. J Am Med Assoc. 2018;320:1985–7.
O’Connor A. More evidence that nutrition studies don’t always add up. New York Times 2018 September 29, 2018.
Bauchner H. Notice of retractions: “first foods most: after 18-hour fast, people drawn to starches first and vegetables last,” “fattening fasting: hungry grocery shoppers buy more calories, not more food,” and “watch what you eat: action-related television content increases food intake” by Brian Wansink. JAMA Intern Med. 2018;178:1450.
Drazen J. Expression of Concern: Beltrami AP et al. Evidence that human cardiac myocytes divide after myocardial infarction. N Engl J Med 2001;344:1750–7 and Quaini F et al. Chimerism of the transplanted heart. N Engl J Med 2002;346:5–15. N Engl J Med. 2018;379:1870.
Anonymous. Statement on NHLBI decision to pause the CONCERT-HF trial. Bethesda, MD: National Institutes of Health 2018 October 29, 2018.
Keown A. Harvard, Brigham and Women’s Hospital Seek retraction of 31 articles with falsified data. BioSpace 2018.
Chien K, Frisén J, Fritsche-Danielson R, Melton D, Murry C, Weissman I. Regenerating the field of cardiovascular cell therapy. Nat Biotechnol. 2019;37:232–7.
Bornemann-Cimenti H, Szilagyi I, Sandner-Kiesling A. Perpetuation of retracted publications using the example of the Scott S. Reuben case: incidences, reasons and possible improvements. Sci Eng Ethics. 2015;22:1063–72.
Hamilton D. Continued citation of retracted radiation oncology literature – do we have a problem? Int J Radiat Oncol Biol Phys. 2018;103:1036–42.
Anonymous. Retraction—ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 2010;375:445.
Suelzer E, Deal J, Hanus K, Ruggeri B, Sieracki R, Witkowski E. Assessment of citations of the retracted article by Wakefield et al with fraudulent claims of an association between vaccination and autism. JAMA Netw Open. 2019;2(11):e1915552.
Fernández L, Vadillo M. Retracted papers die hard: Diederik Stapel and the enduring influence of flawed science. PsyArVix 2019.
Bakker C, Riegelman A. Retracted publications in mental health literature: discovery across bibliographic platforms. J Librarianship Scholar Commun. 2018;6:eP2199.
Mistry V, Grey A, Bolland M. Publication rates after the first retraction for biomedical researchers with multiple retracted publications. Accountab Res – Policies Qual Assur. 2019;26:277–87.
Fanelli D, Moher D. What difference do retractions make? An estimate of the epistemic impact of retractions on recent meta-analyses. bioRxiv. 2019:https://doi.org/10.1101/734137.
Marcus A. Exclusive: Russian site says it has brokered authorships for more than 10,000 researchers. Retraction Watch 2019.
Byrne J, Christopher J. Digital magic, or the dark arts of the 21st century—how can journals and peer reviewers detect manuscripts and publications from paper mills? FEBS Lett. 2020;594:583–9.
Phillips T, Saunders R, Cossman J, Heitman E. Assessing trustworthiness in research: a pilot study on CV verification. J Empir Res Hum Res Ethics. 2019;14:353–64.
Wang M, Yan A, Katz R. Researcher requests for inappropriate analysis and reporting: a U.S. survey of consulting biostatisticians. Ann Intern Med. 2018;169:554–8.
Nelson H, editor. Systematic reviews to answer health care questions. Baltimore, MD: Lippincott Williams & Wilkins; 2014.
Glass G. Primary, secondary, and meta-analysis of research. Educ Res. 1976;10:3–8.
Glasziou P, Irwig L. Meta-analysis of diagnostic tests. In: Armitage P, Colton T, editors. Encyclopaedia of biostatistics, vol. 4. Chichester: Wiley; 1998. p. 2579–85.
Stroup D, Berlin J, Morton S, Olkin L, Williamson G, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. J Am Med Assoc. 2000;283:2008–12.
Pham M, Rajić A, Greig J, Sargeant J, Papadopoulos A, McEwen S. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014;5:371–85.
Munn Z, Peters M, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:143.
Tonin F, Rotta I, Mendes A, Pontarolo R. Network meta-analysis: a technique to gather evidence from direct and indirect comparisons. Pharm Pract. 2017;15:943.
Friedrich M. The Cochrane Collaboration turns 20: assessing the evidence to inform clinical care. J Am Med Assoc. 2013;309:1881–2.
Clancy C, Slutsky J. Advancing excellence in health care: getting to effectiveness. J Investig Med. 2005;53:65–6.
Anonymous. Finding what works in health care: standards for systematic reviews. Washington, DC: Institute of Medicine 2011 March 23, 2011.
Moher D, Liberati A, Tetzlaff J, Altman D. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9.
Guyatt G, Oxman A, Akl E, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64:383–94.
Fergusson D, Glass K, Hutton B, Shapiro S. Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding? Clin Trials. 2005;2:218–29.
Copas J, Shi J. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1:247–62.
Egger M, Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. Br Med J. 1997;315:629–34.
Higgins J, Thompson S. Quantifying heterogeneity in a meta-analysis. Stat Med. 2002;21:1539–58.
Garner P, Hopewell S, Chandler J, MacLehose H, Akl E, Trivella M, et al. When and how to update systematic reviews: consensus and checklist. Br Med J. 2016;354:i3507.
Shekelle P, Shetty K, Newberry S, Maglione M, Motala A. Machine learning versus standard techniques for updating searches for systematic reviews: a diagnostic accuracy study. Ann Intern Med. 2017;167:213–5.
Martin P, Surian D, Bashir R, Bourgeois F, Dunn A. Trial2rev: combining machine learning and crowd-sourcing to create a shared space for updating systematic reviews. JAMIA Open. 2019;1:15–22.
Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. In: Cochrane Library. Update Software. 2003. http://www.cochrane.org/cochrane/mrabstr/mr000010.htm.
von Elm E, Poglia G, Walder B, Tramer M. Different patterns of duplicate publication: an analysis of articles used in systematic reviews. J Am Med Assoc. 2004;291:974–80.
Ioannidis J. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94:485–514.
Page M, Moher D. Mass production of systematic reviews and meta-analyses: an exercise in mega-silliness? Milbank Q. 2016;94:515–9.
Page M, Shamseer L, Altman D, Tetzlaff J, Sampson M, Tricco A, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13(5):e1002028.
Dechartres A, Atal I, Riveros C, Meerpohl J, Ravaud P. Association between publication characteristics and treatment effect estimates – a meta-epidemiologic study. Ann Intern Med. 2018;169:385–93.
Rochon P, Bero L, Bay A, Gold J, Dergal J, Binns M, et al. Comparison of review articles published in peer-reviewed and throwaway journals. J Am Med Assoc. 2002;287:2853–6.
Banzi R, Cinquini M, Liberati A, Moschetti I, Pecoraro V, Tagliabue L, et al. Speed of updating online evidence based point of care summaries: prospective cohort analysis. Br Med J. 2011;343:d5856.
Ketchum A, Saleh A, Jeong K. Type of evidence behind point-of-care clinical information products: a bibliometric analysis. J Med Internet Res. 2011;13(1):e21.
Randhawa A, Babalola O, Henney Z, Miller M, Nelson T, Oza M, et al. A collaborative assessment among 11 pharmaceutical companies of misinformation in commonly used online drug information compendia. Ann Pharmacother. 2016;50:352–9.
Talwar S, Randhawa A, Dankiewicz E, Crudele N, Haddox J. Caveat emptor: erroneous safety information about opioids in online drug-information compendia. J Opioid Manag. 2016;12:281–8.
Piper B, Lambert D, Keefe R, Smukler P, Selemon N, Duperry Z. Undisclosed conflicts of interest among biomedical textbook authors. AJOB Empirical Bioethics. 2018;9(2):59–68.
Field M, Lohr K, editors. Clinical practice guidelines: directions for a new program. Washington, DC: National Academies Press; 1990.
Shiffman R, Brandt C, Liaw Y, Corb G. A design model for computer-based guideline implementation based on information management services. J Am Med Inform Assoc. 1999;6:99–103.
Hibble A, Kanka D, Penchion D, Pooles F. Guidelines in general practice: the new Tower of Babel? Br Med J. 1998;317:862–3.
Cabana M, Rand C, Powe N, Wu A, Wilson M, Abboud P, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. J Am Med Assoc. 1999;282:1458–65.
McAlister F, van Diepen S, Padwal R, Johnson J, Majumdar S. How evidence-based are the recommendations in evidence-based guidelines? PLoS Med. 2007;4(8):e250.
Cherubini A, Oristrell J, Pla X, Ruggiero C, Ferretti R, Diestre G, et al. The persistent exclusion of older patients from ongoing clinical trials regarding heart failure. Arch Intern Med. 2011;171:550–6.
Bennett W, Odelola O, Wilson L, Bolen S, Selvaraj S, Robinson K, et al. Evaluation of guideline recommendations on oral medications for type 2 diabetes mellitus: a systematic review. Ann Intern Med. 2012;156:27–36.
Chen Y, Yang K, Marušić A, Qaseem A, Meerpohl J, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166:128–32.
Maviglia S, Zielstorff R, Paterno M, Teich J, Bates D, Kuperman G. Automating complex guidelines for chronic disease: lessons learned. J Am Med Inform Assoc. 2003;10:154–65.
Friedman C, Flynn A. Computable knowledge: an imperative for learning health systems. Learn Health Syst. 2019;3(4):e10203.
Neuman J, Korenstein D, Ross J, Keyhani S. Prevalence of financial conflicts of interest among panel members producing clinical practice guidelines in Canada and United States: cross sectional study. Br Med J. 2011;343:d5621.
Mendelson T, Meltzer M, Campbell E, Caplan A, Kirkpatrick J. Conflicts of interest in cardiovascular clinical practice guidelines. Arch Intern Med. 2011;171:577–84.
Feuerstein J, Akbari M, Gifford A, Hurley C, Leffler D, Sheth S, et al. Systematic analysis underlying the quality of the scientific evidence and conflicts of interest in interventional medicine subspecialty guidelines. Mayo Clin Proc. 2014;89:16–24.
Khan R, Scaffidi M, Rumman A, Grindal A, Plener I, Grover S. Prevalence of financial conflicts of interest among authors of clinical guidelines related to high-revenue medications. JAMA Intern Med. 2018;178:1712–5.
Ioannidis J. Professional societies should abstain from authorship of guidelines and disease definition statements. Circ Cardiovasc Qual Outcomes. 2018;11(10):e004889.
Qaseem A, Wilt T. Disclosure of interests and management of conflicts of interest in clinical guidelines and guidance statements: methods from the Clinical Guidelines Committee of the American College of Physicians. Ann Intern Med. 2019;171:354–61.
Laine C, Taichman D, Mulrow C. Trustworthy clinical guidelines. Ann Intern Med. 2011;154:774–5.
Graham R, Mancher M, Wolman D, Greenfield S, Steinberg E. Clinical practice guidelines we can trust. Washington, DC: National Academies Press 2011 March 23, 2011.
Jue J, Cunningham S, Lohr K, Shekelle P, Shiffman R, Robbins C, et al. Developing and testing the Agency for Healthcare Research and Quality’s National Guideline Clearinghouse Extent of Adherence to Trustworthy Standards (NEATS) instrument. Ann Intern Med. 2019;170:480–7.
Frankel M, Elliot R, Blume M, Bourgois J, Hugenholtz B, Lundquist M, et al. Defining and certifying electronic publication in science. American Association for the Advancement of Science 2000. http://www.aaas.org/spp/dspp/sfrl/projects/epub/define.htm. Accessed July 1, 2002.
Hersh W, Rindfleisch T. Electronic publishing of scholarly communication in the biomedical sciences. J Am Med Inform Assoc. 2000;7:324–5.
Anonymous. The future of the electronic scientific literature. Nature. 2001;413:1–3.
Fox S, Duggan M. Health Online 2013. Washington, DC: Pew Internet & American Life Project 2013 January 15, 2013.
Silberg W, Lundberg G, Musacchio R. Assessing, controlling, and assuring the quality of medical information on the Internet: caveat lector et viewor – let the reader and viewer beware. J Am Med Assoc. 1997;277:1244–5.
Eysenbach G, Powell J, Kuss O, Sa E-R. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. J Am Med Assoc. 2002;287:2691–700.
Scullard P, Peacock C, Davies P. Googling children’s health: reliability of medical advice on the internet. Arch Dis Child. 2010;95:580–2.
Mirkin J, Lowrance W, Feifer A, Mulhall J, Eastham J, Elkin E. Direct-to-consumer Internet promotion of robotic prostatectomy exhibits varying quality of information. Health Aff. 2012;31:760–9.
Kitchens B, Harle C, Li S. Quality of health-related online search results. Decis Support Syst. 2014;57:454–62.
Kincaid M, Fleisher L, Neuman M. Presentation on US hospital websites of risks and benefits of transcatheter aortic valve replacement procedures. JAMA Intern Med. 2015;175:440–1.
Bruce J, Tucholka J, Steffens N, Neuman H. Quality of online information to support patient decision-making in breast cancer surgery. J Surg Oncol. 2015;112:575–80.
Broniatowski D, Jamison A, Qi S, AlKulaib L, Chen T, Benton A, et al. Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health. 2018;108:1378–84.
Koren M. How Did Astronaut DNA become ‘Fake News’? The Atlantic 2018 March 16, 2018.
Loeb S, Sengupta S, Butaney M, Macaluso J, Czarniecki S, Robbins R, et al. Dissemination of misinformative and biased information about prostate cancer on YouTube. Eur Urol. 2018;75:564–7.
Rothrock S, Rothrock A, Swetland S, Pagane M, Isaak S, Romney J, et al. Quality, trustworthiness, readability, and accuracy of medical information regarding common pediatric emergency medicine-related complaints on the Web. J Emerg Med. 2019;57:469–77.
Overland J, Hoskins P, McGill M. Low literacy: a problem in diabetes education. Diab Med. 1993;10:847–50.
Foltz A, Sullivan J. Reading level, learning presentation preference, and desire for information among cancer patients. J Cancer Educ. 1996;11:32–8.
Williams D, Counselman F, Caggiano C. Emergency department discharge instructions and patient literacy: a problem of disparity. Am J Emerg Med. 1996;14:19–22.
Murphy P. Reading ability of parents compared with reading level of pediatric patient education materials. Pediatrics. 1994;93:460–8.
Gazmararian J, Baker D, Williams M, Parker R, Scott T, Green D, et al. Health literacy among Medicare enrollees in a managed care organization. J Am Med Assoc. 1999;281:545–51.
Flesch R. A new readability yardstick. J Appl Psychol. 1948;32:221–33.
Graber M, Roller C, Kaeble B. Readability levels of patient education material on the World Wide Web. J Fam Pract. 1999;48:58–61.
Berland G, Elliott M, Morales L, Algazy J, Kravitz R, Broder M, et al. Health information on the Internet: accessibility, quality, and readability in English and Spanish. J Am Med Assoc. 2001;285:2612–21.
Cheng C, Dunn M. Health literacy and the Internet: a study on the readability of Australian online health information. Aust N Z J Public Health. 2015;39:309–14.
Murray K, Murray T, O’Rourke A, Low C, Veale D. Readability and quality of online information on osteoarthritis: an objective analysis with historic comparison. Internact J Med Res. 2019;8(3):e12855.
Galesic M, Garcia-Retamero R. Statistical numeracy for health: a cross-cultural comparison with probabilistic national samples. Arch Intern Med. 2010;170:462–8.
Gigerenzer G, Galesic M. Why do single event probabilities confuse patients? Br Med J. 2010;344:e245.
Krouss M, Croft L, Morgan D. Physician understanding and ability to communicate harms and benefits of common medical treatments. JAMA Intern Med. 2016;176:1565–7.
Epstein D. When evidence says no, but doctors say yes ProPublica 2017 February 22, 2017.
Eysenbach G, Diepgen T. Towards quality management of medical information on the internet: evaluation, labelling, and filtering of information. Br Med J. 1998;317:1496–502.
McHenry R. The faith-based encyclopedia tech central station 2004 November 15, 2004.
Shafee T, Masukume G, Kipersztok L, Das D, Häggström M, Heilman J. Evolution of Wikipedia’s medical content: past, present and future. J Epidemiol Commun Health. 2017;71(11):1122.
Tackett S, Gaglani S, Heilman J, Azzam A. The reCAPTCHA of medical education. Med Teach. 2017;41:598–600.
Apollonio D, Broyde K, Azzam A, DeGuia M, Heilman J, Brock T. Pharmacy students can improve access to quality medicines information by editing Wikipedia articles. BMC Med Educ. 2018;18:265.
Clauson K, Polen H, Boulos M, Dzenowagis J. Scope, completeness, and accuracy of drug information in Wikipedia. Ann Pharmacother. 2008;42:1814–21.
Rajagopalan M, Khanna V, Leiter Y, Stott M, Showalter T, Dicker A, et al. Patient-oriented cancer information on the Internet: a comparison of Wikipedia and a professionally maintained database. J Oncol Pract. 2011;7:319–23.
Hasty R, Garbalosa R, Barbato V, Valdes P, Powers D, Hernandez E, et al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc. 2014;114:368–73.
Hwang T, Bourgeois F, Seeger J. Drug safety in the digital age. N Engl J Med. 2014;370:2460–2.
Crocco A, Villasis-Keever M, Jadad A. Analysis of cases of harm associated with use of health information on the internet. J Am Med Assoc. 2002;287:2869–71.
Ferguson T. From patients to end users: quality of online patient networks needs more attention than quality of online health information. Br Med J. 2002;324:555–6.
Tang P, Newcomb C, Gorden S, Kreider N, editors. Meeting the information needs of patients: results from a patient focus group. Proceedings of the 1997 AMIA Annual Fall Symposium; 1997; Nashville, TN: Hanley & Belfus.
Boyer C, Baujard V, Geissbuhler A. Evolution of health web certification through the HONcode experience. Stud Health Technol Inform. 2011;169:53–7.
Starman J, Gettys F, Capo J, Fleischli J, Norton H, Karunakar M. Quality and content of Internet-based information for ten common orthopaedic sports medicine diagnoses. J Bone Joint Surg. 2010;92:1612–8.
Price S, Hersh W, editors. Filtering Web pages for quality indicators: an empirical approach to finding high quality consumer health information on the World Wide Web. Proceedings of the AMIA 1999 Annual Symposium; 1999; Washington, DC: Hanley & Belfus.
Wang Y, Liu Z. Automatic detecting indicators for quality of health information on the Web. Int J Med Inform. 2007;76:575–82.
Shuchman M, Wilkes M. Medical scientists and health news reporting: a case of miscommunication. Ann Intern Med. 1997;126:976–82.
Rennie D. Thyroid storm. J Am Med Assoc. 1997;277:1238–43.
Schwartz L, Woloshin S, Baczek L. Media coverage of scientific meetings: too much, too soon? J Am Med Assoc. 2002;287:2859–63.
Rada R. Retractions, press releases and newspaper coverage. Health Inf Libr J. 2007;24:210–5.
Fishman J, Have T, Casarett D. Cancer and the media: how does the news report on treatment and outcomes? Arch Intern Med. 2010;170:515–8.
Yavchitz A, Boutron I, Bafeta A, Marroun I, Charles P, Mantz J, et al. Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study. PLoS Med. 2012;9(9):e1001308.
Downing N, Cheng T, Krumholz H, Shah N, Ross J. Descriptions and interpretations of the ACCORD-Lipid trial in the news and biomedical literature: a cross-sectional analysis. JAMA Intern Med. 2014;174:1176–82.
Schwitzer G. A guide to reading health care news stories. JAMA Intern Med. 2014;174:1183–6.
Sumner P, Vivian-Griffiths S, Boivin J, Williams A, Venetis C, Davies A, et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. Br Med J. 2014;349:g7015.
Lancaster F, Warner A. Information retrieval today. Arlington, VA: Information Resources Press; 1993.
Cockburn A. Writing effective use cases. Boston: Addison-Wesley; 2001.
Wilkinson R, Fuller M. Integration of information retrieval and hypertext via structure. In: Agosti M, Smeaton A, editors. Information retrieval and hypertext. Norwell, MA: Kluwer; 1996. p. 257–71.
Gorman P. Information needs of physicians. J Am Soc Inf Sci. 1995;46:729–36.
Anonymous. From Screen to Script: The Doctor’s Digital Path to Treatment. New York, NY: Manhattan Research; Google 2012.
Elstein A, Shulman L, Sprafka S. Medical problem solving: an analysis of clinical reasoning. Cambridge, MA: Harvard University Press; 1978.
Schmidt H, Norman G, Boshuizen H. A cognitive perspective on medical expertise: theory and implications. Acad Med. 1990;65:611–21.
Patel V, Evans D, Groen G. Biomedical knowledge and clinical reasoning. In: Evans D, Patel V, editors. Cognitive science in medicine: biomedical modeling. Cambridge, MA: MIT Press; 1989. p. 53–112.
Sox H, Blatt M, Higgins M, Marton K. Medical decision making. Boston, MA: Butterworths; 1988.
Tanenbaum S. Knowing and acting in medical practice: the epistemological politics of outcomes research. J Health Polit Policy Law. 1994;19:27–44.
Huth E. The underused medical literature. Ann Intern Med. 1989;110:99–100.
Kassirer J. Too many books, too few journals. N Engl J Med. 1992;326:1427–8.
Shaughnessy A, Slawson D, Bennett J. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract. 1994;39:489–99.
McDonald C. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996;124:56–62.
Stross J, Harlan W. The dissemination of new medical information. J Am Med Assoc. 1979;241:2622–4.
Stross J, Harlan W. Dissemination of relevant information on hypertension. J Am Med Assoc. 1981;246:360–2.
Williamson J, German P, Weiss R, Skinner E, Bowes F. Health science information management and continuing education of physicians. Ann Intern Med. 1989;110:151–60.
Leigh T, Young P, Haley J. Performances of family practice diplomates on successive mandatory recertification examinations. Acad Med. 1993;68:912–21.
Ramsey P, Carline J, Inui T, Larson E, LoGerfo J, Norcini J, et al. Changes over time in the knowledge base of practicing internists. J Am Med Assoc. 1991;266:1103–8.
Covell D, Uman G, Manning P. Information needs in office practice: are they being met? Ann Intern Med. 1985;103:596–9.
Gorman P, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Mak. 1995;15:113–9.
Ely J, Osheroff J, Ebell M, Bergus G, Levy B, Chambliss M, et al. Analysis of questions asked by family doctors regarding patient care. Br Med J. 1999;319:358–61.
Ely J, Osheroff J, Gorman P, Ebell M, Chambliss M, Pifer E, et al. A taxonomy of generic clinical questions: classification study. Br Med J. 2000;321:429–32.
Ely J, Osheroff J, Ebell M, Chambliss M, Vinson D, Stevermer J, et al. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. Br Med J. 2002;324:710–3.
Ely J, Osheroff J, Chambliss M, Ebell M, Rosenbaum M. Answering physicians’ clinical questions: obstacles and potential solutions. J Am Med Inform Assoc. 2005;12:217–24.
Ely J, Osheroff J, Maviglia S, Rosenbaum M. Patient-care questions that physicians are unable to answer. J Am Med Inform Assoc. 2007;14:407–14.
Ellsworth M, Homan J, Cimino J, Peters S, Pickering B, Herasevich V. Point-of-care knowledge-based resource needs of clinicians: a survey from a large academic medical center. Appl Clin Inform. 2015;6:305–17.
DelFiol G, Workman T, Gorman P. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014;174:710–8.
Curley S, Connelly D, Rich E. Physicians use of medical knowledge resources: preliminary theoretical framework and findings. Med Decis Mak. 1990;10:231–41.
Connelly D, Rich E, Curley S, Kelly J. Knowledge resource preferences of family physicians. J Fam Pract. 1990;30:353–9.
Aakre C, Maggio L, DelFiol G, Cook D. Barriers and facilitators to clinical information seeking: a systematic review. J Am Med Inform Assoc. 2019;26:1129–40.
Cogdill K. Information needs and information seeking in primary care: a study of nurse practitioners. J Med Libr Assoc. 2003;91:203–14.
Roberts P, Hayes W, editors. Information needs and the role of text mining in drug development. Pacific Symposium on Biocomputing; 2008; Big Island, Hawaii: World Scientific Press.
Hemminger B, Lu D, Vaughan K, Adams S. Information seeking behavior of academic scientists. J Am Soc Inf Sci Technol. 2007;58:2205–25.
Roberts K, Demner-Fushman D. Interactive use of online health resources: a comparison of consumer and professional questions. J Am Med Inform Assoc. 2015;23:802–11.
Wartella E, Rideout V, Zupancic H, Beaudoin-Ryan L, Lauricella A. Teens, health, and technology – a national survey. Evanston, IL: Center on Media and Human Development, Northwestern University 2015 June, 2015.
Ioannidis J, Stuart M, Brownlee S, Strite S. How to survive the medical misinformation mess. Eur J Clin Investig. 2017;47:795–802.