Abstract
Billions of dollars are wasted on low-quality research annually. Numerous preventable and systematic biases at the level of the research ecosystem, and lapses at the level of individual study, contribute to such deterioration in research. Clinical research is particularly susceptible to such systematic biases and lapses in conduct, due to the infiltration of financial and intellectual conflict of interest, among other factors. This chapter provides a variety of conceptual frameworks for identifying tendencies of low-quality research and proposes numerous strategies capable of reinforcing high-quality research. To the extent that clinical practice relies on a robust evidence base for high-quality care, low-quality research bears adverse consequences not only for investigators, institutions, and funders, but also for patients.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kohn L. To err is human: an interview with the Institute of Medicine’s Linda Kohn. Jt Comm J Qual Improv. 2000;26(4):227–34.
Makary MA, Daniel M. Medical error-the third leading cause of death in the US. BMJ. 2016;353:i2139.
Evidence-Based Medicine Working G. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–5.
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71–2.
James NT. Evidence based medicine. Scientific method and raw data should be considered. BMJ. 1996;313(7050):169–70. author reply 70-1
Worrall J. What evidence in evidence-based medicine? Philos Sci. 2002;69(3):S316–S30.
Lushington GH, Chaguturu R. Biomedical research: a house of cards? Future Med Chem. 2016;8(1):1–5.
Altman DG. The scandal of poor medical research. BMJ. 1994;308(6924):283–4.
Munafo MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, du Sert NP, et al. A manifesto for reproducible science. Nat Hum Behav. 2017;1(1)
Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.
Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
Freedman LP, Cockburn IM, Simcoe TS. The economics of reproducibility in preclinical research. PLoS Biol. 2015;13(6):e1002165.
Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 2009;4(5):e5738.
Ioannidis JP. How to make more published research true. PLoS Med. 2014;11(10):e1001747.
Moher D, Glasziou P, Chalmers I, Nasser M, Bossuyt PM, Korevaar DA, et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet. 2016;387(10027):1573–86.
Ioannidis JP. Research needs grants, funding and money—missing something? Eur J Clin Investig. 2012;42(4):349–51.
Chalmers I. What do I want from health research and researchers when I am a patient? BMJ. 1995;310(6990):1315–8.
Liberati A. Need to realign patient-oriented and commercial and academic research. Lancet. 2011;378(9805):1777–8.
Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75.
Ioannidis JP. Evolution and translation of research findings: from bench to where? PLoS Clin Trials. 2006;1(7):e36.
Kraus WL. Editorial: do you see what I see? Quality, reliability, and reproducibility in biomedical research. Mol Endocrinol. 2014;28(3):277–80.
Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gotzsche PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66.
Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005.
Young NS, Ioannidis JP, Al-Ubaydli O. Why current publication practices may distort science. PLoS Med. 2008;5(10):e201.
Begley CG, Ioannidis JP. Reproducibility in science: improving the standard for basic and preclinical research. Circ Res. 2015;116(1):116–26.
Rising K, Bacchetti P, Bero L. Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008;5(11):e217. discussion e
Fang FC, Casadevall A. Retracted science and the retraction index. Infect Immun. 2011;79(10):3855–9.
Ioannidis JP, Boyack KW, Klavans R. Estimates of the continuously publishing core in the scientific workforce. PLoS One. 2014;9(7):e101698.
Chiu K, Grundy Q, Bero L. ‘Spin’ in published biomedical literature: a methodological systematic review. PLoS Biol. 2017;15(9):e2002173.
Diong J, Butler AA, Gandevia SC, Heroux ME. Poor statistical reporting, inadequate data presentation and spin persist despite editorial advice. PLoS One. 2018;13(8):e0202121.
Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.
Ioannidis JPA. Meta-research: why research on research matters. PLoS Biol. 2018;16(3):e2005468.
Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii, ix-xi, 1-193
Rosenthal R. The “file drawer problem” and tolerance for null results. Psychol Bull. 1979;86(3):638–41.
Sekula P, Pressler JB, Sauerbrei W, Goebell PJ, Schmitz-Drager BJ. Assessment of the extent of unpublished studies in prognostic factor research: a systematic review of p53 immunohistochemistry in bladder cancer as an example. BMJ Open. 2016;6(8):e009972.
Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13(9):e1002127.
Ioannidis JP. Why science is not necessarily self-correcting. Perspect Psychol Sci. 2012;7(6):645–54.
Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202.
Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4.
Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gulmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.
Chalmers I, Atkinson P, Fenton M, Firkins L, Crowe S, Cowan K. Tackling treatment uncertainties together: the evolution of the James Lind initiative, 2003-2013. J R Soc Med. 2013;106(12):482–91.
Cooper NJ, Jones DR, Sutton AJ. The use of systematic reviews when designing studies. Clin Trials. 2005;2(3):260–4.
Jones AP, Conroy E, Williamson PR, Clarke M, Gamble C. The use of systematic reviews in the planning, design and conduct of randomised trials: a retrospective cohort of NIHR HTA funded trials. BMC Med Res Methodol. 2013;13:50.
Minelli C, Baio G. Value of information: a tool to improve research prioritization and reduce waste. PLoS Med. 2015;12(9):e1001882.
Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ. 1996;5(6):513–24.
Mohseninejad L, van Baal PH, van den Berg M, Buskens E, Feenstra T. Value of information analysis from a societal perspective: a case study in prevention of major depression. Value Health. 2013;16(4):490–7.
Carlson JJ, Thariani R, Roth J, Gralow J, Henry NL, Esmail L, et al. Value-of-information analysis within a stakeholder-driven research prioritization process in a US setting: an application in cancer genomics. Med Decis Mak. 2013;33(4):463–71.
Eeren HV, Schawo SJ, Scholte RH, Busschbach JJ, Hakkaart L. Value of information analysis applied to the economic evaluation of interventions aimed at reducing juvenile delinquency: an illustration. PLoS One. 2015;10(7):e0131255.
Crowe S, Fenton M, Hall M, Cowan K, Chalmers I. Patients’, clinicians’ and the research communities’ priorities for treatment research: there is an important mismatch. Res Involv Engagem. 2015;1:2.
Balas EA, Boren SA. Managing clinical knowledge for health care improvement. Yearb Med Inform. 2000;1:65–70.
Tallon D, Chard J, Dieppe P. Relation between agendas of the research community and the research consumer. Lancet. 2000;355(9220):2037–40.
Freedman LPVG, Wisman R. Reproducibility2020: progress and priorities. F1000Res. 2017;6:604.
Yordanov Y, Dechartres A, Porcher R, Boutron I, Altman DG, Ravaud P. Avoidable waste of research related to inadequate methods in clinical trials. BMJ. 2015;350:h809.
Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One. 2014;9(7):e102670.
Richter SH, Garner JP, Auer C, Kunert J, Wurbel H. Systematic variation improves reproducibility of animal experiments. Nat Methods. 2010;7(3):167–8.
Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. 2014;383(9912):176–85.
Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev. 2011;2011(1):MR000031.
Simmons JP, Nelson LD, Simonsohn U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci. 2011;22(11):1359–66.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.
Ioannidis JP. Perfect study, poor evidence: interpretation of biases preceding study design. Semin Hematol. 2008;45(3):160–6.
Altman DG. Poor-quality medical research: what can journals do? JAMA. 2002;287(21):2765–7.
Perneger TV, Ricou B, Boulvain M, Bovier PA, Herrmann FR, Perrier A, et al. Medical researchers evaluate their methodological skills. J Clin Epidemiol. 2004;57(12):1323–9.
Vandenbroucke JP. 175th anniversary lecture. Medical journals and the shaping of medical knowledge. Lancet. 1998;352(9145):2001–6.
Wynder EL, Higgins IT, Harris RE. The wish bias. J Clin Epidemiol. 1990;43(6):619–21.
Rosenthal R. On the social psychology of the psychological experiment: the experimenter’s hypothesis as unintended determinant of experimental results. Am Sci. 1963;51:268–83.
Djulbegovic B, Kumar A, Magazin A, Schroen AT, Soares H, Hozo I, et al. Optimism bias leads to inconclusive results-an empirical study. J Clin Epidemiol. 2011;64(6):583–93.
Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ. 2003;326(7404):1453–5.
Ioannidis JP. Why most discovered true associations are inflated. Epidemiology. 2008;19(5):640–8.
Panagiotou OA, Ioannidis JP. Primary study authors of significant studies are more likely to believe that a strong association exists in a heterogeneous meta-analysis compared with methodologists. J Clin Epidemiol. 2012;65(7):740–7.
Begley CG, Ellis LM. Drug development: raise standards for preclinical cancer research. Nature. 2012;483(7391):531–3.
Prinz F, Schlange T, Asadullah K. Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov. 2011;10(9):712.
Arrowsmith J. Trial watch: phase II failures: 2008-2010. Nat Rev Drug Discov. 2011;10(5):328–9.
Schork NJ. Personalized medicine: time for one-person trials. Nature. 2015;520(7549):609–11.
Moses H 3rd, Matheson DH, Cairns-Smith S, George BP, Palisch C, Dorsey ER. The anatomy of medical research: US and international comparisons. JAMA. 2015;313(2):174–89.
Iqbal SA, Wallach JD, Khoury MJ, Schully SD, Ioannidis JP. Reproducible research practices and transparency across the biomedical literature. PLoS Biol. 2016;14(1):e1002333.
Grande D, Frosch DL, Perkins AW, Kahn BE. Effect of exposure to small pharmaceutical promotional items on treatment preferences. Arch Intern Med. 2009;169(9):887–93.
Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153(3):158–66.
Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2017;2:MR000033.
Norris SL, Holmer HK, Ogden LA, Burda BU. Conflict of interest in clinical practice guideline development: a systematic review. PLoS One. 2011;6(10):e25153.
Campsall P, Colizza K, Straus S, Stelfox HT. Financial relationships between organizations that produce clinical practice guidelines and the biomedical industry: a cross-sectional study. PLoS Med. 2016;13(5):e1002029.
Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283(20):2701–11.
Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501.
De Meulemeester J, Fedyk M, Jurkovic L, Reaume M, Dowlatshahi D, Stotts G, et al. Many randomized clinical trials may not be justified: a cross-sectional analysis of the ethics and science of randomized clinical trials. J Clin Epidemiol. 2018;97:20–5.
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992;268(2):240–8.
Sawin VI, Robinson KA. Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. J Clin Epidemiol. 2016;69:174–8.
Ker K, Edwards P, Perel P, Shakur H, Roberts I. Effect of tranexamic acid on surgical bleeding: systematic review and cumulative meta-analysis. BMJ. 2012;344:e3054.
Gilbert R, Salanti G, Harden M, See S. Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. Int J Epidemiol. 2005;34(4):874–87.
Rosenblatt M. An incentive-based approach for improving data reproducibility. Sci Transl Med. 2016;8(336):336ed5.
Gelinas L, Lynch HF, Bierer BE, Cohen IG. When clinical trials compete: prioritising study recruitment. J Med Ethics. 2017;43(12):803–9.
Russell JF. If a job is worth doing, it is worth doing twice. Nature. 2013;496(7443):7.
Chalmers I, Nylenna M. A new network to promote evidence-based research. Lancet. 2014;384(9958):1903–4.
von Niederhausern B, Guyatt GH, Briel M, Pauli-Magnus C. Academic response to improving value and reducing waste: a comprehensive framework for INcreasing QUality in patient-oriented academic clinical REsearch (INQUIRE). PLoS Med. 2018;15(6):e1002580.
Bouwmeester W, Zuithoff NP, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, et al. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9(5):1–12.
von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP, et al. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–7.
Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527.
Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. J Pharmacol Pharmacother. 2010;1(2):100–7.
Clayton GL, Smith IL, Higgins JPT, Mihaylova B, Thorpe B, Cicero R, et al. The INVEST project: investigating the use of evidence synthesis in the design and analysis of clinical trials. Trials. 2017;18(1):219.
Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.
Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. 2017;390(10092):415–23.
Harris MR. The librarian’s roles in the systematic review process: a case study. J Med Libr Assoc. 2005;93(1):81–7.
Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
Goodman SN, Schneeweiss S, Baiocchi M. Using design thinking to differentiate useful from misleading evidence in observational research. JAMA. 2017;317(7):705–7.
Altman DG, Goodman SN, Schroter S. How statistical expertise is used in medical research. JAMA. 2002;287(21):2817–20.
Bakobaki JM, Rauchenberger M, Joffe N, McCormack S, Stenning S, Meredith S. The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial. Clin Trials. 2012;9(2):257–64.
Topol EJ. Money back guarantees for non-reproducible results? BMJ. 2016;353:i2770.
Saltelli A, Giampietro M. What is wrong with evidence based policy, and how can it be improved? Futures. 2017;91:62–71.
Fortier I, Burton PR, Robson PJ, Ferretti V, Little J, L’Heureux F, et al. Quality, quantity and harmony: the DataSHaPER approach to integrating data across bioclinical studies. Int J Epidemiol. 2010;39(5):1383–93.
Dong Y, Johnson RA, Chawla NV. Can scientific impact be predicted? IEEE Trans Big Data. 2016;2(1)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Cahan, E.M., Shea, K.G. (2022). Interpreting and Implementing Evidence for Quality Research. In: Samora, J.B., Shea, K.G. (eds) Quality Improvement and Patient Safety in Orthopaedic Surgery. Springer, Cham. https://doi.org/10.1007/978-3-031-07105-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-07105-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07104-1
Online ISBN: 978-3-031-07105-8
eBook Packages: MedicineMedicine (R0)