Case study in major quotation errors: a critical commentary on the Newcastle–Ottawa scale
- 312 Downloads
The Newcastle-Ottawa scale (NOS) is one of many scales used to judge the quality of observational studies in systematic reviews. It was criticized for its arbitrary definitions of quality items in a commentary in 2010 in this journal. That commentary was cited 1,250 times through December 2016. We examined the citation history of this commentary in a random sample of 100 full papers citing it, according to the Web of Science. Of these, 96 were systematic reviews, none of which quoted the commentary directly. All but 2 of the 96 indirect quotations (98%) portrayed the commentary as supporting use of the NOS in systematic reviews when, in fact, the opposite was the case. It appears that the vast majority of systematic review authors who cited this commentary did not read it. Journal reviewers and editors did not recognize and correct these major quotation errors. Authors should read each source they cite to make sure their direct and indirect quotations are accurate. Reviewers and editors should do a better job of checking citations and quotations for accuracy. It might help somewhat for commentaries to include abstracts, so that the basic content can be conveyed by PubMed and other bibliographic resources.
This work was supported by the German Federal Ministry of Education and Science (BMBF) [Grant no. 01ER1704]. The funding source had no role in the study design, in the collection, analysis and interpretation of data, in the writing of the report, and in the decision to submit the paper for publication.
Compliance with ethical standards
Conflict of interest
The authors declare that they have no conflict of interest.
- 1.Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle–Ottawa Scale (NOS) for assessing the quality if nonrandomized studies in meta-analyses. http://www.ohrica/programs/clinical_epidemiology/oxfordasp. 2009.
- 2.Reeves BC, Deeks JJ, Higgins JPT, Wells GA. Chapter 13: Including non-randomized studies. In: Higgins JBT, Green S, editors. Cochrane handbook of systematic reviews of interventions, version 510 (updated March 2011). www.handbook.cochrane.org: The cochrane collaboration. 2011.
- 10.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1–34. https://doi.org/10.1016/j.jclinepi.2009.06.006.CrossRefPubMedGoogle Scholar
- 12.Deeks JJ, Higgins JPT, Altman DG. Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S, editors. Cochrane handbook of systematic reviews of interventions, version 510 (updated March 2011). www.handbook.cochrane.org: The cochrane collaboration. 2011.
- 13.Garvin DA. What does product quality really mean? Sloan Manag Rev. 1984;26:25–43.Google Scholar
- 14.Dickie G. Aesthetics: an introduction. New York: The Bobbs-Merrill Company, Inc.; 1971.Google Scholar
- 16.Crosby PB. Quality is free. New York: McGraw-Hill; 1979.Google Scholar
- 17.Broh RA. Managing quality for higher profits. New York: McGraw-Hill; 1982.Google Scholar
- 18.Kuehn AA, Day RL. Strategy of product quality. Harv Bus Rev. 1962;40:100–10.Google Scholar
- 20.Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. Cochrane handbook for systematic reviews of interventions, version 510 (updated March 2011). www.handbook.cochrane.de: The cochrane collaboration. 2011.