Are other scientific genealogies reporting alternative facts?
- 110 Downloads
Hirshman and colleagues performed a systematic review of publications by American authors that have reported on the survival effect of gross total surgical resection in patients with high-grade gliomas. They identified 108 scientific papers and now publish their fourth paper based on this review of the American glioma literature . Internet searches were performed to determine the authors’ medical subspecialty as well as the timing and location of their medical school, residency, and fellowship training. Links were drawn between authors if one was a faculty member at an institution while another was a trainee in the same discipline. To be connected, mentor and trainee had to be located at the same institution during the same calendar year. From this genealogic approach, the authors identified different scientific herds and previously reported that articles by genealogy members tend to share similar conclusions and that articles by members of the same genealogy tend to publish their results in certain journals . Thus, we scientists seem to stick to our herds and mentors when it comes to both study conclusions and publishing channels. They also reported that neurosurgical publications more often tend to support maximal resection than publications by radiation oncologists or medical oncologists . In the present article, based on the same review of the same 108 American glioma papers, the authors (or nearly the same authors) now quantify the previously reported impact of genealogy on publication results in some greater detail . However, the difference from their previous work is marginal. Their conclusion is (still) that the academic genealogy of scientific authors is predictive of their findings and this contributes to systematic bias in the published literature. Although Hirschman and colleagues are to be congratulated for their creative and important study approach, it still seems reasonable to point out the irony that the authors seem rather eager to salami publish their own study results while they criticize publication bias in the scientific literature. The authors even argue that “the consistency of the results between our previous and current study builds a strong case for the influence of genealogy on published literature.” Although validation of study findings is utterly important, salami-publishing from the same data material does not strengthen the evidence.
Even so, the authors’ findings are startling and deserve to be debated. Are really neurosurgeons and oncologists reaching different conclusions when they study the same thing, and if so why? Are the effects of surgery underestimated by the oncologists, overestimated by the neurosurgeons, or perhaps both? Do we simply inherit scientific truths from our herds or mentors? Is this for example why surgeons preferring microscopes vs. endoscopes in transsphenoidal surgery reach opposite conclusions in their systematic reviews [2, 12]? Or why awake surgery enthusiasts reach different conclusions in systematic reviews on the impact of awake surgery than others [4, 14]? Or is this why radiotherapy usually reduces recurrence rates in atypical meningioma if studies are done by radiation oncologists [18, 20, 21, 22] but not necessarily if the studies are done by neurosurgeons [7, 13, 15, 17]?
Is also our interpretation of the evidence different because we belong to different herds or genealogies where perhaps heretics are sanctioned and only accepted facts prevail? Does this make us read the scientific literature like the devil reads the bible? Is this why most vascular neurosurgeons refuse to believe results from the Aruba trial  or the Scottish AVM cohort study ? Or is this why many spine surgeons still like to fuse degenerative lumbar spines despite the negative trials [5, 6] (under the motto: screw the evidence)?
The present study is an important reminder that not only traditional conflicts of interest matter, but our background, herd, or subspecialty might affect scientific results significantly. However, this may not necessarily always be due to deliberate foul play. When designing studies, our scientific and professional background may affect both why the study was done in the first place, whether a variable is chosen as primary outcome or just included for adjustment in a multivariable model, whether the study was sufficiently powered to find an effect of this variable, and whether the study variable was measured in great detail or just crudely estimated (e.g., volumetric analysis of extents of resection vs. dichotomous gross estimations). Neurosurgeons and oncologists have different perspectives and administer different treatments at different time points. It is therefore perhaps not surprising that when neurosurgeons conduct trials on the effects of surgery in glioma, adjuvant treatment variables may be only crudely assessed or not taken into account  while oncologists fail to measure the extent of surgical resection when assessing the effects of adjuvant treatment . It seems likely that this difference in professional focus may affect study findings.
However, if genealogy has such a profound impact, can we still trust the scientific literature? Are we just preaching to our choirs in parallel scientific genealogies? Nietzsche claimed that “there are no facts, only interpretations.” Feyerabend, the Austrian science philosopher who is famous for his anarchistic view of science, went further. He claimed that modern science has deteriorated into a “stupid religion” where ideas by scientists claiming to represent the truth are held in high regards comparable to how the views of bishops and cardinals were accepted centuries ago. There are today a vast number of scientific journals, scientific boards, special interest groups, and conferences and there may perhaps be a congregation with “alternative facts” for everyone? A more attractive thought (or “alternative fact”) is perhaps that the Hirshman studies could be wrong. (Unless you are in their genealogy, this could work.) Hirshman and colleagues have for example previously reported (based on the same material again) that year of publication is associated with study results as the ratio of articles supporting the survival benefit of surgery has steadily increased over time . This might perhaps have to do with better study methods over time, better surgical tools, better and more widespread postoperative imaging, and advances in adjuvant treatment (synergic effects). From the current publication (Fig. 4 in the paper), one may calculate that median year of publication is significantly different between articles by the two studied genealogies, neurosurgeons (A) and radiation oncologists (B) (A = 2012 vs. B = 1996, p = 0.016; the Mann-Whitney U). This confounder was seemingly not adjusted for. We may therefore not have to deny science just yet. Quod erat demonstrandum (Q.E.D). What a relief.
- 1.Al-Shahi Salman R, White PM, Counsell CE, du Plessis J, van Beijnum J, Josephson CB, Wilkinson T, Wedderburn CJ, Chandy Z, St George EJ, Sellar RJ, Warlow CP (2014) Outcome after conservative management or intervention for unruptured brain arteriovenous malformations. JAMA 311:1661–1669. https://doi.org/10.1001/jama.2014.3200 CrossRefGoogle Scholar
- 3.Buckner JC, Shaw EG, Pugh SL, Chakravarti A, Gilbert MR, Barger GR, Coons S, Ricci P, Bullard D, Brown PD, Stelzer K, Brachman D, Suh JH, Schultz CJ, Bahary JP, Fisher BJ, Kim H, Murtha AD, Bell EH, Won M, Mehta MP, Curran WJ Jr (2016) Radiation plus procarbazine, CCNU, and vincristine in low-grade glioma. N Engl J Med 374:1344–1355. https://doi.org/10.1056/NEJMoa1500925 CrossRefGoogle Scholar
- 6.Ghogawala Z, Dziura J, Butler WE, Dai F, Terrin N, Magge SN, Coumans JV, Harrington JF, Amin-Hanjani S, Schwartz JS, Sonntag VK, Barker FG 2nd, Benzel EC (2016) Laminectomy plus fusion versus laminectomy alone for lumbar spondylolisthesis. N Engl J Med 374:1424–1434. https://doi.org/10.1056/NEJMoa1508788 CrossRefGoogle Scholar
- 7.Graffeo CS, Leeper HE, Perry A, Uhm JH, Lachance DJ, Brown PD, Ma DJ, Van Gompel JJ, Giannini C, Johnson DR, Raghunathan A (2017) Revisiting adjuvant radiotherapy after gross total resection of World Health Organization grade II meningioma. World Neurosurg 103:655–663. https://doi.org/10.1016/j.wneu.2017.04.095 CrossRefGoogle Scholar
- 10.Hirshman BR, Tang JA, Jones LA, Proudfoot JA, Carley KM, Marshall L, Carter BS, Chen CC (2016) Impact of medical academic genealogy on publication patterns: an analysis of the literature for surgical resection in brain tumor patients. Ann Neurol 79:169–177. https://doi.org/10.1002/ana.24569 CrossRefGoogle Scholar
- 11.Hirshman BR, Alattar A, Dhawan S, Carley KM, Chen CC (2019) Association between medical academic genealogy and publication outcome: impact of unconscious bias on scientific objectivity. Acta NeurochirGoogle Scholar
- 15.Masalha W, Heiland DH, Franco P, Delev D, Haaker JG, Schnell O, Scheiwe C, Grauvogel J (2018) Atypical meningioma: progression-free survival in 161 cases treated at our institution with surgery versus surgery and radiotherapy. J Neuro-Oncol 136:147–154. https://doi.org/10.1007/s11060-017-2634-2 CrossRefGoogle Scholar
- 16.Mohr JP, Parides MK, Stapf C, Moquete E, Moy CS, Overbey JR, Al-Shahi Salman R, Vicaut E, Young WL, Houdart E, Cordonnier C, Stefani MA, Hartmann A, von Kummer R, Biondi A, Berkefeld J, Klijn CJ, Harkness K, Libman R, Barreau X, Moskowitz AJ (2014) Medical management with or without interventional therapy for unruptured brain arteriovenous malformations (ARUBA): a multicentre, non-blinded, randomised trial. Lancet 383:614–621. https://doi.org/10.1016/s0140-6736(13)62302-8 CrossRefGoogle Scholar
- 17.Pereira BJA, de Almeida AN, Paiva WS, Teixeira MJ, Marie SKN (2018) Impact of radiotherapy in atypical meningioma recurrence: literature review. Neurosurg Rev. https://doi.org/10.1007/s10143-018-0959-8
- 18.Rydzewski NR, Lesniak MS, Chandler JP, Kalapurakal JA, Pollom E, Tate MC, Bloch O, Kruser T, Dalal P, Sachdev S (2018) Gross total resection and adjuvant radiotherapy most significant predictors of improved survival in patients with atypical meningioma. Cancer 124:734–742. https://doi.org/10.1002/cncr.31088 CrossRefGoogle Scholar
- 19.Stummer W, Pichlmeier U, Meinel T, Wiestler OD, Zanella F, Reulen HJ (2006) Fluorescence-guided surgery with 5-aminolevulinic acid for resection of malignant glioma: a randomised controlled multicentre phase III trial. Lancet Oncol 7:392–401. https://doi.org/10.1016/s1470-2045(06)70665-9 CrossRefGoogle Scholar
- 21.Weber DC, Ares C, Villa S, Peerdeman SM, Renard L, Baumert BG, Lucas A, Veninga T, Pica A, Jefferies S, Ricardi U, Miralbell R, Stelmes JJ, Liu Y, Collette L, Collette S (2018) Adjuvant postoperative high-dose radiotherapy for atypical and malignant meningioma: a phase-II parallel non-randomized and observation study (EORTC 22042-26042). Radiother Oncol 128:260–265. https://doi.org/10.1016/j.radonc.2018.06.018 CrossRefGoogle Scholar
- 22.Zhi M, Girvigian MR, Miller MJ, Chen JC, Schumacher AJ, Rahimian J, Lodin K (2018) Long-term outcomes of newly diagnosed resected atypical meningiomas and the role of adjuvant radiotherapy. World Neurosurg. https://doi.org/10.1016/j.wneu.2018.11.006