Background

Scientifc publishing is an important task for radiation oncologists pursuing an academic career (Holliday et al. 2014). Many of these physicians or physician-scientists undergo yearly rating, and are more or less forced to produce a certain number of publications or surrogate achievements such as impact factor (Holliday et al. 2013). Collaborative groups competing for research funding also feel the pressure of succeeding with clinical trials and publishing their results in prestigious journals (Nieder 2012). Many well-established journals with high impact factor have low acceptance rates of submitted manuscripts. Continuous increases in number of submissions and competition for space result in a need for additional publication channels. Publishing companies launching new journals face the challenges of attracting a sufficient number of scientifically sound manuscripts, achieving high visibility in search engines, and achieving indexing in databases such as PubMed. A rapidly increasing impact factor is also helpful for the continued success of new journals. During the second half of the 1990s, when the field was dominated by the “International Journal of Radiation Oncology, Biology and Physics” and “Radiotherapy and Oncology”, two attempts were made to establish additional specialty journals, “Radiation Oncology Investigations” and “Journal of Radiosurgery”. However, these traditional print journals without page charges did not succeed and were taken off the market after few volumes. We hypothesized that failure was caused by insufficient number of publications that were interesting and important enough to attract readers and impact clinical practice, and that this lack of appeal would be reflected by low number of citations for most published studies. In order to evaluate this hypothesis, patterns of citation for these two journals were analysed.

Materials and methods

A systematic search of the citation database Scopus (Elsevier B.V., http://www.elsevier.com/online-tools/scopus) by use of the function ‘document search’ was performed on 3rd of July 2014. All articles published in “Radiation Oncology Investigations” and the “Journal of Radiosurgery” were selected irrespective of subject area, document and article type (review, clinical study, experimental study, case report etc.). Then, these articles were ranked by number of citations (field ‘times cited’ in the Scopus citation database). As requested during manuscript review, we also performed comparisons to the “International Journal of Radiation Oncology, Biology and Physics”. The respective Scopus search was performed on 6th of February 2015.

Results

During the time period 1996-1999 “Radiation Oncology Investigations” published 205 articles, which achieved a median number of 6 citations (range 0-116). Figure 1 shows the distribution of citations. References (Seymour & Mothersill 1997; Sheridan et al. 1997; Desai et al. 1998; Schmidt-Ullrich et al. 1999; Smith & Haffty 1999; Johnson et al. 1998; Monga et al. 1999; Stickle et al. 1999; Merrick et al. 1998; Roach et al. 1997; Wazer et al. 1999; Joschko et al. 1997; Chidel et al. 1999; Durand & Olive 1997; Peschel et al. 1999; Norman et al. 1997; Kramer et al. 1998; Leborgne et al. 1997; Epperly et al. 1999; Chancy et al. 1998; Prete et al. 1998; Fernandez-Vicioso et al. 1997; Kang & Suh 1999; Banasiak et al. 1999; Nathu et al. 1998; Haffty et al. 1997; Gieger et al. 1997) and Table 1 include the 25 most frequently cited articles (all had at least 30 citations and were published in volumes 5-7, most emanated from the USA). The most frequently cited article in the first 4 volumes achieved 23 citations (Teicher et al. 1996). As also shown in Table 1, the median number of authors was 5 (range 2-10). Both clinical, dosimetric and radiobiological research as well as reviews achieved at least 30 citations. The most common topics were radiobiology (n = 8) and prostate cancer (n = 6). Topics covered all technological developments of that decade, e.g. brachytherapy, stereotactic radiotherapy and intensity-modulated radiotherapy. No final results of randomized studies or phase I-II collaborative group trials were published. The most common type of research were retrospective clinical studies (n = 10, Table 1). The median number of patients in these studies was limited (40.5, range 3-123).

Figure 1
figure 1

Patterns of citation of all articles published in Radiation Oncology Investigations (ROI) and the Journal of Radiosurgery (JRS).

Full size image
Table 1 The most frequently cited articles published in “Radiation Oncology Investigations” (in order of citation frequency, references 4-30)
Full size table

The “Journal of Radiosurgery” published 31 articles, all in the year 1999, which achieved a median number of 1 citation (range 0-11, Figure 1). References (Garell et al. 1999; Maire et al. 1999; Gibon et al. 1999; Solberg et al. 1999; Sanghavi et al. 1999) include the 5 most frequently cited articles (all had at least 5 citations and were published in volume 2). Both clinical and physics research as well as reviews achieved any citations. No randomized studies or phase I-II collaborative group trials were published.

The “International Journal of Radiation Oncology, Biology and Physics” published almost 2200 articles during the time period 1996-1999. These achieved a median number of 22 citations (range 0-1135). Overall, 9% of articles achieved at least 100 citations (p = 0.0002 in Chi-Square test comparing the three journals).

Discussion

The objective of this review was to identify pattern of scientific publication in two radiation oncology journals which were only temporarily available. While Radiation Oncology Investigations covered the whole field of radiation oncology, the Journal of Radiosurgery had a much narrower focus. After arbitrary decisions about which database to search (only those providing citation numbers could be considered for the purpose of this review), we performed a comprehensive evaluation of published research and number of citations accumulated during more than a decade after publication of the final journal volume. These citation numbers can be considered mature because previous analyses demonstrated that citation rate is gradually increasing for the first years after publication, followed by rapid decline (except for landmark randomized studies, which change clinical practice) (Stringer et al. 2010; Kondziolka 2011). Citation rate of published articles was chosen to define the most important contributions. Articles with high numbers of citations are likely those that impressed other clinicians/scientists and had impact on clinical practice or future developments in the field (Shao et al. 2013). It should be noticed that searches in different databases will result in more or less variable citation counts and that the present results therefore provide only a snapshot. This has been illustrated, e.g., in a cohort study of 328 articles published in JAMA, Lancet, or the New England Journal of Medicine between October 1, 1999, and March 31, 2000 (Kulkarni et al. 2009). Total citation counts for each article up to June 2008 were retrieved from Web of Science, Scopus, and Google Scholar. Google Scholar and Scopus retrieved more citations per article with a median of 160 and 149, respectively, than Web of Science (median 122, p < 0.001 for both comparisons). Importantly, Web of Science, Scopus and Google Scholar produced quantitatively and qualitatively different citation counts.

In a previous study of radiosurgery for various conditions, 1.5% of all articles (time period 1951-2010) achieved more than 100 citations (Kondziolka 2011). This figure corresponds well to that of articles in Radiation Oncology Investigations (3 out of 205, 1.5%), while the “International Journal of Radiation Oncology, Biology and Physics” achieved significantly higher proportions during the same time interval (1996-1999; 9%). The Top 100 radiation oncology articles from the time period 1999-2001 achieved a median of 208 citations (range 121-1149) (Nieder et al. 2013a), i.e. more than any of the publications reviewed here. When comparing articles published during different time periods, sources of bias must be acknowledged (variable temporal patterns of citation, increasing use of online databases in recent years, better access to published articles).

An analysis restricted to German radiation oncology publications revealed that most citations per year since publication were recorded for meta-analyses and randomized phase III trials (Nieder 2012). Lower figures were recorded for review articles, non-phase III prospective clinical trials, and retrospective clinical studies. Another analysis demonstrated that pattern of publication of the most influential radiation oncology studies was dominated by only two scientific journals: the Journal of Clinical Oncology and the International Journal of Radiation Oncology Biology and Physics (Nieder et al. 2013a). Several newly launched journals (first issue after 1999) managed to attract highly cited articles (Lancet Oncology, Nature Reviews Clinical Oncology or Nature Reviews Cancer, which had rapidly increasing impact factors). Apparently, despite controversy around impact factors and optimal evaluation of research productivity and quality (Kanaan et al. 2011; Durieux & Gevenois 2010), researchers find it attractive and desirable to publish their most important radiation oncology related work in the top journals of the field. The present findings are in line with these considerations.

Highly cited work is unlikely to be published in the first volumes of new journals. Prospective randomized trials and non-randomized trials performed by influential collaborative groups are prefentially submitted to well-established journals. It has also been shown that five newly established oncology journals, in this case open access publications without print issues, all published less than 50 articles in their first annual volume (Nieder et al. 2013b). First after 4-5 years number of articles per volume increased sharply. At that time, the two journals analysed here were already taken off the market. During the time period in question (1996-1999), no other new radiation oncology journals entered the market. However, afterwards two successful journals were established (in 2006 “Radiation Oncology” (open access, no print issues); in 2011 “Practical Radiation Oncology” (print issues, sister journal to the “International Journal of Radiation Oncology, Biology and Physics”)). Regarding “Radiation Oncology”, the annual volumes 2006-2008 all consisted of less than 50 articles. In 2009, contents expanded to 71 articles, while the most recent volume (2014) featured approximately 300. On the one hand, one could hypothesize that open access fees might prevent authors from submitting to journals like “Radiation Oncology”. On the other hand, advantages such as rapid publication and unlimited distribution and access appear to outweigh financial considerations. Institutional membership or publication funds and waiver of fees for authors from countries with limited resources might also encourage researchers to publish in open access journals.

Conclusions

Highly cited research typically appears in relatively few well-established journals. The vast majority of articles published in temporarily available radiation oncology journals had limited clinical impact and achieved few citations, especially those accepted in the initial phase of marketing.