Introduction

A citation is the acknowledgment that a scientific article has been referenced by another article [1]. The number of citations a published article receives can be considered a measure of its impact in the scientific community [1]. The citation rate directly influences scientometric indicators such as the h-index (an indicator of an author’s research performance [2]) and impact factor (an indicator of a journal’s prestige [3]). Open access refers to the practice of making research outputs such as scientific articles freely available online to all users. A possible benefit is that open access articles may be viewed and cited more frequently than articles that are only available to subscribers. However, open access publishing comes at a price: current prices for open access options in radiology journals range from $750 to $4,000 (median $3,000) [4]. Furthermore, there is currently no unambiguous evidence to support the hypothesis that open access articles are cited more frequently. A randomized prospective trial in physiology journals found no difference in citation rates between open access and subscription access articles in the first year after publication [5]. Studies in the field of oncology [6] and dentistry [7] found conflicting results. A recent study showed a higher citation rate for open access journals in general, but the effect was not uniform across different types of journals [8]. A recent study among radiology journals found no statistically significant differences in journal impact measures (including number of journal citations and impact factors) between journals with and without open access options [4]. To our knowledge, no such research has been performed yet for individual radiological articles. Therefore, the primary objective of our study was to investigate whether there is a difference in citation rate between open access articles and subscription access articles in the field of radiology. In order to measure the potential broader impact of open access publishing, the secondary objective was to investigate its effect with regard to download and share rate.

Methods

Ethics committee approval was not applicable for this literature study.

Data collection

A research fellow (R.H.M.A.) included all consecutive original articles published online in European Radiology between April 23, 2015, and July 6, 2017. Review articles (including guidelines, consensus developments, narrative reviews, and systematic reviews/meta-analyses), editorials, letters to the editor, and case reports were excluded. The following data were extracted for each included article: whether or not the article was published open access, continent of origin of the corresponding author, subspeciality (breast, cardiac, computed tomography, computer applications, contrast media, education, emergency, experimental, forensic medicine, gastrointestinal-abdominal, head-neck, health economy, magnetic resonance, molecular imaging, musculoskeletal, neuroradiology, nuclear medicine, oncology, pediatric, physics, ultrasound, urogenital, or vascular-interventional), mentioning of study findings in the article title, number of authors, number of references, length of the article in pages, number of days the article has been online (calculated as the number of days between the date the article was analyzed for this study and the date the article was published online on the European Radiology website, https://link.springer.com/journal/volumesAndIssues/330), and number of citations, number of downloads, and number of shares (as indicated on the European Radiology website (https://link.springer.com/journal/volumesAndIssues/330) on the date the article was analyzed for this study).

Statistical analysis

Statistical analyses were performed by using IBM SPSS Statistics for Windows (Version 20.0, IBM Corporation). Differences in dichotomous variables between open access and subscription access articles were assessed using the Pearson χ2 test. However, when the number of articles was ≤ five in one cell, Fisher’s exact test was used instead of the Pearson χ2 test. Differences in continuous variables were assessed using the Mann-Whitney U test. Adjustment for multiple testing was done using false positive rate control [9]. Multivariable linear regression analysis was performed to determine the association between open access and citation rate, adjusted for continent of origin of the corresponding author, subspeciality, study findings in the article title, number of authors, number of references, length of the article in pages, and number of days that the article has been online. The enter method was used for regression analysis, i.e., all independent variables were entered in a single step. Based on eight variables, approximately 500 articles were needed to be included to detect a small to medium effect size (f2 of 0.03) with a statistical power of 80% (alpha = 0.05) [10]. We also determined the association between open access and number of downloads and shares. Because articles from Europe and North America may have the benefit from some additional reputation, we also performed separate analyses for articles from these continents. P values less than 0.05 were considered statistically significant.

Results

A total of 500 original studies, of which 86 (17.2%) were open access and 414 were subscription access articles (82.8%), were included. Main characteristics of these studies are displayed in Table 1. Articles with corresponding authors from Europe or North America were significantly more frequently published open access than subscription access (66.3% vs. 52.8%, p = 0.024; and 20.9% vs. 8.6%, p = 0.001, respectively). Articles with corresponding authors from Asia were significantly less frequently published open access than subscription access (12.8% vs. 37.4%, p < 0.001). Open access articles were significantly more frequently cited than subscription access articles (mean number of citations 12.4 vs. 8.5, p = 0.011). Other metrics (subspeciality, mentioning of study findings in title, number of authors, number of references, length of article in pages, and number of days online) were not significantly different between open access and subscription access articles. Results of multivariable linear regression analyses are displayed in Table 2. Open access articles were significantly more frequently cited than subscription access articles (beta coefficient = 3.588, 95% confidence interval [CI] 0.668 to 6.508, p = 0.016). These results remained significant when we limited our analysis to articles from North American and Europe only (beta coefficient = 4.086, 95% CI 0.418 to 7.754, p = 0.029). Open access articles were also significantly more frequently downloaded (beta coefficient = 759.801, 95% CI 630.917 to 888.685, p < 0.001) and shared (beta coefficient = 0.748, 95% CI 0.124 to 1.372, p = 0.019) than subscription access articles.

Table 1 Main characteristics of articles included
Full size table
Table 2 Multivariate linear regression analysis on the association between open access publishing and number of citations, downloads, and shares. Displayed are B coefficients with 95% confidence interval between brackets and p value. Adjustment was also performed for continent of origin of the corresponding author and subspeciality
Full size table

Discussion

The results of our study show that open access articles in European Radiology are significantly and independently more frequently cited than subscription access articles. This can be explained by the facts that open access by definition does not require a journal subscription or payment of a fee to read the article, open access offers potentially faster and easier article access even to subscribers because there is no need to login, and open access articles are also published in PubMed Central, which improves article visibility. Altogether, this may increase the number of article reads and subsequent citations. Interestingly, articles from Europe and North America were significantly more frequently published open access. The opposite was true for articles with corresponding authors from Asia. It can be speculated that there are more European and North American institutions who have an agreement with the publisher to cover open access charges or there may be more study funders from Europe and North America who require open access publishing. Indeed, Springer, the publisher of European Radiology, has open access agreements with many institutions from Europe but none from Asia (https://www.springer.com/gp/open-access/springer-open-choice/springer-compact).

The author’s h-index and journal’s impact factor (which are based on citation rates) are still the most popular measures of research influence [11, 12] and they are frequently used to select candidates for positions, promotions, assignment of research grants, and even financial rewards [13, 14]. However, one may argue whether the number of citations an article receives is a good parameter to determine its scientific impact. Citation rate may rather be a measure of utility rather than of quality, because articles are not always cited because of their scientific merit. For instance, if an article has a flaw, other studies may just cite that article to point out its shortcoming. The number of times an article has been viewed or downloaded may provide a better reflection of article usage and dissemination of knowledge. The results of our study also show that open access articles in European Radiology are significantly and independently more frequently downloaded and shared than subscription access articles. A previous study also confirmed that open access articles are among the most downloaded articles [15]. Interestingly, they showed that there is only small overlap between the most downloaded and most cited articles [15].

Our study has some potential limitations. First, our study was a retrospective, nonrandomized observational study. Hypothetically, authors may have self-selected only those articles of higher than average quality for open access publication. However, a previous study found evidence that the citation advantage of open access articles was not because of a quality bias from authors self-selecting what to make open access [16]. Furthermore, we have attempted to control for several other article parameters which are possibly associated with being more citable. Second, we only included articles from European Radiology. Other renowned general radiology journals, such as Radiology and American Journal of Roentgenology, offer a combined type of open access, i.e., immediate open access for selected articles and delayed open access for all articles after 1 year. This would impede an unconfounded analysis with a follow-up of more than year after article publication. A follow-up time ≤ 1 year may be insufficiently long for published articles to cumulate citations. European Radiology only has one open access option at present, i.e., immediate open access for selected articles. This allowed us to perform a fair comparison with a mean follow-up of 1326 days for open access articles and 1340 days for subscription access articles. Third, our results only apply to original articles. We chose not to include review articles (including guidelines and systematic reviews/meta-analysis and guidelines), because these are more often highly cited and may be more often published open access. Moreover, original articles are the core of scientific research and the primary focus of most journals.

In conclusion, open access publishing is independently associated with an increased citation, download, and share rate in the field of radiology.