Uncited research has received much attention due to its extent and impact on scholarship.1 It has been claimed that articles without a single citation do not have any constructive effect on future research and might be a waste of resources.1 Thus, the percent of uncited articles in any given journal could be considered as a measure of abundance of low-impact articles within that journal, very much like the impact factor (IF) that serves as an index of average impact of all published papers in a given journal.
We investigated rate of uncitedness in five high-impact journals: the Annals of Internal Medicine (AIM), the British Medical Journal (BMJ), the Journal of the American Medical Association (JAMA), the Lancet, and the New England Journal of Medicine (NEJM). Using Web of Science (WOS),2 we calculated the rate of uncitedness in the aforementioned journals in 1990, 2000, 2010, and 2015. To compare, we also assessed the uncitedness for all general medical journals.
The mean rate of uncitedness at five years after publication progressively improved from 8.3 for the 1990 cohort to 7.1 and 0.7 for the 2000 and 2010 cohorts, respectively (Fig. 1). The same pattern was observed for the mean rate of uncited researches in the first year of publication (46.0, 38.0, 18.9, and 15.5 for 1990, 2000, 2010, and 2015 cohorts, respectively) and in the second year (16.1, 13.4, 2.7, and 1.7, respectively). There were no associations between the journals’ IF or the number of published articles, and the rate of uncited research (Spearman’s rho IF vs. uncitedness − 0.46, P = 0.18; Spearman’s rho articles vs. uncitedness 0.43, P = 0.21).
Figure 2 a shows the association between the IF and uncitedness. Overall, there was a negative correlation, in which journals with greater IF had less uncitedness profile. However, this association was nonlinear: the uncitedness dropped sharply in the IF range of 0 to 1, then decreased with less slope for the IF range of 1 to 10, and almost reached a near-zero plateau for the journals with IF > 10. Figure 2 b and c show the longitudinal uncitedness profile for the 2000 and 2010 cohorts, respectively. Top medical journals showed a great leap toward less uncitedness in 2010 cohort compared with 2000, while other journals did not significantly improve
The overall rate of uncited research was low for all top general medical journals during the above mentioned years, except for one journal in 1990. This could be attributed to the individual policy of the journal or less likely misclassification of uncitable papers into citable articles via the WOS system. Providing unintentional self-citation for original articles by encouraging editorial or correspondence pieces for almost all articles might contribute to the very low rate of uncitedness.
We observed significant reduction in the rate of final uncitedness throughout the decades. This finding might have resulted from having more opportunity to access articles in the age of the Internet or due to fast-growing number of journals in medicine (e.g., 47.6% increase in the number of journals in the category of internal/general medicine), both of which could have enhanced the possibility of citation for an article.
A high IF actually reveals the presence of high-impact articles in a giving journal, rather than the absence or scarcity of low-impact articles. We propose that the uncitedness is more sensitive than IF for measuring the low-impact researches within a journal. Generally, journals with higher IF have lower percentage of uncitedness. However, this association cannot always be reproduced within a narrow category of journals with similar IF. Therefore, uncitedness might find its own way to classify the quality of journals, which could be elucidated later.
As a limitation, our results should be interpreted with caution, since it relied on the WOS. Although indexing databases are accurate on the basis of each source’s criteria,3 there are some discrepancies between them, especially on the citation counts.4 In addition, this study was conducted on general medical journals with special focus on the 5 selected high-impact journals. Therefore, the results of this article might not be generalized to the journals in other disciplines.
Van Noorden R. The science that’s never been cited. Nature. 2017;552(7684):162–4. doi:https://doi.org/10.1038/d41586-017-08404-0
2017 Journal Impact Factor, Journal Citation Reports (Clarivate Analytics, 2018).
Nowroozzadeh MH. Bibliometrics: Database differences not citation errors. Nature. 2013;503(7476):342. doi: https://doi.org/10.1038/503342d
Plana NM, Massie JP, Bekisz JM, Spore S, Diaz-Siso JR, Flores RL. Variations in databases used to assess academic output and citation impact. N. Engl. J. Med. 2017;376(25):2489–91. doi: https://doi.org/10.1056/NEJMc1616626
The authors wish to thank Mr. H. Argasi at the Research Consultation Center (RCC) of Shiraz University of Medical Sciences for his invaluable assistance in editing this manuscript.
The datasets of the current study are available from the corresponding author on reasonable request.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Nowroozzadeh, M.H., Salehi-Marzijarani, M. Uncitedness in the Top General Medical Journals. J GEN INTERN MED 34, 2695–2696 (2019). https://doi.org/10.1007/s11606-019-05290-2