Uncitedness in the Top General Medical Journals

Uncited research has received much attention due to its extent and impact on scholarship.1 It has been claimed that articles without a single citation do not have any constructive effect on future research and might be a waste of resources.1 Thus, the percent of uncited articles in any given journal could be considered as a measure of abundance of low-impact articles within that journal, very much like the impact factor (IF) that serves as an index of average impact of all published papers in a given journal.

METHODS

We investigated rate of uncitedness in five high-impact journals: the Annals of Internal Medicine (AIM), the British Medical Journal (BMJ), the Journal of the American Medical Association (JAMA), the Lancet, and the New England Journal of Medicine (NEJM). Using Web of Science (WOS),2 we calculated the rate of uncitedness in the aforementioned journals in 1990, 2000, 2010, and 2015. To compare, we also assessed the uncitedness for all general medical journals.

RESULTS

The mean rate of uncitedness at five years after publication progressively improved from 8.3 for the 1990 cohort to 7.1 and 0.7 for the 2000 and 2010 cohorts, respectively (Fig. 1). The same pattern was observed for the mean rate of uncited researches in the first year of publication (46.0, 38.0, 18.9, and 15.5 for 1990, 2000, 2010, and 2015 cohorts, respectively) and in the second year (16.1, 13.4, 2.7, and 1.7, respectively). There were no associations between the journals’ IF or the number of published articles, and the rate of uncited research (Spearman’s rho IF vs. uncitedness − 0.46, P = 0.18; Spearman’s rho articles vs. uncitedness 0.43, P = 0.21).

Figure 1
figure1

Rate of uncited articles in the five high-impact journals (anonymized as A–E) based on the published articles in 1990, 2000, 2010, and 2015. IF, impact factor; #, number of. Figure 1 and 2 contains poor quality of text inside the artwork. Please do not re-use the file that we have rejected or attempt to increase its resolution and re-save. It is originally poor, therefore, increasing the resolution will not solve the quality problem. We suggest that you provide us the original format. We prefer replacement figures containing vector/editable objects rather than embedded images. Preferred file formats are eps, ai, tiff and pdf. "Figures 2 please confirm if captured correctly.Thank you very much. We provided the highest possible quality of the figures in pdf and tiff formats.

Figure 2
figure2

a The association between the impact factor (IF) and uncitedness. The longitudinal uncitedness profile for the 2000 (b) and 2010 (c) cohorts was divided by the range of IF. #, number of.

Figure 2 a shows the association between the IF and uncitedness. Overall, there was a negative correlation, in which journals with greater IF had less uncitedness profile. However, this association was nonlinear: the uncitedness dropped sharply in the IF range of 0 to 1, then decreased with less slope for the IF range of 1 to 10, and almost reached a near-zero plateau for the journals with IF > 10. Figure 2 b and c show the longitudinal uncitedness profile for the 2000 and 2010 cohorts, respectively. Top medical journals showed a great leap toward less uncitedness in 2010 cohort compared with 2000, while other journals did not significantly improve

DISCUSSION

The overall rate of uncited research was low for all top general medical journals during the above mentioned years, except for one journal in 1990. This could be attributed to the individual policy of the journal or less likely misclassification of uncitable papers into citable articles via the WOS system. Providing unintentional self-citation for original articles by encouraging editorial or correspondence pieces for almost all articles might contribute to the very low rate of uncitedness.

We observed significant reduction in the rate of final uncitedness throughout the decades. This finding might have resulted from having more opportunity to access articles in the age of the Internet or due to fast-growing number of journals in medicine (e.g., 47.6% increase in the number of journals in the category of internal/general medicine), both of which could have enhanced the possibility of citation for an article.

A high IF actually reveals the presence of high-impact articles in a giving journal, rather than the absence or scarcity of low-impact articles. We propose that the uncitedness is more sensitive than IF for measuring the low-impact researches within a journal. Generally, journals with higher IF have lower percentage of uncitedness. However, this association cannot always be reproduced within a narrow category of journals with similar IF. Therefore, uncitedness might find its own way to classify the quality of journals, which could be elucidated later.

As a limitation, our results should be interpreted with caution, since it relied on the WOS. Although indexing databases are accurate on the basis of each source’s criteria,3 there are some discrepancies between them, especially on the citation counts.4 In addition, this study was conducted on general medical journals with special focus on the 5 selected high-impact journals. Therefore, the results of this article might not be generalized to the journals in other disciplines.

References

  1. 1.

    Van Noorden R. The science that’s never been cited. Nature. 2017;552(7684):162–4. doi:https://doi.org/10.1038/d41586-017-08404-0

    CAS  Article  PubMed  Google Scholar 

  2. 2.

    2017 Journal Impact Factor, Journal Citation Reports (Clarivate Analytics, 2018).

  3. 3.

    Nowroozzadeh MH. Bibliometrics: Database differences not citation errors. Nature. 2013;503(7476):342. doi: https://doi.org/10.1038/503342d

    CAS  Article  PubMed  Google Scholar 

  4. 4.

    Plana NM, Massie JP, Bekisz JM, Spore S, Diaz-Siso JR, Flores RL. Variations in databases used to assess academic output and citation impact. N. Engl. J. Med. 2017;376(25):2489–91. doi: https://doi.org/10.1056/NEJMc1616626

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors wish to thank Mr. H. Argasi at the Research Consultation Center (RCC) of Shiraz University of Medical Sciences for his invaluable assistance in editing this manuscript.

Funding

The datasets of the current study are available from the corresponding author on reasonable request.

Author information

Affiliations

Authors

Contributions

The authors contributed equally according to the ICMJE authorship criteria.

Corresponding author

Correspondence to Mohammad Salehi-Marzijarani PhD.

Ethics declarations

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Ethical Approval

Not applicable

Informed Consent

Not applicable

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nowroozzadeh, M.H., Salehi-Marzijarani, M. Uncitedness in the Top General Medical Journals. J GEN INTERN MED 34, 2695–2696 (2019). https://doi.org/10.1007/s11606-019-05290-2

Download citation