Zero impact: a large-scale study of uncitedness

Abstract

This paper presents a large-scale study of the phenomenon ‘uncitedness’. A literature review indicates that uncitedness is related to at least three factors: Field, document type, and time. To explore these factors and their mutual influence further, and at much larger scale than previous studies, the paper focuses on seven subject areas (arts and humanities; social sciences; computer science; mathematics; engineering; medicine; physics and astronomy), seven document types (articles; reviews; notes; letters; conference papers; books; book chapters), and a 20-year publication window (1996–2015). Documents are searched in Scopus, and retrieved year-by-year, discipline-by-discipline, and for each individual document type (total: 29,472,184 documents; 7,508,741 uncited documents). The results show great variance in uncitedness ratios between subject areas and document types. This is probably caused by a somewhat tacitly agreed upon genre hierarchy existing in all subject areas, yet with important local traits and differences. The importance of the time-dimension is documented. Time to first citation varies a great deal between subject areas, and the uncitedness ratio is consequently shown to be quite sensitive to the length of citation windows.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    Article, letter, note and review were searched as DOCUMENT TYPE and further limited to SOURCE TYPE: Journal. Conference paper was searched as DOCUMENT TYPE and further limited to SOURCE TYPE: Conference proceeding. Book and book chapter were searched as DOCUMENT TYPE with no further limitation.

  2. 2.

    https://ufm.dk/en/research-and-innovation/statistics-and-analyses/bibliometric-research-indicator/bfi-rules-and-regulations.

References

  1. Arsenault, C., & Larivière, V. (2015). Is paper uncitedness a function of the alphabet? Paper presented at the 15th international society of scientometrics and informetrics conference, ISSI 2015.

  2. Becher, T. (1989). Academic tribes and territories. Buckingham: Open University Press.

    Google Scholar 

  3. Burrell, Q. L. (2012). Alternative thoughts on uncitedness. Journal of the American Society for Information Science and Technology, 63(7), 1466–1470.

    Article  Google Scholar 

  4. Burrell, Q. L. (2013). A stochastic approach to the relation between the impact factor and the uncitedness factor. Journal of Informetrics, 7(3), 676–682.

    Article  Google Scholar 

  5. Burton, R. E., & Kebler, R. W. (1960). The “half-life” of some scientific and technical literatures. American Documentation, 11(1), 18–22.

    Article  Google Scholar 

  6. Cuenca, A. M. B., Barbosa, M. M. A. L., Oliveira, K., Quinta, F., Alvarez, M. D. C. A., & França, I. J. (2017). Uncited articles in Brazilian public health journals. Revista de Saúde Publica, 51, 114.

    Article  Google Scholar 

  7. Drott, C. M. (1995). Reexamining the role of conference papers in scholarly communication. Journal of the American Society for Information Science, 46(4), 299–305.

    Article  Google Scholar 

  8. Egghe, L. (2013). The functional relation between the impact factor and the uncitedness factor revisited. Journal of Informetrics, 7(1), 183–189.

    Article  Google Scholar 

  9. Egghe, L., Guns, R., & Rousseau, R. (2011). Thoughts on uncitedness: Nobel laureates and fields medalists as case studies. Journal of the American Society for Information Science and Technology, 62(8), 1637–1644.

    Article  Google Scholar 

  10. Frandsen, T. F., & Nicolaisen, J. (2017). Rejoinder: Nobel Prize effects in citation networks. Journal of the Association for Information Science and Technology, 68(12), 2844–2845.

    Article  Google Scholar 

  11. Garvey, W. D., & Griffith, B. C. (1972). Communication and information processing within scientific disciplines: Empirical findings for Psychology. Information Storage and Retrieval, 8(3), 123–136.

    Article  Google Scholar 

  12. Garvey, W. D., Lin, N., Nelson, C. E., & Tomita, K. (1972). Research studies in patterns of scientific communication: II. The role of the national meeting in scientific and technical communication. Information Storage and Retrieval, 8(4), 159–169.

    Article  Google Scholar 

  13. Gopalakrishnan, S., Gopalakrishnan, S., Bathrinarayanan, A. L., & Tamizhchelvan, M. (2015). Uncited publications in MEMS literature: A bibliometric study. DESIDOC Journal of Library and Information Technology, 35(2), 113–123.

    Article  Google Scholar 

  14. Hamilton, D. P. (1990). Publishing by-and for?-the numbers. Science, 250, 1331–1332.

    Article  Google Scholar 

  15. Hamilton, D. P. (1991). Research papers: Who’s uncited now? Science, 251, 25.

    Article  Google Scholar 

  16. Hargens, L. L., & Bott, D. M. (1991). Letter to the editor. Science, 251, 1409.

    Article  Google Scholar 

  17. Heneberg, P. (2013). Supposedly uncited articles of Nobel laureates and Fields medalists can be prevalently attributed to the errors of omission and commission. Journal of the American Society for Information Science and Technology, 64(3), 448–454.

    Article  Google Scholar 

  18. Ho, Y. S., & Hartley, J. (2017). Sleeping beauties in psychology. Scientometrics, 110(1), 301–305.

    Article  Google Scholar 

  19. Hsu, J. W., & Huang, D. W. (2012). A scaling between impact factor and uncitedness. Physica A: Statistical Mechanics and Its Applications, 391(5), 2129–2134.

    MathSciNet  Article  Google Scholar 

  20. Hu, X. J., & Rousseau, R. (2013). Meeting abstracts: A waste of space? Current Science, 105(2), 150–151.

    Google Scholar 

  21. Hu, Z., & Wu, Y. (2014). Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals. Journal of Informetrics, 8(1), 136–146.

    Article  Google Scholar 

  22. Hyland, K. (2000). Disciplinary discourses: Social interactions in academic writing. Harlow: Pearson Education Ltd.

    Google Scholar 

  23. Kamat, P. V. (2018). Most cited versus uncited papers. What do they tell us? ACS Energy Letters, 3(9), 2134–2135.

    Article  Google Scholar 

  24. Koenig, M. E. (1983). Bibliometric indicators versus expert opinion in assessing research performance. Journal of the American Society for Information Science, 34(2), 136–145.

    Article  Google Scholar 

  25. Law, R., Lee, H. A., & Au, N. (2013). Which journal articles are uncited? The case of the Asia Pacific Journal of Tourism Research and the Journal of Travel and Tourism Marketing. Asia Pacific Journal of Tourism Research, 18(6), 661–684.

    Article  Google Scholar 

  26. Li, J. (2013). Uncited SSCI publications in China. Current Science, 104(11), 1462–1463.

    Google Scholar 

  27. Liang, L., Zhong, Z., & Rousseau, R. (2015). Uncited papers, uncited authors and uncited topics: A case study in library and information science. Journal of Informetrics, 9(1), 50–58.

    Article  Google Scholar 

  28. Liu, L., & Danziger, R. S. (1996). Fate of conference abstracts. Nature, 383, 20.

    Article  Google Scholar 

  29. Lou, W., & He, J. (2015). Does author affiliation reputation affect uncitedness? Proceedings of the Association for Information Science and Technology, 52(1), 1–4.

    Article  Google Scholar 

  30. Mackenzie, I. S. (2009). Citedness, uncitedness, and the murky world between. Paper presented at the 27th international conference extended abstracts on human factors in computing systems, CHI 2009, Boston, MA.

  31. Mavrogenis, A. F., Quaile, A., Pećina, M., & Scarlat, M. M. (2018). Citations, non-citations and visibility of International Orthopaedics in 2017. International Orthopaedics, 42(11), 2499–2505.

    Article  Google Scholar 

  32. Nane, T. (2015). Time to first citation estimation in the presence of additional information. In A. Salah, Y. Tonta, A. A. A. Salah, & C. Sugimoto (Eds.), Proceedings of the 15th international society of scientometrics and informetrics conference (ISSI-2015) (pp. 249–260). Istanbul: Bogazici University Printhouse. ISBN: 978-975-518-381-7.

  33. Pendlebury, D. A. (1991). Letter to the editor. Science, 251, 1410–1411.

    Article  Google Scholar 

  34. Price, D. J. D. S. (1965). Networks of scientific papers. Science, 149(3683), 510–515.

    Article  Google Scholar 

  35. Rosenberg, P. (2000). Chardin. London: Royal Academy of Arts.

    Google Scholar 

  36. Rosenkrantz, A. B., Chung, R., & Duszak, R., Jr. (2018). Uncited research articles in popular United States general radiology journals. Academic Radiology, 1, 11. https://doi.org/10.1016/j.acra.2018.04.011.

    Google Scholar 

  37. Sen, R., & Patel, P. (2012). Citation rates of award-winning ASCE papers. Journal of Professional Issues in Engineering Education and Practice, 138(2), 107–113.

    Article  Google Scholar 

  38. Swales, J. M. (2004). Research genres: Exploration and applications. Cambridge: Cambridge University Press.

    Google Scholar 

  39. Tahmasebi, S., Foroughi, Z., & Alizadeh-Navaei, R. (2017). Comparing the levels of non-citation of Iranian journals on health in Persian and English in scopus database. Journal of Mazandaran University of Medical Sciences, 26(146), 165–172.

    Google Scholar 

  40. Tang, R. (2008). Citation characteristics and intellectual acceptance of scholarly monographs. College and Research Libraries, 69(4), 356–369.

    Article  Google Scholar 

  41. Thelwall, M. (2016). Are there too many uncited articles? Zero inflated variants of the discretised lognormal and hooked power law distributions. Journal of Informetrics, 10(2), 622–633.

    Article  Google Scholar 

  42. van Dalen, H. P., & Henkens, K. (2004). Demographers and their journals: Who remains uncited after ten years? Population and Development Review, 30(3), 489–506.

    Article  Google Scholar 

  43. van Leeuwen, T. N., & Moed, H. F. (2005). Characteristics of journal impact factors: The effects of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics, 63(2), 357–371.

    Article  Google Scholar 

  44. van Raan, A. F. (2015). Dormitory of physical and engineering sciences: Sleeping beauties may be sleeping innovations. PLoS ONE, 10(10), e0139786. https://doi.org/10.1371/journal.pone.0139786.

    Article  Google Scholar 

  45. Wallace, M. L., Larivière, V., & Gingras, Y. (2009). Modeling a century of citation distributions. Journal of Informetrics, 3(4), 296–303.

    Article  Google Scholar 

  46. Watson, J. D., & Crick, F. H. C. (1953). Molecular structure of nucleic acids: A structure for deoxyribose nucleic acid. Nature, 171, 737–738.

    Article  Google Scholar 

  47. Yamashita, Y., & Yoshinaga, D. (2014). Influence of researchers’ international mobilities on publication: A comparison of highly cited and uncited papers. Scientometrics, 101(2), 1475–1489.

    Article  Google Scholar 

  48. Zhao, S. X. (2015). Uncitedness of reviews. Current Science, 109(8), 1377–1378.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Jeppe Nicolaisen.

Appendices

Appendix 1

See Table 3.

Table 3 All subject areas (1996–2015)

Appendix 2

See Table 4.

Table 4 Number of publications

Appendix 3

See Table 5.

Table 5 Number of uncited publications

Appendix 4

See Table 6.

Table 6 Uncitedness ratios

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nicolaisen, J., Frandsen, T.F. Zero impact: a large-scale study of uncitedness. Scientometrics 119, 1227–1254 (2019). https://doi.org/10.1007/s11192-019-03064-5

Download citation

Keywords

  • Uncitedness
  • Subject areas
  • Document types
  • Citation windows