Abstract
At the center of this research lies the issue of properly counting international collaborations when assessing countries’ scientific productivity. Much of country-level scientometric research still uses the traditional “total counting” approach, wherein a country receives full credit for its international collaborations, as if it had produced every publication alone. For over a decade, various researchers have been showing how total counting distorts country outputs. However, the alternative, fractional counting methods, designed to eliminate the problem still have not prevailed. Hence more discussion and quantitative evidence is needed. In this article I study 40 average-productivity countries and find that total counting can result in even bigger distortion than the previous studies have shown. Namely, I show that total counting inflates some countries’ scientific impact as much as 12–13 times, rather than about 2 times, as observed with higher productivity countries. I also show that the degree of overcounting varies sharply across countries, often even resulting in a more productive country appearing behind a less productive one or vice versa. Based on the accumulated evidence, I suggest that total counting should be replaced with fractional counting more decisively, in most if not all of the research concerned with scientific productivity of countries.
Similar content being viewed by others
References
Abramo, G., Cicero, T., & D’Angelo, C. A. (2011). Assessing the varying level of impact measurement accuracy as a function of the citation window length. Journal of Informetrics,5(4), 659–667.
Burrows, S., & Moore, M. (2011). Trends in authorship order in biomedical research publications. Journal of Electronic Resources in Medical Libraries,8(2), 155–168.
Cavacini, A. (2016). Recent trends in middle eastern scientific production. Scientometrics,109(1), 423–432.
Coccia, M., & Wang, L. (2016). Evolution and convergence of the patterns of international scientific collaboration. Proceedings of the National Academy of Sciences,113(8), 2057–2061.
de Moya-Anegon, F., Guerrero-Bote, V. P., Lopez-Illescas, C., & Moed, H. F. (2018). Statistical relationships between corresponding authorship, international co-authorship and citation impact of national research systems. Journal of Informetrics,12(4), 1251–1262.
Egghe, L., Rousseau, R., & Van Hooydonk, G. (2000). Methods for accrediting publications to authors or countries: Consequences for evaluation studies. Journal of the American society for information science, 51(2), 145–157.
Gauffriau, M., & Larsen, P. O. (2005). Counting methods are decisive for rankings based on publication and citation studies. Scientometrics,64(1), 85–93.
Gauffriau, M., Larsen, P., Maye, I., Roulin-Perriard, A., & von Ins, M. (2008). Comparisons of results of publication counting using different methods. Scientometrics,77(1), 147–176.
Gazni, A., Sugimoto, C. R., & Didegah, F. (2012). Mapping world scientific collaboration: Authors, institutions, and countries. Journal of the American Society for Information Science and Technology,63(2), 323–335.
Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics,51(1), 69–115.
Gzoyan, E. G., Hovhannisyan, L. A., Aleksanyan, S. A., Ghazaryan, N. A., Hunanyan, S. R., Bourghida, A., et al. (2015). Comparative analysis of the scientific output of Armenia, Azerbaijan and Georgia. Scientometrics,102(1), 195–212.
Huang, M. H., Lin, C. S., & Chen, D. Z. (2011). Counting methods, country rank changes, and counting inflation in the assessment of national research productivity and impact. Journal of the American Society for Information Science and Technology,62(12), 2427–2436.
Ibrahim, B. (2018). Arab Spring’s effect on scientific productivity and research performance in Arab countries. Scientometrics,117(3), 1555–1586.
Jian, D., & Xiaoli, T. (2013). Perceptions of author order versus contribution among researchers with different professional ranks and the potential of harmonic counts for encouraging ethical co-authorship practices. Scientometrics,96(1), 277–295.
Karamourzov, R. (2012). The development trends of science in the CIS countries on the basis of some scientometric indicators. Scientometrics,91(1), 1–14.
Kutlača, D., Babić, D., Živković, L., & Štrbac, D. (2015). Analysis of quantitative and qualitative indicators of SEE countries scientific output. Scientometrics,102(1), 247–265.
Lancho-Barrantes, B. S., & Cantú-Ortiz, F. J. (2019). Science in Mexico: A bibliometric analysis. Scientometrics,118(2), 499–517.
Leydesdorff, L., & Wagner, C. S. (2008). International collaboration in science and the formation of a core group. Journal of informetrics,2(4), 317–325.
Moed, H. F. (2016). Iran’s scientific dominance and the emergence of South-East Asian countries as scientific collaborators in the Persian Gulf Region. Scientometrics,108(1), 305–314.
National Science Foundation. (2018). Science and engineering indicators 2018. www.nsf.gov. Retrieved August 27, 2019.
Osório, A. (2018). On the impossibility of a perfect counting method to allocate the credits of multi-authored publications. Scientometrics, 116(3), 2161–2173.
Parish, A. J., Boyack, K. W., & Ioannidis, J. P. (2018). Dynamics of co-authorship and productivity across different fields of scientific research. PLoS ONE,13(1), e0189742.
Persson, O., Glänzel, W., & Danell, R. (2004). Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics,60(3), 421–432.
Scimago Journal & Country Rank. (2019). Country rankings. https://www.scimagojr.com/. Retrieved December 30, 2019.
Scopus. (2019). Document search. http://scopus.com/. Retrieved May 31, 2019, August 30, 2019, December 30, 2019.
Shashnov, S., & Kotsemir, M. (2018). Research landscape of the BRICS countries: Current trends in research output, thematic structures of publications, and the relative influence of partners. Scientometrics,117(2), 1115–1155.
Strumia, A., & Torre, R. (2019). Biblioranking fundamental physics. Journal of Informetrics,13(2), 515–539.
Trueba, F. J., & Guerrero, H. (2004). A robust formula to credit authors for their publications. Scientometrics, 60(2), 181–204.
United Nations. (2019). Member states. http://un.org/. Retrieved August 28, 2019.
Waltman, L., & van Eck, N. J. (2015). Field-normalized citation impact indicators and the choice of an appropriate counting method. Journal of Informetrics,9(4), 872–894.
Zanotto, S. R., Haeffner, C., & Guimarães, J. A. (2016). Unbalanced international collaboration affects adversely the usefulness of countries’ scientific output as well as their technological and social impact. Scientometrics,109(3), 1789–1814.
Acknowledgements
I would like to thank my master’s thesis supervisor Steven Verheyen, researcher Rati Shubladze, rector of the University of Georgia, Tbilisi Konstantine Topuria, as well as my parents David Tarkhnishvili and Rusudan Mamradze for all the trust, patience, support and valuable advice they have provided at various stages on the rather long road towards producing my first scientific article.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
See Table 2.
Rights and permissions
About this article
Cite this article
Tarkhan-Mouravi, S. Traditional indicators inflate some countries’ scientific impact over 10 times. Scientometrics 123, 337–356 (2020). https://doi.org/10.1007/s11192-020-03372-1
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-020-03372-1