, Volume 116, Issue 1, pp 537–554 | Cite as

Comparison of citation and usage indicators in research assessment in scientific disciplines and journals

  • Pei-Shan ChiEmail author
  • Wolfgang Glänzel


Proceeding from the results of an earlier pilot study usage and citation impact data from the 2013 annual volume of the Web of Science Core Collection were collected and used to build indicators. Besides basic indicators on citation impact, usage counts and international collaboration, we applied the method of Characteristic Scores and Scales to analyse the distributions of citations and usage counts to further test the relation between the usage and citation impact. The results could confirm and extend the results of the previous study to a larger set of subjects and countries. In addition, a new journal metric, the Journals Usage Index, is proposed to supplement journal impact measures by a usage based index.


Usage data Characteristic Scores and Scales Journals Usage Index Relative citation rate Relative usage rate 



This paper is an extended version of a previous work presented at the 16th ISSI Conference in Wuhan, China (Chi and Glänzel 2017b). The authors thank Clarivate Analytics for providing usage data of the 2013 annual volume of the Web of Science Core Collection.


  1. Albarrán, P., & Ruiz-Castillo, J. (2011). References made and citations received by scientific articles. JASIST, 62(1), 40–49.CrossRefGoogle Scholar
  2. Braun, T., Glänzel, W., & Schubert, A. (1985). Scientometric indicators. A 32 country comparison of publication productivity and citation impact. Singapore: World Scientific Publishing.Google Scholar
  3. Chi, P. S., & Glänzel, W. (2017a). An empirical investigation of the associations among usage, scientific collaboration and citation impact. Scientometrics, 112(1), 403–412.CrossRefGoogle Scholar
  4. Chi, P. S., & Glänzel, W. (2017b). Impact and usage indicators for the assessment of research in scientific disciplines and journals. In Proceedings of the ISSI conference 2017. Wuhan University (pp. 203–213).Google Scholar
  5. Clarivate Analytics. (2017). Usage count at web of science core collection help page. Retrieved on December 14, 2017.
  6. Glänzel, W. (2007). Characteristic scores and scales. A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92–102.CrossRefGoogle Scholar
  7. Glänzel, W. (2010). On reliability and robustness of scientometrics indicators based on stochastic models. An evidence-based opinion paper. Journal of Informetrics, 4(3), 313–319.CrossRefGoogle Scholar
  8. Glänzel, W., & Heeffer, S. (2014). Cross-national preferences and similarities in downloads and citations of scientific articles: A pilot study. In E. Noyons (Ed.), Context counts: Pathways to master big and little data. Proceedings of the STI conference 2014. Leiden University (pp. 207–215).Google Scholar
  9. Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.CrossRefGoogle Scholar
  10. Glänzel, W., Thijs, B., & Chi, P. S. (2016). The challenges to expand bibliometric studies from periodical literature to monographic literature with a new data source: The Book Citation Index. Scientometrics, 109(3), 2165–2179.CrossRefGoogle Scholar
  11. Glänzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.CrossRefGoogle Scholar
  12. Glänzel, W., Thijs, B., & Debackere, K. (2018). Citation classes—A distribution-based approach to profiling citation impact for evaluative purposes. In W. Glänzel, H. F. Moed, U. Schmoch, & M. Thelwall (Eds.), Springer handbook of science and technology indicators. Heidelberg: Springer.Google Scholar
  13. Pringle, J., Nefedov, N., & Pop-Lazarov, E. (2015). A new interest indicator based on researcher behavior in the Web of Science. Presentation at ASIS&T SIG/MET metrics 2015 workshop. Retrieved on December 14, 2017.
  14. Schubert, A., & Braun, T. (1986). Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics, 9(5), 281–291.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.ECOOMKU LeuvenLouvainBelgium
  2. 2.ECOOM and Department of MSIKU LeuvenLouvainBelgium
  3. 3.Dept Science Policy and Scientometrics, Library of the Hungarian Academy of SciencesBudapestHungary

Personalised recommendations