Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

The role of baseline granularity for benchmarking citation impact. The case of CSS profiles

  • 144 Accesses

  • 2 Citations

Abstract

In this paper we study the effect of granularity on Characteristic Scores and Scales (CSS). Unlike the traditional indicators that are mostly based on means and quantiles, CSS require the reduction of the citation distributions collaboration of the underlying reference population to four states (classes) and thus higher a different level of granularity. While the question of the choice of granularity is at higher levels of aggregation usually not critical since countries and university have rather multidisciplinary profiles, at lower aggregation levels specialisation becomes more typical. Inappropriate granularity might not warrant the depiction of the publication profiles at these levels in a correct and adequate manner and thus not add accurate citation profiles either. In order to be able to process one complete annual volume of the Web of Science, we decided to calculate CSS thresholds and classes for two levels of granularity, namely sub-fields and WoS Subject Categories. With about 5% deviation, we did not find a real significance. However, we identified journals with similar impact measures but different citation profiles, independently of the granularity. Finally, we have pointed to the limitations in the choice of granularity—in terms of both too broad and too narrow subjects.

This is a preview of subscription content, log in to check access.

Fig. 1

(Data sourced from Clarivate Analytics Web of Science Core Collection)

Fig. 2

(Data sourced from Clarivate Analytics Web of Science Core Collection)

References

  1. Adams, J., Gurney, K., & Jackson, L. (2008). Calibrating the zoom—A test of Zitt’s hypothesis. Scientometrics, 75(1), 81–95.

  2. Adams, J., & Testa, J. (2011). Thomson Reuters book citation index. In E. Noyons, P. Ngulube, J. Leta (Eds.), The 13th conference of the international society for scientometrics and informetrics (Vol. I, pp. 13–18). Durban, South Africa: ISSI, Leiden University and the University of Zululand.

  3. Albarrán, P., & Ruiz-Castillo, J. (2011). References made and citations received by scientific articles. JASIST, 62(1), 40–49.

  4. Bookstein, A. (1997). Informetric distributions. 3. Ambiguity and randomness. Journal of the American Society for Information Science, 48(1), 2–10.

  5. Glänzel, W. (2007). Characteristic scores and scales. A bibliometric analysis of subject characteristics based on long-term citation observation. Journal of Informetrics, 1(1), 92–102.

  6. Glänzel, W. (2009). The multi-dimensionality of journal impact. Scientometrics, 78(2), 355–374.

  7. Glänzel, W. (2011). The application of characteristic scores and scales to the evaluation and ranking of scientific journals. Journal of Information Science, 37(1), 40–48.

  8. Glänzel, W., Meyer, M., Schlemmer, B., du Plessis, M., Thijs, B., Magerman, T., Debackere, K., Veugelers, R. (2003), NanotechnologyAnalysis of an emerging domain of scientific and technologic endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/nanotech_domain_study.pdf.

  9. Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.

  10. Glänzel, W., & Schubert, A. (1988). Characteristic scores and scales in assessing citation impact. Journal of Information Science, 14(2), 123–127.

  11. Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.

  12. Glänzel, W., Schubert, A., Thijs, B., & Debackere, K. (2009). Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics, 78(1), 165–188.

  13. Glänzel, W., & Thijs, B. (2017). The granularity of disciplinary structures for benchmarking citation impact. The case of CSS profiles. In Proceedings of ISSI 2015—The 15th international conference on scientometrics and informetrics (pp. 1190–1200). Wuhan, China.

  14. Glänzel, W., Thijs, B., & Debackere, K. (2014). The application of citation-based performance classes to the disciplinary and multidisciplinary assessment in national comparison and institutional research assessment. Scientometrics, 101(2), 939–952.

  15. Glänzel, W., Thijs, B., & Debackere, K. (2018). Citation classes: A distribution-based approach to profiling citation impact for evaluative purposes. In W. Glänzel et al. (Eds.), Springer handbook of science and technology indicators. Heidelberg: Springer. to be published.

  16. Glänzel, W., Verbeek, A., du Plessis, M., van Looy, B., Magerman, T., Thijs, B., Schlemmer, B., Debackere, K., Veugelers, R. (2004), Stem cellsAnalysis of an emerging domain of scientific and technological endeavour. Accessible via: https://www.ecoom.be/sites/ecoom.be/files/downloads/stemcells_domain_study.pdf.

  17. Glänzel, W., & Zhou, P. (2011). Publication activity, citation impact and bi-directional links between publications and patents in biotechnology. Scientometrics, 86(2), 505–525.

  18. Lehmann, F. (2013). Realität und imagination. Photographie in W. G. Sebalds Austerlitz und Michelangelo Antonionis blow up. Bamberg: University of Bamberg Press.

  19. Schubert, A., Glänzel, W., & Braun, T. (1989). Scientometric datafiles. A comprehensive set of indicators on 2649 journals and 96 countries in all major fields and subfields 1981–1985. Scientometrics, 16(1–6), 3–478.

  20. Thijs, B., Debackere, K., & Glänzel, W. (2017). Improved author profiling through the use of citation classes. Scientometrics, 111(2), 829–839.

  21. Vincze, I. (1974). Mathematical Statistics, 4th edn. University Script, Eötvös University Budapest (in Hungarian).

  22. Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005). Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalization. Scientometrics, 63(2), 373–401.

Download references

Acknowledgements

The present study is an extended version of an article presented at the 16th International Conference on Scientometrics and Informetrics, Wuhan (China), 16–20 October 2017 (Glänzel and Thijs 2017). In particular, the paper has been extended by adding a new option based on new data and results, which are presented in a new section on topic-based granularity. Figure 1 has been reproduced from Glänzel et al. (2009) with permission of the publisher. Table 1 presents data on subfield-based journal indicators shared with figures of Table 9 in Glänzel et al. (2018).

Author information

Correspondence to Wolfgang Glänzel.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Glänzel, W., Thijs, B. The role of baseline granularity for benchmarking citation impact. The case of CSS profiles. Scientometrics 116, 521–536 (2018). https://doi.org/10.1007/s11192-018-2747-1

Download citation

Keywords

  • Granularity
  • Citation impact
  • Characteristic scores and scales
  • Journal citation measures