Abstract
In the present study we discuss the challenge of “Scientometrics 2.0” as introduced by Priem and Hemminger (2010) in the light of possible applications to research evaluation. We use the Web of Science subject category public, environmental and occupational health to illustrate how indicators similar to those used in traditional scientometrics can be built, and we also discuss their opportunities and limitations. The discipline under study combines life sciences and social sciences in a unique manner and provides usable metrics reflecting both scholarly and wider impact. Nonetheless, metrics reflecting social media attention like tweets, retweets and Facebook likes, shares or comments are still subject to limitations in this research discipline as well. Furthermore, Usage metrics clearly point to the manipulation proneness of this measure. Although the counterparts of important bibliometric indicators proved to work for several altmetrics too, their interpretation and application to research assessment requires proper context analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bookstein, A. (1997). Informetric distributions. III. Ambiguity and randomness. JASIS, 48(1), 2–10.
Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. JASIST, 57(8), 1060–1072.
Chi, P.-S., & Glänzel, W. (2018). Comparison of citation and usage indicators in research assessment in scientific disciplines and journals. Scientometrics, 116(1), 537–554.
Chi, P.-S., & Glänzel, W. (2019). Citation and usage indicators for monographic literature in the Book Citation Index in the social sciences. ISSI Newsletter, 14(4), 80–86.
Chi, P.-S., Gorraiz, J., & Glänzel, W. (2019). Comparing capture, usage and citation indicators: An altmetric analysis of journal papers in chemistry disciplines. Scientometrics, 120(3), 1461–1473.
Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. JASIST, 66(10), 2003–2019.
Glänzel, W., & Chi, P.-S. (2016). Scientometrics 2.0—and beyond? Background, promises, challenges and limitations. ISSI Newsletter, 12(3), 33–36.
Glänzel, W., & Chi, P.-S. (2019). Research beyond scholarly communication—The big challenge of scientometrics 2.0. In G. Catalano, C. Daraio, M. Gregori, H. Moed, & G. Ruocco (Eds.), Proceedings of the ISSI conference 2019, Rome, Italy (pp 424–436)
Glänzel, W., & Thijs, B. (2011). Using ‘core documents’ for the representation of clusters and topics. Scientometrics, 88(1), 297–309.
Glänzel, W., & Thijs, B. (2012). Using ‘core documents’ for detecting and labelling new emerging topics. Scientometrics, 91(2), 399–416.
Glänzel, W., & Heeffer, S. (2014). Cross-national preferences and similarities in downloads and citations of scientific articles: A pilot study. In: E. Noyons (Ed.), Context counts: Pathways to master big and little data. Proceedings of the STI conference 2014, Leiden University (pp. 207–215)
Glänzel, W., & Schubert, A. (1988). Characteristic scores and scales in assessing citation impact. Journal of Information Science, 14(2), 123–127.
Gorraiz, J., Gumpenberger, C., & Schloegl, C. (2014). Usage versus citation behaviours in four subject areas. Scientometrics, 101(2), 1077–1095.
Gumpenberger, Ch, Glänzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977–982.
Hoffmann, Ch. P., Lutz, Ch., & Meckel, M. (2014). Impact factor 2.0: Applying social network analysis to scientific impact assessment. SSRN.
Lewison, G. (2004). Citations to papers from other documents. Evaluation of the practical effects of biomedical research. In H. F. Moed, et al. (Eds.), Handbook of quantitative science and technology research (pp. 457–472). Berlin: Springer.
Lewison, G. (2008). The returns to society from medical research (in Spanish). Medicina Clínica, 131(Suppl. 5), 42–47.
Moed, H. F. (2005). Statistical relationships between downloads and citations at the level of individual documents within a single journal. JASIST, 56(10), 1088–1097.
PlumX. (2019). PlumX metrics. Plum analytics. Accessible at: https://plumanalytics.com/learn/about-metrics/. Accessed 22 Nov 2019.
Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday. https://doi.org/10.5210/fm.v15i7.2874.
Sugimoto, C. (2016). Unlocking social data for science indicators. (White paper), NSF Workshop on Bibliometric Indicators, Arlington.
Thelwall, M. (2012). Journal impact evaluation: A webometric perspective. Scientometrics, 92(2), 429–441.
Thelwall, M. (2017a). Are Mendeley reader counts high enough for research evaluations when articles are published? Aslib Journal of Information Management, 69(2), 174–183.
Thelwall, M. (2017b). Are Mendeley reader counts useful impact indicators in all fields? Scientometrics, 113(3), 1721–1731.
Thelwall, M. (2018). Early Mendeley readers correlate with later citation counts. Scientometrics, 115(3), 1231–1240.
Wouters, P., Zahedi, Z., & Costas, R. (2019). Social media metrics for new research evaluation. In W. Glänzel, H. Moed, U. Schmoch, & M. Thelwall (Eds.), Handbook of science and technology indicators (pp. 687–714). Berlin: Springer.
Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513.
Acknowledgements
This paper is an extended version of a previous work presented at the 17th ISSI Conference in Rome, Italy (Glänzel and Chi 2019).
Author information
Authors and Affiliations
Corresponding author
Appendix
Rights and permissions
About this article
Cite this article
Glänzel, W., Chi, PS. The big challenge of Scientometrics 2.0: exploring the broader impact of scientific research in public health. Scientometrics 125, 1011–1031 (2020). https://doi.org/10.1007/s11192-020-03473-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-020-03473-x