Scholarly Output Graph: A Graphical Article-Level Metric Indicating the Impact of a Scholar’s Publications

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10086)


Statistically, top scholars tend to accumulate a large number of publications during their tenure. While the patterns illustrating their scientific impact are monotonous and it is difficult to get a concrete comprehension to the academic development of the scholars’ output. So we address the issue of graphically presenting and comparing the impact of individual scholars’ publications. Besides, with the development of Web 2.0, more information about the social impact of a scholar’s work is becoming increasingly available and relevant. Thus comes the challenge of how to quickly compare among a scholar’s entire collection of publications, and pinpoint those with higher social popularity as well as academic influence. To this end, we propose a graphical article-level metric, namely Scholarly Output Graph (SOG). SOG captures three dimensions including journal impact factor (JIF), scientific impact and social popularity, and reflects not only the quality of the publications but also the immediate responses from social networks. With the visual cues of block length, width and color, users can intuitively locate articles of higher scientific impact, JIF and social popularity. Additionally, SOG proves to be widely applicable, practical and flexible as a navigation tool for filtering publications. To demonstrate the usability of SOG, we design a literature navigation homepage with a list of 50 researchers in computer science with their individual scholarly output graphs and the results can be found at


Graphical article-level metrics Visualization Scholarly Output Graph 



This work was supported in part by the Natural Science Foundation of China grant 61300087,61502069, 61672128; the Natural Science Foundation of Liaoning grant 2015020003; and by the Fundamental Research Funds for the Central Universities grant DUT15QY40, DUT16ZD(G)02.


  1. 1.
    Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., Terliesner, J.: Beyond citations: Scholars’ visibility on the social web. In: Proceedings of 17th International Conference on Science and Technology Indicators. 52900, pp. 98–109 (2012)Google Scholar
  2. 2.
    Haendel, M.A., Vasilevsky, N.A., Wirz, J.A.: Dealing with data: a case study on information and data management literacy. J. PLoS biol. 10(5), e1001339 (2012)CrossRefGoogle Scholar
  3. 3.
    Alexander, R., Sarah, D.R.: Accounting for Impact? the journal impact factor and the making of biomedical research in the Netherlands. J. Minerva 53(2), 117–139 (2015)CrossRefGoogle Scholar
  4. 4.
    Elliott, D.B.: The impact factor: a useful indicator of journal quality or fatally flawed? J. Ophthalmic Physiol. Optics 34(1), 4–7 (2014)CrossRefGoogle Scholar
  5. 5.
    Kamat, P.V., Schatz, G.C.: Journal impact factor and the real impact of your paper. J. Phys. Chem. Lett. 6(15), 3074–3075 (2015)CrossRefGoogle Scholar
  6. 6.
    Wenli, G.: Beyond journal impact and usage statistics: using citation analysis for collection development. Serials Libr. Printed Page Digital Age 70(1–4), 121–127 (2016)Google Scholar
  7. 7.
    Hirsch, J.: An index to quantify an individual’s scientific research output. Proc. Nat. Acad. Sci. 102(46), 16569–16572 (2005)CrossRefGoogle Scholar
  8. 8.
    Ball, P.: Index aims for fair ranking of scientists. Nature 436, 900 (2005)CrossRefGoogle Scholar
  9. 9.
    Glanzel, W.: On the opportunities and limitations of the h-index. Sci. Focus 1, 10–11 (2006)Google Scholar
  10. 10.
    Alonso, S., Cabrerizo, F.J., Herrera-Viedma, E., et al.: h-Index: a review focused in its variants.computation and standardization for different scientific fields. J. Informetrics 3(4), 273–289 (2009)CrossRefGoogle Scholar
  11. 11.
    Bar-Ilan, J., Levene, M.: The hw-rank: An h-index variant for ranking web pages. Scientometrics 102(3), 2247–2253 (2015)CrossRefGoogle Scholar
  12. 12.
    Van Eck, N.J., Waltman, L., van Raan, A.F.J., et al.: Citation analysis may severely underestimate the impact of clinical research as compared to basic research. PLoS One 8(4), e62395 (2013)CrossRefGoogle Scholar
  13. 13.
    Das, A.K., Mishra, S.: Genesis of altmetrics or article-level metrics for measuring efficacy of scholarly communications: Current perspectives. Scientometrics 39(2), 1–16 (2014)Google Scholar
  14. 14.
    Martin, F.: What can article-level metrics do for you? PLoS Biol. 10(11), e1001687 (2013)Google Scholar
  15. 15.
    Neylon, C.: Article-level metrics and the evolution of scientific impact. PLoS Biol. 7(11), e1000242 (2009)CrossRefGoogle Scholar
  16. 16.
    Shema, H., Bar-Ilan, J., Thelwall, M.: Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. J. Assoc. Inf. Sci. Technol. 65(5), 1018–1027 (2014)CrossRefGoogle Scholar
  17. 17.
    Lu, Z.: PubMed and beyond: a survey of web tools for searching biomedical literature. Database 2011(1), 56–65 (2011)Google Scholar
  18. 18.
    Priem, J., Hemminger, B.H.: Scientometrics 2.0: new metrics of scholarly impact on the social web. First Monday 15(7) (2010)Google Scholar
  19. 19.
    Liu, Y., Huang, Z., Yan, Y., et al.: Science Navigation Map: an interactive data mining tool for literature analysis. In: Proceedings of the 24th International Conference on World Wide Web Companion, International World Wide Web Conferences Steering Committee, pp. 591–596 (2015)Google Scholar
  20. 20.
    Adie, E., Roe, W.: Altmetric: enriching scholarly content with article-level discussion and metrics. Learn. Publish. 26(1), 11–17 (2013)CrossRefGoogle Scholar
  21. 21.
    Thelwall, M., Haustein, S., Larivire, V., et al.: Do altmetrics work? Twitter and ten other social web services (2013)Google Scholar
  22. 22.
    Haustein, S., Siebenlist, T.: Applying social bookmarking dat to evaluate journal usage. J. Informetrics 5(3), 446–457 (2011)Google Scholar
  23. 23.
    Gunn, W.: Social signals reflect academic impact: what it means when a scholar adds a paper to Mendeley. Inf. Stand. Q. 25(2), 33–39 (2013)CrossRefGoogle Scholar
  24. 24.
    Liu, Y., Huang, Z., Fang, J., Yan, Y.: An article level metric in the context of research community. In: Proceedings of the companion publication of the 23rd international conference on World Wide Web companion, pp. 1197–1202. International World Wide Web Conferences Steering Committee (2014)Google Scholar
  25. 25.
    Eysenbach, G.: Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. J. Med. Internet Res. 13(4), e123 (2011)CrossRefGoogle Scholar
  26. 26.
    Weller, K., Puschmann, C.: Twitter for scientific communication: how can citations/ references be identified and measured. In: Proceedings of the ACM WebSci 2011, pp. 1–4 (2011)Google Scholar
  27. 27.
    Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., Haustein, S.: Tweeting links to academic articles. Cybermetrics: Int. J. Scientometrics Informetrics Bibliometrics 17, 1–8 (2013)Google Scholar
  28. 28.
    Bollen, J., Van de Sompel, H., Smith, J.A., Luce, R.: Toward alternative metrics of journal impact: a comparison of download and citation data. Inf. Process. Manage. 41(6), 1419–1440 (2005)CrossRefGoogle Scholar
  29. 29.
    Zahedi, Z., Costas, R., Wouters, P.: How well developed are altmetrics? A cross-disciplinary analysis of the presence of alternative metrics in scientific publications. Scientometrics 101(2), 1491–1513 (2014)CrossRefGoogle Scholar
  30. 30.
    Zahedi, Z., Fenner, M., Costas, R.: How consistent are altmetrics providers? Study of 1000 PLoS ONE publications using the PLOS ALM, Mendeley and APIs. In altmetrics 14. Workshop at the Web Science Conference. Bloomington, USA (2014)Google Scholar
  31. 31.
    Li, X., Thelwall, M., Giustini, D.: Validating online reference managers for scholarly impact measurement. Scientometrics 91(2), 461–471 (2012)CrossRefGoogle Scholar
  32. 32.
    Ling, X., Liu, Y., Huang, Z., Shah, P.K., Li, C.: A graphical article-level metric for intuitive comparison of large-scale literatures. Scientometrics 106(1), 41–50 (2015)CrossRefGoogle Scholar
  33. 33.
    Alhoori, H., Kanan, T., Fox, E.A., Furuta, R., Giles, C.L., Pennsylvania, T.: On the relationship between open access and altmetrics. In: iConference 2015 Proceedings, pp. 1–8 (2015)Google Scholar
  34. 34.
    Bornmann, L.: What do altmetrics counts mean? A plea for content analyses. J. Assoc. Inf. Sci. Technol. 67(4), 1016–1017 (2016)CrossRefGoogle Scholar
  35. 35.
    Bornmann, L.: Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics. Scientometrics 103(3), 1123–1144 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.School of SoftwareDalian University of TechnologyDalianChina
  2. 2.Key Laboratory for Ubiquitous Network and Service Software of Liaoning ProvinceDalianChina

Personalised recommendations