, Volume 113, Issue 2, pp 1037–1057 | Cite as

Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data

  • Saeed-Ul HassanEmail author
  • Mubashir Imran
  • Uzair Gillani
  • Naif Radi Aljohani
  • Timothy D. Bowman
  • Fereshteh Didegah


This paper measures social media activities of 15 broad scientific disciplines indexed in Scopus database using data. First, the presence of data in Scopus database is investigated, overall and across disciplines. Second, a zero-truncated negative binomial model is used to determine the association of various factors with increasing or decreasing citations. Lastly, the effectiveness of altmetric indices to identify publications with high citation impact is comprehensively evaluated by deploying area under the curve (AUC)—an application of receiver operating characteristic. Results indicate a rapid increase in the presence of data in Scopus database from 10.19% in 2011 to 20.46% in 2015. It was found that Blog count was the most important factor in the field of Health Professions and Nursing as it increased the number of citations by 38.6%, followed by Twitter count increasing the number of citations by 8% in the field of Physics and Astronomy. The results of receiver operating characteristic show that altmetric indices can be a good indicator to discriminate highly cited publications, with an encouragingly AUC = 0.725 between highly cited publications and total altmetric count. Overall, findings suggest that altmetrics can be used to distinguish highly cited publications. The implications of this research are significant in many different directions. Firstly, they set the basis for a further investigation of altmetrics efficiency to predict publications impact and most significantly promote new insights for the measurement of research outcome dissemination over social media.


Altmetrics Scopus Comparative analysis Research evaluation 



We are thankful to for providing the dataset.


  1. Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1), 11–17.CrossRefGoogle Scholar
  2. Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. arXiv preprint arXiv:1205.5611.
  3. Boyack, K. W. & Klavans, R. (2005). Predicting the importance of current papers. In P. Ingwersen & B. Larsen (Ed.) Proceedings of ISSI 2005 (pp. 335–342). Stockholm, Sweden.Google Scholar
  4. Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the Association for Information Science and Technology, 57(8), 1060–1072.CrossRefGoogle Scholar
  5. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.CrossRefGoogle Scholar
  6. de Winter, J. C. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102(2), 1773–1779.CrossRefGoogle Scholar
  7. Didegah, F., Bowman, T. D., & Holmberg, K. (2017). On the differences between citations and altmetrics: An investigation of factors driving altmetrics vs. citations. Journal of the Association for information Science and Technology (in press).Google Scholar
  8. Didegah, F., & Thelwall, M. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics, 7(4), 861–873.CrossRefGoogle Scholar
  9. Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123. doi: 10.2196/jmir.2012.
  10. Haddawy, P., Hassan, S. U., Abbey, C. W., & Lee, I. B. (2017). Uncovering fine-grained research excellence: The global research benchmarking system. Journal of Informetrics, 11(2), 389–406.CrossRefGoogle Scholar
  11. Hassan, S. U., & Gillani, U. A. (2016). Altmetrics of” altmetrics” using Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki. arXiv preprint arXiv:1603.07992.
  12. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014a). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.CrossRefGoogle Scholar
  13. Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014b). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, 65(4), 656–669.CrossRefGoogle Scholar
  14. Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of informetrics, 5(3), 446–457.Google Scholar
  15. Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042. doi: 10.1007/s11192-014-1229-3.CrossRefGoogle Scholar
  16. Liu, X. Z., & Fang, H. (2017). What we can learn from tweets linking to research papers. Scientometrics, 111(1), 349–369.CrossRefGoogle Scholar
  17. Nielsen, F. (2007). Scientific citations in Wikipedia. arXiv preprint arXiv:0705.2106.
  18. Peoples, B. K., Midway, SR., Sackett, D., Lynch, A., & Cooney, P. B. (2016). Twitter predicts citation rates of ecological research. PLoS ONE, 11(11), e0166570.CrossRefGoogle Scholar
  19. Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7). Accessed 7 July 2017.
  20. Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv preprint arXiv:1203.4745.
  21. Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Available at:
  22. Ringelhan, S., Wollersheim, J., & Welpe, I. M. (2015). I like, I cite? Do facebook likes predict the impact of scientific work? PLoS ONE, 10(8), e0134389.CrossRefGoogle Scholar
  23. Shema, H., Bar-Ilan, J., & Thelwall, M. (2014). Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology, 65(5), 1018–1027.CrossRefGoogle Scholar
  24. Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143.CrossRefGoogle Scholar
  25. Sugimoto, C. R., Russell, T. G., Meho, L. I., & Marchionini, G. (2008). MPACT and citation impact: Two sides of the same scholarly coin? Library & Information Science Research, 30(4), 273–281.CrossRefGoogle Scholar
  26. Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology. doi: 10.1002/asi.23833.Google Scholar
  27. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.CrossRefGoogle Scholar
  28. Wouters, P., & Costas, R. (2012). Users, narcissism and control: tracking the impact of scholarly publications in the 21st century (pp. 847–857). Utrecht: SURF foundation.Google Scholar
  29. Xia, F., Su, X., Wang, W., Zhang, C., Ning, Z., & Lee, I. (2016). Bibliographic analysis of nature based on Twitter and Facebook Altmetrics data. PLoS ONE, 11(12), e0165997.CrossRefGoogle Scholar
  30. Yu, H. (2017). Context of altmetrics data matters: An investigation of count type and user category. Scientometrics, 111(1), 267–283.CrossRefGoogle Scholar
  31. Zahedi, Z., Costas, R., & Wouters, P. (2013, October). What is the impact of the publications read by the different mendeley users? Could they help to identify alternative types of impact? Paper presented at the PLoS ALM Workshop, San Francisco, CA.Google Scholar
  32. Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.Information Technology UniversityLahorePakistan
  2. 2.Faculty of Computing and Information TechnologyKing Abdulaziz UniversityJiddaKingdom of Saudi Arabia
  3. 3.School of Library and Information ScienceWayne State UniversityDetroitUSA
  4. 4.Faculty of Communication, Art and TechnologySimon Fraser UniversityVancouverCanada

Personalised recommendations