Skip to main content
SpringerLink
Log in

Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

This paper measures social media activities of 15 broad scientific disciplines indexed in Scopus database using Altmetric.com data. First, the presence of Altmetric.com data in Scopus database is investigated, overall and across disciplines. Second, a zero-truncated negative binomial model is used to determine the association of various factors with increasing or decreasing citations. Lastly, the effectiveness of altmetric indices to identify publications with high citation impact is comprehensively evaluated by deploying area under the curve (AUC)—an application of receiver operating characteristic. Results indicate a rapid increase in the presence of Altmetric.com data in Scopus database from 10.19% in 2011 to 20.46% in 2015. It was found that Blog count was the most important factor in the field of Health Professions and Nursing as it increased the number of citations by 38.6%, followed by Twitter count increasing the number of citations by 8% in the field of Physics and Astronomy. The results of receiver operating characteristic show that altmetric indices can be a good indicator to discriminate highly cited publications, with an encouragingly AUC = 0.725 between highly cited publications and total altmetric count. Overall, findings suggest that altmetrics can be used to distinguish highly cited publications. The implications of this research are significant in many different directions. Firstly, they set the basis for a further investigation of altmetrics efficiency to predict publications impact and most significantly promote new insights for the measurement of research outcome dissemination over social media.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. This dataset can be obtained by signing an MoU with altmetric.com, for research and development purpose.

  2. The SNIP 2015 data was downloaded from http://www.journalindicators.com.

References

  • Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1), 11–17.

    Article  Google Scholar 

  • Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. arXiv preprint arXiv:1205.5611.

  • Boyack, K. W. & Klavans, R. (2005). Predicting the importance of current papers. In P. Ingwersen & B. Larsen (Ed.) Proceedings of ISSI 2005 (pp. 335–342). Stockholm, Sweden.

  • Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the Association for Information Science and Technology, 57(8), 1060–1072.

    Article  Google Scholar 

  • Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.

    Article  Google Scholar 

  • de Winter, J. C. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102(2), 1773–1779.

    Article  Google Scholar 

  • Didegah, F., Bowman, T. D., & Holmberg, K. (2017). On the differences between citations and altmetrics: An investigation of factors driving altmetrics vs. citations. Journal of the Association for information Science and Technology (in press).

  • Didegah, F., & Thelwall, M. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics, 7(4), 861–873.

    Article  Google Scholar 

  • Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123. doi:10.2196/jmir.2012.

  • Haddawy, P., Hassan, S. U., Abbey, C. W., & Lee, I. B. (2017). Uncovering fine-grained research excellence: The global research benchmarking system. Journal of Informetrics, 11(2), 389–406.

    Article  Google Scholar 

  • Hassan, S. U., & Gillani, U. A. (2016). Altmetrics of” altmetrics” using Google Scholar, Twitter, Mendeley, Facebook, Google-plus, CiteULike, Blogs and Wiki. arXiv preprint arXiv:1603.07992.

  • Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014a). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.

    Article  Google Scholar 

  • Haustein, S., Peters, I., Sugimoto, C. R., Thelwall, M., & Larivière, V. (2014b). Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Science and Technology, 65(4), 656–669.

    Article  Google Scholar 

  • Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of informetrics, 5(3), 446–457.

    Google Scholar 

  • Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042. doi:10.1007/s11192-014-1229-3.

    Article  Google Scholar 

  • Liu, X. Z., & Fang, H. (2017). What we can learn from tweets linking to research papers. Scientometrics, 111(1), 349–369.

    Article  Google Scholar 

  • Nielsen, F. (2007). Scientific citations in Wikipedia. arXiv preprint arXiv:0705.2106.

  • Peoples, B. K., Midway, SR., Sackett, D., Lynch, A., & Cooney, P. B. (2016). Twitter predicts citation rates of ecological research. PLoS ONE, 11(11), e0166570.

    Article  Google Scholar 

  • Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7). http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2874/257. Accessed 7 July 2017.

  • Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv preprint arXiv:1203.4745.

  • Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Available at: http://altmetrics.org/manifesto/.

  • Ringelhan, S., Wollersheim, J., & Welpe, I. M. (2015). I like, I cite? Do facebook likes predict the impact of scientific work? PLoS ONE, 10(8), e0134389.

    Article  Google Scholar 

  • Shema, H., Bar-Ilan, J., & Thelwall, M. (2014). Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. Journal of the Association for Information Science and Technology, 65(5), 1018–1027.

    Article  Google Scholar 

  • Sud, P., & Thelwall, M. (2014). Evaluating altmetrics. Scientometrics, 98(2), 1131–1143.

    Article  Google Scholar 

  • Sugimoto, C. R., Russell, T. G., Meho, L. I., & Marchionini, G. (2008). MPACT and citation impact: Two sides of the same scholarly coin? Library & Information Science Research, 30(4), 273–281.

    Article  Google Scholar 

  • Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology. doi:10.1002/asi.23833.

    Google Scholar 

  • Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.

    Article  Google Scholar 

  • Wouters, P., & Costas, R. (2012). Users, narcissism and control: tracking the impact of scholarly publications in the 21st century (pp. 847–857). Utrecht: SURF foundation.

    Google Scholar 

  • Xia, F., Su, X., Wang, W., Zhang, C., Ning, Z., & Lee, I. (2016). Bibliographic analysis of nature based on Twitter and Facebook Altmetrics data. PLoS ONE, 11(12), e0165997.

    Article  Google Scholar 

  • Yu, H. (2017). Context of altmetrics data matters: An investigation of count type and user category. Scientometrics, 111(1), 267–283.

    Article  Google Scholar 

  • Zahedi, Z., Costas, R., & Wouters, P. (2013, October). What is the impact of the publications read by the different mendeley users? Could they help to identify alternative types of impact? Paper presented at the PLoS ALM Workshop, San Francisco, CA.

  • Zahedi, Z., Costas, R., & Wouters, P. (2014). How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. Scientometrics, 101(2), 1491–1513.

    Article  Google Scholar 

Download references

Acknowledgements

We are thankful to Altmetric.com for providing the dataset.

Author information

Authors and Affiliations

  1. Information Technology University, 346-B, Ferozepur Road, Lahore, Pakistan

    Saeed-Ul Hassan, Mubashir Imran & Uzair Gillani

  2. Faculty of Computing and Information Technology, King Abdulaziz University, Jidda, Kingdom of Saudi Arabia

    Naif Radi Aljohani

  3. School of Library and Information Science, Wayne State University, Detroit, MI, USA

    Timothy D. Bowman

  4. Faculty of Communication, Art and Technology, Simon Fraser University, Vancouver, BC, Canada

    Fereshteh Didegah

Authors
  1. Saeed-Ul Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mubashir Imran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Uzair Gillani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Naif Radi Aljohani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Timothy D. Bowman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fereshteh Didegah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeed-Ul Hassan.

Appendices

Appendix A

See Table 11.

Table 11 Multicollinearity results for the total sample
Full size table

Appendix B

See Figs. 4, 5 and 6.

Fig. 4
figure 4

ROC curve of SNIP and Ac to discriminate HC 1% papers agricultural, biological sciences and veterinary through economics, business and decision sciences

Full size image
Fig. 5
figure 5

ROC curve of SNIP and Ac to discriminate HC 1% papers engineering through medicine and medical sciences

Full size image
Fig. 6
figure 6

ROC curve of SNIP and Ac to discriminate HC 1% papers other life and health sciences through social sciences

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassan, SU., Imran, M., Gillani, U. et al. Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data. Scientometrics 113, 1037–1057 (2017). https://doi.org/10.1007/s11192-017-2512-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2512-x

Keywords

Search

Navigation