Coverage and adoption of altmetrics sources in the bibliometric community

Abstract

Altmetrics, indices based on social media platforms and tools, have recently emerged as alternative means of measuring scholarly impact. Such indices assume that scholars in fact populate online social environments, and interact with scholarly products in the social web. We tested this assumption by examining the use and coverage of social media environments amongst a sample of bibliometricians examining both their own use of online platforms and the use of their papers on social reference managers. As expected, coverage varied: 82 % of articles published by sampled bibliometricians were included in Mendeley libraries, while only 28 % were included in CiteULike. Mendeley bookmarking was moderately correlated (.45) with Scopus citation counts. We conducted a survey among the participants of the STI2012 participants. Over half of respondents asserted that social media tools were affecting their professional lives, although uptake of online tools varied widely. 68 % of those surveyed had LinkedIn accounts, while Academia.edu, Mendeley, and ResearchGate each claimed a fifth of respondents. Nearly half of those responding had Twitter accounts, which they used both personally and professionally. Surveyed bibliometricians had mixed opinions on altmetrics’ potential; 72 % valued download counts, while a third saw potential in tracking articles’ influence in blogs, Wikipedia, reference managers, and social media. Altogether, these findings suggest that some online tools are seeing substantial use by bibliometricians, and that they present a potentially valuable source of impact data.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. 1.

    As reported on mendeley.com on November 15, 2013.

  2. 2.

    Some presenters were omitted either because they had not published in sources covered by Scopus or WoS or due to ambiguous names, for which relevant papers could not be identified. Documents without a doi were not considered as it was needed to identify papers on some of the altmetrics platforms. For a more detailed description of data collection, see Bar-Ilan et al. (2012).

  3. 3.

    http://impactstory.org.

References

  1. Bar-Ilan, J. (2011). Articles tagged by ‘bibliometrics’ on Mendeley and CiteULike. Paper presented at the Metrics 2011 Symposium on Informetric and Scientometric Research.

  2. Bar-Ilan, J. (2012a). JASIST@mendeley. Presented at the ACM Web Science Conference Workshop on Altmetrics. Evanston, IL. Retrieved January 21, 2013 from http://altmetrics.org/altmetrics12/bar-ilan.

  3. Bar-Ilan, J. (2012b). JASIST 2001–2010. Bulletin of the American Society for Information Science and Technology, 38(6), 24–28.

    Google Scholar 

  4. Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. In Proceedings of the 17th International Conference on Science and Technology Indicators, Montréal, Canada (Vol. 1, pp. 98–109).

  5. Bar-Ilan, J., Shema, H., & Thelwall (2014). Bibliographic References in Web 2.0. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance (pp. 307–325). Cambridge, MA: MIT Press.

  6. Cronin, B., & Overfelt, K. (1994). The scholar’s courtesy: A survey of acknowledgement behaviour. Journal of Documentation, 50, 165–196.

    Article  Google Scholar 

  7. Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4). Retrieved January 21, 2013 from http://www.jmir.org/2011/4/e123.

  8. Ganegan, F. (2012, August). Filtering the research record and farming big data. Retrieved January 21, 2013 from http://www.swets.com/blog/filtering-the-research-record-and-farming-big-data#.

  9. Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory. Strategies for Qualitative Research. New Brunswick: Aldine Transactions.

    Google Scholar 

  10. Groth, P., & Gurney, T. (2010). Studying scientific discourse on the Web using bibliometrics: A chemistry blogging case study. Presented at the WebSci10: Extending the Frontiers of Society On-Line, Raleigh, NC, USA.

  11. Haustein, S. (2014). Readership Metrics. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance (pp. 327–344), Cambridge, MA: MIT Press.

  12. Haustein, S., Golov, E., Luckanus, K., Reher, S., & Terliesner, J. (2010). Journal evaluation and science 2.0. Using social bookmarks to analyze reader perception. In Book of Abstracts of the 11th International Conference on Science and Technology Indicators, (pp. 117–119). Leiden, The Netherlands.

  13. Haustein, S., & Peters, I. (2012). Using Social Bookmarks and Tags as Alternative Indicators of Journal Content Description. First Monday, 17(11). Retrieved January 21, 2013 from www.firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/4110/3357.

  14. Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.

    Google Scholar 

  15. Henning, V., & Reichelt, J. (2008). Mendeley: A Last.fm for research? In Proceedings of 4th IEEE International Conference on Escience, (pp. 327–328). Indianapolis, IN, USA.

  16. Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 1–64.

    Article  Google Scholar 

  17. Letierce, J., Passant, A., Decker, S., & Breslin, J.G. (2010). Understanding how Twitter is used to spread scientific messages. In Proceedings of the Web Science Conference, Raleigh, NC, USA.

  18. Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of the 17th International Conference on Science and Technology Indicators, (Vol. 2, pp. 451–551). Montréal, Canada.

  19. Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.

    Article  Google Scholar 

  20. MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis—A critical review. Journal of the American Society for Information Science, 40(5), 342–349.

    Article  Google Scholar 

  21. Mahrt, M., Weller, K., & Peters, I. (2013). Twitter in Scholarly Communication. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 399–410). New York: Peter Lang.

    Google Scholar 

  22. Martin, B. R., & Irvine, J. (1983). Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12(2), 61–90.

    Article  Google Scholar 

  23. Mas-Bleda, A., Thelwall, M., Kousha, K., & Aguillo, I. (2013). European highly cited scientists’ presence on the Web. In Proceedings of the 14th International Conference of the International Society for Scientometrics and Informetrics (Vol. II, pp. 1966–1969). Vienna, Austria

  24. Mendeley (2012). Mendeley Global Research Report. Retrieved January 21, 2013 from http://www.mendeley.com/global-research-report/#.UPxyUqPi58E.

  25. Mohammadi, E. & Thelwall, M. (in press). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the American Society for Information Science and Technology.

  26. Nielsen, F. (2007). Scientific citations in Wikipedia. First Monday, 12(8). Retrieved January 21, 2013 from http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1997/1872.

  27. Piwowar, H. (2013). Value all research products. Nature, 493, 159.

    Google Scholar 

  28. Ponte, D., & Simon, J. (2011). Scholarly communication 2.0: Exploring researchers’ opinions on Web 2.0 for scientific knowledge creation, evaluation and dissemination. Serials Review, 37(3), 149–156.

    Article  Google Scholar 

  29. Price, D., de Solla, J., & Gürsey, S. (1976). Studies in scientometrics I. Transience and continuance in scientific authorship. International Forum on Information and Documentation, 1(2), 17–24.

    Google Scholar 

  30. Priem, J. (2010). Tweet by Jason Priem on September 28, 2010. Retrieved January 21, 2013 from https://twitter.com/#!/jasonpriem/status/25844968813.

  31. Priem, J. (2014). Altmetrics. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance. Cambridge, MA: MIT Press.

  32. Priem, J., & Costello, K. (2010). How and why scholars cite on Twitter. In Proceedings of the 73rd Annual Meeting of the American Society for Information Science and Technology, Pittsburgh, PA, USA. doi: 10.1002/meet.14504701201/full.

  33. Priem, J., Costello, K., & Dzuba, T. (2011). First-year graduate students just wasting time? Prevalence and use of Twitter among scholars. Presented at the Metrics 2011 Symposium on Informetric and Scientometric Research, New Orleans, LA, USA. Retrieved January 21, 2013 from http://jasonpriem.org/self-archived/5uni-poster.png.

  34. Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. Retrieved January 21, 2013 from http://arxiv.org/abs/1203.4745.

  35. Priem, J., Taraborelli, D., Groth, P., & Nylon, C. (2010). alt-metrics: a manifesto. Retrieved January 21, 2013 from http://altmetrics.org/manifesto.

  36. Reher, S., & Haustein, S. (2010). Social bookmarking in STM: Putting services to the acid test. Online - Leading Magazine for Information Professionals, 34(6), 34–42.

    Google Scholar 

  37. Research Councils UK. (2011, March). Types of impact. Retrieved January 21, 2013 from http://www.rcuk.ac.uk/documents/impacts/TypologyofResearchImpacts.pdf.

  38. Rowlands, I., & Nicholas, D. (2007). The missing link: Journal usage metrics. Aslib Proceedings, 59(3), 222–228.

    Article  Google Scholar 

  39. Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013). Download vs. vitiation vs. readership data: The case of an information systems journal. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (Vol. 1, 626–634).

  40. Schlögl, C., & Stock, W. G. (2004). Impact and relevance of LIS journals: A scientometric analysis of international and German-language LIS journals—Citation analysis versus reader survey. Journal of the American Society for Information Science and Technology, 55(13), 1155–1168.

    Article  Google Scholar 

  41. Shema, H., Bar-Ilan, J., & Thelwall, M. (2012). Research blogs and the discussion of scholarly information. PLoS One, 7(5), e35869. doi:10.1371/journal.pone.0035869.

    Article  Google Scholar 

  42. Shema, H., Bar-Ilan, J., & Thelwall, M. (in press). Do blog citations correlate with higher number of future citations? Journal of the American Society for Information Science and Technology.

  43. Tenopir, C., & King, D. W. (2000). Towards electronic journals: Realities for scientists, librarians, and publishers. Washington, DC: Special Libraries Association.

    Google Scholar 

  44. Thelwall, M. (2010). Webometrics: emergent or doomed? Information Research: An International Electronic Journal, 15(4). Retrieved January 21, 2013 from http://informationr.net/ir/15-4/colis713.html.

  45. Thelwall, M. (2012). Journal impact evaluation: A webometric perspective. Scientometrics, 92, 429–441.

    Article  Google Scholar 

  46. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. (2013). Do altmetrics work? Twitter and ten other candidates. PLoS One, 8(5), e64841. doi:10.1371/journal.pone.0064841.

    Article  Google Scholar 

  47. Thelwall, M., Vaughan, L., & Björneborn, L. (2005). Webometrics. Annual Review of Information Science and Technology, 39(1), 81–135.

    Article  Google Scholar 

  48. Weller, K., & Peters, I. (2012). Citations in Web 2.0. In A. Tokar, M. Beurskens, S. Keuneke, M. Mahrt, I. Peters, C. Puschmann, et al. (eds.), Science and the Internet (pp. 211–224). Düsseldorf: Düsseldorf University Press.

  49. Weller, K, & Puschmann, C. (2011). Twitter for scientific communication: How can citations/references be identified and measured? In Proceedings of the 3rd ACM International Conference on Web Science, Koblenz, Germany. Retrieved January 21, 2013 from http://journal.webscience.org/500/1/153_paper.pdf.

  50. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT—information technology.

  51. Zahedi, Z., Costas, R., & Wouters, P. (2013). How well developed are altmetrics? Cross disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (Vol. 1, pp. 876–884).

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Judit Bar-Ilan.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Haustein, S., Peters, I., Bar-Ilan, J. et al. Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics 101, 1145–1163 (2014). https://doi.org/10.1007/s11192-013-1221-3

Download citation

Keywords

  • Altmetrics
  • Social media presence
  • Reference managers
  • Download counts
  • Citation counts