Scientometrics

, Volume 101, Issue 2, pp 1145–1163

Coverage and adoption of altmetrics sources in the bibliometric community

  • Stefanie Haustein
  • Isabella Peters
  • Judit Bar-Ilan
  • Jason Priem
  • Hadas Shema
  • Jens Terliesner
Article

Abstract

Altmetrics, indices based on social media platforms and tools, have recently emerged as alternative means of measuring scholarly impact. Such indices assume that scholars in fact populate online social environments, and interact with scholarly products in the social web. We tested this assumption by examining the use and coverage of social media environments amongst a sample of bibliometricians examining both their own use of online platforms and the use of their papers on social reference managers. As expected, coverage varied: 82 % of articles published by sampled bibliometricians were included in Mendeley libraries, while only 28 % were included in CiteULike. Mendeley bookmarking was moderately correlated (.45) with Scopus citation counts. We conducted a survey among the participants of the STI2012 participants. Over half of respondents asserted that social media tools were affecting their professional lives, although uptake of online tools varied widely. 68 % of those surveyed had LinkedIn accounts, while Academia.edu, Mendeley, and ResearchGate each claimed a fifth of respondents. Nearly half of those responding had Twitter accounts, which they used both personally and professionally. Surveyed bibliometricians had mixed opinions on altmetrics’ potential; 72 % valued download counts, while a third saw potential in tracking articles’ influence in blogs, Wikipedia, reference managers, and social media. Altogether, these findings suggest that some online tools are seeing substantial use by bibliometricians, and that they present a potentially valuable source of impact data.

Keywords

Altmetrics Social media presence Reference managers Download counts Citation counts 

References

  1. Bar-Ilan, J. (2011). Articles tagged by ‘bibliometrics’ on Mendeley and CiteULike. Paper presented at the Metrics 2011 Symposium on Informetric and Scientometric Research.Google Scholar
  2. Bar-Ilan, J. (2012a). JASIST@mendeley. Presented at the ACM Web Science Conference Workshop on Altmetrics. Evanston, IL. Retrieved January 21, 2013 from http://altmetrics.org/altmetrics12/bar-ilan.
  3. Bar-Ilan, J. (2012b). JASIST 2001–2010. Bulletin of the American Society for Information Science and Technology, 38(6), 24–28.Google Scholar
  4. Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. In Proceedings of the 17th International Conference on Science and Technology Indicators, Montréal, Canada (Vol. 1, pp. 98–109).Google Scholar
  5. Bar-Ilan, J., Shema, H., & Thelwall (2014). Bibliographic References in Web 2.0. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance (pp. 307–325). Cambridge, MA: MIT Press.Google Scholar
  6. Cronin, B., & Overfelt, K. (1994). The scholar’s courtesy: A survey of acknowledgement behaviour. Journal of Documentation, 50, 165–196.CrossRefGoogle Scholar
  7. Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4). Retrieved January 21, 2013 from http://www.jmir.org/2011/4/e123.
  8. Ganegan, F. (2012, August). Filtering the research record and farming big data. Retrieved January 21, 2013 from http://www.swets.com/blog/filtering-the-research-record-and-farming-big-data#.Google Scholar
  9. Glaser, B. G., & Strauss, A. L. (1967). The Discovery of Grounded Theory. Strategies for Qualitative Research. New Brunswick: Aldine Transactions.Google Scholar
  10. Groth, P., & Gurney, T. (2010). Studying scientific discourse on the Web using bibliometrics: A chemistry blogging case study. Presented at the WebSci10: Extending the Frontiers of Society On-Line, Raleigh, NC, USA.Google Scholar
  11. Haustein, S. (2014). Readership Metrics. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance (pp. 327–344), Cambridge, MA: MIT Press.Google Scholar
  12. Haustein, S., Golov, E., Luckanus, K., Reher, S., & Terliesner, J. (2010). Journal evaluation and science 2.0. Using social bookmarks to analyze reader perception. In Book of Abstracts of the 11th International Conference on Science and Technology Indicators, (pp. 117–119). Leiden, The Netherlands.Google Scholar
  13. Haustein, S., & Peters, I. (2012). Using Social Bookmarks and Tags as Alternative Indicators of Journal Content Description. First Monday, 17(11). Retrieved January 21, 2013 from www.firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/4110/3357.
  14. Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.Google Scholar
  15. Henning, V., & Reichelt, J. (2008). Mendeley: A Last.fm for research? In Proceedings of 4th IEEE International Conference on Escience, (pp. 327–328). Indianapolis, IN, USA. Google Scholar
  16. Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 1–64.CrossRefGoogle Scholar
  17. Letierce, J., Passant, A., Decker, S., & Breslin, J.G. (2010). Understanding how Twitter is used to spread scientific messages. In Proceedings of the Web Science Conference, Raleigh, NC, USA.Google Scholar
  18. Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of the 17th International Conference on Science and Technology Indicators, (Vol. 2, pp. 451–551). Montréal, Canada.Google Scholar
  19. Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.CrossRefGoogle Scholar
  20. MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis—A critical review. Journal of the American Society for Information Science, 40(5), 342–349.CrossRefGoogle Scholar
  21. Mahrt, M., Weller, K., & Peters, I. (2013). Twitter in Scholarly Communication. In K. Weller, A. Bruns, J. Burgess, M. Mahrt, & C. Puschmann (Eds.), Twitter and Society (pp. 399–410). New York: Peter Lang.Google Scholar
  22. Martin, B. R., & Irvine, J. (1983). Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12(2), 61–90.CrossRefGoogle Scholar
  23. Mas-Bleda, A., Thelwall, M., Kousha, K., & Aguillo, I. (2013). European highly cited scientists’ presence on the Web. In Proceedings of the 14th International Conference of the International Society for Scientometrics and Informetrics (Vol. II, pp. 1966–1969). Vienna, AustriaGoogle Scholar
  24. Mendeley (2012). Mendeley Global Research Report. Retrieved January 21, 2013 from http://www.mendeley.com/global-research-report/#.UPxyUqPi58E.
  25. Mohammadi, E. & Thelwall, M. (in press). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows. Journal of the American Society for Information Science and Technology.Google Scholar
  26. Nielsen, F. (2007). Scientific citations in Wikipedia. First Monday, 12(8). Retrieved January 21, 2013 from http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1997/1872.
  27. Piwowar, H. (2013). Value all research products. Nature, 493, 159.Google Scholar
  28. Ponte, D., & Simon, J. (2011). Scholarly communication 2.0: Exploring researchers’ opinions on Web 2.0 for scientific knowledge creation, evaluation and dissemination. Serials Review, 37(3), 149–156.CrossRefGoogle Scholar
  29. Price, D., de Solla, J., & Gürsey, S. (1976). Studies in scientometrics I. Transience and continuance in scientific authorship. International Forum on Information and Documentation, 1(2), 17–24.Google Scholar
  30. Priem, J. (2010). Tweet by Jason Priem on September 28, 2010. Retrieved January 21, 2013 from https://twitter.com/#!/jasonpriem/status/25844968813.
  31. Priem, J. (2014). Altmetrics. In B. Cronin, & C. Sugimoto (eds.), Beyond Bibliometrics: Harnessing Multi-dimensional Indicators of Performance. Cambridge, MA: MIT Press.Google Scholar
  32. Priem, J., & Costello, K. (2010). How and why scholars cite on Twitter. In Proceedings of the 73rd Annual Meeting of the American Society for Information Science and Technology, Pittsburgh, PA, USA. doi: 10.1002/meet.14504701201/full.
  33. Priem, J., Costello, K., & Dzuba, T. (2011). First-year graduate students just wasting time? Prevalence and use of Twitter among scholars. Presented at the Metrics 2011 Symposium on Informetric and Scientometric Research, New Orleans, LA, USA. Retrieved January 21, 2013 from http://jasonpriem.org/self-archived/5uni-poster.png.
  34. Priem, J., Piwowar, H. A., & Hemminger, B. M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. Retrieved January 21, 2013 from http://arxiv.org/abs/1203.4745.
  35. Priem, J., Taraborelli, D., Groth, P., & Nylon, C. (2010). alt-metrics: a manifesto. Retrieved January 21, 2013 from http://altmetrics.org/manifesto.
  36. Reher, S., & Haustein, S. (2010). Social bookmarking in STM: Putting services to the acid test. Online - Leading Magazine for Information Professionals, 34(6), 34–42.Google Scholar
  37. Research Councils UK. (2011, March). Types of impact. Retrieved January 21, 2013 from http://www.rcuk.ac.uk/documents/impacts/TypologyofResearchImpacts.pdf.
  38. Rowlands, I., & Nicholas, D. (2007). The missing link: Journal usage metrics. Aslib Proceedings, 59(3), 222–228.CrossRefGoogle Scholar
  39. Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013). Download vs. vitiation vs. readership data: The case of an information systems journal. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (Vol. 1, 626–634).Google Scholar
  40. Schlögl, C., & Stock, W. G. (2004). Impact and relevance of LIS journals: A scientometric analysis of international and German-language LIS journals—Citation analysis versus reader survey. Journal of the American Society for Information Science and Technology, 55(13), 1155–1168.CrossRefGoogle Scholar
  41. Shema, H., Bar-Ilan, J., & Thelwall, M. (2012). Research blogs and the discussion of scholarly information. PLoS One, 7(5), e35869. doi:10.1371/journal.pone.0035869.CrossRefGoogle Scholar
  42. Shema, H., Bar-Ilan, J., & Thelwall, M. (in press). Do blog citations correlate with higher number of future citations? Journal of the American Society for Information Science and Technology.Google Scholar
  43. Tenopir, C., & King, D. W. (2000). Towards electronic journals: Realities for scientists, librarians, and publishers. Washington, DC: Special Libraries Association.Google Scholar
  44. Thelwall, M. (2010). Webometrics: emergent or doomed? Information Research: An International Electronic Journal, 15(4). Retrieved January 21, 2013 from http://informationr.net/ir/15-4/colis713.html.
  45. Thelwall, M. (2012). Journal impact evaluation: A webometric perspective. Scientometrics, 92, 429–441.CrossRefGoogle Scholar
  46. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. (2013). Do altmetrics work? Twitter and ten other candidates. PLoS One, 8(5), e64841. doi:10.1371/journal.pone.0064841.CrossRefGoogle Scholar
  47. Thelwall, M., Vaughan, L., & Björneborn, L. (2005). Webometrics. Annual Review of Information Science and Technology, 39(1), 81–135.CrossRefGoogle Scholar
  48. Weller, K., & Peters, I. (2012). Citations in Web 2.0. In A. Tokar, M. Beurskens, S. Keuneke, M. Mahrt, I. Peters, C. Puschmann, et al. (eds.), Science and the Internet (pp. 211–224). Düsseldorf: Düsseldorf University Press.Google Scholar
  49. Weller, K, & Puschmann, C. (2011). Twitter for scientific communication: How can citations/references be identified and measured? In Proceedings of the 3rd ACM International Conference on Web Science, Koblenz, Germany. Retrieved January 21, 2013 from http://journal.webscience.org/500/1/153_paper.pdf.
  50. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT—information technology.Google Scholar
  51. Zahedi, Z., Costas, R., & Wouters, P. (2013). How well developed are altmetrics? Cross disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (Vol. 1, pp. 876–884).Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2013

Authors and Affiliations

  • Stefanie Haustein
    • 1
  • Isabella Peters
    • 2
  • Judit Bar-Ilan
    • 3
  • Jason Priem
    • 4
  • Hadas Shema
    • 3
  • Jens Terliesner
    • 5
  1. 1.École de bibliothéconomie et des sciences de l’informationUniversité de Montréal, Montréal (Canada) and Science-MetrixMontrealCanada
  2. 2.ZBW – German National Library of EconomicsLeibniz Information Centre for EconomicsKielGermany
  3. 3.Department of Information ScienceBar-Ilan UniversityRamat-GanIsrael
  4. 4.School of Information and Library ScienceUniversity of North Carolina at Chapel HillChapel HillUSA
  5. 5.Department of Information ScienceHeinrich Heine UniversityDuesseldorfGermany

Personalised recommendations