Advertisement

Scientometrics

, Volume 116, Issue 3, pp 2123–2138 | Cite as

Reliability and accuracy of altmetric providers: a comparison among Altmetric.com, PlumX and Crossref Event Data

  • José Luis OrtegaEmail author
Article

Abstract

The aim of this study is to measure differences between three relevant altmetric providers: Altmetric.com, PlumX and Crossref Event Data (CED). More than 67,000 research papers, initially extracted from PlumX, were searched in Altmetric.com and CED to compare their counts. Differences between services were analyzed regarding the number of documents with an altmetric event and the counting differences in each metric. Results show that Altmetric.com is the provider with the best coverage of blog posts, news and tweets; while PlumX better collects Mendeley readers; and CED is the site that extracts more Wikipedia citations. The study concludes that there are important counting differences due to technical errors and extracting criteria. The article recommends the use of specific services for the analysis of particular metrics. While, it should be mandatory to employ the combination of several providers, if we want to perform an overall analysis.

Keywords

Altmetric.com PlumX Crossref Event Data Altmetrics Data providers 

References

  1. Adie, E., & Roe, W. (2013). Altmetric: Enriching scholarly content with article-level discussion and metrics. Learned Publishing, 26(1), 11–17.  https://doi.org/10.1087/20130103.CrossRefGoogle Scholar
  2. Altmetric Support (2017). How does Altmetric track mentions on Wikipedia? https://help.altmetric.com/support/solutions/articles/6000060980-how-does-altmetric-track-mentions-on-wikipedia (Visited: May 23, 2018).
  3. Altmetric.com (2018). How it works. https://www.altmetric.com/about-our-data/how-it-works/.
  4. Baessa, M., Lery, T., Grenz, D., & Vijayakumar, J. K. (2015). Connecting the pieces: Using ORCIDs to improve research impact and repositories. F1000Research, 4, 195.  https://doi.org/10.12688/f1000research.6502.1.Google Scholar
  5. Bornmann, L. (2014). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950.  https://doi.org/10.1016/j.joi.2014.09.007.CrossRefGoogle Scholar
  6. CED (2017). Twitter—event data user guide https://www.eventdata.crossref.org/guide/sources/twitter/ (Visited: May 23, 2018).
  7. Champieux, R. (2015). PlumX. Journal of the Medical Library Association, 103(1), 63.  https://doi.org/10.3163/1536-5050.103.1.019.CrossRefGoogle Scholar
  8. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.  https://doi.org/10.1002/asi.23309.CrossRefGoogle Scholar
  9. Elsevier. (2017). Elsevier Acquires Leading ‘Altmetrics’ Provider Plum Analytics. https://www.elsevier.com/about/press-releases/corporate/elsevier-acquires-leading-altmetrics-provider-plum-analytics (Visited: May 23, 2018).
  10. Erdt, M., Nagarajan, A., Sin, S. C. J., & Theng, Y. L. (2016). Altmetrics: An analysis of the state-of-the-art in measuring research impact on social media. Scientometrics, 109(2), 1117–1166.  https://doi.org/10.1007/s11192-016-2077-0.CrossRefGoogle Scholar
  11. Fraumann, G., Zahedi, Z. & Costas, R. (2015). What do we know about Altmetric.com sources? A study of the top 200 blogs and news sites mentioning scholarly output. In Altmetrics workshop, 9 October 2015, Amsterdam Science Park, Amsterdam.Google Scholar
  12. Gumpenberger, C., Glänzel, W., & Gorraiz, J. (2016). The ecstasy and the agony of the altmetric score. Scientometrics, 108(2), 977–982.  https://doi.org/10.1007/s11192-016-1991-5.CrossRefGoogle Scholar
  13. Holmberg, K. J. (2015). Altmetrics for information professionals: Past, present and future. Chandos Publishing. ISBN: 9780081002773.Google Scholar
  14. ImpactStory (2017). New widget and API. ImpactStory blog. http://blog.impactstory.org/new-widget-and-api/ (Visited: May 23, 2018).
  15. Jobmann, A., Hoffmann, C.P., Künne, S., Peters, I., Schmitz, J. & Wollnik-Korn, G. (2014). Altmetrics for large, multidisciplinary research groups: Comparison of current tools. Bibliometrie – Praxis und Forschung, 3, http://www.bibliometrie-pf.de/article/viewFile/205/258.
  16. King, G. (2015). Altmetric and Springer Launch Bookmetrix: A New Platform for Book Impact. Digital Science News Blog. https://www.digital-science.com/blog/news/altmetric-and-springer-launch-bookmetrix-a-new-platform-for-book-impact/ (Visited: May 23, 2018).
  17. Kraker, P., Lex, E., Gorraiz, J., Gumpenberger, C. & Peters, I. (2015). Research data explored II: The anatomy and reception of figshare. arXiv:1503.01298.
  18. Lindsay, J. M. (2016). PlumX from plum analytics: Not just altmetrics. Journal of Electronic Resources in Medical Libraries, 13(1), 8–17.  https://doi.org/10.1080/15424065.2016.1142836.CrossRefGoogle Scholar
  19. Liu, J. (2013) New Feature: News Tracker. Altmetric: https://www.altmetric.com/blog/new-feature-news-tracker/ (Visited: May 23, 2018).
  20. Meschede, C., & Siebenlist, T. (2018). Cross-metric compatability and inconsistencies of altmetrics. Scientometrics, 115(1), 283–297.  https://doi.org/10.1007/s11192-018-2674-1.CrossRefGoogle Scholar
  21. Ortega, J. L. (2016). To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers. Scientometrics, 109(2), 1353–1364.  https://doi.org/10.1007/s11192-016-2113-0.CrossRefGoogle Scholar
  22. Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2015). Research data explored: Citations versus altmetrics. In A. A. Salah, Y. Tonta, A. A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th international conference on scientometrics and informetrics, Istanbul, Turkey (pp. 172–183).Google Scholar
  23. Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107(2), 723–744.  https://doi.org/10.1007/s11192-016-1887-4.CrossRefGoogle Scholar
  24. Plum Analytics (2016). Wikipedia & altmetrics: Calculating mention metrics. https://plumanalytics.com/wikipedia-altmetrics-calculating-mention-metrics/.
  25. Plum Analytics (2018). Coverage: Expanding the world of altmetrics. khttps://plumanalytics.com/learn/about-metrics/coverage/.
  26. Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Altmetrics: A manifesto. http://altmetrics.org/manifesto/.
  27. Robinson-García, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). Nuevos datos, nuevas posibilidades: Revelando el interior de Altmetric.com. El Profesional de la Information, 23(4), 359–366.  https://doi.org/10.3145/epi.2014.jul.03.CrossRefGoogle Scholar
  28. Roemer, R. C. & Borchardt, R. (2015). Altmetrics. American Library Association, ISBN: 0838959652.Google Scholar
  29. Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062.  https://doi.org/10.1002/asi.23833.CrossRefGoogle Scholar
  30. Tattersall, A. (Ed.). (2016). Altmetrics: A practical guide for librarians, researchers and academics. Facet Publishing. ISBN: 178330010.Google Scholar
  31. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.  https://doi.org/10.1371/journal.pone.0064841.CrossRefGoogle Scholar
  32. Torres-Salinas, D., Gumpenberger, C., & Gorraiz, J. (2017). PlumX as a potential tool to assess the macroscopic multidimensional impact of books. Frontiers in Research Metrics and Analytics.  https://doi.org/10.3389/frma.2017.00005.Google Scholar
  33. Trueger, N. S., Thoma, B., Hsu, C. H., Sullivan, D., Peters, L., & Lin, M. (2015). The altmetric score: A new measure for article-level dissemination and impact. Annals of Emergency Medicine, 66(5), 549–553.  https://doi.org/10.1016/j.annemergmed.2015.04.022.CrossRefGoogle Scholar
  34. Williams, C. (2015) Altmetric to provide expanded mainstream media tracking. Altmetric: https://www.altmetric.com/blog/moreover/ (Visited: May 23, 2018).
  35. Williams, C. (2017) Help Altmetric further expand its coverage: Tell us about your favourite blogs. Altmetric: https://www.altmetric.com/blog/expandblogs/ (Visited: May 23, 2018).
  36. Zahedi, Z., & Costas, R. (2018). General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators. PLoS ONE, 13(5), e0197326.CrossRefGoogle Scholar
  37. Zahedi, Z., Fenner, M. & Costas, R. (2015). Consistency among altmetrics data provider/aggregators: What are the challenges? In The 2015 altmetrics workshop, Amsterdam.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Institute for Advanced Social Studies (IESA-CSIC)CórdobaSpain

Personalised recommendations