, Volume 111, Issue 1, pp 349–369 | Cite as

What we can learn from tweets linking to research papers

  • Xuan Zhen Liu
  • Hui FangEmail author


To explore whether there are other factors than count and sentiment that should be incorporated in evaluating research papers with social media mentions, this paper analyses the content of tweets linking to the top 100 papers of 2015 taken from, focusing on the goals, functions and features of research. We discuss three basic issues inherent in using tweets for research evaluation: whose tweets can be used to assess a paper, what objects can be evaluated, and how to score the paper according to each tweet. We suggest that tweets written by those involved in publication of the paper in question should not be included in the paper’s evaluation. Tweets unrelated to the content of the paper should also be excluded. Because controversies in research are inevitable and difficult to resolve, we suggest omitting somewhat supportive and negative tweets in research evaluation. Logically, neutral tweets (such as those linking to, and excerpts from, papers) express a degree of compliment, agreement, interest, or surprise, albeit less so than the tweets explicitly expressing these sentiments. Recommendation tweets also reflect one or more of these sentiments. Expansion tweets, which are inspired by the papers, reflect the function of research. Therefore, we suggest giving a higher weight to praise, agreement, interest, surprise, recommendation and expansion tweets linking to an academic paper than neutral tweets when scoring a paper. Issues related to electronic publishing and social media as learned from tweets are also discussed.


Altmetrics Tweet Academic paper Research evaluation Function of science Electronic publishing 



The authors thank the anonymous reviewers for useful suggestions for improving this contribution.


Humanities and Social Sciences Foundation of the Ministry of Education of China (16YJE870002).


  1. Barber, B. (1961). Resistance by scientists to scientific discovery—This source of resistance has yet to be given scrutiny accorded religious and ideological sources. Science, 134(347), 596–602.CrossRefGoogle Scholar
  2. Bird, A. (2010). The epistemology of science—A bird’s-eye view. Synthese, 175(1), 5–16.MathSciNetCrossRefGoogle Scholar
  3. Bornmann, L. (2014a). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.CrossRefGoogle Scholar
  4. Bornmann, L. (2014b). Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime. Journal of Informetrics, 8(4), 935–950.CrossRefGoogle Scholar
  5. Bornmann, L. (2015). Alternative metrics in scientometrics: A meta-analysis of research into three altmetrics. Scientometrics, 103(3), 1123–1144.CrossRefGoogle Scholar
  6. Bornmann, L. (2016). Letter to the editor what do altmetrics counts mean? A plea for content analyses. Journal of the Association for Information Science and Technology, 67(4), 1016–1017.CrossRefGoogle Scholar
  7. Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behaviour. Journal of Documentation, 64(1), 45–80.CrossRefGoogle Scholar
  8. Bornmann, L., & Haunschild, R. (2016). How to normalize Twitter counts? A first attempt based on journals in the Twitter Index. Scientometrics, 107(3), 1405–1422.CrossRefGoogle Scholar
  9. Chen, J. Y. (2015). Virtue and the scientist: Using virtue ethics to examine science’s ethical and moral challenges. Science and Engineering Ethics, 21(1), 75–94.CrossRefGoogle Scholar
  10. Costas, R., van Leeuwen, T. N., & Bordons, M. (2010). Self-citations at the meso and individual levels: Effects of different calculation methods. Scientometrics, 82(3), 517–537.CrossRefGoogle Scholar
  11. Dallmeier-Tiessen, S., Darby, R., Goerner, B., Hyppoelae, J., Igo-Kemenes, P., Kahn, D., et al. (2011). Highlights from the SOAP project survey. What scientists think about open access publishing. arXiv 1101.5260.Google Scholar
  12. de Winter, J. C. F. (2015). The relationship between tweets, citations, and article views for PLOS ONE articles. Scientometrics, 102(2), 1773–1779.CrossRefGoogle Scholar
  13. Fang, H. (2015). A discussion on governmental research grants. Science and Engineering Ethics, 21(5), 1285–1296.CrossRefGoogle Scholar
  14. Haustein, S. (2014). Readership metrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multi-dimensional indicators of performance (pp. 327–344). Cambridge, MA: MIT Press.Google Scholar
  15. Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495.CrossRefGoogle Scholar
  16. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (2014a). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT Information Technology, 56(5), 207–215.CrossRefGoogle Scholar
  17. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014b). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.CrossRefGoogle Scholar
  18. Holmberg, K., Bowman, T. D., Haustein, S., & Peters, I. (2014). Astrophysicists’ conversational connections on Twitter. PLoS ONE, 9(8), e106086.CrossRefGoogle Scholar
  19. Holmberg, K., & Thelwall, M. (2014). Disciplinary differences in Twitter scholarly communication. Scientometrics, 101(2), 1027–1042.CrossRefGoogle Scholar
  20. Jose, D. (2007). Falsificationism and the structure of theories: The Popper–Kuhn controversy about the rationality of normal science. Studies in History and Philosophy of Science, 38(3), 543–554.CrossRefGoogle Scholar
  21. Karle, J. (1997). The role of motivation in scientific research. 4. The nature of basic research and its implications. Interdisciplinary Science Reviews, 22(1), 78–84.CrossRefGoogle Scholar
  22. Krishnamurthy, B., Gill, P., & Arlitt, M. (2008). A few chirps about Twitter. In Proceedings of the first workshop on Online social networks (pp. 19–24). New York: ACM.Google Scholar
  23. Laakso, M., & Björk, B. C. (2012). Anatomy of open access publishing: A study of longitudinal development and internal structure. BMC Medicine, 10(1), 124.CrossRefGoogle Scholar
  24. Laakso, M., & Björk, B. C. (2013). Delayed open access: An overlooked high-impact category of openly available scientific literature. Journal of the American Society for Information Science and Technology, 64(7), 1323–1329.CrossRefGoogle Scholar
  25. Laakso, M., Welling, P., Bukvova, H., Nyman, L., Björk, B. C., & Hedlund, T. (2011). The development of open access journal publishing from 1993 to 2009. PLoS ONE, 6(6), e20961.CrossRefGoogle Scholar
  26. Mizrahi, M. (2013). What is scientific progress? Lessons from scientific practice. Journal for General Philosophy of Science, 44(2), 375–390.CrossRefGoogle Scholar
  27. Mohammadi, E., Thelwall, M., Haustein, S., & Larivière, V. (2015). Who reads research articles? An altmetrics analysis of Mendeley user categories. Journal of the Association for Information Science and Technology, 66(9), 1832–1846.CrossRefGoogle Scholar
  28. Mutanen, A. (2016). Pedagogy as a framework for a proper dialogue between science and literature. Philosophia, 44(1), 167–180.CrossRefGoogle Scholar
  29. Osterrieder, A. (2013). The value and use of social media as communication tool in the plant sciences. Plant Methods, 9(1), 26.CrossRefGoogle Scholar
  30. Philip, A. E. B. (2012). Anticipatory ethics for emerging technologies. Nanoethics, 6(1), 1–13.CrossRefGoogle Scholar
  31. Popper, K. R. (2002). The logic of scientific discovery. London: Routledge.zbMATHGoogle Scholar
  32. Rousseau, R., & Ye, F. Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(26), 3288–3290.CrossRefGoogle Scholar
  33. Schreiber, M. (2008). The influence of self-citation corrections on Egghe’s g index. Scientometrics, 76(1), 187–200.CrossRefGoogle Scholar
  34. Tahamtan, I., Afshar, A. S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225.CrossRefGoogle Scholar
  35. Thelwall, M. (2014). Heart and soul: Sentiment strength detection in the social web with sentistrength. Retrieved January 16, 2016 from
  36. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013a). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.CrossRefGoogle Scholar
  37. Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013b). Tweeting links to academic articles. Cybermetrics, 17(1), 1–8.Google Scholar
  38. Wang, L. L., Liu, X. Z., & Fang, H. (2015). Investigation of the degree to which articles supported by research grants are published in open access health and life sciences journals. Scientometrics, 104(2), 511–528.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.LibraryNanjing Medical UniversityNanjingChina
  2. 2.State Key Laboratory of Analytical Chemistry for Life Science, School of Electronic Science and EngineeringNanjing UniversityNanjingChina

Personalised recommendations