Abstract
The traditional retraction mechanism's failure to eradicate the retracted papers' continued effects urges for more control and monitoring systems to warn against low-quality and flawed papers. To investigate the potential of Twitter in reflecting social attitudes about retracted papers, this study analyzed the sentiments expressed in the tweets about the papers and contrasted them against two benchmarks: the retraction notes and their tweets respectively serving as authorities’ voices and their social resonance. Using a sentiment analysis method, the study examined a collection of Scopus-indexed retracted papers, their retraction notices, and their tweets. The opinions expressed in the texts were mined using the SentiStrength. The findings revealed a high rate of untweetedness for the retracted papers (91.54%) and retraction notes (90.72%). However, the paper tweets mostly contained texts and were not limited to URLs, except for a low percentage (2.78%). While the retraction notices were mostly negative, followed by neutral polarity, the note and paper tweets were dominated by neutrality followed by negativity. Nevertheless, the paper tweets were more negative either in the pre-, or post-retraction phases. Moreover, negative tweets were comparatively more retweeted than positive and neutral polarities. The research findings implied tweet potentials in increasing the visibility of and awareness about low-quality and erroneous papers, even before being disclosed by official authorities, provided that more users are actively involved in the discussions on the platform. The potential can be regarded as a kind of monitoring applied by social users who feel responsible and show sensitivity towards the quality of science, though they may be scarce in number and selectively react to some papers.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdar, M., Basiri, M. E., Yin, J., Habibnezhad, M., Chi, G., Nemati, S., & Asadi, S. (2020). Energy choices in Alaska: Mining people’s perception and attitudes from geotagged tweets. Renewable & Sustainable Energy Reviews, 124, 109781. https://doi.org/10.1016/j.rser.2020.109781
Abhari, R., Vincent, N., Dambanemuya, H. K., Bodon, H., & Horvát, E. Á. (2022). Twitter engagement with retracted articles: Who, when, and how? ArXiv. ahttps://doi.org/10.48550/arXiv.2203.04228
Al-Shabi, M. A. (2020). Evaluating the performance of the most important Lexicons used to sentiment analysis and opinions mining. IJCSNS, 20(1), 1.
Al-Surimi, K., Khalifa, M., Bahkali, S., EL-Metwally, A., Househ, M. (2016). The potential of social media and internet-based data in preventing and fighting infectious diseases: From internet to Twitter. In Emerging and re-emerging viral infections. Advances in experimental medicine and biology. Springer. https://doi.org/10.1007/5584_2016_132
Alhajji, M.; Al Khalifah, A.; Aljubran, M.; Alkhalifah, M. (2020) Sentiment analysis of tweets in Saudi Arabia regarding governmental preventive measures to contain COVID-19. Preprints, 2020040031 https://doi.org/10.20944/preprints202004.0031.v1
Aljedaani, W., Abuhaimed, I., Rustam, F., Mkaouer, M. W., Ouni, A., & Jenhani, I. (2022a). Automatically detecting and understanding the perception of COVID-19 vaccination: A middle east case study. Social Network Analysis and Mining. https://doi.org/10.1007/s13278-022-00946-0
Aljedaani, W., Saad, E., Rustam, F., De La Torre Díez, I., & Ashraf, I. (2022b). Role of artificial intelligence for analysis of COVID-19 vaccination-related tweets: Opportunities, challenges, and future trends. Mathematics, 10(17), 3199. https://doi.org/10.3390/math10173199
Alvarez-Galvez, J., Suarez-Lledo, V., & Rojas-García, A. (2021). Determinants of infodemics during disease outbreaks: A systematic review. Frontiers in Public Health. https://doi.org/10.3389/fpubh.2021.603603
Araujo, R. E., Sorensen, A. A., Konkiel, S., & Bloem, B. R. (2017). Top Altmetric Scores in the Parkinson’s Disease Literature. Journal of Parkinson’s Disease, 7(1), 81–87. https://doi.org/10.3233/jpd-179000
Bar-Ilan, J., & Halevi, G. (2017). Post retraction citations in context: A case study. Scientometrics, 113(1), 547–565. https://doi.org/10.1007/s11192-017-2242-0
Bar-Ilan, J., & Halevi, G. (2018). Temporal characteristics of retracted articles. Scientometrics, 116(3), 1771–1783. https://doi.org/10.1007/s11192-018-2802-y
Basiri, M. E., Abdar, M., Cifci, M. A., Nemati, S., & Acharya, U. R. (2020). A novel method for sentiment classification of drug reviews using fusion of deep and machine learning techniques. Knowledge Based Systems, 198, 105949. https://doi.org/10.1016/j.knosys.2020.105949
Basiri, M. E., Nemati, S., Abdar, M., Asadi, S., & Acharrya, U. R. (2021). A novel fusion-based deep learning model for sentiment analysis of COVID-19 tweets. Knowledge Based Systems, 228, 107242. https://doi.org/10.1016/j.knosys.2021.107242
Baum, J. (2022). Emotional content in social misinformation affects mind, brain, and judgments [Ph.D. dissertation]. https://edoc.hu-berlin.de/handle/18452/26085
Bik, H. M., & Goldstein, M. C. (2013). an introduction to social media for scientists. PLOS Biology, 11(4), e1001535. https://doi.org/10.1371/journal.pbio.1001535
Blake, B. P., & Agarwal, N. (2017). Modeling user-based modifications to information quality to address privacy and trust related concerns in online social networks. International Journal on Advances in Security, 10(1,2). http://www.iariajournals.org/security
Bornmann, L., & Haunschild, R. (2018a). Allegation of scientific misconduct increases Twitter attention. Scientometrics, 115, 1097–1100. https://doi.org/10.1007/s11192-018-2698-6
Bornmann, L., & Haunschild, R. (2018b). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000 Prime data. PLoS ONE, 13(5), e0197133. https://doi.org/10.1371/journal.pone.0197133
Bornmann, L., & Haunschild, R. (2018c). Normalization of zero-inflated data: An empirical analysis of a new indicator family and its use with altmetrics data. Journal of Informetrics, 12(3), 998–1011. https://doi.org/10.1016/j.joi.2018.01.010
Brainerd, C. J., & Reyna, V. F. (2005). The science of false memory. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195154054.001.0001
Brandt, H. M., Turner-McGrievy, G., Friedman, D. B., Gentile, D., Schrock, C. S., Thomas, T., & West, D. S. (2019). Examining the role of Twitter in response and recovery during and after historic flooding in South Carolina. Journal of Public Health Management and Practice, 25(5), E6–E12. https://doi.org/10.1097/phh.0000000000000841
Briand, S., Cinelli, M., Nguyen, T., Lewis, R. F., Prybylski, D., Valensise, C. M., Colizza, V., Tozzi, A. E., Perra, N., Baronchelli, A., Tizzoni, M., Zollo, F., Scala, A., Purnat, T. D., Czerniak, C., Kucharski, A. J., Tshangela, A., Zhou, L., & Quattrociocchi, W. (2021). Infodemics: A new challenge for public health. Cell, 184(25), 6010–6014. https://doi.org/10.1016/j.cell.2021.10.031
Cambria, E., Das, D., Bandyopadhyay, S., & Feraco, A. (2017). A practical guide to sentiment analysis. Springer. https://doi.org/10.1007/978-3-319-55394-8
Campanario, J. M. (2000). Fraud: Retracted articles are still being cited. Nature, 408(6810), 288. https://doi.org/10.1038/35042753
Cascini, F., Pantovic, A., Al-Ajlouni, Y. A., Failla, G., Puleo, V., Melnyk, A., Lontano, A., & Ricciardi, W. (2022). Social media and attitudes towards a COVID-19 vaccination: A systematic review of the literature. EClinicalMedicine, 48, 101454. https://doi.org/10.1016/j.eclinm.2022.101454
Celebi, A. R. C., Bulut, E., Koprubasi, S., & Dokur, M. (2022). Assessing the relationship between traditional citation-based metrics and new metrics (Altmetrics/Dimensions/Number of Tweets) in COVID-19 & eye research. Research Square. https://assets.researchsquare.com/files/rs-1021430/v1/3d12033e-dab4-446d-b663-d474a48d3b41.pdf?c=1644476162
Chamberlain, J. (2015). Harnessing collective intelligence on social networks [Ph.D. dissertation]. University of Essex. https://repository.essex.ac.uk/15693/
Chauhan, P., Sharma, N., & Sikka, G. (2021). The emergence of social media data and sentiment analysis in election prediction. Journal of Ambient Intelligence and Humanized Computing, 12(2), 2601–2627. https://doi.org/10.1007/s12652-020-02423-y
Chen, R., & Lazer, M. (2013). Sentiment analysis of Twitter feeds for the prediction of stock market movement. Stanf Educ, 25, 1–5. https://doi.org/10.1016/j.ufug.2017.05.003
Cheng, Y., & Chen, Z. F. (2020). Encountering misinformation online: Antecedents of trust and distrust and their impact on the intensity of Facebook use. Online Information Review, 45(2), 372–388. https://doi.org/10.1108/oir-04-2020-0130
Chinnasamy, P., Suresh, V., Ramprathap, K., Jebamani, B., Rao, K. S., & Kranthi, M. S. (2022). COVID-19 vaccine sentiment analysis using public opinions on Twitter. Materials Today: Proceedings, 64, 448–451. https://doi.org/10.1016/j.matpr.2022.04.809
Cillóniz, C., Greenslade, L., Dominedò, C., & Garcia-Vidal, C. (2020). Promoting the use of social networks in pneumonia. Pneumonia, 12, 3. https://doi.org/10.1186/s41479-020-00066-3
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement., 20(1), 37–46. https://doi.org/10.1177/001316446002000104
Collins, K. A. M. (2015). Tweet your science – an analysis of scientists using twitter [M.Sc. thesis]. University of Otago. https://ourarchive.otago.ac.nz/handle/10523/5998
Collins, K. P., Shiffman, D. S., & Rock, J. (2016). How are scientists using social media in the workplace? PLoS ONE, 11(10), e0162680. https://doi.org/10.1371/journal.pone.0162680
Copiello, S. (2020). Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: Tentative findings of a study into the papers retracted by PLoS ONE. Scientometrics, 125(3), 2449–2469. https://doi.org/10.1007/s11192-020-03698-w
Costas, R., Van Honk, J., & Franssen, T. (2017). Scholars on Twitter: who and how many are they? In International conference on scientometrics and informetrics. https://doi.org/10.48550/arXiv.1712.05667
Culpeper, J., Findlay, A., Cortese, B., & Thelwall, M. (2018). Measuring emotional temperatures in Shakespeare’s drama. English Text Construction, 11(1), 10–37. https://doi.org/10.1075/etc.00002.cul
Da Silva, J. A. T., & Dobránszki, J. (2017). Highly cited retracted papers. Scientometrics, 110(3), 1653–1661. https://doi.org/10.1007/s11192-016-2227-4
Didegah, F., Mejlgaard, N., & Sørensen, M. P. (2018). Investigating the quality of interactions and public engagement around scientific papers on Twitter. Journal of Informetrics, 12(3), 960–971. https://doi.org/10.1016/j.joi.2018.08.002
Dubey, A. D. (2021). Twitter sentiment analysis during COVID-19 outbreak. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3572023
Ecker, U. (2015). The psychology of misinformation. Australasian Science, 36(2), 21–23. https://doi.org/10.3316/informit.523665154656184
Ecker, U. K. H., Lewandowsky, S., & Tang, D. T. W. (2010). Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & Cognition, 38(8), 1087–1100. https://doi.org/10.3758/mc.38.8.1087
Ewing, S., & Thomas, J. (2008). Broadband and the “Creative Internet”: Australians as consumers and producers of cultural content online. Observatorio Journal, 6, 187–208. https://doi.org/10.15847/obsOBS232008215
Eysenbach, G. (2012). Correction: Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 14(1), e7. https://doi.org/10.2196/jmir.2041
Fang, Z., Costas, R., Tian, W., Wang, X., & Wouters, P. (2021). How is science clicked on Twitter? Click metrics for Bitly short links to scientific publications. Journal of the Association for Information Science and Technology, 72(7), 918–932. https://doi.org/10.1002/asi.24458
Friedrich, N., Bowman, T. D., & Haustein, S. (2015). Do tweets to scientific articles contain positive or negative sentiments? altmetrics.org. http://altmetrics.org/altmetrics15/friedrich/
Gautam, G. & Yadav, D. (2014). Sentiment analysis of twitter data using machine learning approaches and semantic analysis. In Seventh international conference on contemporary computing (IC3) (pp. 437–442). https://doi.org/10.1109/ic3.2014.6897213
Gebremeskel, G. (2011). A sentiment analysis of Twitter posts about news [M.Sc. thesis]. University of Malta. https://www.um.edu.mt/library/oar/handle/123456789/78379
Geronikolou, S., Drosatos, G., & Chrousos, G. (2021). Emotional analysis of twitter posts during the first phase of the COVID-19 pandemic in Greece: Infoveillance study. JMIR Formative Research, 5(9), e27741. https://doi.org/10.2196/27741
Ghiassi, M., Skinner, J. A., & Zimbra, D. (2013). Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network. Expert Systems with Applications, 40(16), 6266–6282. https://doi.org/10.1016/j.eswa.2013.05.057
Greenspan, R. L., & Loftus, E. F. (2021). Pandemics and infodemics: Research on the effects of misinformation on memory. Human Behavior and Emerging Technologies, 3(1), 8–12. https://doi.org/10.1002/hbe2.228
Guo, X., & Li, J. (2019). A novel Twitter sentiment analysis model with baseline correlation for financial market prediction with improved efficiency. In sixth international conference on Social Networks Analysis, Management and Security (SNAMS) (pp. 472–477). https://doi.org/10.1109/SNAMS.2019.8931720
Hamilton, D. G. (2019). Continued Citation of Retracted Radiation Oncology Literature—Do We Have a Problem? International Journal of Radiation Oncology Biology Physics, 103(5), 1036–1042. https://doi.org/10.1016/j.ijrobp.2018.11.014
Hassan, S., Aljohani, N. R., Idrees, N., Sarwar, R., Nawaz, R., Martínez-Cámara, E., Ventura, S., & Herrera, F. (2020). Predicting literature’s early impact with sentiment analysis in Twitter. Knowledge Based Systems, 192, 105383. https://doi.org/10.1016/j.knosys.2019.105383
Haunschild, R., & Bornmann, L. (2018). Field- and time-normalization of data with many zeros: An empirical analysis using citation and Twitter data. Scientometrics, 116(2), 997–1012. https://doi.org/10.1007/s11192-018-2771-1
Haunschild, R., & Bornmann, L. (2021). Can tweets be used to detect problems early with scientific papers? A case study of three retracted COVID-19/SARS-CoV-2 papers. Scientometrics, 126(6), 5181–5199. https://doi.org/10.1007/s11192-021-03962-7
Haunschild, R., Leydesdorff, L., Bornmann, L., Hellsten, I., & Marx, W. (2019). Does the public discuss other topics on climate change than researchers? A comparison of explorative networks based on author keywords and hashtags. Journal of Informetrics, 13(2), 695–707. https://doi.org/10.1016/j.joi.2019.03.008
Haustein, S., Costas, R., & Larivière, V. (2015). Correction: Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(5), e0127830. https://doi.org/10.1371/journal.pone.0127830
Heibi, I., & Peroni, S. (2021). A qualitative and quantitative analysis of open citations to retracted articles: The Wakefield 1998 et al.’s case. Scientometrics, 126(10), 8433–8470. https://doi.org/10.1007/s11192-021-04097-5
Islam, M. R., & Zibran, M. F. (2018). SentiStrength-SE: Exploiting domain specificity for improved sentiment analysis in software engineering text. Journal of Systems and Software, 145, 125–146. https://doi.org/10.1016/j.jss.2018.08.030
Jamali, H. R., & Alimohammadi, D. (2015). Blog citations as indicators of the societal impact of research: Content analysis of social sciences blogs. International Journal of Knowledge Content Development and Technology, 5(1), 15–32. https://doi.org/10.5865/ijkct.2015.5.1.015
Jan, R., & Zainab, T. (2018a). The impact story of retracted articles: Altmetric it! 2018 5th International Symposium on Emerging Trends and Technologies in Libraries and Information Services (ETTLIS). Published. https://doi.org/10.1109/ettlis.2018.8485245
Jan, R., & Zainab, T. (2018). The impact story of retracted articles Altmetric it! In 5th international symposium on Emerging Trends and Technologies in Libraries and Information Services (ETTLIS). https://doi.org/10.1109/ettlis.2018.8485245
Java, A., Song, X., Finin, T., & Tseng, B. (2007). Why we twitter: understanding microblogging usage and communities. In Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis (pp. 56–65). https://doi.org/10.1145/1348549.1348556
Kahle, K., Sharon, A. J., & Baram-Tsabari, A. (2016). Footprints of fascination: Digital traces of public engagement with particle physics on CERN’s social media platforms. PLoS ONE, 11(5), e0156409. https://doi.org/10.1371/journal.pone.0156409
Kallus, N. (2014). Predicting crowd behavior with big public data. In Proceedings of the 23rd international conference on World Wide Web (pp. 625–630). https://doi.org/10.1145/2567948.2579233
Kaur, C., & Sharma, A. (2020). Twitter Sentiment Analysis on Coronavirus using Textblob. EasyChair Preprint. https://easychair.org/publications/preprint/Fd5m
Kim, S. Y., Ganesan, K., Dickens, P., & Panda, S. (2021). Public sentiment toward solar energy—Opinion mining of Twitter using a transformer-based language model. Sustainability, 13(5), 2673. https://doi.org/10.3390/su13052673
Kochan, C., & Budd, J. M. (1992). The persistence of fraud in the literature: The Darsee case. Journal of the American Society for Information Science, 43(7), 488–493. https://doi.org/10.1002/(SICI)1097-4571(199208)43:7%3C488::AID-ASI3%3E3.0.CO;2-7
Korpela, K. (2010). How long does it take for the scientific literature to purge itself of fraudulent material? The Breuning case revisited. Current Medical Research and Opinion, 26(4), 843–847. https://doi.org/10.1185/03007991003603804
Krouska, A., Troussas, C., & Virvou, M. (2017). Comparative evaluation of algorithms for sentiment analysis over social networking services. Journal of Universal Computer Science, 23(8), 755–768. https://doi.org/10.3217/jucs-023-08-0755
Leitner, S., Gula, B., Jannach, D., Krieg-Holz, U., & Wall, F. (2021). Understanding the dynamics emerging from infodemics: A call to action for interdisciplinary research. SN Business & Economics, 1, 23. https://doi.org/10.1007/s43546-020-00027-4
Lewandowsky, S., Ecker, U. K., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131. https://doi.org/10.1177/1529100612451018
Loftus, E. F. (2005). Planting misinformation in the human mind: A 30-year investigation of the malleability of memory: Figure 1. Learning & Memory, 12(4), 361–366. https://doi.org/10.1101/lm.94705
Marcus, A. I., & Oransky, I. (2011). The paper is not sacred. Nature, 480(7378), 449–450. https://doi.org/10.1038/480449a
Masic, I. (2012). Plagiarism in scientific publishing. Acta Informatica Medica: AIM: Journal of the Society for Medical Informatics of Bosnia & Herzegovina : Časopis Društva Za Medicinsku Informatiku BiH, 20(4), 208–2013. https://doi.org/10.5455/aim.2012.20.208-213
Medford, R. J., Saleh, S. N., Sumarsono, A., Perl, T. M., & Lehmann, C. U. (2020). An “Infodemic”: Leveraging high-volume twitter data to understand early public sentiment for the coronavirus disease 2019 outbreak. Open Forum Infectious Diseases. https://doi.org/10.1093/ofid/ofaa258
Mittal, A., & Goel, A. (2012). Stock prediction using twitter sentiment analysis. Standford University, https://cs229.stanford.edu/proj2011/GoelMittal-StockMarketPredictionUsingTwitterSentimentAnalysis.pdf
Mohan, V. V., & Munigal, A. (2012). Use of community building web technologies in libraries: A study of twitter in American libraries. In Proceedings of informing science & IT education conference, 12 (pp. 205–221). https://doi.org/10.28945/1650
Moussa, S. (2022). The propagation of error: Retracted articles in marketing and their citations. Italian Journal of Marketing, 2022(1), 11–36. https://doi.org/10.1007/s43039-021-00044-7
Moylan, E. C., & Kowalczuk, M. K. (2016). Why articles are retracted: A retrospective cross-sectional study of retraction notices at BioMed Central. British Medical Journal Open, 6(11), e012047. https://doi.org/10.1136/bmjopen-2016-012047
Munigal, A. (2014). Use of microblogs in India: A study of Twitter usage by librarians and in libraries. Journal of Library Administration, 54(7), 590–608. https://doi.org/10.1080/01930826.2014.964021
Musto, J., & Dahanayake, A. (2021). Quality Characteristics for user-generated content. In Information modelling and knowledge bases XXXIII (pp. 244–263). IOS Press. https://doi.org/10.3233/FAIA210490
Nascimento, I. J. B. D., Pizarro, A. M., Almeida, J. M., Azzopardi-Muscat, N., Gonçalves, M. A., Björklund, M., & Novillo-Ortiz, D. (2022). Infodemics and health misinformation: A systematic review of reviews. Bulletin of the World Health Organization, 100(9), 544–561. https://doi.org/10.2471/blt.21.287654
Neethu, M. S. & Rajasree, R. (2013). Sentiment analysis in twitter using machine learning techniques. In Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT) (pp. 1–5). https://doi.org/10.1109/ICCCNT.2013.6726818
Nezhad, Z. B., & Deihimi, M. A. (2022). Twitter sentiment analysis from Iran about COVID 19 vaccine. Diabetes and Metabolic Syndrome: Clinical Research and Reviews, 16(1), 102367. https://doi.org/10.1016/j.dsx.2021.102367
Nistor, S. C., Moca, M., Moldovan, D., Oprean, D. B., & Nistor, R. L. (2021). Building a Twitter sentiment analysis system with recurrent neural networks. Sensors, 21(7), 2266. https://doi.org/10.3390/s21072266
Ortega, J. L. (2017). The presence of academic journals on Twitter and its relationship with dissemination (tweets) and research impact (citations). Aslib Proceedings, 69(6), 674–687. https://doi.org/10.1108/ajim-02-2017-0055
Pagolu, V. S., Reddy, K. N., Panda, G., & Majhi, B. (2016). Sentiment analysis of Twitter data for predicting stock market movements. 2016 international conference on signal processing, communication, power and embedded system (SCOPES). 1345–1350. https://doi.org/10.1109/SCOPES.2016.7955659
Pak, A., & Paroubek, P. (2010). Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of the seventh international conference on language resources and evaluation (pp. 1320–1326).
Paolanti, M., Mancini, A., Frontoni, E., Felicetti, A., Marinelli, L., Marcheggiani, E., & Pierdicca, R. (2021). Tourism destination management using sentiment analysis and geo-location information: A deep learning approach. Information Technology & Tourism, 23(2), 241–264. https://doi.org/10.1007/s40558-021-00196-4
Pastor, C. K. (2020). Sentiment analysis of Filipinos and effects of extreme community quarantine due to coronavirus (COVID-19) pandemic. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3574385
Peng, H., Romero, D. M., & Horvát, E. Á. (2022). Dynamics of cross-platform attention to retracted papers. Proceedings of the National Academy of Sciences, 119(25), e2119086119. https://doi.org/10.1073/pnas.2119086119
Perdana, R. S., & Pinandito, A. (2018). Combining likes-retweet analysis and naive Bayes classifier within twitter for sentiment analysis. Journal of Telecommunication, Electronic and Computer Engineering, 10, 41–46.
Pfeifer, M., & Snodgrass, G. L. (1990). The continued use of retracted, invalid scientific literature. JAMA, 263(10), 1420–1423. https://doi.org/10.1001/jama.1990.03440100140020
Porshnev, A., Redkin, I., & Shevchenko, A. (2013). Machine learning in prediction of stock market indicators based on historical data and data from twitter sentiment analysis. In: IEEE 13th international conference on data mining workshops (pp. 440–444). https://doi.org/10.1109/ICDMW.2013.111
Kaila, R. P., & Prasad, A. K. V. (2020). Informational flow on twitter-corona virus outbreak-topic modelling approach. International Journal of Advanced Research in Engineering and Technology, 11(3), 128–134. https://doi.org/10.34218/IJARET.11.3.2020.011
Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the American Society for Information Science and Technology, 47(1), 1–4. https://doi.org/10.1002/meet.14504701201
Puschmann, C. (2014). (Micro)Blogging science? Notes on potentials and constraints of new forms of scholarly communication. In Opening science. Springer, Cham. https://doi.org/10.1007/978-3-319-00026-8_6
Ra, M., Ab, B., & Kc, S. (2020). COVID-19 outbreak: Tweet based analysis and visualization towards the influence of coronavirus in the World. Gedrag & Organisatie, 33(02), 534–549. https://doi.org/10.37896/gor33.02/062
Rabab'Ah, A. M., Al-Ayyoub, M., Jararweh, Y., & Al-Kabi, M. N. (2016). Evaluating sentistrength for Arabic sentiment analysis. IN IEEE 7th International Conference on Computer Science and Information Technology (CSIT) (pp. 1–6). https://doi.org/10.1109/CSIT.2016.7549458
Rajput, N. K., Grover, B. A., & Rathi, V. K. (2020). Word frequency and sentiment analysis of twitter messages during Coronavirus pandemic. ArXiv. https://doi.org/10.48550/arXiv.2004.03925
Ruz, G. A., Henríquez, P. A., & Mascareño, A. (2020). Sentiment analysis of Twitter data during critical events through Bayesian networks classifiers. Future Generation Computer Systems, 106, 92–104. https://doi.org/10.1016/j.future.2020.01.005
Seddighi, H., Salmani, I., & Seddighi, S. (2020). Saving lives and changing minds with Twitter in disasters and pandemics: A literature review. Journalism and Media, 1(1), 59–77. https://doi.org/10.3390/journalmedia1010005
Serghiou, S., Marton, R. M., & Ioannidis, J. P. A. (2021). Media and social media attention to retracted articles according to Altmetric. PLoS ONE, 16(5), e0248625. https://doi.org/10.1371/journal.pone.0248625
Shema, H., Bar-Ilan, J., & Thelwall, M. (2012). Research blogs and the discussion of scholarly information. PLoS ONE, 7(5), e35869. https://doi.org/10.1371/journal.pone.0035869
Shema, H., Hahn, O., Mazarakis, A., & Peters, I. (2019). Retractions from altmetric and bibliometric perspectives. Information Wissenschaft Und Praxis, 70(2–3), 98–110. https://doi.org/10.1515/iwp-2019-2006
Sidiropoulou, S. (2021). Twitter sentiment analysis on fake news using python and natural language processing [M.Sc. thesis]. University of Macedonia. https://dspace.lib.uom.gr/bitstream/2159/26100/1/SidiropoulouSousanaMsc2021.pdf
Simon, T., Goldberg, A., & Adini, B. (2015). Socializing in emergencies—A review of the use of social media in emergency situations. International Journal of Information Management, 35(5), 609–619. https://doi.org/10.1016/j.ijinfomgt.2015.07.001
Singh, H. P., Mahendra, A., Yadav, B., Singh, H., Arora, N., & Arora, M. (2014). A comprehensive analysis of articles retracted between 2004 and 2013 from biomedical literature: A call for reforms. Journal of Traditional and Complementary Medicine, 4(3), 136–139. https://doi.org/10.4103/2225-4110.136264
Southwell, B. G., Niederdeppe, J., Cappella, J. N., Gaysynsky, A., Kelley, D. E., Oh, A., Chou, W. Y. S., et al. (2019). Misinformation as a misunderstood challenge to public health. American Journal of Preventive Medicine, 57(2), 282–285. https://doi.org/10.1016/j.amepre.2019.03.009
Splendiani, S., & Capriello, A. (2022). Crisis communication, social media and natural disasters: The use of Twitter by local governments during the 2016 Italian earthquake. Corporate Communications: An International Journal., 27(3), 509–526. https://doi.org/10.1108/CCIJ-03-2021-0036
Sugawara, Y., Tanimoto, T., Miyagawa, S., Murakami, M., Tsuya, A., Tanaka, A., Kami, M., & Narimatsu, H. (2017). scientific misconduct and social media: Role of Twitter in the stimulus triggered acquisition of pluripotency cells scandal. Journal of Medical Internet Research. https://doi.org/10.2196/jmir.6706
Swire-Thompson, B., & Lazer, D. (2020). Public Health and Online Misinformation: Challenges and Recommendations. Annual Review of Public Health, 41(1), 433–451. https://doi.org/10.1146/annurev-publhealth-040119-094127
Tangcharoensathien, V., Calleja, N., Nguyen, T., Purnat, T. D., D’Agostino, M. A., García-Saisó, S., Landry, M., Rashidian, A., Hamilton, C., AbdAllah, A., Ghiga, I., Hill, A., Hougendobler, D., Van Andel, J., Nunn, M., Brooks, I. M., Sacco, P. L., De Domenico, M., Mai, P., Briand, S., et al. (2020). Framework for managing the COVID-19 infodemic: Methods and results of an online, crowdsourced WHO technical consultation. Journal of Medical Internet Research, 22(6), e19659. https://doi.org/10.2196/19659
Tavoschi, L., Quattrone, F., D’Andrea, E., Ducange, P., Vabanesi, M., Marcelloni, F., & Lopalco, P. L. (2020). Twitter as a sentinel tool to monitor public opinion on vaccination: An opinion mining analysis from September 2016 to August 2017 in Italy. Human Vaccines & Immunotherapeutics, 16(5), 1062–1069. https://doi.org/10.1080/21645515.2020.1714311
Theis-Mahon, N. R., & Bakker, C. J. (2020). The continued citation of retracted publications in dentistry. Journal of the Medical Library Association, 108(3), 389–397. https://doi.org/10.5195/jmla.2020.824
Thelwall, M. (2017). The heart and soul of the web? Sentiment strength detection in the social web with SentiStrength. In: Holyst, J. (eds) Cyberemotions. Understanding Complex Systems (pp. 119–134). https://doi.org/10.1007/978-3-319-43639-5_7
Thelwall, M., & Buckley, K. (2013). Topic-based sentiment analysis for the social web: The role of mood and issue-related words. Journal of the Association for Information Science and Technology, 64(8), 1608–1617. https://doi.org/10.1002/asi.22872
Thelwall, M., Buckley, K., & Paltoglou, G. (2011). Sentiment in Twitter events. Journal of the Association for Information Science and Technology, 62(2), 406–418. https://doi.org/10.1002/asi.21462
Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013a). Do Altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841
Thelwall, M., Tsou, A., Weingart, S. D., Holmberg, K., & Haustein, S. (2013b). Tweeting links to academic articles. Cybermetrics: International Journal of Scientometrics, Informetrics and Bibliometrics, 17, 1–8.
Unger, K., & Couzin, J. (2006). Even retracted papers endure. Science, 312(5770), 40–41. https://doi.org/10.1126/science.312.5770.40
Van Der Vet, P. E., & Nijveen, H. (2016). Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from, the journal Nature. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-016-0008-5
Van Noorden, R. (2014). Online collaboration: Scientists and the social network. Nature, 512(7513), 126–129. https://doi.org/10.1038/512126a
Vilares, D., Thelwall, M., & Alonso, M. A. (2015). The megaphone of the people? Spanish SentiStrength for real-time analysis of political tweets. Journal of Information Science, 41(6), 799–813. https://doi.org/10.1177/0165551515598926
Vlachos, S. H. (2022). CENSA report: The link between mis-, dis-, and malinformation and domestic extremism. https://censa.net/publications/censa-report-the-link-between-mis-dis-and-malinformation-and-domestic-extremism/
Vuong, Q. (2019). The limitations of retraction notices and the heroic acts of authors who correct the scholarly record: An analysis of retractions of papers published from 1975 to 2019. Learned Publishing, 33(2), 119–130. https://doi.org/10.1002/leap.1282
Vural, A. G., Cambazoglu, B. B., Senkul, P., & Tokgoz, Z. O. (2013). A framework for sentiment analysis in Turkish: Application to polarity detection of movie reviews in Turkish. In: Gelenbe, E., & Lent, R. (Eds) Computer and information sciences III. (pp: 437–445). https://doi.org/10.1007/978-1-4471-4594-3_45
Wang, G., Lu, Q., & Capareda, S. C. (2020). Social network and extension service in farmers’ agricultural technology adoption efficiency. PLoS ONE, 15(7), e0235927. https://doi.org/10.1371/journal.pone.0235927
Wang, Y., McKee, M., Torbica, A., & Stuckler, D. (2019). Systematic literature review on the spread of health-related misinformation on social media. Social Science & Medicine, 240, 112552. https://doi.org/10.1016/j.socscimed.2019.112552
Williams, P., & Wager, E. (2013). Exploring why and how journal editors retract articles: Findings from a qualitative study. Science and Engineering Ethics, 19(1), 1–11. https://doi.org/10.1007/s11948-011-9292-0
Xu, S., & Hu, G. (2018). Retraction notices: Who authored them? Publications, 6(1), 2. https://doi.org/10.3390/publications6010002
Yang, S., Qi, F., Diao, H., & Ajiferukea, I. (2022). Do retraction practices work effectively? Evidence from citations of psychological retracted articles. Journal of Information Science. https://doi.org/10.1177/01655515221097623
Zhu, Y. (2014). Seeking and sharing research information on social media: A 2013 survey of scholarly communication. In Proceedings of European Conference on Social Media ECSM, pp. 705–712.
Zulfiker, M. S., Kabir, N., Biswas, A. A., Zulfiker, S., & Uddin, M. S. (2022). Analyzing the public sentiment on COVID-19 vaccination in social media: Bangladesh context. Array, 15, 100204. https://doi.org/10.1016/j.array.2022.100204
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1: Frequencies of opinion polarities for (re)tweets not containing mentions and # tags
Row | Entities | Frequency | Chi2 test | |||||
|---|---|---|---|---|---|---|---|---|
Polarity | Total | χ | Asymp. Sig | |||||
Negative | Neutral | Positive | ||||||
1 | Note tweets | 149 | 313 | 85 | 547 | 151.69 | 0.00 | |
27.24% | 57.22% | 15.54% | 100% | |||||
2 | Paper tweets | Pre-retraction | 1536 | 2066 | 743 | 4345 | 612.22 | 0.00 |
35.35% | 47.55% | 17.10% | 100% | |||||
3 | Post retraction | 117 | 210 | 61 | 388 | 87.59 | 0.00 | |
30.15% | 54.12% | 15.72% | 100% | |||||
4 | Total | 1653 | 2276 | 804 | 4733 | 692.10 | 0.00 | |
34.92% | 48.09% | 16.99% | 100% | |||||
5 | Note retweet | 262 | 512 | 265 | 1039 | 118.88 | 0.00 | |
25.22% | 49.28% | 25.51% | 100% | |||||
6 | Paper retweet | 3038 | 2006 | 1428 | 6472 | 616.69 | 0.00 | |
46.94% | 31% | 22.06% | 100% | |||||
Appendix 2: Examples of positive note and paper tweets
Theme | Note Tweet | Paper Tweet |
|---|---|---|
Mentioning positive titles or reporting positive results of the paper (retracted or retracted-would-be) | high heels increase women's attractiveness | Our study…suggests that female protégés who remain in academia reap more benefits when mentored by males rather than equally-impactful females |
… the study claiming high heels make women look more attractive was bogus. Looks like the authors manipulated data to get the sexy result. … | … Breathing is one of the most important physiological functions to sustain life and health | |
Retraction of … that concluded having a female mentor is detrimental for young researchers' scientific success | A 'scientific' research paper 'found' female mentors don't lead to academic success! Now the paper stands retracted … | |
Expressing excitement about, suggesting, or praising the act of retracting/ appreciating other tweeters for their points/ challenging peer review or publishing | The paper should never have been accepted in the first place, but glad to see it retracted | The retractation of this paper is an excellent news for those who truely devote time to true mentoring |
Respect to … and other authors for this brave and admirable decision … | You know THAT paper… Ignore it. Better yet, retract it. Here are much better papers on the topic with more solid conceptualization and better supported interpretation. No, women do not need male mentors to be successful. Yes, women mentors support success of women scientists | |
Brave and decent decision by … and colleagues to retract a paper after reanalysis | Glad to see that … has retracted this article! | |
It was retracted. So yeah. It is nonsense. Amazing that your common sense missed that | This paper should be retracted—both due to its flawed methodology and damaging conclusions. My PhD supervisor is a woman and both a stellar scientist as well as an excellent mentor. My achievements are as much down to her as anyone else | |
Whilst I am delighted this article has been retracted, one must wonder how it was allowed to be published in the first place | This paper should be retracted. Shall I wear my suit and/or scrubs when I go to the beach? Shall I not go out to enjoy myself and relax as this will look unprofessional? This paper should never have been published | |
… Glad the debate on how to achieve true equity in science is thriving | The journal must retract the paper and apologize to the scientific community. I am very proud that my research is led by a woman. I can't believe that a journal with this standard would publish such an article | |
Quoting, challenging, or referring to the retraction reasons | … the authors were caught up in the excitement of the moment … | 1. Retract the paper. 2. Publish measures you'll take to ensure this kind of stuff doesn't happen again, preferably through a commitment towards systemic change |
Some examples … where the retractions correct genuine, inadvertent mistakes, point to reanalysis & should be praised | Although the paper has been retracted. I question … why did this paper make it to publication in the first place? How can we trust the validity of a journal if articles with clear bias are being assigned a DOI … | |
Failure in applying negational words in SentiStrength, when decomposing compound terms | not-so-much-exciting response parsed and scored as not[0] so[0] much[0] exciting[2] response[0] | |
Providing positive evidence to challenge or refute the results of the retracted or retracted-would-be paper or mocking them | The … group students absolutely love having a female mentor who is always encouraging inclusivity, diversity and equity and whose strong personality, kindness and brilliant mind serve as an inspiration to all of us | |
What! | ||
Raise your hand if you are a female scientist who had a female mentor who was pivotal to you[r] success. This paper is way off base | ||
| ||
LOVE that feeling when a paper about mentorship, success and gender misunderstands the nature of mentorship…and success…and gender | ||
My ♀ mentors' support, encouragement & examples as role models are why I am still here, standing strong, & love being an academic … | ||
A broader review of existing scientific evidence clearly shows that approved masks with correct certification, and worn in compliance with guidelines, are an effective prevention of COVID-19 transmission | ||
He asked that the vaccine be encouraged. Masking has been extremely helpful as well … | ||
What a joke of a paper! | ||
Confirming the results of or supporting the publication of the retracted or retracted-would-be paper | Wow a real study with references | |
The paper is WOW | ||
Just take … [the facemasks] off. breathe freely. eat well, sleep well, exercise. laugh, hug, love |
How! The mentorship quality of my female supervisors is excellent! Thanks for all your support, hardwork and help … I would have never arrived so far without your supervision
✋several, but most crucial to my success has been the excellent (and ongoing) mentorship from …Appendix 3: Examples of negative and neutral tweets
No. | Theme | Tweet | Polarity |
|---|---|---|---|
1 | Challenging peer reviewing/publishing | They're very aware and concerned and yet they published it anyway, after 4(!!) negative peer-reviews and many, many researchers saying that it should be retracted "¦Seriously"¦ in my country we have … a saying…u better think about what you're doing, instead of having to apologize | Neutral |
2 | Challenging the retraction | Why was this article retracted? What pressure was put on this author? | Negative |
3 | … Feminists lost their shit over this and forced the female author and journal to retract the paper and apologise | Negative | |
4 | … Why is NIH having Med Hypotheses retracted that agree with Fauci and in fact cite Fauci? | Neutral | |
5 | I wonder why this got retracted? | Neutral | |
6 | This paper has now been retracted, no comment on why, but it cites many studies relating to harms from masks. Conclusion section has a good summary. Not something that can just be ignored. Reports on nasty materials in certain masks also now widespread | Negative | |
7 | Why is NIH having medical articles retracted that don't suit their agenda? | Neutral | |
8 | Why retracted?? | Neutral | |
9 | Why was the … mask study retracted? | Neutral | |
10 | Why was the … mask study retracted? Where is the "study" /proof showing it's not true? | Neutral | |
11 | Suggesting retraction and demanding improvement of peer reviewing | The paper should be retracted. But that is nowhere near enough. Journals need to be tracking gender and other group statuses, and ensuring that acceptance rates are equitable. And they need to report those results transparently and on an ongoing basis | Neutral |
12 | Challenging the retraction and peer reviewing/publishing | Question is why editors published it in first place. And maybe first to be retracted under social media pressure | Negative |
13 | Encouraging pressures for retraction | It looks like outraged voices of this appalling … on male vs female mentor paper have been heard. Let's keep the pressure on the journal to retract the paper by commenting on their post and tweeting up a storm! | Negative |
14 | Responding to the tweets challenging the retraction | 99% of those saying the article should not be retracted because "scientific debate"… are white men. Just saying… | Neutral |
15 | Wonder why this source that he mentioned has been retracted. The sheep will still follow their masters to the slaughter and ignore this type of data | Negative | |
16 | Suggesting retraction | "… strongly believes in & supports equality and diversity in research"—What a contradiction with the fact that … published this paper. Want to support equality and diversity? Retract the paper | Neutral |
17 | … I still don't understand why this paper is not retracted yet?? | Neutral | |
18 | Agree this sexist paper should be retracted based on its flawed analysis and unsupported conclusions | Negative | |
19 | An immediate retraction and apology is needed. This is detrimental to the huge efforts to increase the number of women at research institutions. Plus, it is scientifically flawed. Must be explained why this paper was even accepted for publication | Negative | |
20 | Fast retraction and an apology with reasons why it was published in the first place are expected | Neutral | |
21 | Here is the link to this unprofessional study If you are a true #heforshe then you must speak up against this study This should be retracted immediately | Negative | |
22 | Morally and ethically this article should be RETRACTED!!! | Neutral | |
23 | Please, … retract it asap!!!!! And explain to us why???? Why this article was accepted??? A lot of good science rejected and this bullshit published in …! Unbelievable! Disgusting!!! | Negative | |
24 | Question is why editors published it in first place. And maybe first to be retracted under social media pressure | Negative | |
25 | That's unbelievable … retract the paper! | Negative | |
26 | … Time to retract the paper & regain our trust | Neutral | |
27 | This study should be retracted with an apology | Neutral | |
28 | You must retract the paper if you want this not to damage your reputation. Show women in STEM that this was a mistake and you do not support this kind of behavior! | Neutral | |
30 | What an amazing pile of crap. This should be retracted | Neutral | |
31 | Suggesting retraction and challenging peer review/publishing | … 1. 3 doctors thought this was okay (it was not it was stalking) 2. An IRB approved this! 3. A Medical Journal published it. This article should be retracted and an apology given … | Neutral |
32 | Retract the paper…The published peer reviews brought up all of these issues and nothing was done to address them | Neutral | |
33 | The fact this went through peer review, in addition to an editor signing off on the manuscript is shocking. It sends the wrong message to trainees. The right thing to do … is to retract the paper | Negative |
Appendix 4: The partial correlation between tweet-note similarity and tweet opinion strength
Level | Variables | Levenshtein similarity for | ||||||
|---|---|---|---|---|---|---|---|---|
Tweets and notes | Tweets and paper abstracts | Tweets and paper titles | Notes and paper titles | Notes and paper abstracts | ||||
Zero | Tweet Opinion strength | r | − 0.206 | − 0.096 | − 0.170 | 0.027 | 0.004 | |
Sig | 0.000 | 0.040 | 0.000 | 0.565 | 0.928 | |||
Levenshtein similarity for | Tweets and notes | r | 0.384 | 0.474 | 0.039 | − 0.418 | ||
Sig | 0.000 | 0.000 | 0.410 | 0.000 | ||||
Tweets and paper abstracts | r | 0.330 | 0.176 | 0.151 | ||||
Sig | 0.000 | 0.000 | 0.001 | |||||
Tweets and paper titles | r | 0.023 | − 0.101 | |||||
Sig | 0.619 | 0.031 | ||||||
Notes and paper titles | r | 0.435 | ||||||
Sig | 0.000 | |||||||
Control | Tweet Opinion strength | r | − 0.180 | 0.028 | − 0.071 | |||
Sig | 0.000 | 0.548 | 0.132 | |||||
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Amiri, M., Yaghtin, M. & Sotudeh, H. How do tweeters feel about scientific misinformation: an infoveillance sentiment analysis of tweets on retraction notices and retracted papers. Scientometrics 129, 261–287 (2024). https://doi.org/10.1007/s11192-023-04871-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-023-04871-7