Abstract
As the number of retractions of the academic literature increase, there is also a bibliometric interest in appreciating the volume of citations that retracted papers received prior to, and following, a retraction. We assessed the pre- and post-citation counts of one paper each of five classes of retracted papers using the Web of Science Core Collection (WOS-CC). The papers (and their respective DOIs) were: (C1) Data or classification error (https://doi.org/10.1371/journal.pone.0204076); (C2) fraud (https://doi.org/10.1007/s00265-011-1208-0); C3) paper mill (https://doi.org/10.1155/2017/2381482); C4) tortured phrase (https://doi.org/10.1007/s12652-020-02455-4); C5) plagiarism (https://doi.org/10.1016/j.neucom.2012.06.051). C1, C2, C3, C4 and C5 were cited (September 7, 2022) 43, 45, 35, 3, and 16 times, respectively. Separately, noting the shortcomings of the WOS-CC citation classification system for citations, we propose a new four-tier classification system for citation sentiment or use (background; supportive; contradictory; manipulative) that accommodates both the functionality and sentiment of a citation to assess how a retracted paper (or indeed, any paper), is cited. We tested this new classification system using C5. We envision that these two new systems (five classes of retracted papers; four tiers of citation behavior) can further be implemented in citation databases to improve the informative value of these resources.
This is a preview of subscription content, log in via an institution to check access.
Data availability
All data that was used for this research may be found in Suppl. file 1.
Notes
“Background—previously published research that orients the current study within a scholarly area. Basis—references that report the data sets, methods, concepts and ideas that the author is using for her [sic] work directly or on which the author bases her [sic] work. Support—references which the current study reports to have similar results to. This may also refer to similarities in methodology or in some cases replication of results. Differ—references which the current study reports to have differing results to. This may also refer to differences in methodology or differences in sample sizes, affecting results. Discuss—references mentioned because the current study is going into a more detailed discussion.” https://clarivate.com/webofsciencegroup/release-notes/wos/new-wos-may-12-release-notes/ and https://clarivate.com/webofsciencegroup/release-notes/wos/new-wos-february-18-release-notes/.
References
Bar-Ilan, J., & Halevi, G. (2018). Temporal characteristics of retracted articles. Scientometrics, 116(3), 1771–1783. https://doi.org/10.1007/s11192-018-2802-y
Bolboacă, S. D., Buhai, D.-V., Aluaș, M., & Bulboacă, A. E. (2019). Post retraction citations among manuscripts reporting a radiology-imaging diagnostic method. PLoS ONE, 14(6), e0217918. https://doi.org/10.1371/journal.pone.0217918
Bolland, M. J., Grey, A., & Avenell, A. (2022). Citation of retracted publications: A challenging problem. Accountability in Research, 29(1), 18–25. https://doi.org/10.1080/08989621.2021.1886933
Candal-Pedreira, C., Ruano-Ravina, A., Fernández, E., Ramos, J., Campos-Varela, I., & Pérez-Ríos, M. (2020). Does retraction after misconduct have an impact on citations? A pre–post study. BMJ Global Health, 5(11), e003719. https://doi.org/10.1136/bmjgh-2020-003719
Dobránszki, J., & Teixeira da Silva, J. A. (2019). Corrective factors for author- and journal-based metrics impacted by citations to accommodate for retractions. Scientometrics, 121(1), 387–398. https://doi.org/10.1007/s11192-019-03190-0
Gray, R., Al-Ghareeb, A., & McKenna, L. (2019). Why articles continue to be cited after they have been retracted: An audit of retraction notices. International Journal of Nursing Studies, 90, 11–12. https://doi.org/10.1016/j.ijnurstu.2018.10.003
Heibi, I., & Peroni, S. (2022). A protocol to gather, characterize and analyze incoming citations of retracted articles. PLoS ONE, 17(7), e0270872. https://doi.org/10.1371/journal.pone.0270872
Hesselmann, F., Graf, V., Schmidt, M., & Reinhart, M. (2017). The visibility of scientific misconduct: A review of the literature on retracted journal articles. Current Sociology, 65(6), 814–845. https://doi.org/10.1177/0011392116663807
Kunnath, S. N., Herrmannova, D., Pride, D., & Knoth, P. (2022). A meta-analysis of semantic classification of citations. Quantitative Science Studies, 2(4), 1170–1215. https://doi.org/10.1162/qss_a_00159
Liu, X., Wang, C., Chen, D. Z., & Huang, M. H. (2022). Exploring perception of retraction based on mentioned status in post-retraction citations. Journal of Informetrics, 16(3), 101304. https://doi.org/10.1016/j.joi.2022.101304
Lyu, D.-Q., Ruan, X.-M., Xie, J., & Cheng, Y. (2021). The classification of citing motivations: a meta-synthesis. Scientometrics, 126(4), 3243–3264. https://doi.org/10.1007/s11192-021-03908-z
Pérez-Neri, I., Pineda, C., & Sandoval, H. (2022). Threats to scholarly research integrity arising from paper mills: a rapid scoping review. Clinical Rheumatology, 41(7), 2241–2248. https://doi.org/10.1007/s10067-022-06198-9
Tahamtan, I., & Bornmann, L. (2019). What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018. Scientometrics, 121(3), 1635–1684. https://doi.org/10.1007/s11192-019-03243-4
Teixeira da Silva, J. A., & Bornemann-Cimenti, H. (2017). Why do some retracted papers continue to be cited? Scientometrics, 110(1), 365–370. https://doi.org/10.1007/s11192-016-2178-9
Teixeira da Silva, J. A., & Dobránszki, J. (2017). Highly cited retracted papers. Scientometrics, 110(3), 1653–1661. https://doi.org/10.1007/s11192-016-2227-4
Theis-Mahon, N. R., & Bakker, C. J. (2020). The continued citation of retracted publications in dentistry. Journal of the Medical Library Association, 108(3), 389–397. https://doi.org/10.5195/jmla.2020.824
van der Vet, P. E., & Nijveen, H. (2016). Propagation of errors in citation networks: a study involving the entire citation network of a widely cited paper published in, and later retracted from, the journal nature. Research Integrity and Peer Review, 1(1), 3. https://doi.org/10.1186/s41073-016-0008-5
Funding
No funding was received by either author for this research.
Author information
Authors and Affiliations
Contributions
Except for the database searches in WOS-CC, which were conducted by the second author, the authors contributed equally to all other aspects of the paper.
Corresponding authors
Ethics declarations
Conflicts of interest
The authors declare no relevant conflicts of interest.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Teixeira da Silva, J.A., Nazarovets, S. Partial citation analysis of five classes of retracted papers, and devising a new four-tier citation classification system for retracted (and other) papers. Scientometrics 128, 4887–4894 (2023). https://doi.org/10.1007/s11192-023-04769-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-023-04769-4