Background

Inappropriate authors’ self-promoting is a growing mal-practice [1], possibly prompted by the mounting scientific importance of author’s metrics. We recently surveyed the submission guidelines of Anaesthesiology journals for the presence of policies discouraging author self-citations (A-SC) [2]. In parallel, we also evaluated the journals’ self-citation (J-SC) practice, which may hinder attempts of the Editorial Board to increase the journal impact factor (IF).

Although excessive and inappropriate A-SC and J-SC are two independent forms of suboptimal academic practice that have not yet received enough attention, it must be clear from the beginning that not all A-SC and J-SC are synonymous of malpractice.

In the present study, we evaluated the J-SC rate among the Critical Care Medicine (CCM) journals and possible factors influencing it, as well as the presence of policies regarding A-SC in these journals.

Methods

On the 30 October 2020, we evaluated the presence of policies discouraging A-SC in the submission guidelines of CCM journals selected from the InCites Journal Citation Reports 2019 (Clarivate Analytics®) [3]. The J-SC rate was evaluated according to the following formula:

$$ \mathrm{JSC}\ \mathrm{rate}=\frac{\mathrm{IF}-\mathrm{IF}\ \mathrm{without}\ \mathrm{self}\ \mathrm{citations}}{\mathrm{IF}} $$

Our previous investigation suggested that journals focusing on a specific topic of the discipline had a trend towards higher J-SC rate (25% vs 7% as compared to those with broad interest; p = 0.06) [2]. Accordingly, we performed a secondary analysis separating CCM journals in three sub-categories: (1) “multidisciplinary journals” (defined as journals focusing not only on CCM but also on other disciplines as Anaesthesiology, Respiratory Medicine, etc.); (2) “broad interest” CCM journals (defined as those publishing on several aspects of the discipline); (3) “topic-specific” CCM journals (focusing on specific topics of CCM interest). We evaluated also if J-SC is influenced by presence of A-SC policies.

Continuous variables are presented as median (25th–75th percentile), categorical variables as numbers/percentage. Mann-Whitney U test or Kruskal-Wallis test for unrelated samples were performed according to the number of groups. Tests were two-sided; p < 0.05 was considered statistically significant.

Results

A total of 36 CCM journals were found but we removed Human Gene Therapy Clinical Development as its scope clearly falls outside the CCM. Therefore, 35 journals were included. In Table 1, we report J-SC rates and presence of policies regarding A-SC. Five journals (14.3%) discouraged “excessive and inappropriate” self-citations; none proposed an A-SC cut-off. The median IF of CCM journals was 2.7 (2.1–3.8) and was higher in CCM journals with policies on A-SC[4.1 (3–12)] as compared to those without [2.5 (2–3.5); p = 0.02].

Table 1 List of Critical Care Medicine journals according to their rank in Journal Citation Report 2019. For each journal we provide the following: Journal Rank and Full Title, Impact Factor (IF) and IF without self-citation (SC), Journal Self-Citation Rate (J-SC, year 2019), Publisher name, presence and description of policies on limiting SC. Each Journal’s name contains a hyperlink to its “instruction to authors/submission guidelines” so that readers may check them
Full size table

The J-SC rate was highly variable (0–35.4%) with median 8.8% (5.1–17.4%) and was not different between journals with or without A-SC policies [8.1% (5.1–9.5%) vs 9.4% (5–18.7%), respectively; p = 0.32].

CCM journals were sub-categorized as follows: 11 were considered “multidisciplinary” (Table 1: rank 1-3-4-11-18-20-27-28-29-34-35), 14 were classified as “broad interest” (rank 2-5-6-8-9-12-13-15-19-21-22-23-25-31), and 10 were “topic-specific” CCM journals (rank 7-10-14-16-17-24-26-30-32-33). We found significantly different J-SC rate according to the journal interest (p = 0.01), with higher values in “topic-specific” [15.3 (8.8–23.3%)] followed by “broad interest” [11.8 (4.8–17.9%)] and “multidisciplinary” journals [6.1 (3.6–9.1%)]. Conversely, the IF was not different between journals according to sub-categories (p = 0.35): “topic-specific” [2.4 (1.4–3.2)], “broad interest” [2.5 (1.5–8.3)] and “multidisciplinary” journals [3.0 (2.3–4.7)]. The J-SC rate was not significantly influenced by the presence of policies on A-SC (8.1% [5.1–9.5%]) or not (9.4% [5–18.7%]; p = 0.32).

Discussion

Our investigations follows a similar assessment conducted in Anaesthesiology journals [2] and found lower prevalence of A-SC policies among CCM journals as compared to Anaesthesiology ones (14% vs 22%). This finding reinforces our belief that the argument of limiting the malpractice of inappropriate self-referencing is still at embryonic level. Several forms of research misconduct (fabrication, falsification, plagiarism, ghost-writing) have been recognized [4, 5], but “inappropriate self-citation” and “citation farms” have not yet received widespread editorial attention. In our opinion, it is urgent to embrace a debate on the best approach for limiting inappropriate self-referencing, keeping in mind that it remains challenging to define the appropriateness of each A-SC [6] and that a single cut-off is unlikely to fit all the manuscript types. Interestingly, we found a significantly higher IF in CCM journal with policies on A-SC, and this may be a marker of higher publishing standards. Such finding is new, as we did not report differences in IF among Anaesthesiology journals according to presence of A-SC policies [2]. This result may support larger investigations, as the findings of single disciplines are likely influenced by a reduced sample size investigated.

The J-SC rate and its variability were similar between CCM (8.8%, range 0–35.4%) and Anaesthesiology journals (8.4%, range 1.4–37.2%), and J-SC rate was not different according to the presence of A-SC policies.

Several factors may influence J-SC rate. Although it is possible that an excessive J-SC rate may sometimes hinder a sort of editorial malpractice (requests to reference specific articles with the aim to boost IF) [7], this is difficult to assess. Conversely, it must be noted that J-SC rate is also increased by editorials and commentaries introducing the highlights of important articles published by the journal. Therefore, the interpretation of J-SC rate is another challenging aspect of self-promoting. Importantly, whilst the presence of A-SC policies does not seem to influence the J-SC rate, our secondary analysis showed that the focus of interest of CCM journals is a factor significantly influencing the J-SC rate. Indeed, CCM journals with narrower focus of interest had significantly higher J-SC as compared with “broad CCM interest” and “multidisciplinary” journals. Therefore, while a narrower focus of interest should not be regarded as justification for excessive self-promoting, the interpretation of J-SC rates should be paired, among others, with critical estimation of the journal’s scope. Conversely, we found that the focus of interest does not seem to influence the journal IFs.

Considering all the factors influencing the J-SC rates and also the difficulties in critically evaluating the appropriateness of each A-SC or J-SC, an option could be to calculate author’s and journal’s scientific metrics excluding self-citations. Doing so, inappropriate self-referencing will become a useless practice. Importantly, Scopus® and Web of Science® databases offer the opportunity to exclude A-SC when observing scientific metrics. Similarly, it is feasible to calculate the journal’s IF without J-SC contribution (as shown in Table 1 in our study).

Limitations

Overall, our investigation on A-SC policies and J-SC rates over-simplifies complex issues, since we again reinforce that it is challenging to evaluate the appropriateness of each A-SC as well as to show editorial requests to add specific citations. Furthermore, the lack of policies regarding A-SC and the J-SC rate may be greater in “predatory journals” [8, 9]. Another limitation of this study is that several journals publishing in the field of CCM were not included as not listed in the InCites Journal Citation Reports. For instance, several Anaesthesiology journals have dedicated sections on CCM.

Conclusions

We found a very limited number of CCM journals with policies for limiting A-SC. Journals with A-SC policies had higher IF. The J-SC rate was highly variable and greater in journals with narrower focus of interest. Excluding author’s and journal’s self-referencing from the scientific metrics calculation could be a valuable option to tackle these forms of scientific malpractice.