Abstract
Background
The term “JUMPS” was used to describe the impact factor (IF) in an article published in PubMed in 2021, representing an increase of more than 40% of IF.
Aims
In this study, we aimed to compare the growth rate of IF JUMPS in Dermatology in the last 5 years, and particularly the effect of the 2020 COVID-19 pandemic.
Methods
This study evaluated the growth rate (JUMP) in IF from 2016 to 2020. We used the Friedman and Wilcoxon signed ranks tests. We classified JUMPS in negative growth rate; Q1 to Q4 quartiles; and journals with > 100%. A 76–100% growth rate was observed in five (7%) journals, and twelve journals (17%) depicted a 51–75% percentage of change.
Results
Several journals in the Dermatology category increased their IF by 50%. Repeated measures analyses showed a significant difference (p < .001).
Conclusion
Although we found journals with growth rates in the four quartiles, no journals depicted negative growth rates nor > 100% growth. Knowing the growing trends in this category might supplement the assessment of target journals for authors looking to submit their works.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
The interest in reporting on the impact factor (IF) in dermatology journals has been in the medical literature for almost 30 years [1, 2]. However, in the last years, when more technical articles have appeared commenting on self-citation and IF [3], the influence of article type [4], the impact factor as a double-edged sword (for editorial decisions) [5], and the percentage of growth of the IF for selected dermatology journals [6].
The number of citations an article achieves and the impact factor (IF) are the two most generally used metrics for evaluating a research paper’s quality in medical publications. In their decisions to submit their contributions, scientific authors continue to focus particularly value on the Journal IF [7]. The IF was established to let researchers compare journals of different sizes [8]. The Journal Citation Reports (JCR) introduced the journal IF for medical journals, administered by Clarivate Analytics via Web of Science and classified under the science citation index (SCI) and SCI Expanded (SCIE) [9].
The term “JUMPS,” applied to the IF, appeared published for the first time in PubMed in 2010, in the Int J Stroke; this journal experienced a 1-year increase of 43.6% (2.0 to 2.871, during 2008–2009) [10]. At the time of writing this manuscript (March–April 2022), the term IF JUMPS appeared again in PubMed in 2021, in an editorial of J Oral Pathol Med acknowledging that 2020 IF of that Journal had climbed to 4.235, equivalent to 69.7% (a record high for that journal and a significant increase from its 2019 IF of 2.495) [11].
Considering the information mentioned above, we aimed to compare the journals category of Dermatology, the IF, and their growing-rate percentage jump from 2015 to 2020; we believe knowing the growing trends of journals in this category might supplement the assessment of target journals for authors looking to submit their works.
Methods
Study design
The main objective of this study was to evaluate the achievement of journals in the Dermatology category on the Web of Knowledge [9] that recorded the IF values listed in the Journal Citations Reports (JCR) [12] for 5 years (2016–2020). Definitions of the selected bibliometrics from the Web of Knowledge have been recently published [7]. The report of the study followed the STROBE statement guidelines [13]. Because this retrospective study used public, historical data, it did not require the approval of an Institutional Review Board.
Search strategy
The search was performed on the Web Of Knowledge from the Clarivate platform, where the dermatology category was selected. The corresponding files were selected from 2016 to 2020, and an Excel file was downloaded.
Eligibility criteria
The journals included within the dermatology category and have an impact factor for more than three consecutive years were evaluated.
Data extraction
Database and percentages of change were calculated using Microsoft Excel v16.33 (Microsoft Corporation, Redmond, WA, USA). All analyses and graphical representations were performed using IBM® SPSS® Statistics software (version 27.0.1.0 IBM Corporation; Armonk, NY, USA). Boxplots and graphs were drawn with JMP Pro v16.1.0 (SAS Institute Inc., Cary, NC, USA). Statistical significance was indicated by p < 0.05 (two-tailed).
Impact factor measurement
The IF for a publication was calculated by dividing the number of times papers published in a journal were referenced in the previous 2 years by the number of articles published during the same period [14].
Journal selection and measured periods
We chose the IF values of journals in the JCR Science edition Dermatology category that coincidentally appeared between 2015 and 2020 (Table 1). Selected journals included at least three measures of IF in the selected period. The list of 69 chosen journals is available in the supplementary online-only file.
Percentage of growing rate (IF JUMP)
We calculated each year IF percentage (%) change using the formula:
[(IF chosen year − IF previous)/ IF previous] × 100
We calculated five IF growth rates (2015–2016; 2016–2017; 2017–2018; 2018–2019; 2019–2020) for the selected journals; the growth rates of 69 selected journals during 2020 are available in the supplementary online-only file.
Statistical analysis
Changes in IF from 2015 to 2020
We assessed the IF and IF growing rates distribution using the Kolmogorov–Smirnov and Shapiro–Wilk tests; data sets with a non-normal distribution were reported using median, quartiles, and interquartile range (IQR). A significant difference between the IF values and IF growth rates for all the years was calculated using the Friedman test. The specific difference in 2019–2020 IF (to detect JUMPS during the COVID-19 pandemic) was assessed with the Wilcoxon signed ranks test.
Identification of IF JUMPS in growing rates
We calculated the % change in IF (growing rates) from 2015 to 2020 IF; after evidence of a non-normal distribution of IF in each year, the significant difference between years was explored with the Friedman Test, and the specific difference between 2019 and 2020 growing rate (to detect JUMPS during the COVID-19 pandemic) was assessed with the Wilcoxon signed ranks test.
Identification of journals with the highest growth rates
After demonstration of a significant difference in growing rates between 2019 and 2020, we grouped the growing rates (% change) of 2020 in 6 groups based on their quartile values: level 1 = negative grow; level 2 = 1–25%; level 3 = 26–50%; level 4 = 51–75%; and level 5 = 76–100%; and level 6 > 100%.
Software
All analyses and graphical representations were carried out using the IBM® SPSS® Statistics software (version 27.0.1.0 IBM Corporation; Armonk, NY, USA). The percentage of change was calculated using Microsoft Excel v16.33 (Microsoft Corporation, Redmon, WA, USA). Box-plot and graphs were drawn with JMP Pro v16.1.0 (SAS Institute Inc., Cary, NC, USA). Statistical significance was indicated by a p-value < 0.05 (two-tailed).
Results
Impact factor trends
The IF from 2015 to 2020 depicted non-normal distribution for all years, p < 0.001. There was a statistical difference between the impact factor values for all the years (2016–2020); χ2 = 106.295, p < 0.001. In addition, there was a significant difference in the IF of journals between 2019 and 2020; Z = −6.990, p < 0.001. Table 1 shows the quartiles and IQR for selected journals; Fig. 1 shows the IF data distribution of selected journals.
Growing rates trends
The growing rates from 2016 to 2020 (Table 2) also depicted non-normal distribution for all years, p < 0.001, with a statistical difference between the rates values for all the years (2015–2020); χ2 = 36.8, p < 0.001. Also, there was a significant difference in the growth rates of journals, specifically between 2019 and 2020; Z = − 5.189, p < 0.001. Table 2 shows the quartiles and IQR for selected journals; Fig. 2 shows the data distribution of the IF growing rates.
Percentage of change for journals in 2020
From 69 journals evaluated, a negative growth was observed in 4 (6%) of journals; 23 (33%) journals showed 1–25% positive change; a 26–50% growth was observed in 23 (33%); 12 journals (17%) depicted a 51–75% percentage of change; a 76–100% growth rate was observed in 5 (7%) journals; and none of the journals had a growth > 100%. The complete list of journals grouped based on the % change of their 2020 IF is shown in Table 3.
Discussion
Even though it seems contradictory, this manuscript assessed the ability of the IF to detect JUMPS, not endorsing its usage. Several indexed journals compared dermatology journals in 2020 [6, 15, 16]. An unprecedented surge in scientific production has been observed for this speciality during the COVID-19 pandemic [17]. When authors decide where to submit their works, the IF rating of journals becomes a decisive factor. Readers should be aware that the IF has been misused as a proxy for article quality [18]. Usually, researchers look for journals with the highest impact factor instead of journals with the best audience for their research [19]. Our group has published before about the performance of bibliometrics in other specialities (Gastroenterology [20], Neurosciences [21] and Radiology [22]); because our first author will be a fellow of a dermatology program soon, we decided to explore the JUMPs in the IF in this category. For the dermatology community to recognize the new cutaneous associations in patients with coronavirus disease in 2019, dermatologists made significant contributions to COVID-19 and SARS-CoV-2 research during the pandemic [23]. Consequently, during the past 2 years, the trend in research publications in the dermatology field has been focused more on COVID-19 [24].
This study revealed significant IF JUMPS during the 2020 COVID-19 pandemic in selected Dermatology journals. We also grouped journals by quartiles based on their IF growth rate; we discovered that none journals during 2020 increased their IF growth rate by more than 100%.
Some journals have grown since 2016, but it has been much higher since 2020. In 2016, there was a journal with growth > 100%; in 2017, two journals with growth close to 100%, and in 2020, more than 50% of the journals grew > 50%.
Interestingly, the were only four journals with a negative growth rate in 2020; we do not have a confirmed explanation that explains the reason for their negative growth. We hypothesized that because these journals are highly specialized in the topics they published (veterinary dermatology or leprosy, as examples), these might not be common topics for most authors during the covid-19 pandemic. For that reason, those journals received fewer citations.
Academic writers generally agree that the general goal of scientific authors is to produce high-quality, high-impact research (basic or clinical), a task that necessitates endless after-hours work; nevertheless, only 10 to 14% of published information remains valid, and most articles are never cited again, except by the authors who originally published them [25]. Original articles, reviews, editorials, letters to the editor, and other scientific reports are published in the research process. Their influence on authors, journal profiles, and research institutions can be beneficial or harmful [26].
Although recent publications have displayed that the IF has been surpassed by other bibliometrics in journals in the Gastroenterology and Hepatology [20]; Neurosciences [21]; and Radiology, Nuclear Medicine & Medical Imaging [22] categories, we recognize that the success of researchers is nowadays judged by the number of publications they have published in high IF journals [27].
Authors interested in the IF value
The IF value is presently being pursued by several researchers: A) authors who are interested in how to choose a journal that will provide their articles more exposure; B) librarians who identify IF as a selection criterion for a title of greater scientific importance that should be included in the scientific collections of the institutions for which they work; C) medical editors regularly utilize it as a performance metric for their Journals; they agree that IF is an index of impact in fundraising, a way of rating their journals in comparison to their peers, and the attractiveness of high-quality papers to be published [28,29,30,31]. D) journals per se utilize the IF to “promote” their reputation and persuade potential writers to submit high-quality articles [32]. E) Funding organizations utilize IF to identify high-quality researchers and universities that can best address the needs of institutions. [33]. Academic institutions’ promotion committees frequently use the IF to assess the quality of applicants’ publications for promotion and tenure; departmental chairs may use it in the hiring and evaluation of recruits [34]. Journals with a more significant impact factor are often considered more prestigious than those with a lower impact factor [35].
Limitations
Several limitations of this study need to be addressed: a detailed explanation of each bibliometric or comparison between them is beyond the scope of this article; we only report the changes in IF. Other factors to consider: each medical speciality has its own IF threshold; for example, a publication in oncology can have an IF that is more than 30 times higher than the similar figure in forensic medicine [36]. Every journal has different citations, and even the most prestigious periodicals have some pieces that are never referenced [37]. Citations are not evenly distributed, with less than 20% of publications accounting for more than half of the total citations [38]. Despite these facts, the IF is still used to assess science's worth since it significantly rewards individual scientists and publications [39].
Although, if we recognize that in the 17 journals that we reported “JUMPS,” > 50% are journals that typically have an IF from 0.5 to 5, and regarding the periodicity of publication, we see that they are not monthly but bimonthly or quarterly. So there is no clearly defined pattern, and our number of observations is too small to apply statistical analysis. Interestingly, the journals also belong to free or open access journals, without a clear pattern or preference.
Additional elements influencing “citation” (self-citation, semi-mandatory, and mandatory citation) include more extended periods, the number of papers published in each issue, the circulation of each journal, and a slew of other variables that might influence citation calculations and, eventually, the IF. We did not include these possible confounding factors, as the Web of Knowledge does not consider them. We acknowledge that normalizing journal citations by their article count is desirable. However, we used the raw data from the JCR when writing this study. Other criteria that may influence where an author submits their work were mainly outside the scope of this study: topic, affiliation with society, geography, rejection by earlier submission, familiarity with the submission and revision process, turnaround time, and invitation by editors. Readers should be aware that the JCR includes approximately 171 categories in the sciences and 54 in the social sciences; future studies presenting IF JUMPS in other specialities would be desirable.
Future directions
Evaluating publications’ scientific effects (impact) using bibliometrics is a complex, multi-dimensional process. As a result, implementing a single bibliometric index to rank, assess, and value journals is ineffective. Readers should evaluate scientific publications on their merits rather than relying on the impact factor [40]. Preferably, multiple metrics with complementary features provide a comprehensive view of journals and their relative placements in their fields [41].
Conclusions
Knowing the growth rate of IF within each category and discipline can be used as a performance metric for dermatology authors. However, authors must first identify what metric(s) they will consider before submitting their manuscripts and which metric(s) they will use to evaluate and select the fittest journal and track their performance over the years. New metrics to assess the influence, growth, and publishing trends of acceptable research issues, and not only a high-IF growth rate might be a new trend in dermatology research.
References
Saurat JH (1995) The impact factor of dermatology. Dermatology 191:362. https://doi.org/10.1159/000246603
Dubin DB, Arndt KA (1995) The impact of dermatology journals. Arch Dermatol 131:1059–1060
Reiter O, Mimouni M, Mimouni D et al (2016) Analysis of self-citation and impact factor in dermatology journals. Int J Dermatol 55:995–999. https://doi.org/10.1111/ijd.13193
Rodriguez-Lago L, Molina-Leyva A, Pereiro-Ferreiros M, Garcia-Doval I (2018) Influence of article type on the impact factor of dermatology journals. Actas Dermosifiliogr (Engl Ed) 109:432–438. https://doi.org/10.1016/j.ad.2018.01.003
Hernandez Bel P (2018) Editorial decisions in dermatology journals: is the impact factor a double-edged sword? Actas Dermosifiliogr (Engl Ed) 109:389. https://doi.org/10.1016/j.ad.2018.03.001
Schlessinger DI, Jhawar N, Barbieri J, Lipoff JB (2020) Impact factor of major dermatology journals and the increasing influence of dermatology in the house of medicine. Dermatol Online J 26
Roldan-Valadez E, Salazar-Ruiz SY, Ibarra-Contreras R, Rios C (2019) Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics. Ir J Med Sci 188:939–951. https://doi.org/10.1007/s11845-018-1936-5
Adam D (2002) The counting house. Nature 415:726–729. https://doi.org/10.1038/415726a
Clarivate_Analytics (2022) Journal Citation Reports. Clarivate Analytics. https://clarivate.com/webofsciencegroup/solutions/journal-citation-reports/#:~:text=Journal%20Citation%20Reports%E2%84%A2%20(JCR,portfolio%20of%20thousands%20of%20publications. Accessed 27 Sept 2022
Donnan GA (2010) IJS impact factor jumps to 2.871 in only five years! Int J Stroke 5:341. https://doi.org/10.1111/j.1747-4949.2010.00474.x
Brennan PA (2021) Editorial - JOPM impact factor jumps to a record high. J Oral Pathol Med 50:631. https://doi.org/10.1111/jop.13223
Garfield E (1996) How can impact factors be improved? BMJ 313:411–413
Bern I-Uo (2009) Strengthening the reporting of observational studies in epidemiology. https://www.strobe-statement.org/index.php?id=strobe-home.
Kumar V, Upadhyay S, Medhi B et al (2009) Impact of the impact factor in biomedical research: its use and misuse. Singapore Med J 50:752–755
Flege MM, Thomsen SF (2020) Factors associated with citation of original articles in 5 high-impact-factor dermatology journals. J Am Acad Dermatol 83:1822–1824. https://doi.org/10.1016/j.jaad.2020.05.075
Parish LC, Lambert WC (2020) Dermatology Journals and the Impact Factor. Skinmed 18:207–208
Anazco D, Nicolalde B, Espinosa I et al (2021) Publication rate and citation counts for preprints released during the COVID-19 pandemic: the good, the bad and the ugly. PeerJ 9:e10927. https://doi.org/10.7717/peerj.10927
Ending the tyranny of the impact factor (2014) Nat Cell Biol 16:1. https://doi.org/10.1038/ncb2905
Rawat S (2014) How is impact factor impacting our research? Biomedical journal. https://doi.org/10.4103/2319-4170.131388
Roldan-Valadez E, Rios C (2015) Alternative bibliometrics from impact factor improved the esteem of a journal in a 2-year-ahead annual-citation calculation: multivariate analysis of gastroenterology and hepatology journals. Eur J Gastroenterol Hepatol 27:115–122. https://doi.org/10.1097/MEG.0000000000000253
Diaz-Ruiz A, Orbe-Arteaga U, Rios C, Roldan-Valadez E (2018) Alternative bibliometrics from the web of knowledge surpasses the impact factor in a 2-year ahead annual citation calculation: Linear mixed-design models’ analysis of neuroscience journals. Neurol India 66:96–104. https://doi.org/10.4103/0028-3886.222880
Roldan-Valadez E, Orbe-Arteaga U, Rios C et al (2018) Eigenfactor score and alternative bibliometrics surpass the impact factor in a 2-years ahead annual-citation calculation: a linear mixed design model analysis of Radiology, Nuclear Medicine and Medical Imaging journals. Radiol Med 123:524–534. https://doi.org/10.1007/s11547-018-0870-y
Wang Y, Fang R, Zhang H et al (2020) Contributions of dermatologists to COVID-19 research: a brief systematic review. Dermatol Ther 33:e13713. https://doi.org/10.1111/dth.13713
Nayak M, Prabhu SS, Ligade VS et al (2022) Research on COVID-19, SARS-CoV-2 in Dermatology Journals: A Brief Bibliometric Analysis Based on PubMed. Indian J Dermatol 67:96. https://doi.org/10.4103/ijd.ijd_397_21
Weinstein JN (2007) Threats to scientific advancement in clinical practice. Spine 32:S58-62. https://doi.org/10.1097/BRS.0b013e318053d4fc
Gasparyan AY (2010) Thoughts on impact factors and editing of medical journals. Inflamm Allergy Drug Targets 9:2–5
Butler JS, Sebastian AS, Kaye ID et al (2017) Understanding Traditional Research Impact Metrics. Clin Spine Surg 30:164–166. https://doi.org/10.1097/BSD.0000000000000530
Horgan A (2002) BMJ’s impact factor increases by 24%. BMJ 325:8
Parrillo JE (2005) Our Journal, Critical Care Medicine, in 2005: high impact factor, rapid manuscript review, growing submissions, and widespread distribution. Crit Care Med 33:923–924
Tobin MJ (2004) Thirty years of impact factor and the journal. Am J Respir Crit Care Med 170:351–352. https://doi.org/10.1164/rccm.2406005
Wedzicha JA, Johnston SL, Mitchell DM et al (2005) Journal impact factors for 2004: another rise for Thorax. Thorax 60:712. https://doi.org/10.1136/thx.2005.050922
Rizkallah J, Sin DD (2010) Integrative approach to quality assessment of medical journals using impact factor, eigenfactor, and article influence scores. PLoS ONE 5:e10204. https://doi.org/10.1371/journal.pone.0010204
Thomaz PG, Assad RS, Moreira LF et al (2011) Using the impact factor and H index to assess researchers and publications. Arq Bras Cardiol 96:90–93
Saha S, Saint S, Christakis DA et al (2003) Impact factor: a valid measure of journal quality? JMLA 91:42–46
Bergstrom CT, West JD, Wiseman MA et al (2008) The Eigenfactor metrics. J Neurosci : the official J Society Neurosci 28:11433–11434. https://doi.org/10.1523/JNEUROSCI.0003-08.2008
Baethge C (2012) Impact factor–a useful tool, but not for all purposes. Dtsch Arztebl Int 109:267–269. https://doi.org/10.3238/arztebl.2012.0267
Fersht A (2009) The most influential journals: Impact Factor and Eigenfactor. Proc Natl Acad Sci USA 106:6883–6884. https://doi.org/10.1073/pnas.0903307106
Weale AR, Bailey M, Lear PA et al (2004) The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor. BMC Med Res Methodol 4:14. https://doi.org/10.1186/1471-2288-4-14
Casadevall A, Fang FC (2014) Causes for the persistence of impact factor mania. mBio 5:e00064–00014. https://doi.org/10.1128/mBio.00064-14
Reider B (2017) Brace for Impact. Am J Sports Med 45:2213–2216. https://doi.org/10.1177/0363546517721707
Gutierrez FR, Beall J, Forero DA et al (2015) Spurious alternative impact factors: the scale of the problem from an academic perspective. Bioessays 37:474–476. https://doi.org/10.1002/bies.201500011
Acknowledgements
L.M.G.H. was a research fellow at the directorate of research of HGMEL under the supervision of E.R.V. in 2022.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
González-Hermosillo, LM., Roldan-Valadez, E. Impact factor JUMPS after the 2020 COVID-19 pandemic: a retrospective study in Dermatology journals. Ir J Med Sci 192, 1711–1717 (2023). https://doi.org/10.1007/s11845-022-03179-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11845-022-03179-4