Introduction

Many scientific journals have made the transition from print and online to online-only publication. The printed version offers something tangible that carries the distinct brand of the journal and/or the society it represents (Wymbs, 2020). On the other hand, online journals are readily disseminated, offer a reduced environmental impact, faster peer-review, innovative multimedia opportunities, and the possibility of lower production costs (Clarke, 2011; Grech, 2001; Houghton & Oppenheim, 2010; Houghton et al., 2009; Johnson & Luther, 2007; King & Tenopir, 2011; Wymbs, 2020). Online publication allows early access and eliminates one financial limitation on the number of articles per journal issue. However, adopting online publishing is a complicated and arduous task that involves sophisticated technology and should be considered carefully (Adler et al., 2019; Johnson & Luther, 2007).

Print publication incurs significant printing and distribution costs, which most publishers would agree are increasing (Johnson & Luther, 2007). The increased costs are offloaded to the authors as well as readers in the form of increased publication and subscription fees, respectively (Clarke, 2011). The requirements of global funding organizations such as National Institute of Health (NIH) Public Access Policy and European Union’s scientific agencies (EU) for public access to funded research implies a loss of revenue to the publishers from subscriptions (Adler et al., 2019; Enserink, 2018). The cost of online publication is much less than print publication but maintaining a dual publication model in both print and online formats offsets any cost reduction achieved with online publishing (Clarke, 2011; Houghton & Oppenheim, 2010; Houghton et al., 2009; Johnson & Luther, 2007; King & Tenopir, 2011). All these factors combined with convenience and accessibility make online platforms more attractive than print for publishers as well as readers.

The aim of this study is to examine the extent to which transitioning from print and online to online-only publication affects U.S.-based medical journals’ scholarly impact. We hypothesized that moving to online-only publication would have a positive impact on journals’ publication metrics.

Methods

This study was exempted from IRB review as no human subjects were involved. Ulrichsweb Global Serials Directory (Ulrichsweb.com (TM), 2023) was searched for journals that met the following criteria: U.S.-based peer-reviewed journals, in the Medicine and Health Subject Area, that ceased print publication between 2002 and 2020 while continuing to publish in electronic form, and were noted as included in Journal Citation Report (JCR—Clarivate) (Journal Citation Reports, 2023). We chose 2002 as the starting year as the vast majority of U.S. science, technology and medicine journals were publishing online by 2002–2003. (Ware & Mabe, 2015) At the time of writing, the most recent Journal Citation Report (JCR) data was from 2021; hence, 2020 was chosen as the last year in order to see the effects of transition.

Ulrich’s Subject Headings are created by the company to classify the journals in the Ulrich’s database (Frequently Asked Questions (FAQs), n.d.). JCR was chosen a source of publisher-neutral citation data from journals deemed most impactful in their fields via 24 quality criteria for editorial rigor and 4 impact criteria using citation activity. The resulting 94 titles were cross-referenced in JCR’s Science Citation Index-Expanded (SCIE) 2021 edition for data on the total number of citations, citable items, Immediacy Index, and Journal Impact Factor (JIF), and JIF without self cites (JCR, 2023)..

JCR and the Directory of Open Access Journals were searched by title and ISSN number to determine whether any of the journals adopted a fully open access publication model during the study period and if so, what year that took place. We define a fully open access journal as a journal in which every article is open access. There are other ways in which articles from subscription journals become freely available, such as through a hybrid open access model, where authors can pay to make individual articles open access, or green open access, where authors self-archive manuscripts in repositories, often at the requirement of funders such as the National Institutes of Health (NIH). However, there is no readily available data on if and when journals adopted hybrid publication models or how many of their articles were freely available in public repositories, so this study has focused on fully open access journals only.

Outcome measures

Total number of citations

Total number of citations consists of any citation from the journal title in one year, regardless of the publication date of each article, i.e., all citations to a given journal regardless of cited document type (2022 Reference Guide, n.d.).

Citable items

Citable items are all items indexed in the Web of Science™ as articles and reviews in one year. Items with any other document type, including editorial materials, letters, and meeting abstracts, are not included in citable items (2022 Reference Guide, n.d.; Hubbard & McVeigh, 2011).

Journal impact factor (JIF)

JIF is calculated by dividing the total number of citations received by a particular journal title by the total number of citable items in that journal over a specific time period, e.g. 2 years (Immediacy Index, InCites help, Journal Citation Reports JCR, n.d.; Ranjan, 2017)

JIF without self cites

JIF without self cites refers to Impact factor without journal self-citations. Self cites are defined as “citations from the journal to itself, i.e., instances in which an article published in a journal has cited a previously published article in that same journal” (Journal Impact Factor (JIF) Without Self Cites, InCites help, Journal Citation Reports JCR, n.d.). Excessive self-citation by journals is a form of citation manipulation intended solely to increase citations and thereby improve a journal’s reputation, including inflating its Journal Impact Factor (COPE Council., 2019). Comparing the JIF to the JIF without self cites can identify patterns of excessive self-citation.

Immediacy Index

The Immediacy Index is calculated by dividing the number of citations to articles published in a given year by the number of articles published in that year (Immediacy Index, InCites help, Journal Citation Reports JCR, n.d.). It indicates how quickly articles in a journal are cited (Immediacy Index, InCites help, Journal Citation Reports JCR, n.d.)

Statistical analysis

Mixed effects segmented linear regression models were used to examine the average changes in the journal performance prior to and after journals transitioned from print and online to online-only publication. The basic segmented linear regression model takes the form of

$${Y}_{t}={b}_{0}+{b}_{1}{Time}_{t}+{b}_{2}{Transition}_{t}+{b}_{3}{Time}_{t}\times {Transition}_{t}+{\varepsilon }_{t}$$

where Yt represents the average of the outcome (e.g., total number of citations) at year t. Time refers to value of time in years (from 1 to k) from the first year of print and online publication to the last year of online-only publication (within the years of data available, as described above), and equals t. Transition is a dummy coded variable indicating print and online/online-only publication at time t (0 = print, 1 = online). Time x Transition refers to the interaction between Time and Transition; Time x Transition = 0 during print and online publication, and otherwise equals the number of years from the beginning of online-only publication to the last year. b0 is the baseline level of the outcome at time 0, b1 is the slope during the years when journals published in print and online, b2 is the average initial change of the outcome just after journals transitioned to publishing online-only, and b3 is the difference in slopes between online-only and print and online publication.

We expanded the above model to take into account within-journal differences at baseline (time 0) and within-journal correlations over time by including a random intercept and a random slope for journals. As journals differ in the years in which they transitioned from print and online to online-only publication, we centered the data at each journal’s last year of print and online publication (year = 0), such that negative year numbers reflect the amount of time journals published in print and online, and positive year numbers reflect the amount of time journals published online-only. In order to explore the extent to which the results may be influenced by the open access status of some journals, we performed sensitivity analyses by re-running the models once including only journals that never transitioned to be fully open acess. Considering the small number of journals that transitioned to fully open access during the years of data available (as described above), with the majority transitioning to fully open access before transitioning to online-only publication, we did not perform additional analyses to evaluate whether the transition to fully open access had an additional effect on the outcome measures of interest.

Statistical significance level was set at α = 0.05. Mixed effects segmented regression analyses were performed using the “lme4” package (Bates et al., 2015), and all statistical analysis were performed in R 4.1.2 (R Core Team, 2021).

Results

Descriptive statistics

Publication data was extracted from 94 journals. However, no print cessation data was available for 10 journals, thus 84 journals were included in the subsequent analyses. On average, 18.7 years (SD = 6.4, median = 22, IQR = 12) of publication data was available. Of the 84 journals included in this study, most journals (61%) transitioned to online-only publication between 2010 and 2014. Journal of Gambling Studies was the first to transition to online-only publication in 2004, whereas Andrology (Hoboken) was the last one to transition in 2020. (eTable 1) Among the 84 journals included in the analyses, 13 (15.5%) adopted a fully open access model at some point during the years of data available, among which 3 transitioned to be fully open access access after they transitioned to online-only publication.

Total citations

Results in Table 1 showed that there was a statistically significant increase in the total number of citations over time when journals published in print and online (b1 = 547.79, 95% CI = 406.77–688.81). On average, each year is associated with almost 550 additional citations when journals published in print and online. There was no substantial difference in the average number of total citations when journals first transitioned to online-only publication (b2 =  − 328.61, 95% CI =  − 697.72–40.50). Compared to when journals published in print and online, the rate of increase in the total number of citations was slower after journals published online-only (b3 =  − 111.12, 95% CI =  − 181.94 to − 40.30). After journals transitioned to publishing online-only, there is an average of 436.67 (b1 + b3 = 547.79–111.12) additional citations per year. (Fig. 1). Results were consistent after excluding fully open access journals (eTable 2).

Table 1 Mixed effects segmented linear regression models examining the changes in the number of total citations and Immediacy Index as journals transition from publishing in print and online to online-only
Full size table
Fig. 1
figure 1

Estimated and projected change in the total number of citations over time as journals transition from print and online to online-only publication

Full size image

Number of citable items

Inspection of the descriptive statistics of the number of citable items showed that most journals (n = 72; 86%) had ≤ 500 citable items over time, whereas a smaller number of journals (n = 12; 14%) had > 500 citable items in some years. As such, we included four additional fixed effects (b4: main effect of Group: journals with ≤ 500 vs. > 500 citable items, b5: interaction effect of Group x Time, b6: interaction effect of Group x Transition, and b7: interaction effect of Group x Time x Transition) in the mixed effects segmented regression model to examine the changes in the number of citable items prior and after the transition from print and online to online-only publication.

Results are presented in eTable 3. For journals with ≤ 500 citable items, there was an increase in the average number of citable items across the years when journals published in print and online (b1 = 2.50, 95% CI = 1.21–3.80). On average, the number of citable items increased by approximately 2.50 items every additional year. The statistically significant interaction effect (b5 =  − 13.75, 95% CI =  − 16.48 to − 11.03) suggested that, during the same period of time, journals with > 500 citable items showed a decrease (~ 11.2 fewer citable items per year) in the average number of citable items. When journals first transitioned to online-only publication, there was no substantial change in the average citable items for journals with ≤ 500 citable items (b2 =  − 3.60, 95% CI =  − 20.67 to 13.48). The change in the number of citable items when journals published online-only was significantly slower (b3 =  − 5.15, 95% CI =  − 7.72 to − 2.57) compared to the increase over time during the print and online publication. In fact, the average number of citable items decreased (~ 2.64 fewer citable items per year) after journals transitioned to online-only publication. For journals with > 500 citable items, there was a significant decrease in the number of citable items when they first transitioned from print and online to online-only publication (b6 =  − 102.72, 95% CI =  − 147.17 to − 58.28). However, there was no substantial difference in the rate of decrease in citable items over time between when they published in print and online to after the transition to online-only publication (b7 = 1.63, 95% CI =  − 5.51–8.76; ~ 14.8 fewer citable items per year). Results are consistent when excluding fully open access journals, with the exception that journals with > 500 citable items showed a faster increase in the number of citable items over time after they transitioned to online-only publication (eTable 3).

Journal impact factor

Three journals (Journal of Pineal Research, Microbiology and Molecular Biology Reviews, and Pharmacological Reviews) had a JIF ≥ 12 (M = 17, SD = 3.8, range = 12–27), whereas the average JIF of the remaining 81 journals was 3 (SD = 1.8, range = 0.1–12). Therefore, mixed effects segmented linear regression models were performed separately for journals with JIF < 12 (n = 81) and those ≥ 12 (n = 3). For journals with JIF < 12, impact factor increased during print and online publication (b1 = 0.06, 95% CI = 0.03–0.09). Initially, after transition to online-only publication, there was a slight decrease in JIF (b2 =  − 0.35, 95% CI =  − 0.46 to − 0.25). The trajectory of increase was faster after journals transitioned to online-only publication (b3 = 0.05, 95% CI = 0.03 to 0.07). In contrast, journals with JIF ≥ 12 showed relatively stable JIF over time during print and online publication (b1 =  − 0.04, 95% CI =  − 0.41 to 0.33), with no substantial change when initially transitioned to online-only publication (b2 =  − 0.68, 95% CI =  − 3.04–1.69). Once these journals transitioned to publishing online-only, the rate of increase in JIF increased substantially (b3 = 0.50, 95% CI = 0.12–0.87). Compared to the relatively stable trajectory of JIF during print and online publication, there was ~ 0.46 unit increase in JIF per year for journals with JIF ≥ 12 once they transitioned to online-only publication. These results were consistent for JIF without self cites, and after excluding fully open access journals (Table 2, eTable 4 and Fig. 2).

Table 2 Mixed effects segmented linear regression models examining the changes in the journal impact factor as journals transition from publishing in print and online to online-only
Full size table
Fig. 2
figure 2

Estimated and projected change in journal impact factor over time as journals transition from print and online to online-only publication

Full size image

Immediacy index

As shown in Table 1, there was a slow increase in Immediacy Index over time when journals published in print and online (b1 = 0.02, 95% CI = 0.01–0.03). On average, each additional year is associated with a 0.02 unit increase in Immediacy Index when journals published in print and online. Although there was an initial decrease in Immediacy Index when journals transitioned to publishing online-only (b2 =  − 0.07, 95% CI =  − 0.12 to − 0.01), there was a higher rate of increase in Immediacy Index when journals published online-only as compared to when they published in print and online (b3 = 0.04, 95% CI = 0.03 to 0.05). This effect is clearly illustrated in Fig. 3, where the slope of the Immediacy Index showed a sharper increase over time after journals transitioned to publishing online-only (grey line: ~ 0.02 unit increase per year prior to transition; tan line: ~ 0.06 unit increase per year after transition). After excluding fully open access journals, results were mostly consistent, with the exception that there was no immediate effect of transitioning to online-only publication on Immediacy Index (eTable 2).

Fig. 3
figure 3

Estimated and projected change in the Immediacy Index over time as journals transition from print and online to online-only publication

Full size image

Discussion

Our results showed there was a slower rate of increase in the total number of citations after journals transitioned from print and online (~ 550 additional citations per year) to online-only (~ 440 additional citations per year) publications, and a decrease in the number of citable items. The citation of article types not included in Citable items such as editorials and letters (Ranjan, 2017) could be one of the plausible reason for increase in the number of citations even with a decrase in number of Citable items. Some studies have shown that online journals receive more citations than print journals (Groote & Barrett, 2010; Groote et al., 2005; Tahamtan et al., 2016). Ease of access and portability of online publication could be another explanation for increase in number of citations even with a decrease in the number of citable items. Choosing to read a journal in print versus online is a personal preference. However, multiple authors have noted preferential use of online version even when a particular journal was available both in print and online format. (Brennan et al., 2002; Curtis et al., 1997; Groote & Dorsch, 2003; Groote et al., 2005)

JIF is the single most popular criterion for evaluating a journal’s performance (Ranjan, 2017). Our results showed a positive impact on JIF when journals transitioned to online-only format. Most journals, regardless of their initial JIF, showed a faster increase in JIF after transitioning to online-only publication. Curtis et al., reported similar results in their study on biomedical scientific journals (Curti et al., 2001).

There has been a debate on the inflated importance of JIF (Lariviere & Sugimoto, 2019; Wouters et al., 2019). Various metrics have been suggested to gauge the scholarly impact of journals (Leydesdorff, 2012; Roldan-Valadez et al., 2019). Immediacy Index, calculated by dividing the number of total citations of articles published in a given year divided by the number of total articles published in that year, is a measure of the speed of citation of articles published in a journal (2022 Reference Guide n.d.). Consistent with our findings with JIF, there was a positive impact on the rate of increase of Immediacy Index when journals published online-only. One of the critiques of JIF is that it does not exclude self cites and is thus subject to manipulation by unscrupulous journals (Ranjan, 2017). However, we found a consistent positive impact of online-only publication for JIF even without counting journal self cites.

The current study has several limitations. various circumstances such as the rise of academic search engines and bibliographic databases, and changing funder public access policies and article access models that took place during the study time period and may affect all the performance parameters discussed. Additionally, the focus of the study was only journals published in the U.S. in the health and medicine sector and so the outcomes may not apply to journals from other fields and countries. Thirdly, our measure of total citations does not account for typical citation patterns over the life of an article. Finally, this study did not account for ways that citation patterns may be affected by hybrid publication models, in which journals provide some articles through subscription and others through open access.

Conclusion

Transition to online-only publication has the advantage of ease of accessibility and immediate dissemination of research with a potential to reach a wider audience. Positive impact on citation patterns of online-only publication, such as faster increase in JIF and Immediacy Index, further supports transition of medical journals from print and online to online-only publication.