We study the impact of the changes to the British ABS ranking on publication outcomes. The ABS ranking is widely used for assessing the reputation of both individual researchers and their institutions (e.g., Salter et al. 2017). Walker et al. (2019) carried out a large-scale survey of UK business academics and collected responses from 8002 academics from 90 UK business and management schools. The basic descriptive statistics suggest that 67% of researchers always/almost every time use the ABS ranking system when preparing to submit. In addition, about 76% (79%) of academics, at least occasionally, use the ABS list to judge the research outputs of other academics (when assessing a promotion case).
In our analysis, we exploit the plausibly exogenous change in the ABS ranking in 2015 (published in February that year) from the ranking’s previous version in 2010. As we can see in Fig. 1, the ranking experienced a small revolution. In 2015, a new journal grade (4*) was added,Footnote 1 168 journals were upgraded, 42 were downgraded, 590 journals were added and only 579 maintained their grade. Overall, there was a substantial grade inflation, due to which one should perceive a ranking decrease in 2015 as a more significant change than in previous ranking updates.Footnote 2 This makes the 2015 ABS ranking change ideal for our study.
As shown in Fig. 1, the ABS ranking also experienced a change in 2010, however, the change was less pronounced. In total, 49 journals had their ranking revised, 77 journals were added and one journal was removed. Another change came in 2018 and although none of those journals that were already ranked had their ranking revised, 177 journals were added to the list which significantly expanded the authors’ choice.
To study the impact of the ranking change on publication outlet choices, we analyze data on academic papers from the upload of a working paper to IDEAS/RePEc repository through to journal publication.Footnote 3 In our sample, we keep only those papers with at least one UK-based author registered at the repository that were uploaded in IDEAS/RePEc during the 2010–2014 period, i.e. prior to the ABS ranking change in 2015. 11,557 papers authored by 1054 UK-based researchers satisfy all criteria, of which 6,294 papers (54%) were published in an ABS-ranked journal.Footnote 4 This is consistent with Bauman et al. (2020a).Footnote 5 We follow those papers through to 2017, i.e. the last year before the subsequent ABS ranking change. In total, our unbalanced panel has 41,143 observations in annual frequency. We provide summary statistics of our data in the Appendix, Table 5.
We begin the analysis by calculating the share of papers published in a particular journal category (constant/downgraded/upgraded) conditional on paper age. We interpret this as the publication probability. Figure 2 shows that the paper’s age (i.e. the time since the first upload) is a key determinant of the publication probability. The figure highlights the strong impact of journal downgrading. The share of papers published in the journals that were downgraded in 2015 significantly drops after the 2015 ABS ranking was published.Footnote 6
To explore the link between changes in journal ranking and publication outcomes, we employ the following linear probability model:
$${Outcome}_{p,t}=\alpha +{ABS15}_{t}{\beta }_{1}+{Controls}_{p,t}+{\lambda }_{t}+ {\theta }_{p}+{\varepsilon }_{p,t}$$
where Outcome variable is a scaled dummy variable indicating publication in a downgraded/upgraded/unchanged journal in a year t. To facilitate the presentation of the results, we set the variable to take value 100 if a paper is published in that year in the journal category of interest and zero otherwise. This allows us to interpret the regression coefficients as percentage points. ABS15 is a dummy variable (on a zero–one scale) indicating the years after the ranking change, i.e. 2015, 2016 and 2017. Control variables include the number of versions that the paper has had.Footnote 7\({\lambda }_{t}\) and \({\theta }_{p}\) represent paper age and paper fixed effectsFootnote 8 respectively, where age is defined as the number of years since the paper was first posted on IDEAS/RePEc. Our model is estimated using fixed effect estimators which allows us to control for paper and time unobservable characteristics.
Baseline results are reported in Table 1. The key coefficient of interest is ABS15, which tells us how the share of working papers published in a given journal category changed after 2015, controlling for a paper’s age and other covariates. The results suggest that since 2015, UK-based scholars are less likely to publish in the downgraded journals. The share of papers published in that journal category declines by 0.17 percentage points, or around a quarter given the unconditional probability of 0.66%. We observe virtually no change for the two other journal categories. However, once we disaggregate the results, as we show in Table 2, we find that an increased share of papers is published in journals upgraded to ABS4 after the ranking change. The results remain qualitatively unchanged if we replace paper fixed effects with author fixed effects, as we show in the Appendix Table 9.Footnote 9
Table 1 Regression Results: Change in ABS Journal Ranking and Publication Outcomes Table 2 Regression Results: Change in Journal Ranking and Publication Outcome, a detailed view We also notice, perhaps unsurprisingly, that published papers which are subject to more revisions are more likely to be published as indicated by the coefficients on the variables indicating the number of versions. This is the case, particularly for those journals that were not downgraded.
When we review the ABS journal categories in more detail, as we show in Table 2, we find that journals downgraded to grades 3 or 2 suffer the most, while increases to grade 4 are associated with the highest increases in publication probability. This is to be expected, as publications in journals with grades 3 and 4 are typically crucial for research evaluation. We also observe a small, albeit insignificant, decrease in publication probability in journals upgraded to grade 4*. This grade was only created in 2015 and the increase is a likely manifestation of increasing global competition for publication in top journals.