1 Introduction

As Bayesian methods gain popularity in Psychology [10], there has also been a parallel interest in investigating their scientific prominence [3, 7, 13]. The studies consistently showed a growing use of Bayesian methodologies, with some variations across different research domains. However, none of these works tackled the issue of Bayesian vs Frequentist contributions, i.e. how Frequentist-informed and Bayesian-informed papers are relative to each other in terms of prominence. The current research aims to evaluate the popularity of Bayesian versus Frequentist techniques  in the fields of Psychometrics and Statistical Psychology from the 1960s onward.

2 Method

We investigated the online database PsycINFO (see also [4, 9, 11, 12, 14,15,16]). In particular, our focus was directed at three primary sources: the Journal of Mathematical Psychology (JMP), chosen for its exemplification of Statistical and Mathematical Psychology ([6]), all papers categorized under "Statistics and Mathematics" by PsycINFO (PsycINFO code 2240); and all papers related to Psychometrics broadly defined according to PyscINFO codes (see [1]).

Relying on the PsycINFO database has been considered the optimal approach for accessing psychometric literature across various levels of specialization. Access to all psychometric codes enables exploration of a diverse range of literature, including papers from traditional psychometric journals as well as those published in applied fields such as Clinical Psychology or Psychiatry. The "Statistics and Mathematics" code focuses on a more specialized area of pure Psychometrics, and eventually the JMP focus even more specifically in the field of Mathematical Psychology strictly speaking. The choice is also motivated by the ease of result interpretation: new techniques often emerge in specialized literature and, if successful, gradually extend to less specialized areas.

The operationalization of trends, specifically Bayesian versus Frequentist approaches, was implemented through keyword searches. For the Bayesian method, the selected keywords were "bayes*", "prior distribution*", "prior probabil*", "posterior distribution*", and "posterior probabil*".Footnote 1 The keywords were connected by the operator "OR."

On the other hand, the Frequentist approach was operationalized using the keywords "stochastic*", "probabil*", “null hypothes*”, “alternative hypothes*”, “type I error”, connected by the operator "OR." Additionally, to ensure the Frequentist query could identify all stochastic-probabilistic contributions not explicitly labeled as Bayesian, results identified by the Bayesian search were excluded using the (AND) NOTFootnote 2 operator.Footnote 3 Please consult the Online Supplementary Material for the detailed syntaxes: https://osf.io/arh3v/.

Our approach focused on specific field codes in PsycInfo, including Abstract [AB], Keywords/Key Concepts/Identifiers [KW], Tests and Measures [TM], Title [TI], and Subjects/Subject Headings/Index Terms [DE].Footnote 4 This targeted strategy aimed to enhance the precision and consistency of the search process, in contrast to the “unqualified search” which is often used [15].

We examined the period spanning from 1964 to 2023, categorizing papers in five-year intervals (e.g., 1964–1968, 1969–1973, etc.). To create a comparable measure over time, in each time fraction we divided the number of papers identified by the query by the overall number of papers found in the same source.

3 Results and discussion

Detailed data might be consulted in the Online Supplementary Material (https://osf.io/arh3v/), while results are displayed graphically in Fig. 1. The order of the figures is to reflect the trends across different level of specialization. First, the source of all psychometrics-related codes (M =23 237.25, Median = 20 045.5, SD = 14 259.61 for papers investigated for quinquennium) is reported. Then the “Statistics and Mathematics” code is depicted (M =3052.91, Median = 2827.5, SD = 2413.88 for papers investigated for quinquennium) and eventually the JMP (M = 159.83, Median = 134.5, SD = 62.32 for papers investigated for quinquennium).Footnote 5

Fig. 1
figure 1

Bayesian vs Frequentist trends in psychometrics and statistical psychology

Full size image

All the graphs consistently indicate a) the early adoption of the Bayesian approach in the 1980s, b) its further consolidation throughout the 1990s, and c) its definitive establishment in the early 2000s (consistent to [3, 7, 13]).

The Frequentist approach steadily demonstrates its greater prevalence (as compared to the Bayesian approach) and its gradual increase throughout the documented period in the two most comprehensive sources (excluding an anomalous ‘peak’ during 1969–1973). However, in very recent years Frequentism was approached by Bayesian contributions.

Specifically, the Bayesian contribution took over in 2009–2013 in JMP, and after a brief resurgence of the Frequentist contributions in 2014–2018 they became more prevalent again in 2019–2023. In the Statistics and Mathematics code, Bayesian contributions steadily grew from the 80s on, had an exponential upward trend after 2000, and eventually surpassed the Frequentist ones in 2019–2023. In all psychometrics codes, Bayesian contributions have never surpassed the Frequentist contributions, but the trend points toward a close alignment, with prevalence percentages almost similar between 2019 and 2023 (specifically 3.42% versus 3.95%, see https://osf.io/arh3v/).

Despite the overall consistency in trends across the sources, the graphs reveal significant variations in the percentages of the Bayesian contributions versus  the Frequentist ones. Specifically, the JMP shows percentages ranging around 30–40%, the "Statistics and Mathematics" code hovers around 15–20%, and all psychometric codes range from 4 to 8%. This underscores a distinct emphasis of the three sources on technical aspects.

4 Discussion and Limitations

Consistent with prior research [3, 7, 13], the Bayesian approach is gaining prominence, establishing itself as a viable alternative to Frequentist school of thought. While Frequentism likely remains the predominant approach in Psychology at large, our study underscores the growing importance of Bayesian contributions to the extent that it is not unreasonable to envision a distant future where Bayesian techniques might surpass Frequentist ones in popularity.

Furthermore, the trends analyzed show an overtaking of Frequentist contributions first in highly specialized fields (JMP) and then in more general areas (Statistics and Mathematics code). That aligns with the trajectory of an approach that initially gained traction among specialists before gaining ground among applied researchers. Future research will determine whether Bayesian methods will be more popular than Frequentist ones also in the broader psychometric and psychological literature or if this trend will remain confined to a specific segment of literature.

There could be several reasons contributing to this increased interest in the Bayesian approach, including its perceived intuitiveness [5] and alignment with the ideal of cumulative science [8]. What is certain is that Bayesian methods are becoming increasingly prominent, even featuring in introductory textbooks on psychometric statistics [3].

Our exploratory work does not come without limitations. One potential challenge is that our query may have underestimated the prevalence of the Frequentist approach.Footnote 6 Indeed, it is much more likely for Bayesian approaches to be explicitly identified as such (given their contrast to the default Frequentist practice), whereas many studies that routinely employ predominantly Frequentist analyses (e.g., regression, ANOVA, confirmatory factor analysis) may not have been captured by our search query. Moreover, the exclusion of Bayesian keywords in the Frequentist query provides an advantage to the former. All papers that compare Bayesian methods with the traditional Frequentist ones are automatically classified as NOT Frequentist (on the contrary, they are detected in the Bayesian search query).

However, we chose to focus on terms inherently associated with either approach, erring on the side of caution, rather than on the side of inclusiveness. Expanding the search to include a range of models often (but not exclusively) Frequentist was deemed unreasonable and unsound. That is, we aimed to minimize false negatives rather than false positives.

However, the consistency of trends across three different sources and the temporal lag in the rise of the Bayesian approach confirm the overall robustness of our methodology. Additionally, our analysis focused on the specialized domains of Psychometrics, where Bayesian approaches are more likely to be overrepresented compared to broader, applied, or clinical disciplines. Furthermore, the Frequentist peak in 1969–1973 remains unexplained.

Nonetheless, our research indicates the growing popularity and credibility of the Bayesian approach, especially among specialists, as compared to Frequentist works. Such a phenomenon, to our knowledge, had not been objectively quantified previously.