Abstract
The increasing popularity of the Bayesian approach in Psychology has prompted metascientific efforts to quantify its prevalence. However, despite enduring debates between proponents of Frequentist and Bayesian schools of thought, no systematic comparison of their prominence has been conducted in existing literature. This brief report fills this gap, examining Bayesian and Frequentist trends in the period from 1964 to 2023 through a meticulous search in PsycINFO. The findings reveal that the Frequentist approach has consistently been more popular than the Bayesian approach in the realm of Psychometrics and Statistical Psychology. However, Bayesian contributions steadily increased from the 80’s onward and appear to be almost as important or even surpassing the Frequentist counterparts in the latest years investigated (2019–2023). Although this observation applies primarily to specialized literature rather than the entire domain of Psychology, it underscores the growing prevalence of the Bayesian approach, signaling attention among specialists in the field.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
As Bayesian methods gain popularity in Psychology [10], there has also been a parallel interest in investigating their scientific prominence [3, 7, 13]. The studies consistently showed a growing use of Bayesian methodologies, with some variations across different research domains. However, none of these works tackled the issue of Bayesian vs Frequentist contributions, i.e. how Frequentist-informed and Bayesian-informed papers are relative to each other in terms of prominence. The current research aims to evaluate the popularity of Bayesian versus Frequentist techniques in the fields of Psychometrics and Statistical Psychology from the 1960s onward.
2 Method
We investigated the online database PsycINFO (see also [4, 9, 11, 12, 14,15,16]). In particular, our focus was directed at three primary sources: the Journal of Mathematical Psychology (JMP), chosen for its exemplification of Statistical and Mathematical Psychology ([6]), all papers categorized under "Statistics and Mathematics" by PsycINFO (PsycINFO code 2240); and all papers related to Psychometrics broadly defined according to PyscINFO codes (see [1]).
Relying on the PsycINFO database has been considered the optimal approach for accessing psychometric literature across various levels of specialization. Access to all psychometric codes enables exploration of a diverse range of literature, including papers from traditional psychometric journals as well as those published in applied fields such as Clinical Psychology or Psychiatry. The "Statistics and Mathematics" code focuses on a more specialized area of pure Psychometrics, and eventually the JMP focus even more specifically in the field of Mathematical Psychology strictly speaking. The choice is also motivated by the ease of result interpretation: new techniques often emerge in specialized literature and, if successful, gradually extend to less specialized areas.
The operationalization of trends, specifically Bayesian versus Frequentist approaches, was implemented through keyword searches. For the Bayesian method, the selected keywords were "bayes*", "prior distribution*", "prior probabil*", "posterior distribution*", and "posterior probabil*".Footnote 1 The keywords were connected by the operator "OR."
On the other hand, the Frequentist approach was operationalized using the keywords "stochastic*", "probabil*", “null hypothes*”, “alternative hypothes*”, “type I error”, connected by the operator "OR." Additionally, to ensure the Frequentist query could identify all stochastic-probabilistic contributions not explicitly labeled as Bayesian, results identified by the Bayesian search were excluded using the (AND) NOTFootnote 2 operator.Footnote 3 Please consult the Online Supplementary Material for the detailed syntaxes: https://osf.io/arh3v/.
Our approach focused on specific field codes in PsycInfo, including Abstract [AB], Keywords/Key Concepts/Identifiers [KW], Tests and Measures [TM], Title [TI], and Subjects/Subject Headings/Index Terms [DE].Footnote 4 This targeted strategy aimed to enhance the precision and consistency of the search process, in contrast to the “unqualified search” which is often used [15].
We examined the period spanning from 1964 to 2023, categorizing papers in five-year intervals (e.g., 1964–1968, 1969–1973, etc.). To create a comparable measure over time, in each time fraction we divided the number of papers identified by the query by the overall number of papers found in the same source.
3 Results and discussion
Detailed data might be consulted in the Online Supplementary Material (https://osf.io/arh3v/), while results are displayed graphically in Fig. 1. The order of the figures is to reflect the trends across different level of specialization. First, the source of all psychometrics-related codes (M =23 237.25, Median = 20 045.5, SD = 14 259.61 for papers investigated for quinquennium) is reported. Then the “Statistics and Mathematics” code is depicted (M =3052.91, Median = 2827.5, SD = 2413.88 for papers investigated for quinquennium) and eventually the JMP (M = 159.83, Median = 134.5, SD = 62.32 for papers investigated for quinquennium).Footnote 5
All the graphs consistently indicate a) the early adoption of the Bayesian approach in the 1980s, b) its further consolidation throughout the 1990s, and c) its definitive establishment in the early 2000s (consistent to [3, 7, 13]).
The Frequentist approach steadily demonstrates its greater prevalence (as compared to the Bayesian approach) and its gradual increase throughout the documented period in the two most comprehensive sources (excluding an anomalous ‘peak’ during 1969–1973). However, in very recent years Frequentism was approached by Bayesian contributions.
Specifically, the Bayesian contribution took over in 2009–2013 in JMP, and after a brief resurgence of the Frequentist contributions in 2014–2018 they became more prevalent again in 2019–2023. In the Statistics and Mathematics code, Bayesian contributions steadily grew from the 80s on, had an exponential upward trend after 2000, and eventually surpassed the Frequentist ones in 2019–2023. In all psychometrics codes, Bayesian contributions have never surpassed the Frequentist contributions, but the trend points toward a close alignment, with prevalence percentages almost similar between 2019 and 2023 (specifically 3.42% versus 3.95%, see https://osf.io/arh3v/).
Despite the overall consistency in trends across the sources, the graphs reveal significant variations in the percentages of the Bayesian contributions versus the Frequentist ones. Specifically, the JMP shows percentages ranging around 30–40%, the "Statistics and Mathematics" code hovers around 15–20%, and all psychometric codes range from 4 to 8%. This underscores a distinct emphasis of the three sources on technical aspects.
4 Discussion and Limitations
Consistent with prior research [3, 7, 13], the Bayesian approach is gaining prominence, establishing itself as a viable alternative to Frequentist school of thought. While Frequentism likely remains the predominant approach in Psychology at large, our study underscores the growing importance of Bayesian contributions to the extent that it is not unreasonable to envision a distant future where Bayesian techniques might surpass Frequentist ones in popularity.
Furthermore, the trends analyzed show an overtaking of Frequentist contributions first in highly specialized fields (JMP) and then in more general areas (Statistics and Mathematics code). That aligns with the trajectory of an approach that initially gained traction among specialists before gaining ground among applied researchers. Future research will determine whether Bayesian methods will be more popular than Frequentist ones also in the broader psychometric and psychological literature or if this trend will remain confined to a specific segment of literature.
There could be several reasons contributing to this increased interest in the Bayesian approach, including its perceived intuitiveness [5] and alignment with the ideal of cumulative science [8]. What is certain is that Bayesian methods are becoming increasingly prominent, even featuring in introductory textbooks on psychometric statistics [3].
Our exploratory work does not come without limitations. One potential challenge is that our query may have underestimated the prevalence of the Frequentist approach.Footnote 6 Indeed, it is much more likely for Bayesian approaches to be explicitly identified as such (given their contrast to the default Frequentist practice), whereas many studies that routinely employ predominantly Frequentist analyses (e.g., regression, ANOVA, confirmatory factor analysis) may not have been captured by our search query. Moreover, the exclusion of Bayesian keywords in the Frequentist query provides an advantage to the former. All papers that compare Bayesian methods with the traditional Frequentist ones are automatically classified as NOT Frequentist (on the contrary, they are detected in the Bayesian search query).
However, we chose to focus on terms inherently associated with either approach, erring on the side of caution, rather than on the side of inclusiveness. Expanding the search to include a range of models often (but not exclusively) Frequentist was deemed unreasonable and unsound. That is, we aimed to minimize false negatives rather than false positives.
However, the consistency of trends across three different sources and the temporal lag in the rise of the Bayesian approach confirm the overall robustness of our methodology. Additionally, our analysis focused on the specialized domains of Psychometrics, where Bayesian approaches are more likely to be overrepresented compared to broader, applied, or clinical disciplines. Furthermore, the Frequentist peak in 1969–1973 remains unexplained.
Nonetheless, our research indicates the growing popularity and credibility of the Bayesian approach, especially among specialists, as compared to Frequentist works. Such a phenomenon, to our knowledge, had not been objectively quantified previously.
Data availability
The data and the syntaxes can be publicly accessed on the Open Science Framework (https://osf.io/arh3v/).
Notes
PsycINFO is not case-sensitive.
PyscINFO routinely considers the boolean operator NOT as equivalent to AND NOT.
An attempt was made to include the keyword “p value” for the Frequentist approach. However, PsycINFO could not read the query, likely because “p value” is a bigram.
Note that the acronym for the field code might slightly change from platform to platform. However, the field codes should be consistent across different platforms [2]. Note also that the search on PsycINFO through EBSCO presents a minor issue with the DE code. Theproblem leads to searches within the DE field code inadvertently encompassing Medical Subject Heading [MESH] terms (MA field code) as well [1, 15]. MESH terms are the offical lexicon adopted by PubMED. Since MESH terms are also controlled for lexicon, this should have no substantial consequences on the search.
In the Online Supplementary Material, the reader may find also a sample of 50 papers identified as Frequentist and 50 papers identified as Bayesian in the 2014–2018 quinquennium in all psychometrics-related codes, to check for their theoretical content
There is also the more unlikely possibility that some work identified by our query as Frequentist may not have adopted that approach.
References
American Psychological Association. APA PsycInfo Classification Categories and Codes. https://www.apa.org/pubs/databases/training/class-codes?_gl=1*ev1jwt*_ga*MjQwMjQzMDE0LjE2OTk4Mjk0NTI.*_ga_SZXLGDJGNB*MTY5OTgyOTQ1Mi4xLjAuMTY5OTgyOTQ1NC4wLjAuMA. Accessed on 12 Nov 2023.
American Psychological Association. Guide to the fields in APA Database records. 2020. https://www.apa.org/pubs/databases/training/field-guide. Accessed 03 Jan 2024.
Andrews M, Baguley T. Prior approval: the growth of Bayesian methods in psychology. Br J Math Stat Psychol. 2013;66(1):1–7. https://doi.org/10.1111/bmsp.12004.
Friman PC, Allen KD, Kerwin ML, Larzelere R. Changes in modern psychology: a citation analysis of the Kuhnian displacement thesis. Am Psychol. 1993;48(6):658. https://doi.org/10.1037/0003-066X.48.6.658.
Gurrin LC, Kurinczuk JJ, Burton PR. Bayesian statistics in medical research: an intuitive alternative to conventional data analysis. J Eval Clin Pract. 2000;6(2):193–204. https://doi.org/10.1046/j.1365-2753.2000.00216.x.
Falmagne JC. Mathematical psychology—a perspective. J Math Psychol. 2005;49(6):436–9. https://doi.org/10.1016/j.jmp.2005.06.007.
Jevremov T, Pajić D. Bayesian method in psychology: a bibliometric analysis. Curr Psychol. 2023. https://doi.org/10.1007/s12144-023-05003-3v.
König C, van de Schoot R. Bayesian statistics in educational research: a look at the current state of affairs. Educ Rev. 2018;70(4):486–509. https://doi.org/10.1080/00131911.2017.1350636.
Robins RW, Gosling SD, Craik KH. An empirical analysis of trends in psychology. Am Psychol. 1999;54(2):117–28. https://doi.org/10.1037/0003-066X.54.2.117.
Schmalz X, Biurrun Manresa J, Zhang L. What is a Bayes factor? Psychol Methods. 2021;28(3):705. https://doi.org/10.1037/met0000421.
Spear JH. Prominent schools or other active specialties? A fresh look at some trends in psychology. Rev Gen Psychol. 2007;11(4):363–80. https://doi.org/10.1037/1089-2680.11.4.363.
Tracy JL, Robins RW, Gosling SD. Tracking trends in psychological science. In: Dalton TC, Evans RB, editors. The life cycle of psychological ideas. Boston: Springer; 2004. p. 105–30.
Van De Schoot R, Winter SD, Ryan O, Zondervan-Zwijnenburg M, Depaoli S. A systematic review of Bayesian articles in psychology: the last 25 years. Psychol Methods. 2017;22(2):217. https://doi.org/10.1037/met0000100.
Webster GD. Evolutionary theory in cognitive neuroscience: a 20-year quantitative review of publication trends. Evol Psychol. 2007;5(3):147470490700500300. https://doi.org/10.1177/14747049070050030.
Zagaria A, Lombardi L. A new perspective at trends in psychology. New Ideas Psychol. 2024;74: 101078. https://doi.org/10.1016/j.newideapsych.2024.101078.
Zagaria A. Is evolutionary psychology a scientific revolution? A bibliometric analysis. Adapt Hum Behav Physiol. 2024. https://doi.org/10.1007/s40750-024-00234-5.
Acknowledgements
The authors express their gratitude to the European Mathematical Psychology Group (EMPG) for accepting an initial draft of this work as a poster at the EMPG 2022 conference held in September 2022 in Rovereto, Italy.
Funding
No financial support has been received from any agency.
Author information
Authors and Affiliations
Contributions
Conceptualization, data curation, formal analysis, investigation, methodology, software: Andrea Zagaria ideated the study, crafted the syntaxis used in PsycInfo, collected and analyzed the data, and produced the graphs. All these activities were closely supervised and prompted by Luigi Lombardi, especially regarding the multiphasic selection of descriptors. Project administration, resources, validation: not applicable. Writing, original draft: the original draft was written by Andrea Zagaria. Writing, review and editing: Andrea Zagaria produced the final draft.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
This article does not contain any studies with human participants or animals performed by any of the authors.
Consent for publication
All the authors mentioned in the manuscript have agreed for authorship, read and approved the manuscript, and given consent for submission and subsequent publication of the manuscript.
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zagaria, A., Lombardi, L. Bayesian versus Frequentist approaches in Psychometrics: a bibliometric analysis. Discov Psychol 4, 61 (2024). https://doi.org/10.1007/s44202-024-00164-z
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s44202-024-00164-z