Categorical study outcomes are omnipresent across the prevention sciences, and data analysts in prevention are routinely tasked with modeling binary (e.g., identifying factors that promote medication adherence; Phillips et al., 2019), polytomous (e.g., evaluating risk factors of polysubstance use; Jongenelis et al., 2019), and ordinal outcome data (e.g., identifying risk factors for dating violence victimization; Rostad et al., 2020). Similarly, preventive interventions are usually delivered based on categorical decisions such as who should receive selective or indicated intervention, or who is most likely to benefit from some specific intervention (see Supplee et al., 2013). Categorical data analysis constitutes a broad dynamic research area, and advances in categorical data analytic methods hold promise for improving prevention science outcome analyses and decision-making. In addition, many standard techniques to evaluate categorical data have well-documented limitations in terms of performance (e.g., in case of small samples or violated statistical assumptions) and interpretability of results.

A goal of this Special Issue, Advanced Categorical Data Analysis in Prevention Science, was to bring together leading scholars in the fields of measurement and categorical methods research to present and discuss recent methodological developments that can help overcome many of the shortcomings of existing statistical tools. Before giving an overview of the contributions of the Special Issue, we provide a bit of background on the status quo of measurement and categorical data analysis practices in the prevention sciences. Given the various excellent sources on categorical data analysis, such as introductory textbooks by Agresti (2013), Hosmer et al. (2013), Fullerton and Xu (2016), to name a few, we believe it is helpful to provide additional context and rationale for the current special issue.

Background and Rationale for the Special Issue

To get a sense of the status quo of categorical data modeling in the area of prevention science, we performed a literature review of all articles (n = 233; 184 empirical studies, 22 reviews, 13 commentaries, 12 editorials, and 2 case studies; note that we have excluded one addendum and two corrections) published in Prevention Science in 2019 and 2020. Overall, 402 study outcomes (the number of outcomes per study ranged from 1 to 16; M = 2.19, SD = 1.91) were reviewed with respect to scale level, measurement approach, and type of outcome analysis. The vast majority of study outcomes were analyzed in the context of quantitative (93.0%; 4.2% mixed methods and 2.7% qualitative) observational studies (57.7%; 46.3% randomized designs). About 62.9% of the study outcomes were evaluated longitudinally (37.1% cross-sectional designs) with measurement occasions ranging from 2 to 30 time points (Md = 3.0, Q1 = 2.0, Q3 = 4.0). Level 1 sample sizes ranged from n = 7 to 470,795 with a median sample size of 774 (Q1 = 245.0; Q3 = 2630.0), level 2 sample sizes ranged from 4 to 706 with a median number of clusters of 42 (Q1 = 20.0, Q3 = 68.0), and level 3 sample sizes varied from 8 to 96 with a median sample size of 36 (Q1 = 15.0, Q3 = 96.0).

The majority of study outcomes were measured as either interval- or ratio-scaled variables (49.3%), about 30.4% of the outcomes were binary in nature, 9.6% were assessed in the form of counts, 8.1% were ordinal, and 2.6% were polytomous. As a whole, categorical outcomes comprised approximately half of the outcomes investigated. Binary outcomes were most often analyzed using standard logistic regression models (76.9%), questions concerning polytomous outcomes were most often addressed using multinomial logistic regression (80.0%), and count outcomes were often evaluated using standard Poisson regression (32.4%; 24.3% used negative binomial regressions and another 24.3% treated counts as interval-scaled and applied linear regression models). Ordered categorical outcomes were most often treated as continuous, resulting in the use of a standard linear regression model or structural equation models (36.6%); about 33.3% of these outcomes were analyzed using an ordered regression model (such as ordered logistic or ordered probit models), and 16.7% applied multinomial logistic regression.

Further, issues of measurement were rarely addressed thoroughly. Reliability information was often provided for study outcomes (59.7% report reliability properties of outcomes based on primary or secondary data), and, as expected, Cronbach’s alpha (93.1%) was the most widely used reliability index, despite its limitations (Sijtsma, 2009). However, many studies suffered from some lack of comprehensive presentation of measurement properties of instruments. For example, confidence intervals for reliability estimates were reported for only 3.2% of the studied outcomes and few studies used sophisticated measurement modeling strategies (only 5.9% of the studied outcomes were analyzed using advanced measurement modeling techniques such as item response theory or factor analysis).

Although the present literature review is neither intended to be exhaustive with respect to the number of considered outlets of prevention science research (we only focused on articles published in Prevention Science) nor intended to thoroughly describe potential time trends (our review focused on the two recent volumes at the time of conceptualizing the content of the Special Issue), several interesting observations can be drawn from the presented results. First, decisions concerning statistical tools to analyze categorical data were overall in good accordance with standard textbook recommendations. For example, binary study outcomes were mostly analyzed using some variant of logistic regression, polytomous outcomes were evaluated using multinomial (logistic) models, and counts tended to be analyzed using either Poisson or negative binomial regression. Second, despite the adherence to standard statistical practice, researchers in prevention science may not have fully considered ongoing refinements of existing categorical data modeling techniques and the development of new approaches to evaluate categorical data. For example, alternatives to logistic regression were rarely considered and, in line with previous method surveys in the social sciences (cf. Liddell & Kruschke, 2018), ordinal outcomes were often treated as being sufficiently interval-scaled. Third, our results suggest that there might be room for improvement in thoroughly analyzing and reporting psychometric properties of measurement scales. Taken together, these findings highlight several gaps in the coverage of alternative and advanced methods for analyzing categorical data in the prevention sciences. The contributions to this Special Issue, as detailed below, cover a range of methodologies and statistical techniques that we hope will address these gaps in the literature.

Overview of the Special Issue

The article by Huang (2023) focuses on dichotomous study outcomes. Although standard logistic regression models are the default choice in prevention science (see the results of the literature review reported above) when outcomes come in the form of two states (e.g., a disease is present or absent), results from logistic regressions are often hard to interpret. Several alternative modeling approaches, such as linear probability and modified Poisson regression models, are available which are equally valid (in the context of an experimental study) and easier to interpret (see Huang, 2022). This paper introduces model alternatives for binary outcomes with a focus on cluster randomized trials. Specifically linear probability and modified Poisson models with standard error adjustment are introduced. Simulation results suggest that these alternatives give unbiased point estimates as well as adequate Type I error and power rates even when the number of level 2 clusters is low.

The article by Rijnhart et al. (2023) discusses approaches to test mediation mechanisms when mediators and outcomes are measured in binary form. Here, emphasis is put on similarities and differences of traditional mediation analysis (based on the standard approach of decomposing the total effect an intervention exposure has on the outcome into direct and indirect effect components) and recently developed causal mediation analysis rooted in the potential-outcome framework of causation (Holland, 1986; Pearl, 2001). Data from a randomized-controlled trial on primary prevention of alcohol, cigarette, and marijuana use among adolescents is used to illustrate similarities and differences of traditional and causal mediation analysis for binary mediators and binary outcomes.

In the article by von Eye et al. (2023), a person-oriented perspective to the evaluation of intervention effectiveness for categorical data is taken. Configural frequency analysis (CFA; von Eye & Gutiérrez-Pena, 2004; von Eye & Wiedermann, 2021) is a cross-classification approach to detect patterns (i.e., types and antitypes) in multiple (binary) variables which contradict a particular chance model. Although CFA is one of the prime methods in person-oriented research (Wiedermann et al., 2016), it is still under-utilized in prevention science. The authors, therefore, introduce CFA in the context of non-randomized as well as randomized intervention contexts and demonstrate that an integration of person- and (standard) variable-oriented perspectives (e.g., CFA and subsequent logistic regression) allows one to characterize the effectiveness of interventions in a more rigorous manner.

The article by Fullerton and Anderson (2023) provides an example where outcomes are measured on an ordinal scale. Specifically, an ordered regression modeling framework is introduced that provides a flexible toolkit to approach the analysis of ordinal outcome variables in a systematic and rigorous way. The authors introduce variations of cumulative, adjacent, and stage models and highlight their underlying assumptions. Furthermore, the authors address the important topic of properly modeling ordinal predictor variables in the analysis of ordinal outcomes. An empirical example using data from the 2018 General Social Survey illustrates potential differences of the various ordered regression models when analyzing an ordinally scaled (self-rated health) outcome and discusses consequences of coding decisions made for ordinal predictors.

The next article by Wiedermann, Frick, and Merkle (2023) is devoted to comparative measures such as paired comparisons and rankings, which are often used to quantify health states and quality of life. The authors introduce the log-linear Bradley–Terry (LLBT; Dittrich et al., 2002; Sinclair, 1982) model and propose a combination of the LLBT model and a data-driven moderation analysis tool known as model-based recursive partitioning (MOB; Zeileis et al., 2008). The proposed approach enables researchers to scale items on a latent continuum and to detect differential treatment effects on choice behavior (for a recent empirical application, see Tallon et al., 2022). To illustrate the MOB-LLBT approach in the context of prevention science, the authors use data from a split-ballot experiment to assess perceived drug harm among music festival visitors (Wiedermann et al., 2014). While standard LLBT suggests a trivialization effect of drug harm, a re-analysis using MOB LLBT indicates that this trivialization effect is highly context-dependent.

The next two articles in this Special Issue are concerned with the evaluation of statistical models for categorical data, a task which is often constrained by sparseness in the contingency table. The paper by Cai et al. (2023) confronts this issue by developing the TLIRT: an adaptation of the Tucker–Lewis incremental fit index (a statistic that is widely used in structural equation modeling to assess the degree to which the model fits the data) for use in item response theory (IRT) modeling. Through a simulation study and an empirical analysis of smoking cessation data, Cai et al. demonstrate that the TLIRT, when compared to more familiar metrics, offers a unique perspective on model-data fit.

The article by Bonifay and Depaoli (2023) also addresses the sparse contingency table problem, but rather than proposing a novel goodness-of-fit statistic for categorical data modeling, they focus on an alternative approach to model evaluation. Specifically, they describe the Bayesian statistical technique known as posterior predictive model checking, which enables researchers not only to inspect overall goodness-of-fit but also to scrutinize any other feature of their model and/or data. As a didactic example of Bayesian model checking, the authors examine several features of an IRT model of empirical data from a psychopathology screening instrument.

The article by Tan et al. (2023) also takes a measurement perspective. Because measurement error is known to compromise the performance of statistical methods, study outcomes in prevention science must be precisely measured. Although modern psychometric frameworks such as IRT are well-suited to describe participants’ proficiencies on a latent scale, standard IRT methods are, however, of limited use when one is interested in explaining why participants possess certain proficiencies. Instead of focusing on quantifying general trait levels, cognitive diagnosis models (CDMs; cf. de la Torre et al., 2018) remedy these potential shortcomings and allow for a more fine-grained analysis of item-level measurement data in terms of symptom profiles. To introduce CDMs to the audience of prevention scientists, the authors analyzed data from a multisite longitudinal prevention study on alcohol-related problems and clinically relevant psychological symptoms.

The article by Bray et al. (2023) presents new developments of latent class moderation (Lanza & Rhoades, 2013) and demonstrates how multiple factors can be combined to form a multidimensional moderator and thereby improve the detection of differential intervention effects. Using large sample data from a study on adolescent cannabis use, the authors illustrate the advantages of this sophisticated mixture modeling technique and provide evidence of the particular substance use services that would be most beneficial for youth in different latent subgroups.

The article by McNeish et al. (2023) focuses on latent categorical variables in the context of describing growth trajectories of longitudinal data. Growth mixture models are known to be difficult to estimate, which often results in oversimplifications of observed data to obtain model results. In addition, large sample sizes are usually required to guarantee stable results. The authors present an approach based on covariance pattern analysis (Liu et al., 2012) to fit growth mixture models with small to moderate sample sizes. Data on juvenile diabetes are used to illustrate that the proposed approach is able to give meaningful results with sample sizes as small as 90 observations.

Finally, the Special Issue closes with a commentary by Wood (2023). In this commentary, the author carefully evaluates the potential merits of adopting recent methodological advances in the daily routine of analyzing data and interpreting model results. Specifically, the contributions made in this Special Issue are discussed with an emphasis on their level of parsimony to data, their potential to generate new hypotheses, and their ability to provide counterarguments to existing findings.

The articles in this Special issue present recent advances in the theory and application of categorical data analysis in prevention science research. These recent developments showcase that quantitative methodologies focused on categorical data continue to be a dynamic area of research. The contributions herein comprise a selection of state-of-the-art methods that have yet to be widely adopted by the prevention science research community. We anticipate that the diverse techniques presented in this Special Issue will enable future prevention scientists to analyze, model, and evaluate their categorical data in ways that will yield deeper insights and stronger inferences. We also hope that the methods and examples in this issue will inspire future prevention scientists to refine their research designs in light of the availability of tools to answer specific research questions (e.g., awareness of cognitive diagnosis modeling could compel prevention researchers to design studies that target more fine-grained measurement). Overall, this issue demonstrates that methodologists who specialize in generalized linear modeling, item response theory and categorical factor analysis, mixture modeling, and other analytic tools are continually refining the available methods and developing novel techniques. Prevention scientists can certainly benefit from these developments by enriching their standard methodological toolbox to critically evaluate and refine theories in the field of prevention science. The challenge then is for prevention researchers (as well as journal and grant reviewers) to understand, review, and interpret the results that these different methods provide; we hope that the breadth of topics, the accessibility of the writing, and the relevance of the empirical illustrations presented in this Special Issue will assist in those endeavors.