A mixed-effects location-scale model for ordinal questionnaire data

Abstract

In health studies, questionnaire items are often scored on an ordinal scale, for example on a Likert scale. For such questionnaires, item response theory (IRT) models provide a useful approach for obtaining summary scores for subjects (i.e., the model’s random subject effect) and characteristics of the items (e.g., item difficulty and discrimination). In this article, we describe a model that allows the items to additionally exhibit different within-subject variance, and also includes a subject-level random effect to the within-subject variance specification. This permits subjects to be characterized in terms of their mean level, or location, and their variability, or scale, and the model allows item difficulty and discrimination in terms of both random subject effects (location and scale). We illustrate application of this location-scale mixed model using data from the Social Subscale of the Drinking Motives Questionnaire assessed in an adolescent study. We show that the proposed model fits the data significantly better than simpler IRT models, and is able to identify items and subjects that are not well-fit by the simpler models. The proposed model has useful applications in many areas where questionnaires are often rated on an ordinal scale, and there is interest in characterizing subjects in terms of both their mean and variability.

Appendix: SAS PROC NLMIXED syntax

Below is a sample of syntax necessary to run the mixed-effects ordinal location-scale model described in this article. Uppercase letters are used for SAS specific syntax and lowercase letters are used for user defined entities. In terms of the variables used in this syntax, id is a subject identifier, y denotes the ordinal outcome and x1 to x3 are item indicators. Here, for simplicity, we illustrate the syntax consdering only three items and three response categories. The two cumulative logits are clogit1 and clogit2, and the two cumulative probabilities are cprob1 and cprob2. The random location effect is named theta, the random scale effect is zeta, and their correlation is corr. The location model is summarized by loc, while the scale model is given by scale.

Users must provide starting values for all parameters on the PARMS statement. To do so, it is beneficial to run the model in stages using estimates from a prior stage as starting values and setting the additional parameters to zero or some small value. For example, one can start by estimating a random-intercepts ordinal model with item difficulty (beta1-beta3), item discrimination (delta1-delta3), and threshold parameter (alpha2). Estimates of these parameters can then be specified as starting values in a model that adds in the WS variance parameters (gamma1-gamma3). Finally, the full model with the additional parameters (tau1-tau3 and cor) can be estimated. In practice, this approach works well with PROC NLMIXED, which sometimes has difficulties in converging to a solution for complex models. Furthermore, for complex models, it is sometimes the case that the default convergence criteria is not strict enough. In the above syntax, the convergence criteria is specified as GCONV=1e-12 on the PROC NLMIXED statement.

Fig. 1

DMQ social subscale items: category response proportions. Items: (1) as a way to celebrate; (2) because it is what most of my friends do when we get together; (3) to be sociable; (4) because it is customary on special occasions; (5) because it makes a social gathering more enjoyable. Categories: $\textit{1}=\hbox {almost}$ never, $\textit{2}=\hbox {some}$ of the time, $\textit{3}=\hbox {half}$ of the time, $\textit{4}=\hbox {most}$ of the time, $\textit{5}=\hbox {almost}$ always

Table 1 Simulation results for 4-parameter location-scale model

Table 2 Simulation results for 2- and 3-parameter models

Table 3 4-parameter location-scale model estimates (95 % confidence intervals)

Table 4 Subjects with high, near-zero, and low scale: Random effect estimates and observed item responses

Table 5 4-parameter location-scale model: Male gender DIF estimates (95 % confidence intervals)