An Exploratory Diagnostic Model for Ordinal Responses with Binary Attributes: Identifiability and Estimation

Abstract

Diagnostic models (DMs) provide researchers and practitioners with tools to classify respondents into substantively relevant classes. DMs are widely applied to binary response data; however, binary response models are not applicable to the wealth of ordinal data collected by educational, psychological, and behavioral researchers. Prior research developed confirmatory ordinal DMs that require expert knowledge to specify the underlying structure. This paper introduces an exploratory DM for ordinal data. In particular, we present an exploratory ordinal DM, which uses a cumulative probit link along with Bayesian variable selection techniques to uncover the latent structure. Furthermore, we discuss new identifiability conditions for structured multinomial mixture models with binary attributes. We provide evidence of accurate parameter recovery in a Monte Carlo simulation study across moderate to large sample sizes. We apply the model to twelve items from the public-use, Early Childhood Longitudinal Study, Kindergarten Class of 1998–1999 approaches to learning and self-description questionnaire and report evidence to support a three-attribute solution with eight classes to describe the latent structure underlying the teacher and parent ratings. In short, the developed methodology contributes to the development of ordinal DMs and broadens their applicability to address theoretical and substantive issues more generally across the social sciences.

Appendix: Gibbs Sampling Algorithm and Full Conditional Distributions

This section discusses the full conditional distributions used to approximate the posterior distribution of the ordinal diagnostic model parameters with Gibbs sampling. For iteration $t=1,\ldots , T$ we sample:

• 1. For $i=1,\ldots ,n$,

  • (a) ${\varvec{\alpha }}_i^{(t)}$ from the multinomial full conditional distribution ${\varvec{\alpha }}_i^{(t)}|{\varvec{Y}}_i,\mathbf B ^{(t-1)},{\varvec{\alpha }}_1^{(t)},\ldots ,{\varvec{\alpha }}_{i-1}^{(t)},{\varvec{\alpha }}_{i+1}^{(t-1)},\ldots ,{\varvec{\alpha }}_{n}^{(t-1)}$ where the conditional probability ${\varvec{\alpha }}_i^{(t)}$ is classified as profile c is,

    $$\begin{aligned} P({\varvec{\alpha }}_i^{(t)\top }{\varvec{v}}&=c|{\varvec{Y}}_i,\mathbf B ^{(t-1)},{\varvec{\alpha }}_1^{(t)},\ldots ,{\varvec{\alpha }}_{i-1}^{(t)},{\varvec{\alpha }}_{i+1}^{(t-1)},\ldots ,{\varvec{\alpha }}_{n}^{(t-1)})\nonumber \\&=\frac{(n_{ci}+n_{c0})\prod _{j=1}^J \theta _{jc,y_{ij}}^{(t-1)} }{\sum _{c=0}^{2^K-1} (n_{ci}+n_{c0}) \prod _{j=1}^J \theta _{jc,y_{ij}}^{(t-1)} } \end{aligned}$$

    (A1)

    where $\theta _{jc,y_{ij}}^{(t-1)}=\Phi \left( \tau _{jc,y_{ij}+1}-{\varvec{a}}_c^\top {\varvec{\beta }}_j^{(t-1)}\right) -\Phi \left( \tau _{jc,y_{ij}}-{\varvec{a}}_c^\top {\varvec{\beta }}_j^{(t-1)}\right)$. Notice that we integrate ${\varvec{\pi }}$ from the prior distribution $p(\mathbf A ,{\varvec{\pi }})=p({\varvec{\alpha }}_1,\ldots ,{\varvec{\alpha }}_n|{\varvec{\pi }})p({\varvec{\pi }})$ and instead use the conditional prior distribution $p({\varvec{\alpha }}_i|{\varvec{\alpha }}_1^{(t)},\ldots ,{\varvec{\alpha }}_{i-1}^{(t)},{\varvec{\alpha }}_{i+1}^{(t-1)},\ldots ,{\varvec{\alpha }}_{n}^{(t-1)})$ which implies the usual $\pi _c$ (e.g., see Equation 7 of Culpepper, 2019) is replaced with $n_{ci}+n_{c0}$ where $n_{ci}$ is the number of respondents other than i that are classified in class c (e.g., see Jain & Neal, 2004) and $n_{c0}$ is the prior Dirichlet parameter (note $n_{c0}=1$ for a uniform prior).

  • (b) For $j=1,\ldots ,J$ update the latent augmented data from the full conditional distribution

    $$\begin{aligned} Y_{ij}^{*(t)}|Y_{ij},{\varvec{\alpha }}_i^{(t)},{\varvec{\beta }}_j^{(t-1)}\sim \mathcal N({\varvec{a}}_i^{(t)\top }{\varvec{\beta }}_j^{(t-1)},1) \mathcal I(\tau _{jc,y_{ij}}<Y_{ij}^{*(t)}<\tau _{jc,y_{ij}+1}) \end{aligned}$$

    (A2)

    where $\tau _{jc,y_{ij}}$ and $\tau _{jc,y_{ij}+1}$ are lower and upper thresholds for the observed value of $Y_{ij}$ for class c and item j. Recall we follow previously discussed strategies and fix the thresholds as ${\varvec{\tau }}_j=(0,2,\ldots , 2(M_j-2))^\top$.

• 2. Update the latent class probabilities (i.e., the mixing weights) as ${\varvec{\pi }}^{(t)}|\mathbf{A }^{(t)}$ from the Dirichlet full conditional distribution (e.g., see Culpepper, 2015).

• 3. For $j=1,\ldots ,J$,

  • (a) For $k=1,\ldots ,K$ sample $q_{jk}^{(t)}$ from the Bernoulli full conditional distribution $q_{jk}^{(t)}|{\varvec{\beta }}_j^{(t-1)}, q_{j1}^{(t)},\ldots ,q_{j,k-1}^{(t)},q_{j,k+1}^{(t-1)},\ldots ,q_{jK}^{(t-1)},\omega ^{(t-1)}$ (e.g., see Culpepper, 2019).

  • (b) For $p=1,\ldots ,P$ sample $\beta _{jp}^{(t)}$ from the truncated normal full conditional distribution $\beta _{jp}^{(t)}|Y_{1j}^{*(t)},\ldots ,Y_{nj}^{*(t)},\mathbf A ^{(t)},\beta _{j1}^{(t)},\ldots ,\beta _{j,p-1}^{(t)},\beta _{j,p+1}^{(t-1)},\ldots ,,\beta _{j,P+1}^{(t-1)}, {\varvec{q}}_j^{(t)}$ (e.g., see Culpepper, 2019).

• 4. Sample $\omega ^{(t)}$ from the Beta full conditional distribution $\omega ^{(t)}|\mathbf{Q }^{(t)}$ (e.g., see Culpepper, 2019).