Bayes Factor Covariance Testing in Item Response Models

Abstract

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.

Appendices

Appendix A: (Orthogonal) Helmert Transformation Matrix

An orthogonal matrix $\mathbf {H}$ has the property that $\mathbf {H}^t\mathbf {H} = \mathbf {H}\mathbf {H}^t=\mathbf {I}$, where the rows of $\mathbf {H}$ are mutually orthogonal and each row has a unit norm. A particular $(p\times p)$ orthogonal matrix is the Helmert matrix, where the first row has elements $p^{-\frac{1}{2}}$, and all zeroes of the triangle above the main diagonal and below the first row. The remaining elements below the main diagonal are positive, where row j $(j=2,\ldots ,p)$ has elements $\left[ \frac{1}{\sqrt{j(j+1)}} \mathbf {1}^t_j,\frac{-j}{\sqrt{j(j+1)}},\mathbf {0}\right]$. Lancaster (1965) referred to it as Helmertian in the strict sense and showed various properties of Helmert matrices (see also, Searle, 1971, pp. 31–33). Subsequently, the Helmert matrix of order p is given by

$$\begin{aligned} \mathbf {H} = \left[ \begin{array}{cccccc} \frac{1}{\sqrt{p}} &{} \frac{1}{\sqrt{p}} &{} \frac{1}{\sqrt{p}} &{} \cdots &{} \frac{1}{\sqrt{p}}\\ \frac{1}{\sqrt{2}} &{} -\frac{1}{\sqrt{2}} &{} 0 &{} \cdots &{} 0\\ \frac{1}{\sqrt{6}} &{} \frac{1}{\sqrt{6}} &{} -\frac{2}{\sqrt{6}} &{} \ddots &{} \vdots \\ \vdots &{} \vdots &{} \vdots &{} \ddots &{} 0 \\ \frac{1}{\sqrt{p(p-1)}} &{} \frac{1}{\sqrt{p(p-1)}} &{} \frac{1}{\sqrt{p(p-1)}} &{} \cdots &{} -\frac{p-1}{\sqrt{p(p-1)}} \end{array} \right] . \end{aligned}$$

(26)

Appendix B: Helmert Transformed Normal Random Variables

Consider a multivariate normally distributed random variable $\mathbf {z}_i=(z_{i1},\ldots ,z_{ip})^t \sim \mathcal {N}\left( \mu \mathbf 1 _p,\varvec{\Sigma }\right)$, where the covariance matrix has a compound symmetry structure represented by $\varvec{\Sigma }=\sigma ^2 \mathbf {I}_p + \tau \mathbf {J}_p$. The $\mathbf {z}_i$ are transformed using Helmert, and the transformed variable is given by $\tilde{\mathbf {z}}_i=\mathbf {H}\mathbf {z}_i$. The components of the transformed variable $\tilde{\mathbf {z}}_i$ are independently normally distributed. The first component of the Helmert transformed variable, $\tilde{z}_{i1}$, is normally distributed with mean and variance equal to,

$$\begin{aligned} E\left( \tilde{z}_{i1}\right)= & {} E\left( \sqrt{p}\bar{z}_i\right) = \sqrt{p}E\left( \sum {z}_{ij}/p\right) = \sqrt{p}\mu \\ { Var}\left( \tilde{z}_{i1}\right)= & {} { Var}\left( \sqrt{p}\bar{z}_i\right) = p { Var}\left( \bar{z}_i\right) = { Var}\left( \sum _{j=1}^{p} z_{ij}\right) /p \\= & {} \left[ \sum _{j=1}^p \left( \sigma ^2+\tau \right) + \sum _{k=1}^p\sum _{j\ne k} \tau \right] /p \\= & {} \left[ p(\sigma ^2+\tau ) + p(p-1)\tau \right] /p = \sigma ^2 + p\tau , \end{aligned}$$

respectively.

Consider a sample $\mathbf {z} = \left( \mathbf {z}^t_{1},\ldots ,\mathbf {z}^t_{n}\right) ^t$, where the components are identically and independently multivariate normally distributed. Subsequently, let $\lambda = \sigma ^2 + p\tau$, the probability density function of the first Helmert transformed component, $\tilde{\mathbf {z}}_{1}$, is given by

$$\begin{aligned} p\left( \tilde{\mathbf {z}}_{1} \mid \mu , \lambda \right)= & {} \left( 2\pi \lambda \right) ^{-n/2}\exp \left( \frac{-\sum _i \left( \tilde{z}_{i1} - \mu \sqrt{p}\right) ^2/2}{\lambda } \right) \\= & {} \left( 2\pi \lambda \right) ^{-n/2}\exp \left( \frac{-p\sum _i\left( \overline{z}_i - \mu \right) ^2/2}{\lambda } \right) \\= & {} \left( 2\pi \lambda \right) ^{-n/2} \exp \left( \frac{-p S^2_B/2}{\lambda } \right) , \end{aligned}$$

where $S^2_B = \sum _i\left( \overline{z}_i - \mu \right) ^2$. The probability density function of the $\tilde{\mathbf {z}}_{1}$ can be expressed as the density of $\overline{\mathbf {z}}_i$ given $\sigma ^2$ and $\tau$. It follows that

$$\begin{aligned} p\left( \overline{z}_1,\ldots ,\overline{z}_n \mid \sigma ^2,\tau ,\mu \right)= & {} \left( 2\pi (\sigma ^2 + p\tau )\right) ^{-n/2}\exp \left( \frac{-pS^2_B/2}{\sigma ^2 + p\tau } \right) \\= & {} (2\pi p)^{-n/2}\left( \sigma ^2/p + \tau )\right) ^{-n/2}\exp \left( \frac{-S^2_B/2}{\sigma ^2/p + \tau } \right) , \end{aligned}$$

where $\tau > \sigma ^2/p$, since $\lambda = \sigma ^2 + p\tau > 0$ when considering $\sigma ^2$ a constant.

The remaining $n(p-1)$ components $\left( \tilde{\mathbf {z}}_{2},\ldots ,\tilde{\mathbf {z}}_{p}\right)$ are distributed according to

$$\begin{aligned} p\left( \tilde{\mathbf {z}}_{2},\ldots ,\tilde{\mathbf {z}}_{p} \mid \mu ,\sigma ^2 \right)= & {} \left( 2\pi \sigma ^2\right) ^{-n(p-1)/2} \exp \left( \frac{-\sum _{i=1}^n\sum _{j=2}^p \tilde{z}_{ij}^2}{2\sigma ^2} \right) \\= & {} \left( 2\pi \sigma ^2\right) ^{-n(p-1)/2} \exp \left( \frac{-S^2_W}{2\sigma ^2} \right) \end{aligned}$$

where $S^2_W = \sum _{i=1}^{n}\sum _{j=2}^{p}\tilde{z}_{ij}^2 = \sum _{i=1}^{n}\sum _{j=1}^{p}\left( z_{ij} -\overline{z}_i\right) ^2$.