Consistency of Cluster Analysis for Cognitive Diagnosis: The Reduced Reparameterized Unified Model and the General Diagnostic Model

Abstract

The asymptotic classification theory of cognitive diagnosis (ACTCD) provided the theoretical foundation for using clustering methods that do not rely on a parametric statistical model for assigning examinees to proficiency classes. Like general diagnostic classification models, clustering methods can be useful in situations where the true diagnostic classification model (DCM) underlying the data is unknown and possibly misspecified, or the items of a test conform to a mix of multiple DCMs. Clustering methods can also be an option when fitting advanced and complex DCMs encounters computational difficulties. These can range from the use of excessive CPU times to plain computational infeasibility. However, the propositions of the ACTCD have only been proven for the Deterministic Input Noisy Output “AND” gate (DINA) model and the Deterministic Input Noisy Output “OR” gate (DINO) model. For other DCMs, there does not exist a theoretical justification to use clustering for assigning examinees to proficiency classes. But if clustering is to be used legitimately, then the ACTCD must cover a larger number of DCMs than just the DINA model and the DINO model. Thus, the purpose of this article is to prove the theoretical propositions of the ACTCD for two other important DCMs, the Reduced Reparameterized Unified Model and the General Diagnostic Model.

Appendix: The Limited Legitimacy of \(\varvec{W}\) for the GDM

Recall the IRF of the GDM and the expected response to item j (see Equation 2):

$$\begin{aligned} P(Y_j=1 \mid \varvec{\alpha }) = \frac{ \exp \big \{ \beta _j + \sum ^K_{k=1}\gamma _{jk} q_{jk} \alpha _k \big \} }{ 1 + \exp \big \{ \beta _j + \sum ^K_{k=1} \gamma _{jk} q_{jk} \alpha _k \big \} } = S_j(\varvec{\alpha }), \end{aligned}$$

where $\varvec{\gamma }_j > \varvec{0}$. For the GDM, the sum-score statistic $\varvec{W}$ cannot guarantee well-separated proficiency-class centers for distinct $\varvec{\alpha }$, which invalidates Lemma 2. As an example, consider a test with 17 items involving $K=3$ attributes. The Q-matrix and the parameter settings are

Frequency of item type	Item-attribute profile pattern			Parameter
				$\beta _j$	$\gamma _{j1}$	$\gamma _{j2}$	$\gamma _{j3}$
2	1	0	0	$-$1.00	1.40	–	–
2	0	1	0	$-$1.00	–	1.50	–
2	0	0	1	$-$0.20	–	–	0.70
2	1	1	0	$-$0.20	0.20	0.10	–
2	1	0	1	$-$1.76	4.00	–	1.00
2	0	1	1	$-$1.64	–	3.00	0.50
5	1	1	1	0.22	0.10	0.10	5.00

For attribute profiles $\varvec{\alpha }=(0,0,1)^{\prime }$ and $\varvec{\alpha }^{*}=(1,1,0)^{\prime }$, the proficiency-class centers $\varvec{T}(\varvec{\alpha }) = \varvec{T}(\varvec{\alpha }^{*}) = (7.1, 6.9, 7.3)^{\prime }$ are identical. Detailed calculations are provided only for $\varvec{T}(\varvec{\alpha })$:

$$\begin{aligned} T_1(\varvec{\alpha })= & {} 2 \Big ( \frac{\exp \{\beta _1\}}{1+\exp \{\beta _1\}}\Big ) + 2 \Big (\frac{\exp \{\beta _4\}}{1+\exp \{\beta _4\}}\Big ) + 2 \Big (\frac{\exp \{\beta _5+\gamma _{53}\}}{1+\exp \{\beta _5+\gamma _{53}\}}\Big ) + 5 \Big (\frac{\exp \{\beta _7+\gamma _{73}\}}{1+\exp \{\beta _7+\gamma _{73}\}}\Big )\\= & {} 2 \Big (\frac{\exp \{-1\}}{1+\exp \{-1\}}\Big ) + 2 \Big (\frac{\exp \{-.20\}}{1+\exp \{-.20\}}\Big ) + 2 \Big (\frac{\exp \{-1.76+1\}}{1+\exp \{-1.76+1\}}\Big )\\&+\, 5 \Big (\frac{\exp \{.22+5\}}{1+\exp \{.22+5\}}\Big )\\= & {} 7.10, \\ T_2(\varvec{\alpha })= & {} 2\Big (\frac{\exp \{\beta _2\}}{1+\exp \{\beta _2\}}\Big ) + 2\Big (\frac{\exp \{\beta _4\}}{1+\exp \{\beta _4\}}\Big ) + 2\Big (\frac{\exp \{\beta _6+\gamma _{63}\}}{1+\exp \{\beta _6+\gamma _{63}\}}\Big ) + 5\Big (\frac{\exp \{\beta _7+\gamma _{73}\}}{1+\exp \{\beta _7+\gamma _{73}\}}\Big )\\= & {} 2\Big (\frac{\exp \{-1\}}{1+\exp \{-1\}}\Big ) + 2\Big (\frac{\exp \{-.20\}}{1+\exp \{-.20\}}\Big ) + 2\Big (\frac{\exp \{-1.64+.50\}}{1+\exp \{-1.64+.50\}}\Big ) \\&+\, 5\Big (\frac{\exp \{.22+5\}}{1+\exp \{.22+5\}}\Big )\\= & {} 6.90, \\ T_3(\varvec{\alpha })= & {} 2\Big (\frac{\exp \{\beta _3+\gamma _{33}\}}{1+\exp \{\beta _3 +\gamma _{33}\}}\Big ) + 2\Big (\frac{\exp \{\beta _5+\gamma _{53}\}}{1+\exp \{\beta _5+\gamma _{53}\}}\Big ) + 2\Big (\frac{\exp \{\beta _6+\gamma _{63}\}}{1+\exp \{\beta _6+\gamma _{63}\}}\Big )\\&+\, 5\Big (\frac{\exp \{\beta _7+\gamma _{73}\}}{1+\exp \{\beta _7+\gamma _{73}\}}\Big )\\= & {} 2\Big (\frac{\exp \{-.20+.70\}}{1+\exp \{-.20+.70\}}\Big ) + 2\Big (\frac{\exp \{-1.76+1\}}{1+\exp \{-1.76+1\}}\Big ) + 2\Big (\frac{\exp \{-1.64+.50\}}{1+\exp \{-1.64+.50\}}\Big ) \\&+ \, 5\Big (\frac{\exp \{.22+5\}}{1+\exp \{.22+5\}}\Big )\\= & {} 7.30. \end{aligned}$$

Given two attribute profiles, $\varvec{\alpha }\ne \varvec{\alpha }^{*}$, $\varvec{\alpha }$ is said to be nested within $\varvec{\alpha }^{*}$ if there exist some k and $k^{\prime }$ ($k\ne k^{\prime }$; $k, k^{\prime } \in \{1,2,\ldots ,K\}$) such that $\alpha _k=0$, $\alpha ^{*}_k=1$, and $\alpha _{k^{\prime }} \le \alpha ^{*}_{k^{\prime }}$. Assume that the Q-matrix contains at least one of each of the possible $2^K-1$ item-attribute profiles.

1.1 Condition: $\varvec{\alpha }\ne \varvec{\alpha }^{*}$ Is Nested Within $\varvec{\alpha }^{*}$

From the equation for the expected response $S_j(\varvec{\alpha })$ for the GDM, it is known that

$$\begin{aligned} T_k(\varvec{\alpha })= & {} \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1 + \exp \big \{\beta _j+\sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk}=1), \end{aligned}$$

(14)

$$\begin{aligned} T_k(\varvec{\alpha }^{*})= & {} \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\sum _{l=1}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1 + \exp \big \{\beta _j+\sum _{l=1}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk}=1). \end{aligned}$$

(15)

Based on the assumptions that the slope $\gamma _{_{jk}}>0$, $\alpha ^{*}_k=1$ and that $\alpha _{k'} \le \alpha ^{*}_{k'}$ for all $k' \ne k$, the term $\sum _{l=1}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}$ in Eq. 15 can be decomposed as

$$\begin{aligned} \sum _{l=1}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l} = \sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l} + \gamma _{_{jk}} \ge \sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}+\gamma _{_{jk}} > \sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}. \end{aligned}$$

Because the logistic function, $\exp \{\beta \}/(1 + \exp \{\beta \})$, increases monotonically, $T_k(\varvec{\alpha }) < T_k(\varvec{\alpha }^{*})$, which implies that $\varvec{T}(\varvec{\alpha }) \ne \varvec{T}(\varvec{\alpha }^{*})$. Hence, as long as the attribute profiles of two proficiency classes are nested, their centers are guaranteed to be separated.

1.2 Condition: $\varvec{\alpha }\ne \varvec{\alpha }^{*}$ Is Not Nested Within $\varvec{\alpha }^{*}$

If $\varvec{\alpha }$ is not nested within $\varvec{\alpha }^{*}$, then there must be some k and $k^{\prime }$ ($k\ne k^{\prime }$; $k, k^{\prime } \in \{1,2,\ldots ,K\}$) such that $\alpha _k=1$ and $\alpha _{k^{\prime }}=0$; $\alpha ^{*}_k=0$ and $\alpha ^{*}_{k^{\prime }}=1$. Then the $k^{th}$ components of $\varvec{T}(\varvec{\alpha })$ and $\varvec{T}(\varvec{\alpha }^{*})$ are

$$\begin{aligned} \begin{aligned} T_k(\varvec{\alpha })&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk}=1)\\&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1, l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk}=1,q_{jk'}=0)\\&\quad +\, \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk}=1,q_{jk'}=1), \\ T_k(\varvec{\alpha }^{*})&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j + \sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j + \sum _{l=1,l \ne k}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk}=1)\\&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk}=1,q_{jk'}=0) \\&\quad +\, \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk}=1,q_{jk'}=1). \end{aligned} \end{aligned}$$

(16)

The components $k^{\prime }$ of $\varvec{T}(\varvec{\alpha })$ and $\varvec{T}(\varvec{\alpha }^{*})$ are

$$\begin{aligned} \begin{aligned} T_{k'}(\varvec{\alpha })&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j + \sum _{l=1,l \ne k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j + \sum _{l=1,l \ne k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk^{\prime }}=1) \\&= \sum _{j=1}^J \frac{\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk'}=1,q_{jk}=0) \\&\quad +\,\frac{\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\,\, \text{ I }(q_{jk'}=1,q_{jk}=1), \\ T_{k'}(\varvec{\alpha }^{*})&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j + \gamma _{_{jk^{\prime }}} + \sum _{l=1,l \ne k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j + \gamma _{_{jk^{\prime }}} + \sum _{l=1,l \ne k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk^{\prime }}=1)\\&= \sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk'}=1,q_{jk}=0) \\&\quad +\,\sum _{j=1}^J\frac{\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\,\, \text{ I }(q_{jk'}=1,q_{jk}=1). \end{aligned} \end{aligned}$$

(17)

Note that the right-hand side summation terms in the exponential expressions of the pairs of equations in 16 and 17 involve triple products depending on the remaining entries, $\alpha _l$ and $\alpha ^{*}_l$, with $l \ne k, k^{\prime }$. However, there is no way to predict which specific elements, $\alpha _l$, $\alpha ^{*}_l$, equal 0 or 1. Hence, it is not possible (as it was for the case of the nested attribute profile patterns) to determine whether any reliable equality relation exists between the right-hand side terms of the pair of equations in either 16 or 17. Consequently, no conclusions about the equality or inequality of the components $T_k(\varvec{\alpha })$ and $T_k(\varvec{\alpha }^{*})$, or the components $T_{k^{\prime }}(\varvec{\alpha })$ and $T_{k^{\prime }}(\varvec{\alpha }^{*})$, can be drawn, and no conclusive statement as to whether $\varvec{T}(\varvec{\alpha })$ equals $\varvec{T}(\varvec{\alpha }^{*})$ is possible when attribute profiles are not nested, with the exception of two special cases.

1.3 Special Case: $||\varvec{\alpha }||^2=||\varvec{\alpha }^{*}||^2= K^{*} = 1,2, \ldots , K-1$; $\varvec{\alpha }\ne \varvec{\alpha }^{*}$ Is Not Nested Within $\varvec{\alpha }^{*}$

If two attribute profiles, $\varvec{\alpha }$ and $\varvec{\alpha }^{*}$, have the same length, $||\varvec{\alpha }||^2 = ||\varvec{\alpha }^{*}||^2 = K^{*} = 1,2, \ldots , K-1$, but differ in only two elements, k and $k^{\prime }$, (i.e., all other elements in $\varvec{\alpha }$ and $\varvec{\alpha }^{*}$ must be identical), then the following equality holds:

$$\begin{aligned} \sum _{l=1,l \ne k, k'}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l} = \sum _{l=1,l \ne k, k'}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}. \end{aligned}$$

(18)

This equality implies that, for the pair of equations representing the $k^{th}$ components of $\varvec{T}(\varvec{\alpha })$ and $\varvec{T}(\varvec{\alpha }^{*})$ in 16,

$$\begin{aligned} \frac{\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}} > \frac{\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}, \end{aligned}$$

(19)

and that, for the pair of equations representing components $k^{\prime }$ of $\varvec{T}(\varvec{\alpha })$ and $\varvec{T}(\varvec{\alpha }^{*})$ in 17,

$$\begin{aligned} \frac{\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{ \beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}} > \frac{\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{ \beta _j+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}. \end{aligned}$$

(20)

Recall that whether $T_k(\varvec{\alpha })$ equals $T_k(\varvec{\alpha }^{*})$ or whether $T_{k^{\prime }}(\varvec{\alpha })$ equals $T_{k^{\prime }}(\varvec{\alpha }^{*})$ (and thus, whether $\varvec{T}(\varvec{\alpha })$ equals $\varvec{T}(\varvec{\alpha }^{*})$) cannot be determined. But if it is assumed that $T_k(\varvec{\alpha }) \ne T_k(\varvec{\alpha }^{*})$, then no further examination of $T_{k^{\prime }}(\varvec{\alpha })$ and $T_{k^{\prime }}(\varvec{\alpha }^{*})$ is necessary because then $\varvec{T}(\varvec{\alpha })\ne \varvec{T}(\varvec{\alpha }^{*})$ by assumption. If it is also assumed that $T_k(\varvec{\alpha })= T_k(\varvec{\alpha }^{*})$, then from Eq. 19, it follows that

$$\begin{aligned} \begin{aligned}&\sum _{j=1}^J\frac{\exp \big \{\beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}{1+\exp \big \{\beta _j+\gamma _{_{jk}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha _{_l}\big \}}\quad \text{ I }(q_{jk}=1,q_{jk'}=1) \\&< \\&\sum _{j=1}^J \frac{\exp \big \{\beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}{1+\exp \big \{\beta _j+\gamma _{_{jk'}}+\sum _{l=1,l \ne k, k^{\prime }}^K \gamma _{_{jl}}q_{_{jl}}\alpha ^{*}_{_l}\big \}}\quad \text{ I }(q_{jk}=1,q_{jk'}=1) \end{aligned} \end{aligned}$$

(21)

must be true; otherwise, this assumption would be violated. Inspection of $T_{k^{\prime }}(\varvec{\alpha })$ and $T_{k^{\prime }}(\varvec{\alpha }^{*})$ in Equations 20 and 21 suggests that $T_{k'}(\varvec{\alpha }) < T_{k'}(\varvec{\alpha }^{*})$, which implies $\varvec{T}(\varvec{\alpha })\ne \varvec{T}(\varvec{\alpha }^{*})$.