Commentary on “Extending the Basic Local Independence Model to Polytomous Data” by Stefanutti, de Chiusole, Anselmi, and Spoto

Abstract

The Polytomous Local Independence Model (PoLIM) by Stefanutti, de Chiusole, Anselmi, and Spoto, is an extension of the Basic Local Independence Model (BLIM) to accommodate polytomous items. BLIM, a model for analyzing responses to binary items, is based on Knowledge Space Theory, a framework developed by cognitive scientists and mathematical psychologists for modeling human knowledge acquisition and representation. The purpose of this commentary is to show that PoLIM is simply a paraphrase of a DINA model in cognitive diagnosis for polytomous items. Specifically, BLIM is shown to be equivalent to the DINA model when the BLIM-items are conceived as binary single-attribute items, each with a distinct attribute; thus, PoLIM is equivalent to the DINA for polytomous single-attribute items, each with a distinct attribute.

4. Appendix: The Polytomous DINA Model

The section “The Polytomous DINA Model” only describes the special case of single-attribute items. For multi-attribute items, the polytomous DINA is more complex. This appendix provides a description of technical details of the polytomous DINA when used for modeling responses to polytomous items.

Recall that the q-vector of a polytomous single-attribute item j is written as $\mathbf {q}^{(a)}_j = (0, \ldots , 0, q_{ja}, 0, \ldots , 0)$. Let $L_j$ denote the highest level of the polytomous ideal response $\xi _{ij}$, with different levels $l \in \{0,1, \ldots , L_j\}$. For a single-attribute item having q-vector $\mathbf {q}^{(a)}_j$, $L_j = q_{ja}$ is true, and so the ideal response is

$$\begin{aligned} \xi _{ij} = \max _{l \in \{0,1,\ldots , q_{ja}\}} \left\{ l \times \Big (I \big [ \alpha _{ia} \ge l \big ] \Big ) \right\} \end{aligned}$$

(10)

So, for a single-attribute polytomous item, the relation between levels l and the item attribute vector $\mathbf {q}$ is relatively straightforward; this is not the case for polytomous multi-attribute items. The relation is far more complex and requires adjustment of the notation.

1.1 The “Star” Notation

First, Consider the entries $q_{ja}$ of the item attribute vector $\mathbf {q}_j = (q_{j1}, q_{j2}, \ldots , q_{jA})^{\prime }$ that document which attributes an examinee must have mastered to answer item j correctly. Suppose item j requires $A_j \le A$ attributes. Also, assume that the entries of the item attribute vector have been rearranged such that the attributes required for item j have been shifted to the first $A_j$ positions of $\mathbf {q}_j$; the remaining entries beyond $A_j$ are all zero. The shuffled item attribute vector is denoted as $\mathbf {q}^{*}_j = (q^{*}_{j1}, q^{*}_{j2}, \ldots , q^{*}_{jA_j})^{\prime }$. Notice that the zero entries in $\mathbf {q}_j$ have been deleted in $\mathbf {q}^{*}_j$. Thus, distinct from the $q_{ja}$, the entries of $\mathbf {q}^{*}_j$, $q^{*}_{ja}$, take on values ranging only from 1 to $H_a$ denoting the different levels of attributes required for item j.

1.2 Polytomous Ideal Response Categories

Switching from $\mathbf {q}_j$ to $\mathbf {q}^{*}_j$ allows for expressing the number of categories of $\eta _{ij}$—that is, of the polytomous ideal response to item j— as the product of the entries of $\mathbf {q}^{*}_j$: define $L_j = q^{*}_{j1} q^{*}_{j2} \cdots q^{*}_{jA_j}$; then, the number of categories is $L_j +1$, where the additional category refers to the zero response. The ideal response categories of $\eta _{ij}$ are indexed by $l = 0,1, \ldots , L_j$. (As $\eta _{ij}$ is the latent counterpart of the manifest response, $L_j + 1$ also denotes the number of response categories of the observable random variable $Y_{ij}$.) Notice the terminology: “response categories”—and not “response levels.” The latter would have implied ordered response categories. However, here, the general case is concerned that also includes non-ordered response categories; hence, the indices $l = 0,1, \ldots , L_j$ should be merely interpreted as category labels.

1.3 Identifying Polytomous Ideal Response Categories

The different combinations of values of the entries of $\mathbf {q}^{*}_j$ form a partially ordered set. Therefore, a formal rule is needed how to establish the relation between these different combinations of $q^{*}_{ja}$ and the categories $l= 1,2, \ldots , L_j$ of the polytomous ideal response to item j in an unambiguous manner. (The zero-category, indexed $l=0$, denoting non-mastery of the attributes required for item j, is a special case and is discussed later.) Let $\mathbf {q}^{*\, (l)}_j = \Big ( q^{*\, (l)}_{j1}, q^{*\, (l)}_{j2}, \ldots , q^{*\, (l)}_{jA_j} \Big )$ denote the specific combination of values in $\mathbf {q}^{*}_j$ that correspond to a particular response category l. The aforementioned formal rule is best explained by an illustrative example; so, consider $\mathbf {q}^{*}_j = (q^{*}_{j1}, q^{*}_{j2}, q^{*}_{j3}) = (3,2,2)$; hence, $L_j = q^{*}_{j1} q^{*}_{j2} q^{*}_{j3} = 3(2)2 = 12$. In disregarding $l=0$ for now, the different combinations $\mathbf {q}^{*\,(l)}_j$ associated with $l = 1,2, \ldots , 12$ spelled out explicitly are:

l	$\mathbf {q}^{*\,(l)}_j$
	$q^{*\, (l)}_{j1}$	$q^{*\, (l)}_{j2}$	$q^{*\, (l)}_{j3}$
1	1	1	1
2	1	1	2
3	1	2	1
4	1	2	2
5	2	1	1
6	2	1	2
7	2	2	1
8	2	2	2
9	3	1	1
10	3	1	2
11	3	2	1
12	3	2	2

Notice that the different $\mathbf {q}^{*\, (l)}_j$ have been arranged in lexicographic order. For this example, the rule for identifying category l associated with a particular $\mathbf {q}^{*\, (l)}_j$ is

$$\begin{aligned} l = \big ( q^{*\, (l)}_{j1} - 1 \big ) q^{*}_{j2} q^{*}_{j3} + \big ( q^{*\, (l)}_{j2} - 1 \big ) q^{*}_{j3} + q^{*\, (l)}_{j3} \end{aligned}$$

(11)

Of course, the values of $q^{*\, (l)}_{j1}$, $q^{*\, (l)}_{j2}$, and $q^{*\, (l)}_{j3}$ must be known. So, if $\mathbf {q}^{*\, (l)}_j = (3,1,2)$; then, $l = (3-1)2(2) + (1-1)2 + 2 = 10$. The general expression of Eq. (11) is

$$\begin{aligned} l = \sum _{a=1}^{A_j-1} \Big ( q^{*\, (l)}_{ja} - 1 \Big ) \prod _{b=a+1}^{A_j} q^{*}_{jb} + q^{*(l)}_{jA_j} \end{aligned}$$

1.4 Identifying $\mathbf {q}^{*\, (l)}$ Underlying a Specific Polytomous Ideal Response Category l

If, however, for a given l, the associated $\mathbf {q}^{*\, (l)}_j$ is sought, then the inverse of this equation must be used. For our earlier example with $\mathbf {q}^{*}_j = (3,2,2)$, the inverse of Eq. (11) is

$$\begin{aligned} q^{*(l)}_{j1} = \frac{ \frac{ l - q^{*(l)}_{j3} }{ q^{*}_{j3} } + 1 - q^{*(l)}_{j2} }{ q^{*}_{j2} } + 1 \end{aligned}$$

subject to

$$\begin{aligned} 1 \le q^{*(l)}_{ja} \le q^{*}_{ja} \forall a \in \{1, 2, \ldots , A_j \} \end{aligned}$$

Now, given $l=6$, what is the associated $\mathbf {q}^{*(l)}$? Use

$$\begin{aligned} q^{*(6)}_{j1} = \frac{\frac{6 - q^{*(6)}_{j3}}{2} + 1 - q^{*(6)}_{j2}}{2} +1 \end{aligned}$$

subject to

$$\begin{aligned} 1 \le&q^{*(6)}_{j1}&\le 3 \\ 1 \le&q^{*(6)}_{j2}&\le 2 \\ 1 \le&q^{*(6)}_{j3}&\le 2 \end{aligned}$$

Begin with the term $\frac{6 - q^{*(6)}_{j3}}{2}$ in the numerator, which must be an integer. Hence, $q^{*(6)}_{j3}$ can only equal 2 due to the constraint $1 \le q^{*(6)}_{j3} \le 2$. As a result, $\frac{6 - q^{*(6)}_{j3}}{2} = 2$ and

$$\begin{aligned} q^{*(6)}_{j1} = \frac{2 + 1 - q^{*(6)}_{j2}}{2} + 1 = \frac{3 - q^{*(6)}_{j2}}{2} +1 \end{aligned}$$

But $\frac{3 - q^{*(6)}_{j2}}{2}$ must also be an integer. Thus, $q^{*(6)}_{j2}$ can only be 1 due to the constraint $1 \le q^{*(6)}_{j2} \le 2$. Hence, $\frac{3 - q^{*(6)}_{j2}}{2} = 1$ and

$$\begin{aligned} q^{*(6)}_{j1} = \frac{3 - q^{*(6)}_{j2}}{2} +1 = 1 + 1 = 2 \end{aligned}$$

Therefore, the final result is $\mathbf {q}^{*(6)} = (2,1,2)$, which is, indeed, the $\mathbf {q}^{*(l)}$-vector listed for category $l=6$ in the table presented earlier.

1.5 The Polytomous Ideal Item Response

With these preliminaries in place, the polytomous ideal item response can then be defined as

$$\begin{aligned} \xi _{ij} = \max _{l \in \{0,1,\ldots , L_j\}} \left\{ l \prod _{a=1}^{A_j} I \Big [ \alpha ^{*}_{ia} \ge q^{*(l)}_{ja} \Big ] \right\} \end{aligned}$$

(12)

where $\varvec{\alpha }^{*}_i = (\alpha ^{*}_{i1}, \alpha ^{*}_{i2}, \ldots , \alpha ^{*}_{iA_j})^{\prime }$ is obtained by rearranging the entries of the examinee attribute vector $\varvec{\alpha }_i$ into the same order of the attributes as in $\mathbf {q}^{*}_j$; hence, $\varvec{\alpha }^{*}_i$ is of the same length as $\mathbf {q}^{*}_j$. Notice that the form of the term $\prod _{a=1}^{A_j} I \Big [ \alpha ^{*}_{ia} \ge q^{*(l)}_{ja} \Big ]$ is familiar from the definition of the conjunctive ideal response as it was introduced earlier for the binary DINA model (see Eq. (2). Furthermore, if for just a single attribute, $\alpha ^{*}_{ia} < q^{*(l)}_{ja}$, then the value of the indicator function is 0. Hence, regardless of the results of the comparisons for the remaining attributes, the term $\prod _{a=1}^{A_j} I \Big [ \alpha ^{*}_{ia} \ge q^{*(l)}_{ja} \Big ]$ equals zero, so that the category of the ideal response $\xi$ is $l=0$. (As was mentioned earlier, the polytomous DINA is a conjunctive model: all attributes required for an item must be mastered at least at level 1 for maximum probability of a correct response.) As an example, consider the attribute profile $\varvec{\alpha }_i = (1,2,3)$; hence, $\varvec{\alpha }^{*}_i = (1,2,3)$. The ideal response to item j having $\mathbf {q}^{*}_j = (q^{*}_{j1}, q^{*}_{j2}, q^{*}_{j3}) = (3,2,2)$ is then computed as:

$$\begin{aligned} \xi _{ij}= & {} \max _{l \in \{1,2,\ldots , L_j\}} \left\{ l \prod _{a=1}^{A_j}I \Big [ \alpha ^{*}_{ia} \ge q^{*(l)}_{ja} \Big ] \right\} \\= & {} \max \big \{ 1(1)1(1), 2(1)1(1), 3(1)1(1), 4(1)1(1), 5(0)1(1), 6(0)1(1), \ldots , 11(0)1(1), 12(0)1(1) \big \} \\= & {} 4 \end{aligned}$$

1.6 Perturbations (Formerly Known as “Slips” and “Guesses”

If the attributes $\alpha _1, \alpha _2, \ldots , \alpha _A$—and thus, $\xi$ and Y—have more than two levels, then describing the item parameters as “slips” and “guesses” does not adequately account for the increased complexity of potential discrepancies between $\xi$ and Y. Instead, whenever ideal and observed responses disagree, the more general term “perturbation” should be preferred, which also calls for adjusting the notation. Let l, as already established, denote the category of the ideal response $\xi _{ij}$ of examinee i to item j; also, let $l^{\prime }$ denote the category of her manifest response $Y_{ij}$ to this item. Two scenarios are possible: (i) the ideal response category disagrees with that of the manifest response to item j; (ii) the ideal response category matches the manifest response category. The corresponding probabilities are

$$\begin{aligned} \begin{array}{lcl} P(Y_{ij} = l^{\prime } \mid \xi _{ij} = l) &{} = &{} \left\{ \begin{array}{l} \epsilon _{jll^{\prime }} \text { if }l \ne l^{\prime }\\ 1-{\sum }_{l \ne l^{\prime }} \epsilon _{jll^{\prime }} \text { if }l = l^{\prime } \end{array} \right. \end{array} \end{aligned}$$

The IRF of the polytomous DINA model can then be expressed as

$$\begin{aligned} P(Y_{ij} = l^{\prime } \mid \varvec{\alpha }_i) = \left( 1 - \sum _{m \ne l^{\prime }} \epsilon _{j l^{\prime } m} \right) ^{I[\xi _{ij} = l^{\prime }]} \prod _{m \ne l^{\prime }} \epsilon _{j m l^{\prime }}^{I[\xi _{ij} = m]} \end{aligned}$$

where $m \in \{ 0,1, \ldots , L_j \}$. As an illustration, consider again the previous example. The table below summarizes for all combinations of the categories of l and $l^{\prime }$ the corresponding item parameters $\epsilon _j$:

	$l^{\prime }$ of $Y_{ij}$
l of $\xi _{ij}$	0	1	2	$\ldots$	12
0	$1-\sum _{m \ne 0} \epsilon _{j 0 m}$	$\epsilon _{j01}$	$\epsilon _{j02}$	$\ldots$	$\epsilon _{j0(12)}$
1	$\epsilon _{j10}$	$1-\sum _{m \ne 1} \epsilon _{j 1 m}$	$\epsilon _{j12}$	$\ldots$	$\epsilon _{j1(12)}$
2	$\epsilon _{j20}$	$\epsilon _{j21}$	$1-\sum _{m \ne 2} \epsilon _{j 2 m}$	$\ldots$	$\epsilon _{j2(12)}$
$\vdots$	$\vdots$			$\ddots$
12	$\epsilon _{j(12)0}$	$\epsilon _{j(12)1}$	$\epsilon _{j(12)2}$	$\ldots$	$1-\sum _{m \ne 12} \epsilon _{j (12) m}$

where the diagonal entries spelled out in full read as

$$\begin{aligned} l = l^{\prime } = 0:&1-\epsilon _{j01}-\epsilon _{j02}-\epsilon _{j03}-\epsilon _{j04}-\epsilon _{j05} -\epsilon _{j06}-\epsilon _{j07}-\epsilon _{j08}-\epsilon _{j09}\\&-\epsilon _{j0(10)} -\epsilon _{j0(11)}-\epsilon _{j0(12)}\\ l = l^{\prime } = 1:&1-\epsilon _{j10}-\epsilon _{j12}-\epsilon _{j13} - \cdots - \epsilon _{j1(12)}\\ l = l^{\prime } = 2:&1-\epsilon _{j20}-\epsilon _{j21}-\epsilon _{j23} - \cdots - \epsilon _{j2(12)}\\ \vdots&\\ l = l^{\prime } = 12:&1-\epsilon _{j(12)0}-\epsilon _{j(12)2}-\epsilon _{j(12)3} - \cdots - \epsilon _{j(12)(12)} \end{aligned}$$

As an aside, notice that the form of the main diagonal entries, $1-\sum _{m \ne l^{\prime }} \epsilon _{j l^{\prime }m}$, is slightly reminiscent of the term $1-s_j$ in the binary DINA model. In returning to the previous example, with $\xi _{ij} = 4$, given $\mathbf {q}_j = (3,2,2)$ and $\varvec{\alpha }_i = \varvec{\alpha }^{*}_i = (1,2,3)$, the IRF of the polytomous DINA model returns the following probabilities for the response categories of $Y_{ij}$:

$$\begin{aligned} P(Y_{ij} = 0 \mid \varvec{\alpha }_i)= & {} \left( 1-\sum _{m \ne 0} \epsilon _{j 0 m} \right) ^{I[\xi _{ij} = 0]} \epsilon _{j10}^{I[\xi _{ij} = 1]} \epsilon _{j20}^{I[\xi _{ij} = 2]} \epsilon _{j30}^{I[\xi _{ij} = 3]} \epsilon _{j40}^{I[\xi _{ij} = 4]} \epsilon _{j50}^{I[\xi _{ij} = 5]} \epsilon _{j60}^{I[\xi _{ij} = 6]} \\&\times \epsilon _{j70}^{I[\xi _{ij} = 7]} \epsilon _{j80}^{I[\xi _{ij} = 8]} \epsilon _{j90}^{I[\xi _{ij} = 9]} \epsilon _{j(10)0}^{I[\xi _{ij} = 10]} \epsilon _{j(11)0}^{I[\xi _{ij} = 11]} \epsilon _{j(12)0}^{I[\xi _{ij} = 12]} \\= & {} \epsilon _{j40}\\ P(Y_{ij} = 1 \mid \varvec{\alpha }_i)= & {} \epsilon _{j01}^{I[\xi _{ij}=0]} \left( 1-\sum _{m \ne 1} \epsilon _{j 1 m} \right) \epsilon _{j21}^{I[\xi _{ij} = 2]} \epsilon _{j31}^{I[\xi _{ij} = 3]} \epsilon _{j41}^{I[\xi _{ij} = 4]} \epsilon _{j51}^{I[\xi _{ij} = 5]} \ldots \epsilon _{j(11)1}^{I[\xi _{ij} = 11]} \epsilon _{j(12)1}^{I[\xi _{ij} = 12]} \\= & {} \epsilon _{j41}\\ P(Y_{ij} = 2 \mid \varvec{\alpha }_i)= & {} \epsilon _{j02}^{I[\xi _{ij}=0]} \epsilon _{j12}^{I[\xi _{ij}=1]} \left( 1-\sum _{m \ne 2} \epsilon _{j 2 m}\right) ^{I[\xi _{ij}=2]} \epsilon _{j32}^{I[\xi _{ij}=3]} \epsilon _{j42}^{I[\xi _{ij}=4]} \epsilon _{j52}^{I[\xi _{ij} = 5]} \ldots \epsilon _{j(11)2}^{I[\xi _{ij} = 11]} \epsilon _{j(12)2}^{I[\xi _{ij}=12]} \\= & {} \epsilon _{j42}\\ P(Y_{ij} = 3 \mid \varvec{\alpha }_i)= & {} \epsilon _{j43}\\ P(Y_{ij} = 4 \mid \varvec{\alpha }_i)= & {} \epsilon _{j04}^{I[\xi _{ij}=0]} \epsilon _{j14}^{I[\xi _{ij}=1]} \epsilon _{j24}^{I[\xi _{ij}=2]} \epsilon _{j34}^{I[\xi _{ij}=3]} \left( 1-\sum _{m \ne 4} \epsilon _{j 4 m} \right) ^{I[\xi _{ij}=4]} \epsilon _{j54}^{I[\xi _{ij}=5]} \ldots \epsilon _{j(11)4}^{I[\xi _{ij} = 11]} \epsilon _{j(12)4}^{I[\xi _{ij}=12]} \\= & {} \left( 1-\sum _{m \ne 4} \epsilon _{j 4 m}\right) ^{I[\xi _{ij}=4]}\\ \vdots&\\ P(Y_{ij} = 11 \mid \varvec{\alpha }_i)= & {} \epsilon _{j4(11)}\\ P(Y_{ij} = 12 \mid \varvec{\alpha }_i)= & {} \epsilon _{j4(12)} \end{aligned}$$