Latent Variable Selection for Multidimensional Item Response Theory Models via $L_{1}$ Regularization

Abstract

We develop a latent variable selection method for multidimensional item response theory models. The proposed method identifies latent traits probed by items of a multidimensional test. Its basic strategy is to impose an $L_{1}$ penalty term to the log-likelihood. The computation is carried out by the expectation–maximization algorithm combined with the coordinate descent algorithm. Simulation studies show that the resulting estimator provides an effective way in correctly identifying the latent structures. The method is applied to a real dataset involving the Eysenck Personality Questionnaire.

Appendix

The cyclical coordinate descent algorithm for solving the optimization (12) is introduced as follows. For each item j,  there are one difficulty parameter $b_j$ and K discrimination parameters $\mathbf {a}_j=(a_{j1},\ldots ,a_{jK}).$ The algorithm update each of the $K+1$ variables iteratively according to the following updating rule. For the difficulty parameter, there is no $L_1$ penalty and it is updated by

$$\begin{aligned} {\hat{b}_{j}}=b_{j}^{*}-\frac{\partial _{b_{j}}\hat{Q}_{j}({\mathbf {a}_{j}},\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})}{\partial ^2 _{b_{j}}\hat{Q}_{j}({\mathbf {a} _{j}},\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})}, \end{aligned}$$

where $\partial \hat{Q}_{j}$ denotes derivative of $\hat{Q}_{j}({\mathbf {a}_{j}} ,\,b_{j}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})$ with respect to $b_{j}$ or $a_{jk}$ as labeled by the subscript and $\partial ^{2}{\hat{Q}_{j}}$ is the second derivative. During the above update, the discrimination vector $\mathbf {a}_j$ takes its most up-to-date value. The above update employs a local quadratic approximation of $\hat{Q}_{j}({\mathbf {a}_{j}},\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})$ as a function of $b_j$ with all the other variables fixed. For each discrimination parameter $a_{jk},$ an $L_1$ penalty is imposed and it is updated by

$$\begin{aligned} {\hat{a}_{jk}}={-}\frac{{S(- \partial ^{2}{_{a{{_{jk}}}}\hat{Q}_{j}({\mathbf {a}_{j} },\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})\times a_{jk}^{*}+\partial _{a{ {_{jk}}}}\hat{Q}_{j}({\mathbf {a}_{j}},\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)}),\,{ \eta }}) }}{\partial ^{2}{_{a{{_{jk}}}}\hat{Q}_{j}({\mathbf {a}_{j}},\,b_{j}^{*}|\mathbf {a}_{j}^{(t)},\,b_{j}^{(t)})}}, \end{aligned}$$

The function S is the soft threshold operator (Donoho & Johnstone, 1995):

$$\begin{aligned} S(\delta ,\,\eta )={\text {sign}}(\delta ){(|\delta |-\eta )_{+}}=\left\{ \begin{array}{ll} \delta -\eta , &{} \text {if}\,\delta >0\,\text {and}\,\eta<|\delta |, \\ \delta +\eta , &{} \text {if}\,\delta<0\,\text {and}\,\eta <|\delta |, \\ 0, &{} \text {if}\,\eta \ge |\delta |. \end{array} \right. . \end{aligned}$$

To obtain the above updating rule, we approximate a generic univariate function f(x) by a quadratic function

$$\begin{aligned} f(x) \approx f\left( x_0\right) + f^{\prime }\left( x_0\right) \left( x-x_0\right) + \frac{f^{\prime \prime }(x_0)}{2} \left( x-x_0\right) ^2, \end{aligned}$$

where $f^{\prime \prime }(x_0)$ is negative. Furthermore, the $L_1$-penalized maximization with the approximated function

$$\begin{aligned} \sup _x \left\{ f\left( x_0\right) + f^{\prime }\left( x_0\right) \left( x-x_0\right) + \frac{f^{\prime \prime }(x_0)}{2} \left( x-x_0\right) ^2 - \eta |x|\right\} , \end{aligned}$$

is solved at

$$\begin{aligned} -\frac{S(- f^{\prime \prime }(x_0) x_0 + f^{\prime }(x_0),\,\eta )}{f^{\prime \prime }(x_0)}. \end{aligned}$$