Saddlepoint Approximations of the Distribution of the Person Parameter in the Two Parameter Logistic Model

Abstract

Large sample theory states the asymptotic normality of the maximum likelihood estimator of the person parameter in the two parameter logistic (2PL) model. In short tests, however, the assumption of normality can be grossly wrong. As a consequence, intended coverage rates may be exceeded and confidence intervals are revealed to be overly conservative. Methods belonging to the higher-order-theory, more specifically saddlepoint approximations, are a convenient way to deal with small-sample problems. Confidence bounds obtained by these means hold the approximate confidence level for a broad range of the person parameter. Moreover, an approximation to the exact distribution permits to compute median unbiased estimates (MUE) that are as likely to overestimate as to underestimate the true person parameter. Additionally, in small samples, these MUE are less mean-biased than the often-used maximum likelihood estimator.

Appendices

Appendix A. Derivation of the Saddlepoint Approximation for the 2PL

The probability of the response vector x _v in the 2PL is

$$ P(\mathbf{x}_v|\theta_v,{\boldsymbol{\beta}}, {\boldsymbol{\alpha}})=\frac{\exp(\theta_v \sum_{i=1}^n x_{vi}\alpha_i-\sum_{i=1}^n x_{vi} \alpha_i \beta_i)}{\prod_{i=1}^n [1+\exp\{\alpha_{i} (\theta_v-\beta_i)\}]}, $$

(A.1)

and the probability to obtain the weighted sum score $\sum_{i=1}^{n} x_{vi}\alpha_{i}$ is

$$ \begin{aligned}[b] P\Biggl(\sum_{i=1}^n x_{vi}\alpha_i\Big|\theta_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}}\Biggr)=&\sum_{\mathbf{x}_v|\sum x_{vi}\alpha_i}P(\mathbf{x}_v| \theta_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}}) \\ =&\frac{\exp(\theta_v\sum_{i=1}^nx_{vi}\alpha_i)\sum_{\mathbf{x}_v|\sum x_{vi}\alpha_i}\exp(-\sum_{i=1}^nx_{vi}\alpha_i\beta_i)}{\prod_{i=1}^n[1+\exp\{\alpha_{i}(\theta_v-\beta_i)\}]}, \end{aligned} $$

(A.2)

where $\sum_{\mathbf{x}_{v}|\sum x_{vi}\alpha_{i}}$ means summation over all response vectors having the same weighted sum score $\sum_{i=1}^{n} x_{vi}\alpha_{i}$. In the context of the Rasch model, this sum is called $\gamma_{r_{v}}$, the elementary symmetric function. If the weighted sums corresponding to the response vectors are all unique, formulae (A.1) and (A.2) agree exactly. Here we can observe an interesting feature of the exponential family, namely that the density of the data points x _v is of the same family as the density of the sufficient statistic (e.g., Casella & Berger 2002, p. 217)

$$ p(x;\theta)=\exp \bigl\{ \theta x-K(\theta) \bigr\} f_0(x). $$

(A.3)

The density of the sufficient statistic is then

$$ p(s;\theta)=\exp \bigl\{ \theta s-K(\theta) \bigr\} F_0(s), $$

(A.4)

with the sole difference given in functions f ₀(x) and F ₀(s). Comparing this to Equations (A.1) and (A.2), we see that $\exp(-\sum_{i}^{n}x_{vi}\alpha_{i}\beta_{i})=f_{0}(\mathbf{x}_{v})$ and $\sum_{\mathbf{x}_{v}|\sum x_{vi}\alpha_{i}}\exp(-\sum_{i=1}^{n} x_{vi}\alpha_{i}\beta_{i})=F_{0}(r_{v})$. In order to simplify the notation, the weighted sum score $\sum_{i=1}^{n} x_{vi}\alpha_{i}$ will be abbreviated in the sequel as w _v . Because in exponential family theory (e.g., Lehmann & Casella 1998; Davison 2003) the log of the normalizing constants, i.e., the log of the denominators of Equations (A.2) and (A.1), are the cumulant generating functions, we abbreviate $\log \prod_{i=1}^{n} [1+\exp \{\alpha_{i}(\theta_{v}-\beta_{i}) \} ]$ by K(θ _v ).

Exponential tilting is a technique that allows any density to be embedded in an exponential family, and, by tilting an exponential family, a density of the same family is obtained (Davison 2003, p. 168). Any arbitrary density can be multiplied by exp(sϕ), and, by renormalizing this product, a valid density function is obtained.

The distribution of a response vector x _v in the 2PL model is given in Equation (A.1). Multiplying this density by exp(w _v ϕ _v ) and normalizing the product yields

$$ P(\mathbf{x}_v|\theta_v+\phi_v,{\boldsymbol{\beta}}, {\boldsymbol{\alpha}})=\frac{\exp (\theta_v w_v-K(\theta_v) ) f_0(\mathbf{x}_v) \exp(w_v \phi_v)}{ \sum_{\mathbf{x}_v} \exp (\theta_v w_v-K(\theta_v) ) f_0(\mathbf{x}_v) \exp(w_v \phi_v)}. $$

(A.5)

The summation in the denominator takes place over all possible response vectors. The above equation can be expressed in a slightly different way by writing the term in the denominator not depending on w _v before the summation sign

$$ P(\mathbf{x}_v|\theta_v+ \phi_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}})=\frac{\exp(-K(\theta_v)) \exp ((\theta_v+\phi_v) w_v ) f_0(\mathbf{x}_v)}{ \exp(-K(\theta_v))\sum_{\mathbf{x}_v} \exp ((\theta_v+\phi_v) w_v ) f_0(\mathbf{x}_v)}. $$

(A.6)

Now, the first factors in the enumerator and denominator cancel, and from Equation (A.1) we have the result that the remaining term in the denominator must be

$$ \sum_{\mathbf{x}_v} \exp \bigl((\theta_v+ \phi_v) w_v \bigr) f_0( \mathbf{x}_v)=\prod_{i=1}^n \bigl[1+\exp \bigl\{ \alpha_i\bigl((\theta_v+ \phi_v)-\beta_i\bigr) \bigr\} \bigr]=\exp \bigl(K( \theta_v+\phi_v) \bigr). $$

(A.7)

Starting from Equation (A.6), the probability P(x _v |θ _v +ϕ _v ,β,α) can hence be written as

$$ P(\mathbf{x}_v|\theta_v+\phi_v,{\boldsymbol{\beta}}, {\boldsymbol{\alpha}})=P(\mathbf{x}_v|\theta_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}}) \exp \bigl\{ w_v \phi_v -\bigl(K(\theta_v+ \phi_v)-K(\theta_v)\bigr) \bigr\} . $$

(A.8)

This equation can be solved for the initial density P(x _v |θ _v ,β,α), and one obtains

$$ P(\mathbf{x}_v|\theta_v,{\boldsymbol{\beta}}, {\boldsymbol{\alpha}})=P(\mathbf{x}_v|\theta_v+\phi_v, {\boldsymbol{\beta}},{\boldsymbol{\alpha}}) \exp \bigl\{ K(\theta_v+\phi_v)-K( \theta_v)-w_v \phi_v \bigr\} . $$

(A.9)

By comparing Equation (A.9) to Equations (A.3) and (A.4), one sees that, in order to obtain an approximation to the density of the sufficient statistic w _v , we have to exchange P(x _v |θ _v ,β,α) with P(w _v |θ _v ,β,α) and likewise for P(x _v |θ _v +ϕ _v ,β,α). This last term can be approximated by the first term of an Edgeworth expansion of a standardized sum $S_{n}^{*}=(S_{n}-n\mu)/\sqrt{n}\sigma$ of independent variables Y ₁,…,Y _n (Pace & Salvan 1997). Here $S_{n} =\sum_{i}^{n} Y_{i}$, μ=E(Y) and σ ²=Var(Y). The Edgeworth expansion for such a sum is given by

$$ p_{S_n^*}(x)=\phi(x) \biggl(1+\frac{\rho_3}{6\sqrt{n}}H_3(x)+ \frac{\rho_4}{24n}H_4(x)+\frac{\rho_3^2}{72n}H_6(x)+O \bigl(n^{-\frac{3}{2}}\bigr) \biggr), $$

(A.10)

where $\rho_{r}(S_{n}^{*})=\kappa_{r}$ are the respective standardized cumulants. The first two cumulants are given by ρ ₁=0 and ρ ₂=1 and H _r (x) are Tchebycheff–Hermite Polynomials (e.g., Stuart & Ord 1987). The odd-ordered Hermite Polynomials vanish at x=0, the mean of the standardized sum. As can be seen, including only the first term of the above expansion approximates the density of the normalized sum as a normal density with error of order O(n ^−1/2) in accordance with the central limit theorem (e.g., Young & Smith 2005). Including the second and third term corrects for skewness and kurtosis and the error will drop to O(n ⁻¹), and with the fourth term to O(n ^−3/2). Since the Edgeworth expansion achieves greatest accuracy at the mean, where H ₃(x) vanishes and leaves an error of O(n ⁻¹), the mean of P(w _v |θ _v +ϕ _v ,β,α) is needed. Again, from the theory of exponential families we have that the expectation of a variable is given by the first derivative of the cumulant generating function. Equating the expectation to the sufficient statistic gives the so-called maximum likelihood equation

$$ w_v=\frac{\partial}{\partial\theta_v} K(\theta_v)=\sum _{i=1}^n\frac{\exp \{\alpha_i(\theta_v-\beta_i) \}}{1+\exp \{\alpha_i(\theta_v-\beta_i) \}} \alpha_i, $$

(A.11)

which is solved by the maximum likelihood estimator $\hat{\theta}_{v}$. Since the distribution P(w _v |θ _v +ϕ _v ,β,α) has the same sufficient statistic w _v , it is also maximized at $\hat{\theta}_{v}$, and hence $\phi_{v}=\hat{\theta}_{v}-\theta_{v}$. For standardized sums the first term of the Edgeworth expansion (Pace & Salvan 1997, Chap. 10) is the standard normal density. The Jacobian of the transformation (Casella & Berger 2002, pp. 120, 158) for the univariate case is the reciprocal of the standard deviation, which in turn is obtained as the square root of the second derivative of the cumulant generating function. Hence, the approximation to the desired density is written as

$$ P(w_v|\hat{\theta}_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}})= \frac{1}{\sqrt{2\pi K''(\hat{\theta}_v)}}+ \mathit{Rest}, $$

(A.12)

where Rest stands for the terms not accounted for. Note that $K''(\hat{\theta}_{v})$, the variance of variable w _v , may not be confounded with the asymptotic variance of the estimate $\hat{\theta}_{v}$, which of course is given by the reciprocal of $K''(\hat{\theta}_{v})$. Keep in mind that $\phi_{v}=\hat{\theta}_{v}-\theta_{v}$ so that Equation (A.9) can now be written as

$$ P(w_v|\theta_v,{\boldsymbol{\beta}},{\boldsymbol{\alpha}})=\frac{\exp \{w_v \theta_v-w_v\hat{\theta}_v+K(\hat{\theta}_v)-K(\theta_v) \}}{\sqrt{2\pi K''(\hat{\theta}_v)}} + \mathit{Rest}. $$

(A.13)

This is the saddlepoint approximation to the probability of a certain weighted sum score w _v for a given ability level θ _v and item parameters β, α.

Appendix B. Polynomial Interpolation

In order to obtain stable values of r ^∗ (Equation (16)), polynomial interpolation is done for values of θ that lie in an interval $\hat{\theta}\pm 0.5 \cdot j(\hat{\theta})^{-1/2}$. First, the term r for all values of θ (Equation (10)) is modeled as a tenth order polynomial in u (Equation (14))

$$ r=a_1 u+ a_2 u^2+ \cdots +a_{10} u^{10}. $$

(B.1)

The coefficients are fitted by the least-squares criterion. Next the fraction r/u is determined by

$$ \frac{r}{u}=a_1+a_2 u^1 + \cdots +a_{10} u^9. $$

(B.2)

After that the logarithm of this fraction in turn is modeled by a polynomial regression model with predictors given by the powers of r

$$ \log \frac{r}{u}=a_1 r+a_2 r^2+ \cdots+ a_{10} r^{10}. $$

(B.3)

The fraction of the logarithmic term and r is estimated by

$$ \log \biggl(\frac{r}{u} \biggr)\frac{1}{r}=a_1+a_2 r^1 +\cdots +a_{10} r^9. $$

(B.4)

The modified likelihood ratio is finally given by r minus the above term

$$ r^*=r-\log \biggl(\frac{r}{u} \biggr)\frac{1}{r}. $$

(B.5)