Methods for Estimating Conditional Standard Errors of Measurement and Some Critical Reflections

Abstract

Educational assessments can have far-reaching consequences for individuals. To allow test users to make valid decisions, it is important to provide evidence about the uncertainties in the observed scores on which the individual decisions are based. In this chapter we examine standard errors of measurement defined for specific score groups, which are referred to as conditional standard errors of measurement. In particular, we study the foundations of the ANOVA method proposed by Feldt et al. (Appl Psychol Meas 9:351–361, 1985) within the context of classical test theory. In addition, we suggest some variations and study their practical usefulness including sample size requirements.

Appendix

1.1 Proof of Eq. 11.8

We start from the well-known definition of the standard error of measurement; that is,

$$ {S}_E^2\left({\lambda}_3\right)=\left(1-\alpha \right)\bullet {S}_X^2, $$

(11.A1)

where α is coefficient alpha and ${S}_X^2$ the variance of the total scores across persons. Because $\alpha \equiv ICC\left(3,J\right)=\left[1-\frac{M{S}_{N\times J}}{M{S}_s}\right]$ (Eq. 11.7), substituting the definition of ICC(3, J) for α gives

$$ {\sigma}_E^2\left({\lambda}_3\right)=\frac{M{S}_{N\times J}}{M{S}_s}\bullet {S}_X^2. $$

(11.A2)

Furthermore, we have $M{S}_s=\frac{J{\sum}_v{\overline{X}}_v^2- nJ{\overline{X}}^2}{n-1}$ (e.g., Brennan, 2001, p. 41), where ${\overline{X}}_v^2$ is the square average test score for an arbitrary person v. It can be shown – after some tedious algebra – that MS _s is equivalent with $\frac{\sum_v{X}_{+}^2-n{\overline{X}}_{+}^2}{J\left(n-1\right)}=\frac{S_X^2}{J}$, showing that MS _s can be conceived as the average variance of subjects across items. Substituting $\frac{S_X^2}{J}$ for MS _s in Eq. (11.A2) gives $\frac{M{S}_{N\times J}}{\frac{S_X^2}{J}}\bullet {S}_X^2= JM{S}_{N\times J}$, and that completes the proof.