New Approaches to Equating With Small Samples

Abstract

The purpose of this chapter is to introduce the reader to some recent innovations intended to solve the problem of equating test scores on the basis of data from small numbers of test takers. We begin with a brief description of the problem and of the techniques that psychometricians now use in attempting to deal with it. We then describe three new approaches to the problem, each dealing with a different stage of the equating process: (1) data collection, (2) estimating the equating relationship from the data collected, and (3) using collateral information to improve the estimate. We begin with Stage 2, describing a new method of estimating the equating transformation from small-sample data. We also describe the type of research studies we are using to evaluate the effectiveness of this new method. Then we move to Stage 3, describing some procedures for using collateral information from other equatings to improve the accuracy of an equating based on small-sample data. Finally, we turn to Stage 1, describing a new data collection plan in which the new form is introduced in a series of stages rather than all at once.

Chapter 7 Appendix

1.1 A.1 Formulas for Circle-Arc Equating

In the symmetric circle-arc method, the estimated equating curve is an arc of a circle. Let $({x_1},{y_1})$ represent the lower end point of the equating curve, let $({x_2},{y_2})$ represent the empirically determined middle point, and let $({x_3},{y_3})$ represent the upper end point. Let r represent the radius of the circle, and label the coordinates of its center $({x_c},{y_c})$.

The equation of the circle is ${(X - {x_c})^2} + {(Y - {y_c})^2} = {r^2}$ or, equivalently, $\left| {Y - {y_c}} \right| = \sqrt {{r^2} - {{(X - {x_c})}^2}}$. If the new form is harder than the reference form, the middle point will lie above the line connecting the lower and upper points, so that the center of the circle will be below the arc. For all points $(X,Y)$ on the arc, $Y > {y_c}$, so that $\left| {Y - {y_c}} \right| = Y - {y_c}$, and the formula for the arc will be

$$Y = {y_c} + \sqrt {{r^2} - {{(X - {x_c})}^2}}.$$

(7.A.1)

If the new form is easier than the reference form, the middle point will lie below the line connecting the lower and upper end points, so that the center of the circle will be above the arc. For all points (X,Y) on the arc, $Y < {y_c}$, so that $\left| {Y - {y_c}} \right| = {y_c} - Y$, and the formula for the arc will be

$$Y = {y_c} - \sqrt {{r^2} - {{(X - {x_c})}^2}}.$$

(7.A.2)

A simple decision rule is to use Equation 7.A.1 if ${y_2} > {y_c}$ and Equation 7.A.2 if ${y_2} < {y_c}$.

The formulas for x _c and y _c in the symmetric circle-arc method are a bit cumbersome:

$${x_c} = \frac{{(x_1^2 + y_1^2)({y_3} - {y_2}) + (x_2^2 + y_2^2)({y_1} - {y_3}) + (x_3^2 + y_3^2)({y_2} - {y_1})}}{{2\left[ {{x_1}({y_3} - {y_2}) + {x_2}({y_1} - {y_3}) + {x_3}({y_2} - {y_1})} \right]}}$$

(7.A.3)

and

$${y_c} = \frac{{(x_1^2 + y_1^2)({x_3} - {x_2}) + (x_2^2 + y_2^2)({x_1} - {x_3}) + (x_3^2 + y_3^2)({x_2} - {x_1})}}{{2\left[ {{y_1}({x_3} - {x_2}) + {y_2}({x_1} - {x_3}) + {y_3}({x_2} - {x_1})} \right]}},$$

(7.A.4)

but the formula for r ² is simply

$${r^2} = {({x_1} - {x_c})^2} + {({y_1} - {y_c})^2}.$$

(7.A.5)

In the simplified circle-arc method, the transformed points to be connected by a circle arc are $({x_1},0)$, $({x_2},{y_2}^*)$, and $({x_3},0)$, where

$${y_2}^* = {y_2} - \left( {\frac{{{y_3} - {y_1}}}{{{x_3} - {x_1}}}} \right)\left( {{x_2} - {x_1}} \right).$$

(7.A.6)

The transformation of the data points results in a much simpler set of formulas for the coordinates of the center of the circle:

$${x_c} = \frac{x_1 +x_3}{2},$$

(7.A.7)

$${y_c} = \frac{{(x_1^2)({x_3} - {x_2}) - (x_2^2 + {{({y_2}^*)}^2})({x_3} - {x_1}) + (x_3^2)({x_2} - {x_1})}}{{2\left[ {{y_2}^*({x_1} - {x_3})} \right]}},$$

(7.A.8)

and a slightly simpler formula for r ²:

$${r^2} = {({x_1} - {x_c})^2} + {y_c}^2.$$

(7.A.9)

Author Note: Any opinions expressed in this chapter are those of the author and not necessarily of Educational Testing Service.