Abstract
Replenishing item pools for on-line ability testing requires innovative and efficient data collection designs. By generating localD-optimal designs for selecting individual examinees, and consistently estimating item parameters in the presence of error in the design points, sequential procedures are efficient for on-line item calibration. The estimating error in the on-line ability values is accounted for with an item parameter estimate studied by Stefanski and Carroll. LocallyD-optimaln-point designs are derived using the branch-and-bound algorithm of Welch. In simulations, the overall sequential designs appear to be considerably more efficient than random seeding of items.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramowitz, M., & Stegun, I. A. (Eds.). (1970).Handbook of mathematical functions. New York: Dover.
Berger, Martijn P. F. (1991). On the efficiency of IRT models when applied to different sampling designs.Applied Psychological Measurement, 15, 293–306.
Berger, Martijn P. F., & van der Linden, W. J. (1991). Optimality of sampling designs in item response theory models. In M. Wilson (Ed.),Objective measurement: Theory into practice. Norwood NJ: Ablex Publishing.
Box, G. E. P., & Draper, N. R. (1959). A basis for the selection of a response surface design.Journal of the American Statistical Association, 54, 622–653.
Donev, A. N., & Atkinson, A. C. (1988). An adjustment algorithm for the construction of exact D-optimum experimental designs.Technometrics, 30, 429–433.
Federov, V. V. (1972).Theory of optimal experiments. New York: Academic Press.
Ford, I. (1976).Optimal static and sequential design: A critical review. Unpublished doctoral dissertation, University of Glasgow.
Ford, I., Kitsos, C. P., & Titterington, D. M. (1989). Recent advances in nonlinear experimental design.Technometrics, 31, 49–60.
Ford, I., & Silvey, S. D. (1980). A sequentially constructed design for estimating a nonlinear parametric function.Biometrika, 67, 381–388.
Ford, I., Titterington, D. M., & Wu, C. F. J. (1985). Inference and sequential design.Biometrika, 72, 545–551.
Fuller, W. A. (1987).Measurement error models. New York: John Wiley & Sons.
Haines, L. M. (1987). The application of the annealing algorithm to the construction of exact optimal designs for linear-regression models.Technometrics, 29, 439–447.
Lord, F. M. (1971). Tailored testing, an application of stochastic approximation.Journal of the American Statistical Association, 66, 707–711.
Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillside, NJ: Erlbaum.
Mitchell, T. J. (1974). An algorithm for the construction of D-optimal experimental designs.Technometrics, 16, 203–210.
Silvey, S. D. (1980).Optimal design. New York: Chapman and Hall.
Stefanski, L. A., & Carroll, R. J. (1985). Covariate measurement error in logistic regression.Annals Statistics, 13, 1335–1351.
Steinberg, D. M., & Hunter, W. G. (1984). Experimental design: Review and comment.Technometrics, 26, 71–130.
Stocking, M. L. (1990). Specifying optimum examinees for item parameter estimation in item response theory.Psychometrika, 55, 461–475.
Vale, C. D. (1986). Linking item parameters onto a common scale.Applied Psychological Measurement, 10, 333–344.
van der Linden, W. J. (1988).Optimizing incomplete sampling designs for item response model parameters (Research Report No. 88-5). Enschede, The Netherlands: University of Twente.
Welch, W. J. (1982). Branch-and-bound search for experimental designs based on D optimality and other criteria.Technometrics, 24, 41–48.
Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures.Applied Psychological Measurement, 8, 347–364.
Wu, C. F. J. (1985). Asymptotic inference from sequential design in a nonlinear situation.Biometrika, 72, 553–558.
Author information
Authors and Affiliations
Additional information
This report was prepared under the Navy Manpower, Personnel, and Training R&D Program of the Office of the Chief of Naval Research under Contract N00014-87-0696. The authors wish to acknowledge the valuable advice and consultation given by Ronald Armstrong, Charles Davis, Bradford Sympson, Zhaobo Wang, Ing-Long Wu and three anonymous reviewers.
Rights and permissions
About this article
Cite this article
Jones, D.H., Jin, Z. Optimal sequential designs for on-line item estimation. Psychometrika 59, 59–75 (1994). https://doi.org/10.1007/BF02294265
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294265