Psychometrika

, Volume 59, Issue 1, pp 59–75 | Cite as

Optimal sequential designs for on-line item estimation

  • Douglas H. Jones
  • Zhiying Jin
Article
Received:
Revised:

Abstract

Replenishing item pools for on-line ability testing requires innovative and efficient data collection designs. By generating localD-optimal designs for selecting individual examinees, and consistently estimating item parameters in the presence of error in the design points, sequential procedures are efficient for on-line item calibration. The estimating error in the on-line ability values is accounted for with an item parameter estimate studied by Stefanski and Carroll. LocallyD-optimaln-point designs are derived using the branch-and-bound algorithm of Welch. In simulations, the overall sequential designs appear to be considerably more efficient than random seeding of items.

Key words

branch-and-bound computerized adaptive test exactn-pointD-optimal integer programming item response theory measurement errors model on-line testing sequential design 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abramowitz, M., & Stegun, I. A. (Eds.). (1970).Handbook of mathematical functions. New York: Dover.Google Scholar
  2. Berger, Martijn P. F. (1991). On the efficiency of IRT models when applied to different sampling designs.Applied Psychological Measurement, 15, 293–306.Google Scholar
  3. Berger, Martijn P. F., & van der Linden, W. J. (1991). Optimality of sampling designs in item response theory models. In M. Wilson (Ed.),Objective measurement: Theory into practice. Norwood NJ: Ablex Publishing.Google Scholar
  4. Box, G. E. P., & Draper, N. R. (1959). A basis for the selection of a response surface design.Journal of the American Statistical Association, 54, 622–653.Google Scholar
  5. Donev, A. N., & Atkinson, A. C. (1988). An adjustment algorithm for the construction of exact D-optimum experimental designs.Technometrics, 30, 429–433.Google Scholar
  6. Federov, V. V. (1972).Theory of optimal experiments. New York: Academic Press.Google Scholar
  7. Ford, I. (1976).Optimal static and sequential design: A critical review. Unpublished doctoral dissertation, University of Glasgow.Google Scholar
  8. Ford, I., Kitsos, C. P., & Titterington, D. M. (1989). Recent advances in nonlinear experimental design.Technometrics, 31, 49–60.Google Scholar
  9. Ford, I., & Silvey, S. D. (1980). A sequentially constructed design for estimating a nonlinear parametric function.Biometrika, 67, 381–388.Google Scholar
  10. Ford, I., Titterington, D. M., & Wu, C. F. J. (1985). Inference and sequential design.Biometrika, 72, 545–551.Google Scholar
  11. Fuller, W. A. (1987).Measurement error models. New York: John Wiley & Sons.Google Scholar
  12. Haines, L. M. (1987). The application of the annealing algorithm to the construction of exact optimal designs for linear-regression models.Technometrics, 29, 439–447.Google Scholar
  13. Lord, F. M. (1971). Tailored testing, an application of stochastic approximation.Journal of the American Statistical Association, 66, 707–711.Google Scholar
  14. Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillside, NJ: Erlbaum.Google Scholar
  15. Mitchell, T. J. (1974). An algorithm for the construction of D-optimal experimental designs.Technometrics, 16, 203–210.Google Scholar
  16. Silvey, S. D. (1980).Optimal design. New York: Chapman and Hall.Google Scholar
  17. Stefanski, L. A., & Carroll, R. J. (1985). Covariate measurement error in logistic regression.Annals Statistics, 13, 1335–1351.Google Scholar
  18. Steinberg, D. M., & Hunter, W. G. (1984). Experimental design: Review and comment.Technometrics, 26, 71–130.Google Scholar
  19. Stocking, M. L. (1990). Specifying optimum examinees for item parameter estimation in item response theory.Psychometrika, 55, 461–475.Google Scholar
  20. Vale, C. D. (1986). Linking item parameters onto a common scale.Applied Psychological Measurement, 10, 333–344.Google Scholar
  21. van der Linden, W. J. (1988).Optimizing incomplete sampling designs for item response model parameters (Research Report No. 88-5). Enschede, The Netherlands: University of Twente.Google Scholar
  22. Welch, W. J. (1982). Branch-and-bound search for experimental designs based on D optimality and other criteria.Technometrics, 24, 41–48.Google Scholar
  23. Wingersky, M. S., & Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures.Applied Psychological Measurement, 8, 347–364.Google Scholar
  24. Wu, C. F. J. (1985). Asymptotic inference from sequential design in a nonlinear situation.Biometrika, 72, 553–558.Google Scholar

Copyright information

© The Psychometric Society 1994

Authors and Affiliations

  • Douglas H. Jones
    • 1
    • 2
  • Zhiying Jin
    • 1
    • 2
  1. 1.Graduate School of ManagementRutgers
  2. 2.The State UniversityNewark

Personalised recommendations