Using response times to detect aberrant responses in computerized adaptive testing
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
A lognormal model for response times is used to check response times for aberrances in examinee behavior on computerized adaptive tests. Both classical procedures and Bayesian posterior predictive checks are presented. For a fixed examinee, responses and response times are independent; checks based on response times offer thus information independent of the results of checks on response patterns. Empirical examples of the use of classical and Bayesian checks for detecting two different types of aberrances in response times are presented. The detection rates for the Bayesian checks outperformed those for the classical checks, but at the cost of higher false-alarm rates. A guideline for the choice between the two types of checks is offered.
- Bradlow, E.T., Weiss, R. E., & Cho, M. (1998). Bayesian detection of outliers in computerized adaptive tests.Journal of the American Statistical Association, 93, 910–919.
- Drasgow, F., Levine, M.V., Williams, E.A. (1985). Appropriateness measurement with polytomous item reponse models and standardized indices.British Journal of Mathematical and Statistical Psychology, 38, 67–86.
- Gelman, A., Carlin, J.B., Stern, H., & Rubin, D.B. (1995).Bayesian data analysis. London, U.K.: Chapman & Hall.
- Johnson, V.E., & Albert, J.H. (1999).Ordinal data modeling. New York, NY: Springer-Verlag.
- Levine, M.V., & Rubin, D.B. (1979). Measuring the appropriateness of multiple-choice test scores.Journal of Educational Statistics, 4, 269–290.
- Meijer, R.R., & Sijtsma, K. (1995). Detection of aberrant item response patterns: A review of recent developments.Applied Measurement in Education, 8, 261–272.
- Mislevy, R.J., & Chang, H. (2000). Does adaptive testing violate local independence?Psychometrika, 65, 149–156.
- Mislevy, R.J., & Wu, P.-K. (1996).Missing responses and Bayesian IRT estimation: Omits, choice, time limits, and adaptive testing (Research Rep. RR-96-30-ONR). Princeton, NJ: Educational Testing Service.
- Molenaar, I.W., & Hoijtink, H. (1990). The many null distributions of person-fit statistics.Psychometrika, 55, 75–106.
- Neter, J., Wasserman, W., & Kutner, M.H. (1985).Applied linear statistical models: Regression, analysis of variance, and experimental designs. Homewood, IL: Richard D. Irwin.
- Segall, D.O., Moreno, K.E., & Hetter, D.H. (1997). In W.A. Sands, B.K. Waters, & J.R. McBride (Eds.),Computerized adaptive testing: From inquiry to operation (pp. 117–130). Washington, DC: American Psychological Association.
- Schnipke, D.L., & Scrams, D.J. (1997).Representing response time information in item banks (LSAC Computerized Testing Rep. No. 97-09). Newtown, PA: Law School Admission Council.
- Thissen, D. (1983). Timed testing: An approach using item resonse theory. In D.J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179–203). New York, NY: Academic Press.
- Trabin, T.E., & Weiss, D.J. (1983). The person response curve: Fit of individuals to item response theory models. In D.J. Weiss (Ed.),New horizons in testing: Latent trait theory and computerized adaptive testing. New York, NY: Academic Press.
- van der Linden, W.J. (2002).A model for speed and accuracy on tests. Unpublished manuscipt.
- van der Linden, W.J., & Pashley, P.J. (2000). Item selection and ability estimation in adaptive testing. In W.J. van der Linden & C.A.W. Glas (Eds.),Computerized adaptive testing: Theory and practice (pp. 1–25). Norwell, MA: Kluwer Academic Publishers.
- van der Linden, W.J., Scrams, D.J., & Schnipke, D.L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing.Applied Psychological Measurement, 23, 195–210.
- van Krimpen-Stoop, E.M.L.A., & Meijer, R.R. (1999). Simulating the null distribution of person-fit statistics for conventional and adaptive tests.Applied Psychological Measurement, 23, 327–345.
- van Krimpen-Stoop, E.M.L.A., & Meijer, R.R. (2000). Detecting person misfit in adaptive testing using statistical process control techniques. In W.J. van der Linden & C.A.W. Glas. (Eds.),Computerized adaptive testing: Theory and practice (pp. 221–219). Norwell, MA: Kluwer Academic Publishers.
- Using response times to detect aberrant responses in computerized adaptive testing
Volume 68, Issue 2 , pp 251-265
- Cover Date
- Print ISSN
- Online ISSN
- Additional Links
- aberrant response patterns
- computerized adaptive testing
- posterior predictive checks
- person misfit
- residual analysis
- response times
- Industry Sectors