Abstract
In order to identify aberrant response-time patterns on educational and psychological tests, it is important to be able to separate the speed at which the test taker operates from the time the items require. A lognormal model for response times with this feature was used to derive a Bayesian procedure for detecting aberrant response times. Besides, a combination of the response-time model with a regular response model in an hierarchical framework was used in an alternative procedure for the detection of aberrant response times, in which collateral information on the test takers’ speed is derived from their response vectors. The procedures are illustrated using a data set for the Graduate Management Admission Test® (GMAT®). In addition, a power study was conducted using simulated cheating behavior on an adaptive test.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albert, J.H. (1992). Bayesian estimation of normal-ogive item response curves using Gibbs sampling. Journal of Educational and Behavioral Statistics, 17, 261–269.
Bradlow, E.T., Weiss, R.E., & Cho, M. (1998). Bayesian detection of outliers in computerized adaptive tests. Journal of the American Statistical Association, 93, 910–919.
Casella, G., & Berger, R.L. (2002). Statistical inference (2nd ed.). Pacific Grove: Duxbury.
Chang, H.-H., & Stout, W. (1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37–52.
Fisher, R.A. (1925). Statistical methods for research workers. Edinburgh: Oliver & Boyd.
Gelman, A., Carlin, J.B, Stern, H., & Rubin, D.B. (1995). Bayesian data analysis. London: Chapman & Hall.
Glas, C.A.W., & Meijer, R.R. (2003). A Bayesian approach to person fit analysis in item response theory models. Applied Psychological Measurement, 27, 217–233.
Johnson, V.E., & Albert, J.H. (1999). Ordinal data modeling. New York: Springer.
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
Meijer, R.R., & Sijtsma, K. (1995). Detection of aberrant item response patterns: A review of recent developments. Applied Measurement in Education, 8, 261–272.
Meijer, R.R., & Sijtsma, K. (2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107–135.
Miller, G.A. (1956). The magic number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97.
Owen, R.J. (1969). A Bayesian approach to tailored testing (Research Report 69-92). Princeton, NJ, Educational Testing Service.
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351–356.
van der Linden, W.J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.
van der Linden, W.J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.
van der Linden, W.J. (2008). Using response times for item selection in adaptive tests. Journal of Educational and Behavioral Statistics, 33. In press.
van der Linden, W.J., & van Krimpen-Stoop, E.M.L.A. (2003). Using response times to detect aberrant response patterns in computerized adaptive testing. Psychometrika, 68, 251–265.
van der Linden, W.J., Scrams, D.J., & Schnipke, D.L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210.
van Krimpen-Stoop, E.M.L.A., & Meijer, R.R. (2001). CUSUM-based person fit statistics for adaptive testing. Journal of Educational and Behavioral Statistics, 26, 199–218.
Author information
Authors and Affiliations
Corresponding author
Additional information
The authors have relied upon data supplied by the Graduate Management Admission Council® (GMAC®) to conduct the independent research that forms the basis for the findings and conclusions stated in this article. These findings and conclusions are the opinion of the authors only, and do not necessarily reflect the opinion of the GMAC®. The authors are indebted to Wim M.M. Tielen and Rinke H. Klein Entink for their computational support.
Rights and permissions
About this article
Cite this article
van der Linden, W.J., Guo, F. Bayesian Procedures for Identifying Aberrant Response-Time Patterns in Adaptive Testing. Psychometrika 73, 365–384 (2008). https://doi.org/10.1007/s11336-007-9046-8
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-007-9046-8