Using Deterministic, Gated Item Response Theory Model to Detect Test Cheating due to Item Compromise
 Zhan Shu,
 Robert Henson,
 Richard Luecht
 … show all 3 hide
Rent the article at a discount
Rent now* Final gross prices may vary according to local VAT.
Get AccessAbstract
The Deterministic, Gated Item Response Theory Model (DGM, Shu, Unpublished Dissertation. The University of North Carolina at Greensboro, 2010) is proposed to identify cheaters who obtain significant score gain on tests due to item exposure/compromise by conditioning on the item status (exposed or unexposed items). A “gated” function is introduced to decompose the observed examinees’ performance into two distributions (the true ability distribution determined by examinees’ true ability and the cheating distribution determined by examinees’ cheating ability). Test cheaters who have score gain due to item exposure are identified through the comparison of the two distributions. Hierarchical Markov Chain Monte Carlo is used as the model’s estimation framework. Finally, the model is applied in a real data set to illustrate how the model can be used to identify examinees having preknowledge on the exposed items.
 Angoff, W.H. (1974) The development of statistical indices for detecting cheaters. Journal of the American Statistical Association 69: pp. 4449 CrossRef
 Bellezza, F., Bellezza, S. (1995) Detection of copying on multiplechoice tests: an update. Teaching of Psychology 22: pp. 180182 CrossRef
 Cizek, G.J. (1999) Cheating on tests: how to do it, detect it, and prevent it. Lawrence Erlbaum Associates, Mahway
 Drasgow, F., Levine, M.V., Williams, E.A. (1985) Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical & Statistical Psychology 38: pp. 6786 CrossRef
 Drasgow, F., Levine, M.V. (1986) Optimal detection of certain forms of inappropriate test scores. Applied Psychological Measurement 10: pp. 5967 CrossRef
 Drasgow, F., Luecht, R.M., Bennett, R. Technology and testing. In: Brennan, R.L. eds. (2006) Educational measurement. American Council on Education/Praeger Publishers, Washington, pp. 471515
 Dwyer, D.J., Hecht, J.B. (1996) Using statistics to catch cheaters: methodological and legal issues for students personnel administrators. NASPA Journal 33: pp. 125135
 Frary, R., Tideman, N., Watts, T. (1977) Indices of cheating on multiple choice tests. Journal of Educational Statistics 2: pp. 235256 CrossRef
 Hanson, B., Harris, D., & Brennan, R. (1987). A comparison of several methods for examining allegations of copying (ACT Research Report No. 8715). Iowa City: American College Testing.
 Holand, P. (1996). Assessing unusual agreement between the incorrect answers of two examinees using the Kindex: statistical theory and empirical support ETS (Technical Report No. 964). Princeton: Educational Testing Service.
 Levine, M.V., Rubin, D.B. (1979) Measuring the appropriateness of multiplechoice test scores. Journal of Educational Statistics 4: pp. 269290 CrossRef
 Lewis, C., & Thayer, D.T. (1998). The power of Kindex to detect test copying (Research Report No. 08541). Princeton: Educational Testing Service.
 Luecht, R.M. (1998). A framework for exploring and controlling risks associated with test item exposure over time. Paper presented at the national council on measurement in education annual meeting, San Diego.
 Luecht, R.M. (2005). Some useful costbenefit criteria for evaluating computerbased test delivery models and systems. Journal of Applied Testing Technology. http://www.testpublishers.org/assets/documents/Volum%207%20Some%20useful%20cost%20benefit.pdf.
 Meijer, R.R. (Ed.) (1996). Personfit research: theory and applications. Applied Measurement in Education, 9(1), 9–18, [Special issue].
 Mcleod, L., Lewis, C. (1999) Detecting item memorization in CAT environment. Applied Psychological Measurement 23: pp. 147159 CrossRef
 Mcleod, L., Lewis, C., Thissen, D. (2003) A Bayesian method for the detection of item preknowledge in computerized adaptive testing. Applied Psychological Measurement 27: pp. 121137 CrossRef
 Nering, M.L. (1996). The effects of person misfit in computerized adaptive testing. Unpublished doctoral dissertation, University of Minnesota, Minneapolis.
 Nering, M.L. (1997) The distribution of indexes of person fit within the computerized adaptive testing environment. Applied Psychological Measurement 21: pp. 115127 CrossRef
 Patz, R.J., Junker, B.W. (1999) A straightforward approach to Markov chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics 24: pp. 146178
 Patz, R.J., Junker, B.W. (1999) A straightforward approach to Markov Chain Monte Carlo methods for item response models. Journal of Educational and Behavioral Statistics 24: pp. 146178
 Rost, J. (1990) Rasch models in latent classes: an integration of two approaches to item analysis. Applied Psychological Measurement 14: pp. 271282 CrossRef
 Rasch, G. (1960) Probabilistic models of some intelligence and attainment tests. Danish Institute for Educational Research, Copenhagen
 Stocking, M.L., Ward, W.C., Potenza, M.T. (1998) Simulating the use of disclosed items in computerized adaptive testing. Journal of Educational Measurement 35: pp. 4868 CrossRef
 Segall, D. (2002) An item response model for characterizing test comprise. Journal of Educational and Behavioral Statistics 27: pp. 163179 CrossRef
 Segall, D. (2004) A sharing item response theory model for computerized adaptive testing. Journal of Educational and Behavioral Statistics 29: pp. 439460 CrossRef
 Shu, Z. (2010). Using the deterministic, gated item response model detecting test cheating. Unpublished Dissertation. The University of North Carolina at Greensboro.
 Sotaridona, L.S., Meijer, R.R. (2003) Two new statistics to detect answer copying. Journal of Educational Measurement 40: pp. 5369 CrossRef
 Sotaridona, L.S. (2003) Statistical methods for the detection of answer copying on achievement tests. Twente University Press, AE Enschede
 Tatsuoka, K. (1996) Use of generalized personfit indexes, zetas for statistical pattern classification. Applied Measurement in Education 9: pp. 6575 CrossRef
 Linden, W.J., Sotaridona, L.S. (2006) Detecting answer copying when the regular response process follows a known response model. Journal of Educational and Behavioral Statistics 31: pp. 283304 CrossRef
 Linden, W.J., Sotaridona, L.S. (2004) A statistical test for detecting answer copying on multiplechoice tests. Journal of Educational Measurement 41: pp. 361377 CrossRef
 Linden, W.J., Jeon, M. (2012) Modeling answer changes on test items. Journal of Educational and Behavioral Statistics 37: pp. 180199 CrossRef
 Watson, S.A., Iwamoto, C.K., Nungester, R.J., & Luecht, R.M. (1998). The use of response similarity statistics to study examinees who benefit from copying. Paper presented at the national council on measurement in education annual meeting, San Diego.
 Wollack, J.A. (1997) A nominal response model approach for detecting answer copying. Applied Psychological Measurement 21: pp. 307320 CrossRef
 Wollack, A., Cohen, A. (1998) Detection of answer copying with unknown item and trait parameters. Applied Measurement in Education 22: pp. 144152
 Wollack, A., Cohen, A., Serlin, R. (2001) Defining error rate and power for detecting answer copying. Applied Psychological Measurement 25: pp. 385404 CrossRef
 Wollack, J.A. (2006) Simultaneous use of multiple answer copying indexes to improve detection rates. Applied Measurement in Education 19: pp. 265288 CrossRef
 Title
 Using Deterministic, Gated Item Response Theory Model to Detect Test Cheating due to Item Compromise
 Journal

Psychometrika
Volume 78, Issue 3 , pp 481497
 Cover Date
 20130701
 DOI
 10.1007/s1133601293113
 Print ISSN
 00333123
 Online ISSN
 18600980
 Publisher
 SpringerVerlag
 Additional Links
 Topics
 Keywords

 cheating
 model estimation
 Industry Sectors
 Authors

 Zhan Shu ^{(1)}
 Robert Henson ^{(2)}
 Richard Luecht ^{(2)}
 Author Affiliations

 1. Educational Testing Service, 660 Rosedale Road, Princeton, NJ, 08541, USA
 2. The University of North Carolina at Greensboro, Greensboro, USA