Abstract
Currently, test operation using Item Response Theory (IRT) requires test items to undergo parameter estimation using examinee data. Furthermore, after equating, the items may be included in an item pool that can be used for several tests. However, this test operation method contains the probability of item content leakage. Thus, estimating item parameters while keeping the item contents secret would be useful. In this study, to make such a situation possible, a model in which item parameters are estimated using a paired comparison from the perspective of the difficulty of items by a rater familiar with the field is proposed. The estimation accuracy of this model was confirmed in a simulation study, and the feasibility of its use in practical settings is demonstrated using actual data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrich, D. (1978). Relationships between the Thurstone and Rasch approaches to item scaling. Applied Psychological Measurement, 2, 449–460.
Andrich, D. (1988). The application of an unfolding model of the PIRT type to the measurement of attitude. Applied Psychological Measurement, 12, 33–51.
Andrich, D. (1989). A probabilistic IRT model for unfolding preferene data. Applied Psychological Measurement, 13, 193–216.
Andrich, D. (1995). Hyperbolic cosine latent trait models for unfolding direct responses and pairwise preferences. Applied Psychological Measurement, 19, 269–290.
Bejar, I.I. (1983). Subject matter experts’ assessment of item statistics. Applied Psychological Measurement, 3, 303–310.
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.
Bock, R.D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.
Bockenholt, U. (2001). Thresholds and intransitivities in pairwise judgments: A multilevel analysis. Journal of Educational and Behavioral Statistics, 26, 269–282.
Fischer, G.H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37, 359–374.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer-Nijhoff.
Jansen, P.G.W. (1984). Relationships between the Thurstone, Coombs, and Rasch approaches to item scaling. Applied Psychological Measurement, 8, 373–383.
Lord, F.M. (1980). Applications of item response theory to practical testing problesm. Hillsdale NJ: Erlbaum.
von Davier, M., & von Davier, A.A. (2004). A unified approach to IRT scale linking and scale transformations. ETS Research Report, 04-09, Princeton, NJ: Educational Testing Service.
Thurstone, L.L. (1927). A law of comparative judgement. Pschological Review, 34, 273–286.
Toyoda, H. (2001). Parameter estimation for Rasch type items that have never been used. The Japanese Journal of Psychology. 73, 26–33 (in Japanese).
Toyoda, H. (2002). Koumoku hannou riron nyumon hen (Introduction to item response theory). Tokyo: Asakura Shoten (in Japanese).
Author information
Authors and Affiliations
Corresponding author
About this article
Cite this article
Ozaki, K., Toyoda, H. A Paired Comparison IRT Model Using 3-Value Judgment: Estimation of Item Difficulty Parameters Prior to the Administration of the Test. Behaviormetrika 33, 131–147 (2006). https://doi.org/10.2333/bhmk.33.131
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.2333/bhmk.33.131