Skip to main content
SpringerLink
Log in

Optimal Sampling Design for IRT Linking with Bimodal Data

  • Conference paper
Quantitative Psychology Research

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 89))

  • 2131 Accesses

Abstract

Optimal sampling designs for an IRT linking with improved efficiency are often sought in analyzing assessment data. In practice, the skill distribution of an assessment sample may be bimodal, and this warrants special consideration when trying to create these designs. In this study we explore optimal sampling designs for IRT linking of bimodal data. Our design paradigm is modeled and presents a formal setup for optimal IRT linking. In an optimal sampling design, the sample structure of bimodal data is treated as being drawn from a stratified population. The optimum search algorithm proposed is used to adjust the stratum weights and form a weighted compound sample that minimizes linking errors. The initial focus of the current study is the robust mean–mean transformation method, though the model of IRT linking under consideration is adaptable to generic methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Angoff WH (1984) Scales, norms, and equivalent scores. Educational Testing Service, Princeton

    Google Scholar 

  • Berger MPF (1991) On the efficiency of IRT models when applied to different sampling designs. Appl Psychol Meas 15:293–306

    Article  Google Scholar 

  • Berger MPF (1997) Optimal designs for latent variable models: a review. In: Rost J, Langeheine R (eds) Application of latent trait and latent class models in the social sciences. Waxmann, Muenster, pp 71–79

    Google Scholar 

  • Berger MPF, van der Linden WJ (1992) Optimality of sampling designs in item response theory models. In: Wilson M (ed) Objective measurement: theory into practice, vol 1. Ablex, Norwood, pp 274–288

    Google Scholar 

  • Berger MPF, King CYJ, Wong WK (2000) Minimax D-optimal designs for item response theory models. Psychometrika 65:377–390

    Article  MATH  MathSciNet  Google Scholar 

  • Beveridge GSG, Schechter RS (1970) Optimization: theory and practice. McGraw-Hill, New York

    MATH  Google Scholar 

  • Buyske S (2005) Optimal design in educational testing. In: Berger MPF, Wong WK (eds) Applied optimal designs. Wiley, New York, pp 1–19

    Chapter  Google Scholar 

  • Cochran WG (1977) Sampling techniques, 3rd edn. Wiley, New York

    MATH  Google Scholar 

  • Dorans NJ, Holland PW (2000) Population invariance and equitability of tests: basic theory and the linear case. J Educ Meas 37:281–306

    Article  Google Scholar 

  • Duong M, von Davier AA (2012) Observed-score equating with a heterogeneous target population. Int J Test 12:224–251

    Article  Google Scholar 

  • Haberman SJ (2009) Linking parameter estimates derived from an item response model through separate calibrations (research report 09–40). Educational Testing Service, Princeton

    Google Scholar 

  • Haberman SJ, Lee Y, Qian J (2009) Jackknifing techniques for evaluation of equating accuracy (research report 09–39). Educational Testing Service, Princeton

    Google Scholar 

  • Haebara T (1980) Equating logistic ability scales by a weighted least squares method. Jpn Psychol Res 22(3):144–149

    Google Scholar 

  • Hua L-K, Wang Y, Heijmans JGC (1989) Optimum seeking methods (single variable). In: Lucas WF, Thompson M (eds) Mathematical modelling, vol 2. Springer, New York, pp 57–78

    Google Scholar 

  • Jones DH, Jin Z (1994) Optimal sequential designs for on-line item estimation. Psychometrika 59:59–75

    Article  MATH  Google Scholar 

  • Kish L (1965) Survey sampling. Wiley, New York

    MATH  Google Scholar 

  • Kolen MJ, Brennan RL (2004) Test equating, scaling, and linking: methods and practices. Springer, New York

    Book  Google Scholar 

  • Kuhn HW, Tucker AW (1951) Nonlinear programming. In: Neyman J (ed) Proceedings of the second Berkeley symposium on mathematical statistics and probability, University of California Press, Berkeley, pp 481–492

    Google Scholar 

  • Lord FM (1980) Applications of item response theory to practical testing problems. Erlbaum, Hillsdale

    Google Scholar 

  • Lord MF, Wingersky MS (1985) Sampling variances and covariances of parameter estimates in item response theory. In: Weiss DJ (ed) Proceedings of the 1982 IRT/CAT conference, Department of Psychology, CAT Laboratory, University of Minnesota, Minneapolis

    Google Scholar 

  • Loyd BH, Hoover HD (1980) Vertical equating using the Rasch model. J Educ Meas 17:179–193

    Article  Google Scholar 

  • Mislevy RJ, Bock RD (1990) BILOG 3, 2nd edn. Scientific Software, Mooresville

    Google Scholar 

  • Muraki E, Bock RD (2002) PARSCALE (Version 4.1) [Computer software]. Scientific Software, Lincolnwood

    Google Scholar 

  • Nocedal J, Wright SJ (2006) Numerical optimization. Springer, New York

    MATH  Google Scholar 

  • Qian J, Spencer B (1994). Optimally weighted means in stratified sampling. In: Proceedings of the section on survey research methods, American Statistical Association, pp 863–866

    Google Scholar 

  • Qian J, von Davier AA, Jiang Y (2013) Achieving a stable scale for an assessment with multiple forms: weighting test samples in IRT linking. In: Millsap RE, van der Ark LA, Bolt DM, Woods CM (eds) Springer proceedings in mathematics & statistics, new developments in quantitative psychology. Springer, New York, pp 171–185

    Chapter  Google Scholar 

  • Silvey SD (1970) Statistical inference. Penguin Books, Baltimore

    MATH  Google Scholar 

  • Stocking ML (1990) Specifying optimum examinees for item parameter estimation in item response theory. Psychometrika 55:461–475

    Article  Google Scholar 

  • Stocking ML, Lord FM (1983) Developing a common metric in item response theory. Appl Psychol Meas 7:201–210

    Article  Google Scholar 

  • van der Linden WJ, Luecht RM (1998) Observed-score equating as a test assembly problem. Psychometrika 63:401–418

    Article  MATH  MathSciNet  Google Scholar 

  • von Davier M, von Davier AA (2011) A general model for IRT scale linking and scale transformation. In: von Davier AA (ed) Statistical models for test equating, scaling, and linking. Springer, New York, pp 225–242

    Chapter  Google Scholar 

  • von Davier AA, Wilson C (2008) Investigating the population sensitivity assumption of item response theory true-score equating across two subgroups of examinees and two test formats. Appl Psychol Meas 32:11–26

    Article  MathSciNet  Google Scholar 

  • von Davier AA, Holland PW, Thayer DT (2004) The kernel method of test equating. Springer, New York

    MATH  Google Scholar 

  • Wilde DJ (1964) Optimum seeking methods. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  • Wolter K (2007) Introduction to variance estimation, 2nd edn. Springer, New York

    MATH  Google Scholar 

  • Zumbo BD (2007) Validity: foundational issues and statistical methodology. In: Rao CR, Sinharay S (eds) Handbook of statistics, vol 26, Psychometrics. Elsevier Science B.V, Amsterdam, pp 45–79

    Google Scholar 

Download references

Acknowledgments

The authors thank Jim Carlson, Shelby Haberman, Yi-Hsuan Lee, Ying Lu, and Daniel Bolt for their suggestions and comments. The authors also thank Shuhong Li and Jill Carey for their assistance in assembling data and Kim Fryer for editorial help. Any opinions expressed in this paper are solely those of the authors and not necessarily those of ETS.

Author information

Authors and Affiliations

  1. Educational Testing Service, Research and Development, Rosedale Rd, MS 02-T, Princeton, NJ, 08541, USA

    Jiahe Qian & Alina A. von Davier

Authors
  1. Jiahe Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alina A. von Davier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiahe Qian .

Editor information

Editors and Affiliations

  1. Department of Psychology, Arizona State University, Tempe, Arizona, USA

    Roger E. Millsap

  2. Dept. of Educational Psychology, University of Wisconsin, Madison, USA

    Daniel M. Bolt

  3. University of Amsterdam, Amsterdam, The Netherlands

    L. Andries van der Ark

  4. Department of Psychological Studies, The Hong Kong Institute of Education, Hong Kong, Hong Kong SAR

    Wen-Chung Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Qian, J., von Davier, A.A. (2015). Optimal Sampling Design for IRT Linking with Bimodal Data. In: Millsap, R., Bolt, D., van der Ark, L., Wang, WC. (eds) Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol 89. Springer, Cham. https://doi.org/10.1007/978-3-319-07503-7_10

Download citation

Publish with us

Policies and ethics

Search

Navigation