Skip to main content
SpringerLink
Log in

Handling Missing Data in Item Response Theory. Assessing the Accuracy of a Multiple Imputation Procedure Based on Latent Class Analysis

  • Published:
Journal of Classification Aims and scope Submit manuscript

Abstract

A critical issue in analyzing multi-item scales is missing data treatment. Previous studies on this topic in the framework of item response theory have shown that imputation procedures are in general associated with more accurate estimates of item location and discrimination parameters under several missing data generating mechanisms. This paper proposes a model-based multiple imputation procedure for multiple categorical items (dichotomous, multinomial or Likert-type) which relies on the results of latent class analysis to impute missing item responses. The effectiveness of the proposed technique is assessed in the estimation of item response theory parameters using a range of ad hoc measures. The accuracy of the method is assessed with respect to other single and multiple imputation procedures, under different missing data generating mechanisms and different rate of missingness (5% to 30%). The simulation results indicate that the proposed technique performs satisfactorily under all conditions and has the greatest potential with severe rates of missingness and under non ignorable missing data mechanisms. The method was implemented in R code with a function that calls scripts from a latent class analysis routine.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • AGRESTI, A. (2002), Categorical Data Analysis, Hoboken: Wiley-Interscience.

    Book  MATH  Google Scholar 

  • AKE, C.F. (2005), “Rounding After Multiple Imputation with Non-Binary Categorical Covariates”, paper presented at the annual meeting of the SAS User Group International, Philadelphia.

  • BAKER, F.B., and KIM, S.H. (2004), Item Response Theory: Parameter Estimation Techniques, New York: Dekker.

    MATH  Google Scholar 

  • BARALDI, A.N., and ENDERS, C.K. (2010), “An Introduction to Modern Missing Data Analyses”, Journal of School Psychology, 48, 5–37.

    Article  Google Scholar 

  • BERNAARDS, C.A., and SIJTSMA, K. (1999), “Factor Analysis of Multidimensional Polytomous Item Response Data from Ignorable Item Non Response”, Multivariate Behavioral Research, 34, 277–314.

    Article  Google Scholar 

  • BIRNBAUM, A. (1968), “Statistical Theories of Mental Test Scores”, in Some Latent Trait Models and Their Use in Inferring an Examinee’S Ability, eds F.M. Lord and M.R. Novick, Reading: Addsion-Wesley, pp. 395–497.

  • CARPITA, M., and MANISERA, M. (2011), “On the Imputation of Missing Data in Surveys with Likert-Type Scales”, Journal of Classification, 28, 93–112,

    Article  MathSciNet  Google Scholar 

  • EDELEN, M.O., and REEVE, B.B. (2007), “Applying Item Response Theory (IRT) Modeling to Questionnaire Development, Evaluation and Refinement”, Quality of Life Researches, 16, 5–18

    Article  Google Scholar 

  • ENDERS, G.K. (2004), “The Impact of Missing Data on Sample Reliability Estimates: Implications of Reliability Reporting Practices”, Educational and Psychological Measurement, 64(3), 419–436,

    Article  MathSciNet  Google Scholar 

  • FINCH, H. (2008), “Estimation of Item Response Theory Parameters in the Presence of Missing Data”, Journal of Educational Measurement, 45(3), 225–245.

    Article  Google Scholar 

  • FINCH, H. (2010), “Imputation Methods for Missing Categorical Questionnaire Data: A Comparison of Approaches”, Journal of Data Science, 8(8), 361–378.

    Google Scholar 

  • FINCH, H. (2011), “The Impact of Missing Data on the Detection of Nonuniform Differential Item Functioning”, Educational and Psychological Measurement, 71(4), 663–683.

    Article  MathSciNet  Google Scholar 

  • HUISMAN, M. (1999), Item Nonresponse: Occurence, Causes, and Imputation of Missing Answers to Test Items, Leiden, The Netherlands: DSWO Press.

    MATH  Google Scholar 

  • LINZER, D.A., and LEWIS, J. (2011), “poLCA: Polytomous Variable Latent Class Analysis”, Journal of Statistical Software, 42(10), 1–29.

    Article  Google Scholar 

  • LITTLE, R.J.A., and RUBIN, D.B. (2002), Statistical Analysis with Missing Data (2nd. ed.), New York: John Wiley.

  • MULLIS, I.V.S., MARTIN, M.O., FOY, P., and DRUCKER, K.T. (eds.) (2012), “PIRLS 2011 International Results in Reading”, 2012 International Association for the Evaluation of Educational, Chestnut Hill MA: TIMSS & PIRLS International Study Center Boston College.

  • NYLUND, K., ASPAROUHOV, T., and BENGT, O.M. 2007, “Deciding on the Number of Classes in Latent Class Analysis and Growth Mixture Modeling: A Monte Carlo Simulation Study”, Structural Equation Modeling, 14(4), 535–569.

    Article  MathSciNet  Google Scholar 

  • RAAIJMAKERS, A.W. (1999), “Effectiveness of Different Missing Data Treatments in Surveys with Likert-Type Data: Introducing the Relative Mean Substitution Approach”, Educational and Psychological Measurement, 59(5), 725–748.

    Article  Google Scholar 

  • RAGHUNATHAN, T.E., LEPKOWSKI, J.M., VAN HOEWYK, J., and SOLENBERGER, P. (2001), “A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models”, Survey Methodology, 27, 85–95.

    Google Scholar 

  • RIZOPOULOS, D. (2006), “ltm: An R package for Latent Variable Modelling and Item Response Theory Analyses”, Journal of Statistical Software, 17(5), 1–25.

    Article  Google Scholar 

  • RUBIN, D. (1976), “Inference and Missing Data”, Biometrika, 63, 581–592.

    Article  MathSciNet  MATH  Google Scholar 

  • SAMEJIMA, F. (1969), “Estimation Of Ability Using a Response Pattern of Graded Scores”, Psychometrika Monograph, 17.

  • SCHAFER, J. (1997), Analysis of Incomplete Multivariate Data, London: Chapman and Hall.

    Book  MATH  Google Scholar 

  • SCHAFER, J., and GRAHAM, J.W. (2002), “Missing Data: Our View of the State of the Art”, Psychological Methods, 7(2), 147–177.

    Article  Google Scholar 

  • SIJTSMA, K., and VAN DER ARK, L.A., (2003), “Investigation and Treatment of Missing Item Scores in Test and Questionnaire Data”, Multivariate Behavioral Research, 38(4), 505–528.

    Article  Google Scholar 

  • SULIS, I. (2013), “A Further Proposal to Perform Multiple Imputation on a Bunch of Polythomous Items based on Latent Class Analysis”, in Statistical Models for Data Analysis: Studies in Classification, Data Analysis, and Knowledge Organization, eds. P. Giudici, S. Ingrassia, and M. Vichi, Heidelberg: Springer-Verlag.

  • SULIS, I., and PORCU, P. (2008), “Assessing the Effectiveness of a Stochastic Regression Imputation Method for Ordered Categorical Data”, CRENoS Working Papers, 4.

  • VAN BUUREN, S., and OUDSHOORN, C.G.M. (2011), “MICE: Multivariate Imputation by Chained Equations”, Journal of Statistical Software, 45(3), 1–67.

    Article  Google Scholar 

  • VERMUNT, J.K, VAN GINKEL, J.R., VAN DER ARK, L.A., and SIJTSMA, K. (2008), “Multiple Imputation of Categorical Data Using Latent Class Analysis”, Sociological Methodology, 33, 269–297.

  • WU, W., JIA, F., and ENDERS, C. (2015), “A Comparison of Imputation Strategies for Ordinal Missing Data on Likert Scale Variables”, Multivariate Behavioral Research, 50, 484–503.

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Scienze Sociali e delle Istituzioni, Università degli Studi di Cagliari, Cagliari, Italy

    Isabella Sulis

  2. Università degli Studi di Cagliari, Cagliari, Italy

    Mariano Porcu

Authors
  1. Isabella Sulis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariano Porcu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Isabella Sulis.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 94 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sulis, I., Porcu, M. Handling Missing Data in Item Response Theory. Assessing the Accuracy of a Multiple Imputation Procedure Based on Latent Class Analysis. J Classif 34, 327–359 (2017). https://doi.org/10.1007/s00357-017-9220-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00357-017-9220-3

Keywords

Search

Navigation