Journal of Classification

, Volume 29, Issue 1, pp 91–116 | Cite as

Handling Missing Values with Regularized Iterative Multiple Correspondence Analysis

  • Julie Josse
  • Marie Chavent
  • Benot Liquet
  • François Husson
Article

Abstract

A common approach to deal with missing values in multivariate exploratory data analysis consists in minimizing the loss function over all non-missing elements, which can be achieved by EM-type algorithms where an iterative imputation of the missing values is performed during the estimation of the axes and components. This paper proposes such an algorithm, named iterative multiple correspondence analysis, to handle missing values in multiple correspondence analysis (MCA). The algorithm, based on an iterative PCA algorithm, is described and its properties are studied. We point out the overfitting problem and propose a regularized version of the algorithm to overcome this major issue. Finally, performances of the regularized iterative MCA algorithm (implemented in the R-package named missMDA) are assessed from both simulations and a real dataset. Results are promising with respect to other methods such as the missing-data passive modified margin method, an adaptation of the missing passive method used in Gifi’s Homogeneity analysis framework.

Keywords

Multiple correspondence analysis Categorical data Missing values Imputation Regularization 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. BENZÉCRI, J-P. (1973), L’Analyse des Données, Tome II: L’Analyse des Correspondances, Paris: Dunod.MATHGoogle Scholar
  2. BRO, R., KJELDAHL, K., SMILDE, A. K., and KIERS, H. A. L. (2008), “Cross-validation of Component Models: A Critical Look at Current Methods”, Analalytical and Bioanalytical Chemistry, 390, 1241–1251.CrossRefGoogle Scholar
  3. DE LEEUW, J., and VAN DER HEIJDEN, P. G. M. (1988), “Correspondence Analysis of Incomplete Contingency Tables”, Psychometrika, 53, 223–233.MathSciNetMATHCrossRefGoogle Scholar
  4. DEMPSTER, A. P., LAIRD, N. M., and RUBIN, D. B. (1977), “Maximum Likelihood from Incomplete Data via the Em Algorithm”, Journal of the Royal Statistical Society B, 39, 1–38.MathSciNetMATHGoogle Scholar
  5. ESCOFIER, B. (1987), “Traitement des Questionnaires avec Non Réponse, Analyse des Correspondances avec Marges Modifiée et Analyse Multicanonique avec Contrainte”, Publications de l’Institut de Statistique de l’Université de Paris, 32, 33–70.MATHGoogle Scholar
  6. ESCOUFIER, Y. (1973), “Le Traitement des Variables Vectorielles”, Biometrics, 29, 751–760.MathSciNetCrossRefGoogle Scholar
  7. GABRIEL, K.R.,and ZAMIR, S. (1979), “Lower Rank Approximation of Matrices by Least Squares with Any Choice of Weights”, Technometrics, 21, 236–246.CrossRefGoogle Scholar
  8. GIFI, A. (1981), Non-linear Multivariate Analysis, Leiden: D.S.W.O.-Press.Google Scholar
  9. GREENACRE, M. (1984), Theory and Applications of Correspondence Analysis, London: Acadamic Press.MATHGoogle Scholar
  10. GREENACRE, M. (1988), “Correspondence Analysis of Multivariate Categorical Data by Weighted Least-squares”, Biometrika, 75, 457–477.MathSciNetMATHCrossRefGoogle Scholar
  11. GREENACRE, and BLASIUS, J. (2006), Multiple Correspondence Analysis and Related Methods, London: Chapman & Hall/CRC.MATHCrossRefGoogle Scholar
  12. GREENACRE, M. and PARDO, R. (2006), “Subset Correspondence Analysis: Visualizing Relationships Among a Selected Set of Response Categories from a Questionnaire Survey”, Sociological Methods and Research, 35 (2): 193–218.MathSciNetCrossRefGoogle Scholar
  13. HASTIE, T., TIBSHIRANI, R., and FRIEDMAN, J. (2001), The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer Series in Statistics.Google Scholar
  14. HOERL, A.F., and KENNARD, R.W. (1970), “Ridge Regression: Biased Estimation for Nonorthogonal Problems”, Technometrics, 12, 55–67.MATHCrossRefGoogle Scholar
  15. HUSSON, F. and JOSSE, J. (2010), missMDA: Handling Missing Values With/In Multivariate Data Analysis (Principal Component Methods), R package version 1.2, http://www.agrocampus-ouest.fr/math/husson, http://www.agrocampus-ouest.fr/math/josse.
  16. HUSSON, F., JOSSE, J., LÊ, S., and MAZET, J. (2011), FactoMineR: Multivariate Exploratory Data Analysis and Data Mining with R, R package version 1.16, http://factominer.free.fr, http://www.agrocampus-ouest.fr/math/.
  17. ILIN, A., and RAIKO, T. (2010), “Practical Approaches to Principal Component Analysis in the Presence of Missing Values”, Journal of Machine Learning Research, 11, pp. 1957-2000.MathSciNetGoogle Scholar
  18. JOSSE, J., PAGÈS, J., and HUSSON, F. (2008), “Testing the Significance of the Rv Coefficient”, Computational Statistics and Data Analysis, 53, 82–91.MathSciNetMATHCrossRefGoogle Scholar
  19. JOSSE, J., PAGÈS, J., and HUSSON, F. (2009), “Gestion des DonnÉes Manquantes en Analyse en Composantes Principales”, Journal de la Société Française de Statistique, 150, 28–51.Google Scholar
  20. KIERS, H.A.L. (1997), “Weighted Least Squares Fitting Using Ordinary Least Squares Algorithms”, Psychometrika, 62, 251–266.MathSciNetMATHCrossRefGoogle Scholar
  21. LÊ, S., JOSSE, J. and HUSSON, F. (2008), “Factominer: An R Package for Multivariate Analysis”, Journal of Statistical Software, 25(1), 1–18.Google Scholar
  22. LEBART, L., MORINEAU, A., and WARWICK, K.M. (1984), Multivariate Descriptive Statistical Analysis, New York: Wiley.MATHGoogle Scholar
  23. LITTLE, R.J.A., and RUBIN, D.B. (1987, 2002), Statistical Analysis with Missing Data, New York: Wiley Series in Probability And Statistics.Google Scholar
  24. MEULMAN, J. (1982), Homgeneity Analysis of Incomplete Data, Leiden: D.S.W.O.-Press.Google Scholar
  25. NISHISATO, S. (1980), Analysis of Categorical Data: Dual Scaling and its Applications, Toronto: University of Toronto Press, Toronto.MATHGoogle Scholar
  26. NORA-CHOUTEAU, C. (1974), Une Méthode de Reconstitution et d’Analyse de Données IncomplÈtes, unpublished PhD thesis, Université Pierre et Marie Curie.Google Scholar
  27. R DEVELOPMENT CORE TEAM, (2010), R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0, http://www.R-project.org/.
  28. RUBIN, D.B. (1976), “Inference and Missing Data”, Biometrika, 63, 581–592.MathSciNetMATHCrossRefGoogle Scholar
  29. SCHAFER, J.L. (1997), Analysis of Incomplete Multivariate Data, Chapman & Hall/CRC.Google Scholar
  30. SCHAFER, J.L., and GRAHAM, J.W. (2002), “Missing Data: Our View of the State of the Art”, Psychological Methods, 7, 147–177.CrossRefGoogle Scholar
  31. SMILDE, A.K., KIERS, H.A.L., BIJLSMA, S., RUBINGH, C.M. and VAN ERK, M.J. (2009), “Matrix Correlations for High-dimensional Data: The Modified RV-coefficient”, Bioinformatics, 25, 401–405.CrossRefGoogle Scholar
  32. TAKANE, Y., and HWANG, H. (2002), “Generalized Constrained Canonical Correlation Analysis”, Multivariate Behavioral Research, 37, 163–195.CrossRefGoogle Scholar
  33. TAKANE, Y,. and HWANG, H. (2006), “Regularized Multiple Correspondence Analysis”, in Multiple Correspondence Analysis and Related Methods, eds. J. Blasius and M. J. Greenacre, Chapman & Hall, pp. 259–279.Google Scholar
  34. TAKANE, Y., and OSHIMA-TAKANE, Y. (2003), “Relationships Between Two Methods for Dealing with Missing Data in Principal Component Analysis”, Behaviormetrika, 30, 145–154.MathSciNetMATHCrossRefGoogle Scholar
  35. TENENHAUS, M., and YOUNG, F.W. (1985), “An Analysis and Synthesis of Multiple Correspondence Analysis, Optimal Scaling, Dual Scaling, Homogeneity Analysis and Other Methods for Quantifying Categorical Multivariate Data”, Psychometrika, 50, 91–119.MathSciNetMATHCrossRefGoogle Scholar
  36. TIPPING, M., and BISHOP, C.M. (1999), “Probabilistic Principal Component Analysis”, Journal of the Royal Statistical Society B, 61, 611–622.MathSciNetMATHCrossRefGoogle Scholar
  37. VAN DER HEIJDEN, P.G.M., and ESCOFIER, B. (2003), “Multiple Correspondence Analysis with Missing Data”, in Recherches sur l’Analyse des Correspondances, pp. 152–170.Google Scholar
  38. VERMUNT, J.K., VAN GINKEL, J.R., VAN DER ARK, L.A., and SIJTSMA, K. (2008), “Multiple Imputation of Incomplete Categorical Data Using Latent Class Analysis”, Sociological Methodology, 33, 369–397.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Julie Josse
    • 1
  • Marie Chavent
    • 2
  • Benot Liquet
    • 3
  • François Husson
    • 1
  1. 1.Agrocampus RennesRennesFrance
  2. 2.Université V. Segalen Bordeaux 2BordeauxFrance
  3. 3.Equipe Biostatistique de l’U897 INSERM ISPEDBordeauxFrance

Personalised recommendations