Advances in Data Analysis and Classification

, Volume 5, Issue 3, pp 231–246 | Cite as

Multiple imputation in principal component analysis

  • Julie JosseEmail author
  • Jérôme Pagès
  • François Husson
Regular Article


The available methods to handle missing values in principal component analysis only provide point estimates of the parameters (axes and components) and estimates of the missing values. To take into account the variability due to missing values a multiple imputation method is proposed. First a method to generate multiple imputed data sets from a principal component analysis model is defined. Then, two ways to visualize the uncertainty due to missing values onto the principal component analysis results are described. The first one consists in projecting the imputed data sets onto a reference configuration as supplementary elements to assess the stability of the individuals (respectively of the variables). The second one consists in performing a principal component analysis on each imputed data set and fitting each obtained configuration onto the reference one with Procrustes rotation. The latter strategy allows to assess the variability of the principal component analysis parameters induced by the missing values. The methodology is then evaluated from a real data set.


Principal component analysis Missing values EM algorithm Multiple imputation Bootstrap Procrustes rotation 

Mathematics Subject Classification (2000)

62H25 62G09 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Adams E, Walczak B, Vervaet C, Risha PG, Massart D (2002) Principal component analysis of dissolution data with missing elements. Int J Pharm 234: 169–178CrossRefGoogle Scholar
  2. Bro R (1998) Multi-way analysis in the food industry—models, algorithms, and applications. Tech. rep., MRI, EPG and EMA, Proc ICSLP 2000Google Scholar
  3. Bro R, Kjeldahl K, Smilde AK, Kiers HAL (2008) Cross-validation of component models: A critical look at current methods. Anal Bioanal Chem 5: 1241–1251CrossRefGoogle Scholar
  4. Caussinus H (1986) Models and uses of principal component analysis. In: de Leeuw J, Heiser W, Meulman J, Critchley F (eds) Multidimensional data analysis. DSWO Press, Ram, pp 149–178Google Scholar
  5. Chateau F, Lebart L (1996) Assessing sample variability in the visualization techniques related to principal componenet analysis: Bootstrap and alternative simulation methods. In: COMPSTAT, pp 205–210Google Scholar
  6. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J Royal Stat Soc B 39: 1–38MathSciNetzbMATHGoogle Scholar
  7. Denis JB (1991) Ajustements de modèles linéaires et bilinéaires sous contraintes linéaires avec données manquantes. Revue de Statistique Appliquée 39: 5–24Google Scholar
  8. Dray S (2008) On the number of principal components: A test of dimensionality based on measurements of similarity between matrices. Comput Stat Data Anal 52: 2228–2237MathSciNetzbMATHCrossRefGoogle Scholar
  9. Escofier B, Pagès J (2008) Analyses factorielles simples et multiples, 4th edn. Economica, ParisGoogle Scholar
  10. Gabriel KR, Zamir S (1979) Lower rank approximation of matrices by least squares with any choice of weights. Technometrics 21: 236–246CrossRefGoogle Scholar
  11. Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, BaltimorezbMATHGoogle Scholar
  12. Gower JC, Dijksterhuis GB (2004) Procrustes problems. Oxford University Press, New YorkzbMATHCrossRefGoogle Scholar
  13. Greenacre M (1984) Theory and applications of correspondence analysis. Acadamic Press, LondonzbMATHGoogle Scholar
  14. Grung B, Manne R (1998) Missing values in principal component analysis. Chemiometr Intell Lab Syst 42: 125–139CrossRefGoogle Scholar
  15. Husson F, Josse J (2010) missMDA: Handling missing values with/in multivariate data analysis (principal component methods).,, R package version 1.2
  16. Ilin A, Raiko T (2010) Practical approaches to principal component analysis in the presence of missing values. J Mach Learn Res 11: 1957–2000MathSciNetGoogle Scholar
  17. Josse J, Pagès J, Husson F (2009) Gestion des données manquantes en analyse en composantes principales. J de la Société Française de Statistique 150: 28–51Google Scholar
  18. Josse J, Pagès J, Husson F (2011) Selecting the number of components in principal component analysis using cross-validation approximations (submitted)Google Scholar
  19. Kiers HAL (1997) Weighted least squares fitting using ordinary least squares algorithms. Psychometrica 62: 251–266MathSciNetzbMATHCrossRefGoogle Scholar
  20. Kroonenberg PM (2008) Applied Multiway data analysis (chap.7). Wiley series in probability and statistics, New YorkCrossRefGoogle Scholar
  21. Little RJA, Rubin DB (1987) 2002) Statistical analysis with missing data. Wiley series in probability and statistics, New YorkGoogle Scholar
  22. Milan M (1995) Application of the parametric bootstrap to models that incorporate a singular value decomposition. J Royal Stat Soc Ser C 44: 31–49zbMATHGoogle Scholar
  23. Netflix (2009) Netflix challenge.
  24. Nora-Chouteau C (1974) Une méthode de reconstitution et d’analyse de données incomplètes. PhD thesis, Université Pierre et Marie CurieGoogle Scholar
  25. Peres-Neto PR, Jackson DA, Somers KM (2005) How many principal components? stopping rules for determining the number of non-trivial axes revisited. Comput Stat Data Anal 49: 974–997MathSciNetzbMATHCrossRefGoogle Scholar
  26. R Development Core Team (2009) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria,, ISBN 3-900051-07-0
  27. Raiko T, Ilin A, Karhunen J (2007) Principal component analysis for sparse high-dimensional data. In: Neural Information Processing, pp 566–575Google Scholar
  28. Rubin DB (1987) Multiple imputation for non-response in survey. Wiley, New YorkCrossRefGoogle Scholar
  29. Schafer JL (1997) Analysis of incomplete multivariate data. Chapman & Hall/CRC, LondonzbMATHCrossRefGoogle Scholar
  30. Schafer JL, Olsen MK (1998) Multiple imputation for missing-data problems: A data analyst’s perspective. Multivar Behav Res 33: 545–571CrossRefGoogle Scholar
  31. Song J (1999) Analysis of incomplete high-dimensional multivariate normal data using a common factor model. PhD thesis, Dept. of Biostatistics, UCLA, Los AngelesGoogle Scholar
  32. Tanner MA, Wong WH (1987) The calculation of posterior distributions by data augmentation. J Am Stat Assoc 82: 805–811MathSciNetGoogle Scholar
  33. Timmerman ME, Kiers HAL, Smilde AK (2007) Estimating confidence intervals for principal component loadings: a comparaison between the bootstrap and asymptotic results. Br J Math Stat Psychol 60: 295–314CrossRefGoogle Scholar
  34. Tipping M, Bishop CM (1999) Probabilistic principal component analysis. J Royal Stat Soc B 61: 611–622MathSciNetzbMATHCrossRefGoogle Scholar
  35. van Buuren S (2007) Multiple imputation of discrete and continuous data by fully conditional specification. Stat Methods Med Res 16: 219–242MathSciNetzbMATHCrossRefGoogle Scholar
  36. Wold H (1966) Nonlinear estimation by iterative least squares procedures. In: David FN (eds) Research Papers in Statistics: Festschrift for Jerzy Neyman. Wiley, New York, pp 411–444Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Julie Josse
    • 1
    Email author
  • Jérôme Pagès
    • 1
  • François Husson
    • 1
  1. 1.Agrocampus OuestRennesFrance

Personalised recommendations