Quality of Life Research

, Volume 20, Issue 2, pp 287–300 | Cite as

Missing data methods for dealing with missing items in quality of life questionnaires. A comparison by simulation of personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques applied to the SF-36 in the French 2003 decennial health survey

  • Hugo Peyre
  • Alain Leplège
  • Joël CosteEmail author



Missing items are common in quality of life (QoL) questionnaires and present a challenge for research in this field. It remains unclear which of the various methods proposed to deal with missing data performs best in this context. We compared personal mean score, full information maximum likelihood, multiple imputation, and hot deck techniques using various realistic simulation scenarios of item missingness in QoL questionnaires constructed within the framework of classical test theory.


Samples of 300 and 1,000 subjects were randomly drawn from the 2003 INSEE Decennial Health Survey (of 23,018 subjects representative of the French population and having completed the SF-36) and various patterns of missing data were generated according to three different item non-response rates (3, 6, and 9%) and three types of missing data (Little and Rubin’s “missing completely at random,” “missing at random,” and “missing not at random”). The missing data methods were evaluated in terms of accuracy and precision for the analysis of one descriptive and one association parameter for three different scales of the SF-36.


For all item non-response rates and types of missing data, multiple imputation and full information maximum likelihood appeared superior to the personal mean score and especially to hot deck in terms of accuracy and precision; however, the use of personal mean score was associated with insignificant bias (relative bias <2%) in all studied situations.


Whereas multiple imputation and full information maximum likelihood are confirmed as reference methods, the personal mean score appears nonetheless appropriate for dealing with items missing from completed SF-36 questionnaires in most situations of routine use. These results can reasonably be extended to other questionnaires constructed according to classical test theory.


Quality of life SF-36 Missing data Imputation methods 



Missing completely at random


Missing at random


Missing not at random


Personal mean score


Multiple Imputation


Hot deck


Quality of life


Medical outcome study 36-item short-form health survey


Full information maximum likelihood



We thank Jean Louis Lanoë for allowing us to exploit the data of the 2003 Decennial Health Survey. We also thank David Jegou and Vivian Viallon for assistance with simulations.


  1. 1.
    Enders, C. (2006). A primer on the use of modern missing-data methods in psychosomatic medicine research. Psychosomatic Medicine, 68, 427–436.CrossRefPubMedGoogle Scholar
  2. 2.
    Graham, J. W. (2009). Missing data analysis: Making it works in the real world. Annual Review of Psychology, 60, 549–576.CrossRefPubMedGoogle Scholar
  3. 3.
    Baraldi, A. N., & Enders, C. K. (2010). An introduction to modern missing data analyses. Journal of School Psychology, 48(1), 5–47.CrossRefPubMedGoogle Scholar
  4. 4.
    Zwinderman, A. H. (1992). Statistical analysis of longitudinal quality of life data with missing measurements. Quality of Life Research, 1(3), 219–224.CrossRefPubMedGoogle Scholar
  5. 5.
    Fairclough, D. L., Peterson, H. F., & Chang, V. (1998). Why are missing quality of life data a problem in clinical trials of cancer therapy? Statistics in Medicine, 17(5–7), 667–677.CrossRefPubMedGoogle Scholar
  6. 6.
    Troxel, A. B., Fairclough, D. L., Curran, D., & Hahn, E. A. (1998). Statistical analysis of quality of life with missing data in cancer clinical trials. Statistics in Medicine, 17(5–7), 653–666.CrossRefPubMedGoogle Scholar
  7. 7.
    Little, R., & Rubin, D. (1987). Statistical analysis with missing data. New York: John Wiley and Sons.Google Scholar
  8. 8.
    Donaldson, G. W., & Moinpour, C. M. (2005). Learning to live with missing quality-of-life data in advanced-stage disease trials. Journal of Clinical Oncology, 23, 447–453.CrossRefGoogle Scholar
  9. 9.
    Fielding, S., Fayers, P. M., & Ramsay, C. R. (2009). Investigating the missing data mechanism in quality of life outcomes: A comparison of approaches. Health and Quality of Life Outcomes, 8(16), 1477–7525.Google Scholar
  10. 10.
    Peyre, H., Coste, J., & Leplège, A. (2010). Identifying type and determinants of missing items in quality of life questionnaires: Application to the SF-36 French version of the 2003 decennial health survey. Health and Quality of Life Outcomes, 8, 16.CrossRefPubMedGoogle Scholar
  11. 11.
    Morris, J., & Coyle, D. (1994). Quality of life questionnaires in cancer clinical trials: Imputing missing values. Psycho-Oncology, 3(3), 215–222.CrossRefGoogle Scholar
  12. 12.
    Fairclough, D. L., & Cella, D. F. (1996). Functional assessment of cancer therapy (FACT-G): Non-response to individual questions. Quality of Life Research, 5(3), 321–329.CrossRefPubMedGoogle Scholar
  13. 13.
    Fayers, P. M., Curran, D., & Machin, D. (1998). Incomplete quality of life data in randomized trials: Missing items. Statistics in Medicine, 17(5–7), 679–696.CrossRefPubMedGoogle Scholar
  14. 14.
    Hawthorne, G., & Elliott, P. (2005). Imputing cross-sectional missing data: Comparison of common techniques. The Australian and New Zealand Journal of Psychiatry, 39(7), 583–590.PubMedGoogle Scholar
  15. 15.
    Croy, C. D., & Novins, D. K. (2005). Methods for addressing missing data in psychiatric and developmental research. Journal of the American Academy of Child and Adolescent Psychiatry, 44(12), 1230–1240.CrossRefPubMedGoogle Scholar
  16. 16.
    Schafer, J., & Graham, J. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177.CrossRefPubMedGoogle Scholar
  17. 17.
    Stuart, E. A., Azur, M., Frangakis, C., & Leaf, P. (2009). Multiple imputation with large data sets: A case study of the children’s mental Health initiative. American Journal of Epidemiology, 169, 1133–1139.CrossRefPubMedGoogle Scholar
  18. 18.
    Perneger, T. V., & Burnand, B. (2005). A simple imputation algorithm reduced missing data in SF-12 health surveys. Journal of Clinical Epidemiology, 58(2), 142–149.CrossRefPubMedGoogle Scholar
  19. 19.
    Lin, T. H. (2006). Missing data imputation in quality-of-life assessment: Imputation for WHOQOL-BREF. Pharmacoeconomics, 24(9), 917–925.CrossRefPubMedGoogle Scholar
  20. 20.
    Sande, I. (1983). Hot Deck imputation procedures, incomplete data in samples surveys. New York: Academic Press. (Book).Google Scholar
  21. 21.
    Rubin, D. (1987). Multiple imputation for nonreponse in surveys. New York: John Wiley and Sons. (Book).CrossRefGoogle Scholar
  22. 22.
    Donders, A. R., van der Heijden, G. J., Stijnen, T., & Moons, K. G. (2006). Review: A gentle introduction to imputation of missing values. Journal of Clinical Epidemiology, 59(10), 1087–1091.CrossRefPubMedGoogle Scholar
  23. 23.
    Shrive, F. M., Stuart, H., Quan, H., & Ghali, W. A. (2006). Dealing with missing data in a multi-question depression scale: A comparison of imputation methods. BMC Medical Research Methodology, 6, 57.CrossRefPubMedGoogle Scholar
  24. 24.
    Lanoë, J., Makdessi-Raynaud, Y. (2005). L’état de santé en France en 2003. Santé perçue, morbidité déclarée et recours aux soins à travers l’enquête décennale santé. Etudes et résultats (DRESS), 436,1–12.Google Scholar
  25. 25.
    Ware, J. E., Jr, & Sherbourne, C. D. (1992). The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Medical Care, 30(6), 473–483.CrossRefPubMedGoogle Scholar
  26. 26.
    Stewart, A. L., & Ware, J. E. (1992). Measuring functioning and well-being. Durham and London: D.U. Press. (Book).Google Scholar
  27. 27.
    Leplege, A., Ecosse, E., Verdier, A., Perneger, T. V. (1998). The French SF-36 Health Survey: Translation, cultural adaptation and preliminary psychometric evaluation. Journal of Clinical Epidemiology, 51(11), 1013–23.Google Scholar
  28. 28.
    Leplege, A., Ecosse, E., Pouchot, J., Coste, J., & Perneger, T. V. (2001). Le questionnaire MOS SF-36, manuel de l’utilisation et guide d’interprétation des scores. Paris: ESTEM. (Book).Google Scholar
  29. 29.
    Ware, J. E., & Gandek, B. (1998). Overview of the SF-36 health survey and the international quality of life assessment (IQOLA). Journal of Clinical Epidemiology, 51(11), 903–911.CrossRefPubMedGoogle Scholar
  30. 30.
    Fayers, P. M., Hand, D. J., Bjordal, K., & Groenvold, M. (1997). Causal indicators in quality of life research. Quality of Life Research, 6, 393–406.CrossRefPubMedGoogle Scholar
  31. 31.
    Fayers, P. M., & Hand, D. J. (1997). Factor analysis, causal indicators and quality of life. Quality of Life Research, 6, 139–150.PubMedGoogle Scholar
  32. 32.
    Gandek, B., Ware, J. E., Aaronson, N. K., Alonso, J., Apolone, G., Bjorner, J., et al. (1998). Tests of data duality, scaling assumptions, and reliability of the SF-36 in Eleven Countries: Results from the IQOLA Project. Journal of Clinical Epidemiology, 51(11), 1149–1158.CrossRefPubMedGoogle Scholar
  33. 33.
    Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8(1), 128–141.CrossRefGoogle Scholar
  34. 34.
    Sentas, P., & Angelis, L. (2006). Categorical missing data imputation for software cost estimation by multinomial logistic regression. Journal of Systems and Software, 79(3), 404–414.CrossRefGoogle Scholar
  35. 35.
    Vermunt, J. K., Van Ginkel, J. R., Van der Ark, L. A., & Sijtsma, K. (2008). Multiple imputation of incomplete categorical data using latent class analysis. Sociological Methodology, 38, 369–397.Google Scholar
  36. 36.
    Reiter, J., Raghunathan, T. E., & Kinney, S. K. (2006). The importance of modeling the sampling design in multiple imputation for missing data. Survey Methodology, 32(2), 143–149.Google Scholar
  37. 37.
    Binder, D. A., Sun, W. (1996). Frequency valid multiple imputation for surveys with a complex design. Proceedings of the survey research methods section, ASA, 281–286.Google Scholar
  38. 38.
    Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., et al. (2009). Multiple imputation for missing data in epidemiological and clinical research. Potential and Pitfalls BMJ, 338, b2393.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  1. 1.Biostatistics and Epidemiology Unit, Assistance Publique-Hôpitaux de Paris, Hôpital CochinNancy-Université, Université Paris-Descartes, Université Metz Paul Verlaine, Research unit APEMAC, EA 4360Paris Cedex 14France
  2. 2.Department of History and Philosophy of SciencesUniversity of Paris DiderotParis 7France

Personalised recommendations