Advertisement

Quality and Quantity

, Volume 35, Issue 2, pp 147–160 | Cite as

Reducing Missing Data in Surveys: An Overview of Methods

  • Edith D. de Leeuw
Article

Abstract

Although item nonresponse can never be totally prevented, it can be considerably reduced, and thereby provide the researcher with not only more useable data, but also with helpful auxiliary information for a better imputation and adjustment. To achieve this an optimal data collection design is necessary. The optimization of the questionnaire and survey design are the main tools a researcher has to reduce the number of missing data in any such survey. In this contribution a concise typology of missing data patterns and their sources of origin are presented. Based on this typology, the mechanisms responsible for missing data are identified, followed by a discussion on how item nonresponse can be prevented.

item nonresponse causes of missingness cognitive pretest data collection mode ignorability question wording questionnaire development sensitive questions survey 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arbuckle, J. (1996). Full information estimation in the presence of missing data. In: G. A. Marcoulides & R. E. Schumacker (eds), Advanced Structural Equation Modeling. Mahwah, NY: Erlbaum.Google Scholar
  2. Billiet, J.& Loosveldt, G. (1988). Improvement of the quality of responses to factual survey questions by interviewer training. Public Opinion Quarterly 52: 190-211.Google Scholar
  3. Campanelli, P. (1997). Testing survey questions: new directions in cognitive interviewing. BMS 55: 5-17 (special issue on the cognitive interview).Google Scholar
  4. Carton, A. (1999). Selectie, Training en Evaluatie van Interviewers binnen een Interviewernetwerk [In Dutch: An Interviewer Network: Constructing a Procedure to Evaluate Interviewers]. Leuven: Garant.Google Scholar
  5. Colsher, P. L. & Wallace, R. B. (1989). Data quality and age: health and psychobehavioral correlates of item nonresponse and inconsistent responses. Journal of Gerontology 44: 45-52.Google Scholar
  6. Conan Doyle, Sir Arthur. (1981). The Copper Beeches: The Adventures of Sherlock Holmes. London: Penguin Books, p. 268.Google Scholar
  7. Couper, M. P., Hansen, S. E. & Sadovsky, S. A. (1997). Evaluating interviewer use of CAPI technology. In: L. Lyberg et al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 267-285.Google Scholar
  8. De Leeuw, E. D. (1992). Data Quality in Mail, Telephone, and Face to Face Surveys. Amsterdam: TT publikaties.Google Scholar
  9. De Leeuw, E. D. & Collins, M. (1997). Data collection method and survey quality: an overview. In: L. Lyberg et al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 199-220.Google Scholar
  10. De Leeuw, E. D., Hox, J. J. & Snijkers, G. (1998). The effect of computer-assisted interviewing on data quality. In: B. Blyth (ed.), Market Research and Information Technology. Application and Innovation. Esomar Monograph 6. Amsterdam: Esomar, pp. 173-198.Google Scholar
  11. De Leeuw, E. D., Hox, J., Kef, S. & Van Hattum, M. (1997). Overcoming the problems of special interviews on sensitive topics: computer assisted self-interviewing tailored for young children and adolescents. Sawtooth Software Conference Proceedings. Sequim, WA: Sawtooth.Google Scholar
  12. Dillman, D. A. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley (a fully revised and updated version is now in press).Google Scholar
  13. Dippo, C. S. (1997). Survey measurement and process improvement: concepts and integration. In: L. Lyberget al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 457-474.Google Scholar
  14. Dykema, J., Lepkowski, J. M. & Blixt, S. (1997). The effect of interviewer and respondent behavior on data quality: an analysis of interaction coding in a validation study. In: L. Lyberg et al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 287-310.Google Scholar
  15. Engel, U. & Reinecke, J. (1994). Panelanalyse: Grundlagen, Techniken, Beispiele. Berlin: Walter de Gruyter.Google Scholar
  16. Fowler, F. J., Jr. (1991). Reducing interviewer related error through interviewer training, supervision and other means. In: P. Biemer et al. (eds), Measurement Errors in Surveys. New York: Wiley, pp. 259-278.Google Scholar
  17. Freedman, D. S., Thornton, A. & Camburn, D. (1980). Maintaining response rates in longitudinal studies. Sociological Methods & Research 9: 87-98.Google Scholar
  18. Forsyth, B. H. & Lessler, J. T. (1991). Cognitive laboratory methods: a taxonomy. In: P. Biemer et al. (eds), Measurement Errors in Surveys. New York: Wiley, pp. 393-418.Google Scholar
  19. Groves, R. M. (1989). Survey Errors and Survey Costs. New York: Wiley.Google Scholar
  20. Groves, R. M. & Couper, M. P. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.Google Scholar
  21. Hermkens, P. L. J. (1983). Oordelen over de rechtvaardigheid van inkomens [In Dutch: Judgements on the Fairness of Income]. Amsterdam: Kobra.Google Scholar
  22. Herzog, A. R. & Rodgers, W. L. (1992). The use of survey methods in research on older Americans. In: R. B. Wallace & R. F. Woolson (eds). The Epidemiological Study of the Elderly. Oxford: Oxford University Press.Google Scholar
  23. Hippler, H.-J., Schwarz, N. & Singer, E. (1990). Der influess von Datenschutzzusagen auf die teilnamebereitschaft an Umfragen[In German: The influence of dataprotection reassurance on the willingness to participate in a survey]. ZUMA nachrichten 27: 54-67.Google Scholar
  24. Hox, J. J. (1999). A review of current software for handling missing data. Kwantitieve Methoden (in press).Google Scholar
  25. Hox, J. J., Kreft, I. G. G. & Hermkens, P. L. J. (1991). The analysis of factorial surveys. Sociological Methods & Research 19: 493-510.Google Scholar
  26. Huisman, M., Krol, B. & Van Sonderen, F. L. P. (1998). Handling missing data by reapproaching nonrespondents. Quality and Quantity 32: 77-91.Google Scholar
  27. Huisman, M. (1999). Item Nonresponse: Occurrence, Causes, and Imputation of Missing Answers to Test Items. Leiden: DSWO Press.Google Scholar
  28. Jansen, M. G. H. (1997). The rasch model for speed tests and some extensions with applications to incomplete designs. Journal of Educational and Behavioral Statistics 22: 125-140.Google Scholar
  29. Jenkins, C. R. & Dillman, D. A. (1997). Towards a theory of self-administered questionnaire design. In: L. Lyberg et al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 165-196.Google Scholar
  30. Kasprzyk, D., Duncan, G. J., Kalton, G. & Singh, M. P. (1989). Panel Surveys. New York: Wiley.Google Scholar
  31. Krosnick, J. A. & Fabrigar, L. R. (1997). Designing rating scales for effective measurement in surveys. In: L. Lyberg et al. (eds), Survey Measurement and Process Quality. New York: Wiley, pp. 141-164.Google Scholar
  32. Lavrakas, P. (1999). Personal communication to AAPOR-net, June 11.Google Scholar
  33. Leigh, J. H. & Martin, C. R. (1987). Do-not-know item nonresponse in telephone surveys: effects of question form and respondent characteristics. Journal of Marketing Research 24: 418-424.Google Scholar
  34. Lessler, J. T. & Kalsbeek, W. D. (1992). Nonsampling Error in Surveys. New York: Wiley.Google Scholar
  35. Little, R. J. A. & Rubin, D. B. (1987). Statistical Analysis with Missing Data. New York: Wiley.Google Scholar
  36. Martin, J. et al. (1996). Task Force on Imputation, Report on Imputation. Government Statistical Services, Methodology Series, #3, UK: GSS/ONS.Google Scholar
  37. Morton-Williams, J. (1993). Interviewer Approaches. Aldershot: Darthmouth PublicationsGoogle Scholar
  38. McCrossan, L. (1991). A Handbook for Interviewers. London: HMSO.Google Scholar
  39. Nicholls, W. L. II., Baker, R. P. & Martin, J. (1997). The effect of new data collection technologies on survey data quality. In: L. Lyberg et al. (eds). Survey Measurement and Process Quality. New York: Wiley, pp. 221-248.Google Scholar
  40. Saris, W. E. (1998). Ten years of interviewing without interviewers. In: M. P. Couper et al. (eds), Computer Assisted information Collection. New York: Wiley.Google Scholar
  41. Skinner, C. (1999). Developing an imputation strategy, with illustrations from a self-completion survey of local authorities. Lecture presented at the Survey Methods Centre seminar on item nonresponse in surveys. London: Royal Statistical Society, March 1999.Google Scholar
  42. Strack, F. & Martin, L. (1987). Thinking judging and communicating: a process account of context effects in attitude surveys. In: H. J. Hippler et al. (eds), Social Information Processing and Survey Methodology. New York: Springer Verlag, pp. 123-148.Google Scholar
  43. Sudman, S. & Bradburn, N. M. (1974). Response Effects in Surveys. Chicago: Aldine.Google Scholar
  44. Schuman, H. H. & Presser, S. (1981). Questions &Answers in Attitude Surveys. New York: Academic Press.Google Scholar
  45. Scherpenzeel, A. & Saris, W. (1997). The validity and reliability of survey questions: a meta-analysis of MTMM studies. Sociological Methods and Research 25: 341-383.Google Scholar
  46. Schwarz, N. (1997). Questionnaire design: the rocky road from concepts to answers. In: L. Lyberg et al. (eds). Survey Measurement and Process Quality. New York: Wiley, pp. 29-45.Google Scholar
  47. Snijkers, G., Akkerboom, H., Kuijpers, I, De Leeuw, E. (1996). Computer-assisted qualitative interviewing: an intermediate technology of quality assessment. Paper presented at INTERCASIC '96, San Antonio, Texas, 11-12 December 1996.Google Scholar
  48. Tourangeau, R. (1984). Cognitive science and survey methods: a cognitive perspective. In: T. Jabine et al. (eds), Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines. Washington DC: National Academy Press, pp. 73-100.Google Scholar
  49. Van Hattum, M. & Leeuw, E. D. (1999). A disk-by-mail survey of teachers and pupils in Dutch primary schools: logistics and data quality. Journal of Official Statistics 3(in press).Google Scholar
  50. Van de Pol, F. J. R. (1989). Issues of Design and Analysis of Panels. Amsterdam: Sociometric Research Foundation.Google Scholar
  51. Van der Zouwen, J., Dijkstra, W. & Smith, J. (1991). Studying respondent-interviewer interaction: the relationship between interviewer style, interviewer behavior and response behavior. In: P. Biemer et al. (eds), Measurement Errors in Surveys. New York: Wiley, pp. 419-438.Google Scholar
  52. Vermunt, J. (1996). Causal log-linear modeling with latent variables and missing data. In: U. Engel & J. Reinecke (eds). Analysis of Change. Advanced Techniques in Panel Data Analysis. New York: De Gruyter.Google Scholar
  53. Weisband, S. & Kiesler, S. (1996). Self Disclosure on Computer Forms: Meta-Analysis and Implications. Tucson: University of Arizona. (http://www.al.arizona.edu_weisband/chi/chi96.html)Google Scholar

Copyright information

© Kluwer Academic Publishers 2001

Authors and Affiliations

  • Edith D. de Leeuw
    • 1
  1. 1.MethodikA AmsterdamCN AmsterdamThe Netherlands

Personalised recommendations