Advertisement

Social Indicators Research

, Volume 127, Issue 3, pp 1005–1020 | Cite as

Correction for Measurement Errors in Survey Research: Necessary and Possible

  • Willem E. Saris
  • Melanie Revilla
Article

Abstract

Survey research is the most frequently used data collection method in many disciplines. Nearly, everybody agrees that such data contain serious measurement errors. However, only few researchers try to correct for them. If the measurement errors in the variables vary, the comparison of the sizes of effects of these variables on each other will be wrong. If the sizes of the measurement errors are different across countries, cross-national comparisons of relationships between variables cannot be made. There is ample evidence for these differences in measurement errors across variables, methods and countries (Saris and Gallhofer in Design, evaluation and analysis of questionnaires for survey. Wiley, Hoboken, 2007; Oberski in Measurement errors in comparative surveys. PhD thesis, University of Tilburg, 2011). Therefore, correction for measurement errors is essential for the social sciences. The correction for measurement errors can be made in a simple way, but it requires that the sizes of the error variances are known for all observed variables. Many experiments are carried out to determine the quality of questions. The relationship between the quality and the characteristics of the questions has been studied. Because this relationship is rather strong, one can also predict the quality of new questions. A program SQP has been developed to predict the quality of questions. Using this program, the quality of the questions (complement of error variance) can be obtained for nearly all questions measuring subjective concepts. For objective variables, other research needs to be used (e.g., Alwin in Margins of error: a study of reliability in survey measurement. Wiley, Hoboken, 2007). Using these two sources of information, making correction for measurement error in survey research is possible. We illustrate here that correction for measurement errors can and should be performed.

Keywords

Correction for measurement errors Quality SQP 2.0 European Social Survey 

References

  1. Alwin, D. F. (2007). Margins of error: A study of reliability in survey measurement. Hoboken: Wiley.CrossRefGoogle Scholar
  2. Alwin, D. F., & Krosnick, I. A. (1991). The reliability of survey attitude measurement. The influence of question and respondent attributes. Sociological Methods and Research, 20, 139–181.CrossRefGoogle Scholar
  3. Andrews, F. M. (1984). Construct validity and error components of survey measures: A structural equation approach. Public Opinion Quarterly, 48, 409–442.CrossRefGoogle Scholar
  4. Belson, W. (1981). The design and understanding of survey questions. London: Gower.Google Scholar
  5. Biemer, P. R. (2011). Latent class analysis of survey errors. Hoboken: Wiley.Google Scholar
  6. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.CrossRefGoogle Scholar
  7. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrices. Psychological Bulletin, 56, 81–105.CrossRefGoogle Scholar
  8. De Castellarnau, A., & Saris, W. E. (2014). A simple way to correct for measurement errors. European Social Survey Education Net (ESS EduNet). http://essedunet.nsd.uib.no/cms/topics/measurement/
  9. Dillman, D. A. (2000). Mail and internet surveys: The tailored design method. Hoboken: Wiley.Google Scholar
  10. Goldberger, A. S., & Duncan, O. D. (Eds.). (1973). Structural equation models in the social sciences. New York: Seminar Press.Google Scholar
  11. Hagenaars, J. (1988). Latent structure model with direct effects between indicators; local dependency models. Sociological Methods and Research, 16, 379–405.CrossRefGoogle Scholar
  12. Hambleto, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. London: Sage.Google Scholar
  13. Harman, H. H. (1976). Modern factor analysis (3rd ed.). Chicago: University of Chicago Press.Google Scholar
  14. Heise, D. R. (1969). Separating reliability and stability in test–retest-correlation. American Sociological Review, 34, 93–101.CrossRefGoogle Scholar
  15. Jöreskog, K. G. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences. New York: Academic Press.Google Scholar
  16. Költringer, R. (1993). Gültigkeit von Umfragendaten. Wien: Bohlau.Google Scholar
  17. Költringer, R. (1995). Measurement quality in Austrian personal interview surveys. In W. E. Saris & A. Münnich (Eds.), The multitrait-multimethod approach to evaluate measurement instruments (pp. 207–225). Budapest: Eötvös University Press.Google Scholar
  18. Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method. London: Butterworth.Google Scholar
  19. Lord, F., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesly.Google Scholar
  20. Mokken, R. J. (1971). A theory and procedure of scale analysis: With applications in political research. The Hague: Mouton.CrossRefGoogle Scholar
  21. Molenaar, N. I. (1986). Formuleringseffecten in survey-interviews. PhD thesis, Amsterdam: Free University.Google Scholar
  22. Oberski, D.T. (2011). Measurement errors in comparative surveys. PhD thesis, University of Tilburg.Google Scholar
  23. Oberski, D.T., Gruner, T., & Saris, W. E. (2011). The prediction procedure the quality of the questions based on the present data base of questions In W. E Saris, D. Oberski, M. Revilla, D. Zavalla, L. Lilleoja, I. Gallhofer, & T. Grüner. (Eds.), The development of the Program SQP 2.0 for the prediction of the quality of survey questions. RECSM Working paper 24, chapter 6. Google Scholar
  24. Saris, W. E, Oberski, D., Revilla, M., Zavalla, D., Lilleoja, L., Gallhofer, I., & Grüner, T. (2011). The development of the Program SQP 2.0 for the prediction of the quality of survey questions. RECSM Working paper 24. Google Scholar
  25. Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen: Danish Institute for Education Research.Google Scholar
  26. Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation and analysis of questionnaires for survey research. Hoboken: Wiley.CrossRefGoogle Scholar
  27. Saris, W. E., & Gallhofer, I. N. (2014). Design, evaluation and analysis of questionnaires for survey research (2nd ed.). Hoboken: Wiley.CrossRefGoogle Scholar
  28. Saris, W. E., Satorra, A., & Coenders, G. (2004). A new approach for evaluating quality of measurement instruments. Sociological Methodology, 3, 311–347.CrossRefGoogle Scholar
  29. Oberski, D. T., & Satorra, A. (2013). Measurement error models with uncertainty about the error variance. Structural Equation Modeling, 20, 409–428.Google Scholar
  30. Scherpenzeel, A. C. (1995). A question of quality. Evaluating survey questions by multitrait-multimethod studies. Leidschendam: KPN Research.Google Scholar
  31. Schuman, H., & Presser, S. (1981). Questions and answers in attitude survey: Experiments on question form, wording and context. New York: Academic Press.Google Scholar
  32. Sudman, S., & Bradburn, N. M. (1982). Asking questions: A practical guide to questionnaire design. San Francisco: Jossey Bass.Google Scholar
  33. Torgerson, W. S. (1958). Theory and methods of scaling. London: Wiley.Google Scholar
  34. Tourangeau, R., Rips, J., & Rasinski, K. (2000). The psychology of survey response. Cambridge, MA: Cambridge University Press.CrossRefGoogle Scholar
  35. Van Schuur, W. H. (1997). Nonparametric IRT models for dominance and proximity data. In M. Wilson, G. Engelhard Jr, & K. Draney (Eds.), Objective measurement: Theory into practice (Vol. 4, pp. 313–331). Greenwich: Ablex Publishing Corporation.Google Scholar
  36. Vermunt, J. K. (2003). Multilevel latent class models. Sociological Methodology, 33, 213–239.CrossRefGoogle Scholar
  37. Wiley, D. E., & Wiley, I. A. (1970). The estimation of measurement error in panel data. American Sociological Review, 35, 112–117.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.RECSMUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations