Abstract
Using administrative data as validating standard, we studied the combined effects of two sources of survey error – nonresponse and recall errors – on distributional and substantive bias in a mail survey of absence because of illness among the employees of a Dutch road building company (response rate 77%). No distributional bias was found in five socio-demographic variables (sex, age, years of service, function, and district), but both nonresponse bias and recall bias occurred in our central dependent variables: frequency and duration of absence because of illness. Nonrespondents were on sick leave more frequently and longer than respondents. Furthermore, the self-reports of absence because of illness of our respondents proved to be rather inaccurate. Underreporting of frequency and duration of sick leave was more common than overreporting. Therefore, both sources of error had a cumulative effect.
While nonresponse did not result in biased relationships, recall errors had clearly biasing consequences: seven out of 30 correlation coefficients analyzed were too biased to produce valid outcomes; another six were substantially biased. Multiple regression used for predicting recent absence because of illness among our respondents also led to different outcomes depending on the choice of data source (administration or questionnaire) for our absence variables.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Blalock, H. M. (1979). Social Statistics(Revised 2nd edn). New York: McGraw-Hill.
Cohen, G. & Java, R. (1995). Memory for medical history: Accuracy of recall. Applied Cognitive Psychology9: 273–288.
Cohen, J. (1994). The earth is round (p <0:05). American Psychologist49: 997–1003.
De Graaf, P. M., Poortman, A. & Ultee, W. C. (1996). De kwaliteit van retrospectieve beroepsgegevens; een onderzoek op basis van huwelijksaktes. Sociale Wetenschappen39(3): 1–15.
DeMaris, A. & Jackson, J. (1986). Nonrespondent characteristics and bias in a study of batterers. Social Service Review60: 460–474.
Dex, S. (1995). The reliability of recall data: A literature review. Bulletin de Méthodologie Sociologique49 (December): 58–89.
Dijkstra, A. (1981). Ziekteverzuim en non-response: Representativiteit van deelname aan enquê teonderzoek in arbeidsorganisaties ten aanzien van een te verklaren variabele. Gezondheid en Samenleving2: 266–273.
Dillman, D. A. (1991). The design and administration of mail surveys. Annual Review of Sociology17: 225–249.
Evans, D. S. & Leighton, L. S. (1995). Retrospective bias in the Displaced Worker Surveys. The Journal of Human Resources30: 386–396.
Goudy, W. J. (1976). Nonresponse effects on relationships between variables. Public Opinion Quarterly40: 360–369.
Goudy, W. J. (1978). Interim response to a mail questionnaire: Impacts on variable relationships. Sociological Quarterly19: 253–265.
Goyder, J. (1987). The Silent Minority: Nonrespondents on Sample Surveys. Oxford: Polity Press.
Gray, P. G. (1955). The memory factor in social surveys. Journal of the American Statistical Association50: 344–363.
Green, J.M. (1996).Warning that reminders will be sent increased response rate. Quality & Quantity30: 449–450.
Green, K. E. (1991). Reluctant respondents: Differences between early, late, and nonresponders to a mail survey. Journal of Experimental Education59: 268–276.
Groves, R. M. (1987). Research on survey data quality. Public Opinion Quarterly51: S156–S172.
Hawkins, D. F. (1975). Estimation of nonresponse bias. Sociological Methods and Research3: 461–488.
Hessing, D. J., Elffers, H. & Weigel, R. H. (1988). Exploring the limits of self-reports and reasoned action: An investigation of the psychology of tax evasion behavior. Journal of Personality and Social Psychology54: 405–413.
Loftus, E. F., Smith, K. D., Klinger, M. R. & Fiedler, J. (1992). Memory and mismemory for health events. In: J. M. Tanur (ed.), Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation, pp. 102- 137.
McKee, D. O. (1992). The effect of using a questionnaire identification code and message about nonresponse follow-up plans on mail survey response characteristics. Journal of theMarket Research Society34: 179–191.
Martin, Ch. L. (1994). The impact of topic interest on mail survey response behaviour. Journal of the Market Research Society36: 327–338.
O'Muircheartaigh, C. A. (1976). Response errors in an attitudinal sample survey. Quality & Quantity10: 97–115.
O'Neill, M. J. (1979). Estimating the nonresponse bias due to refusals in telephone surveys. Public Opinion Quarterly43: 218–232.
Pavalko, R. M. & Lutterman, K. G. (1973). Characteristics of willing and reluctant respondents. Pacific Sociological Review16: 463–476.
Pearson, R.W., Ross, M. & Dawes, R.M. (1992). Personal recall and the limits of retrospective questions in surveys. In: J. M. Tanur (ed.), Questions about Questions: Inquiries into the Cognitive Bases of Surveys. New York: Russell Sage Foundation, pp. 65–94.
Popping, R. (1995). Computing Agreement on Nominal Data: The Computer Program AGREE 6.0. Groningen: iec ProGAMMA.
Presser, S. & Traugott, M. (1992). Little white lies and social science models: Correlated response errors in a panel study of voting. Public Opinion Quarterly56: 77–86.
Silver, B. D., Anderson, B. A. & Abramson, P. R. (1986). Who overreports voting? American Political Science Review80: 613–624.
Stinchcombe, A. L., Jones, C. & Sheatsley, P. (1981). Nonresponse bias for attitude questions. Public Opinion Quarterly45: 359–375.
Sudman, S. & Bradburn, N. M. (1974). Response Effects in Surveys: A Review and Synthesis. Chicago: Aldine.
Van Goor, H. & Stuiver, B. (1998a). A wave-analysis of distributional bias, substantive bias and data quality in a mail survey among Dutch municipalities. Acta Politica33: 179–196.
Van Goor, H. & Stuiver, B. (1998b). Can weighting compensate for nonresponse bias in a dependent variable? An evaluation of weighting methods to correct for substantive bias in a mail survey among Dutch municipalities. Social Science Research27: 481–499.
Van Goor, H. & Verhage, A. L. (MS). Recall errors in a study of absence because of illness: Differences in the accuracy of self-reports of sick-leave as a consequence of problems of self-presentation, memory effects, and working conditions.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Van Goor, H., Verhage, A.L. Nonresponse and Recall Errors in a Study of Absence because of Illness: An Analysis of Their Effects on Distributions and Relationships. Quality & Quantity 33, 411–428 (1999). https://doi.org/10.1023/A:1004732502598
Issue Date:
DOI: https://doi.org/10.1023/A:1004732502598