Abstract
So far, certain problems of web surveys have received great attention: Coverage problems, sampling issues and response rates as well as screen design and usability issues. With some of the disadvantages mentioned diminishing, web surveys are increasingly adopted by survey researchers. In terms of the validity of web surveys early studies suggested that — controlling for other sources of survey error — little differences are to be expected in terms of response distributions.
However, little insight is available regarding the cognitive processes at work in a respondent’s mind when answering a survey question on the internet. We assume that the cognitive capacity available when answering a survey question is impaired by the respondent’s navigational efforts when working through the questionnaire. Comparing the navigational tasks in a paper and pencil survey and a web survey, the later is considered to be more demanding compared with a paper questionnaire. Reading and flipping through pages is a task that can be done with little distraction from the question answer process. By contrast, browsing through an online form using mouse and buttons while reading is more demanding and distracting for the respondent. Thus, working through an online survey reduces the respondent’s attention for the question answer process.
In this paper we present results from a large scale field experiment (n = 5,000) on measurement error in mixed mode survey (p&p and online) among the German youth conducted in spring of 2001. In this survey, 5 experiments on measurement error were incorporated: 1 experiment on question order, 2 experiments on response categories, and 1 experiment on response order in long lists and 1 experiment on social desirability. The results suggest that measurement effects are generally smaller in the web portion of the survey. In the conclusions the paper discusses the question whether or not we need to worry about the respondent’s redued atteniton to the survey questions under web conditions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Literatur
ADM, ASI, BVM (1999): Standards zur Qualitätssicherung in der Markt-und Sozialforschung. Arbeitskreis deutscher Markt-und Sozialforschungsinstitute e. V., Frankfurt a. M.
ADM ASI; Statistisches Bundesamt (Hrsg.) (2003): Online-Erhebungen. 5. Wissenschaftliche Tagung. Sozialwissenschaftliche Tagungsberichte Band 7, IZ-Sozialwissenschaften, Bonn.
Bandilla, W.; Bosnjak, M.; Altdorfer, P. (2001): Effekte des Erhebungsverfahrens? Ein Vergleich zwischen einer Web-basierten und einer schriftlichen Befragung zum ISSP-Modul Umwelt. In: ZUMA-Nachrichten, 49: 7–27.
Borgers, N.; de Leeuw, E.; Hox, J. (2000): Children as respondents in survey research: Cognitive development and response quality. In: Bulletin de Methodologie Sociologique (BMS), 66, 2000, 60–75.
Bailar, B. A: (1984): The quality of survey data. 1984 Proceedings of the section on survey research methods of the American Statistical Association. Alexandria (USA), 43–52.
Bosnjak, M. (2003): Web-basierte Fragebogenuntersuchungen — Methodische Möglichkeiten, aktuelle Themen und Erweiterungen. In: ADM ASI; Statistisches Bundesamt (Hrsg.): Online-Erhebungen. 5. Wissenschaftliche Tagung. Sozialwissenschaftliche Tagungsberichte Band 7, IZ-Sozialwissenschaften, Bonn.
Burr, M. A.; Levin, K. Y.; Becher, A. (2001): Examining web vs. paper mode effects in a Federal government customer satisfaction study. Paper presented at 2001 AAPOR Annual Conference, Montreal, Quebec, Canada, May 17–20, 2001.
Couper, M. P. (2001): Web surveys: A review of issues and approaches. In: Public Opinion Quarterly, 64(4): 464–494.
Couper, M. P. (2002): Designing a strategy for reducing ‘no opinion’ responses in web-based surveys. In: Social Science Computer Review, 20(1): 3–9.
Couper, M. P.; Tourangeau, R.; Steiger, D. M. (2001): Social presence in web surveys. Paper presented at the CHI 2001, Conference on Human Factors in Computing Systems, April 2001, Seattle, USA.
Crawford, S. D.; Couper, M. P.; Lamias, M. J. (2001): Web Surveys: perception of burden. In: Social Science Computer Review, 19(2): 146–162.
Crowne, D.; Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. In: Journal of Consulting Psychology, 24, 349–354.
Danielson, S. (2002): The propensity score and estimation in nonrandom surveys — an overview. Report No. 18 from the project ‘Modern statistical survey methods’, University of Linköping, Department of Statistics, Sweden.
Dillman, D. A. (2000): Mail and internet surveys. The tailored design method. New York.
Dillman, D. A., Phelps, G.; Tortora, R.; Swift, K.; Kohrell, J.; Berck, J. (2001): Response rate and measurement differences in mixed mode surveys using mail, telephone, interactive voice response and the internet. Paper presented at 2001 AAPOR Annual Conference, Montreal, Quebec, Canada, May 17–20, 2001.
Forsman, G.; Varedian, M. (2002): Mail and web surveys: a cost and response rate comparison in a study of students housing conditions. Paper presented at the ICIS, Copenhagen, Denmark.
Fowler, F. J.; Mangione, T. W. (1990): Standardized survey interviewing. Newbury Park.
Fuchs, M. (2000): Screen design and question order in a CAI Instrument — Results from a usability field experiment. In: Survey Methodology, 26(2): 199–207.
Fuchs, M. (2002): The impact of technology on interaction in computer-assisted interviews. In: Maynard, D. W., Houtkoop-Steenstra, H., Schaeffer, N. C., Zouwen, H. van der (Hrsg.): Standardization and tacit knowledge. Interaction and practice in the survey interview. New York, 471–491.
Fuchs, M. (2003): Data quality in surveys among the youth. Item non-response and response errors. Vortrag anlässlich der internationalen Fachtagung „Methoden und Techniken der Kindheitsforschung“, Schloss Hohenkammer, April 2003.
Fuchs, M. (2004): Kinder und Jugendliche als Befragte. Feldexperimente zur Befragung Minderjähriger. In: ZUMA-Nachrichten, Nr. 54, 2004 (zur Publikation angenommen).
Fuchs, M.; Couper, M. P.; Hansen, S. E. (2000): Technology effects: Do CAPI or PAPI interviews take longer? Journal of Official Statistics, 16(3): 273–286.
Göritz, A. S.; Moser, K. (2000): Repräsentativität im Online-Panel. In: der markt, Jg. 39, Nr. 155, 156–162.
Göritz, A. S. (2001): Online-Panels. In: Theobald, A., Dreyer, M., Starsetzki, Th. (Hrsg.): Online-Marktforschung. Theoretische Grundlagen und praktische Erfahrungen. Wiesbaden, 67–78.
Groves, R. (1989): Survey error and survey costs. New York.
Fuchs, M. (2003): Querschläger. Jugendliche zwischen rechter Ideologie und Gewalt. Opladen.
Hippler, H. J.; Schwarz, N.; Noelle-Neumann, E. (1989): Response order effects in dichotomous questions: The impact of administrations mode. ZUMA-Arbeitsberichte 89/17. Mannheim.
Hippler, Hans-J.; Schwarz, Norbert (1992): The impact of administration mode on response effects in Surveys. ZUMA-Arbeitsberichte 92/14. Mannheim, Germany.
Krosnick, J. A.; Alwin, D. (1987): An evaluation of cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51: 201–219.
deLeeuw, E.; Collins, M. (1997): Data collection methods and survey quality: An overview. In: Lyberg, L.; Biemer, P.; Collins, M.; deLeeuw, E.; Dippo, C.; Schwarz, N.; Trewin, D. (Hrsg.): Survey measurement and process quality. New York, 199–220.
deLeeuw, E.; Otter, M. (1995): The reliability of children’s responses to questionnaire items: question effects in children’s questionnaire data. In: Hox, J.; van der Meulen, B.; Kanssens, J.; ter Laak, J.; Tavecchio, L. (Hrsg.), Advances in family research. Amsterdam, 251–257.
Lozar Manfreda, K.; Vehovar, V.; Batagelj, Z. (2000): Mode effects in web surveys. Paper presented at the International Conference on Methodology and Statistics, Preddvor, Slovenia, September 24–27, 2000.
Lück, H.; Timaeus, E. (1969): Skalen zur Messung manifester Angst (MAS) und sozialer Wünschbarkeit (SDS-E und SDS-CM). In: Diagnostica, 15, 134–141.
Nielsen, J. (2000): Designing web usability: The practice of simplicity. Indianapolis.
Reinecke, Jost (1991): Interviewer-und Befragtenverhalten: Theoretische Ansätze und methodische Konzepte. Opladen.
Reips, U.-D.; Bosnjak, M. (2001): Dimensions of internet science. Lengerich.
Schonlau M.; Fricker R. D.; Elliott M. N. (2002): Conducting research surveys via e-mail and the web. Rand Corporation, Santa Monica.
Schuman, H.; Presser, S. (1981): Questions and answers in attitude surveys: Experiments in question form, wording, and context. New York.
Schwarz, N.; Hippler, H.-J. (1991): Response alternatives. The impact of their choice and ordering. In: Biemer, P., Groves, R., Mathiowetz, N., Sudman, S. (Hrsg.), Measurement error in surveys. Chichester, 41–56.
Schwarz, N.; Hipper, H.-J. (1995): The numeric values of rating scales: a comparison of their impact in mail surveys and telephone interviews. In: International Journal of Public Opinion Research, 7, 72–74.
Schwarz, N.; Hippler, H.-J.; Deutsch, B.; Strack, F. (1985): Response scales. Effects of category range on reported behaviour and comparative judgements. In: Public Opinion Quarterly, 49, 388–395.
Schwarz, N.; Hippler, H. J.; Noelle-Neumann, E. (1991): A cognitive model of response order effects in survey measurement. In: Schwarz, N.; Sudman, S. (Hrsg.): Context effects in social and psychological research. New York, 187–201.
Scott, J. (1997): Children as respondents: Methods for improving data quality. In: Lyberg, L.; Biemer, P.; Coolins, M.; deLeeuw, E.; Dippo, C.; Schwarz, N.; Trewin, D. (Hrsg.), Survey measurement and process quality. Wiley, New York, 331–350.
Sudman, S.; Bradbrun, N.; Schwarz, N. (1996): Thinking about answers. The application of cognitive processes to survey methodology. San Francisco.
Terhanian G.; Smith, R.; Bremer, J.; Thomas, R. K. (2001): Exploiting analytical advances: minimizing the biases associated with internet-based surveys of non-random samples, ARF/ESCOMAR: Worldwide Online Measurement, ESOMAR Publication Services, Vol. 248, 247–272.
Theobald, A.; Dreyer, M.; Stanetzki, Th. (Hrsg.) (2002): Online-Marktforschung. Theoretische Grundlagen und praktische Erfahrungen. Wiesbaden.
Tourangeau, R.; Rips, L. J.; Rasinski, K. (2000): The psychology of survey response. Cambridge.
Tourangeau, R.; Smith, T. W. (1998): Collecting sensitive information with different modes of data Collection. In: Couper, M. P.; Baker, R. P.; Bethlehem, J.; Clark, C. Z. F.; Martin, J.; Nicholls, W. L.; O’Reilly, J. M. (Hrsg.): Computer assisted survey information collection. New York, 431–453.
Author information
Authors and Affiliations
Corresponding author
Additional information
PD Dr. rer. pol., geb. 1964, Vertretung der Professur für empirische Sozialforschung an der Universität Kassel. Forschungsgebiete: Methodenforschung (insbesondere Computer-gestützte Verfahren), abweichendes Verhalten (insbesondere Gewalt an Schulen, Rechtsextremismus), Migration (insbesondere Aussiedlerzuzug) und Familienforschung (insbesondere Mehrgenerationenhaushalte). Neuere Veröffentlichungen: Devianz und andere gesellschaftliche Probleme, Opladen 2003 (hrsg. mit J. Luedtke), Querschläger — Jugendliche zwischen Gewalt und rechter Ideologie, Opladen 2003 (mit S. Lamnek und R. Wiederer).
Rights and permissions
About this article
Cite this article
Fuchs, M. Kognitive Prozesse und Antwortverhalten in einer Internet-Befragung. ÖZS 28, 19–45 (2003). https://doi.org/10.1007/s11614-003-0020-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11614-003-0020-0