Abstract
Purpose: Research on survey methodology has demonstrated that seemingly innocuous aspects of a survey’s design, such as the order of questions, can produce biased results. The current investigation extends this work by testing whether standard survey introductions alter the observed associations between variables. Methods: In two experimental studies, we invited Parkinson’s disease (PD) patients to participate in a telephone survey of (a) Parkinson’s patients, conducted by a regional medical center, or (b) the general population, conducted by a regional university. The survey in Study 1 (n = 156) first assessed life-satisfaction, and subsequently health satisfaction. In Study 2 (n = 99), we reversed the order of the two questions, asking the health questions first. Results: When the introduction focused on Parkinson’s disease, we observed an increased correlation between life-satisfaction and a later question about health satisfaction (r = 0.34 vs. 0.63 after general population versus Parkinson’s introduction, respectively; Study 1). In Study 2, asking the health questions first resulted in high correlations regardless of the introduction; in addition, judgments of life-satisfaction were lower after the Parkinson’s-focused introduction. Conclusions: When participants were informed prior to the survey that its purpose was to examine well-being in PD, health satisfaction was a much more important component of life-satisfaction, accounting for three times as much variation. We hypothesize that the survey introduction primed participants’ health status, resulting in an artificially large correlation with life-satisfaction.
Similar content being viewed by others
References
N Schwarz (1999) ArticleTitleSelf-reports: How the questions shape the answers Am Psychol 54 93–105 Occurrence Handle10.1037/0003-066X.54.2.93
R Tourangeau LJ Rips K Rasinski (2000) The Psychology of Survey Response Cambridge University Press Cambridge, UK
S Sudman N Bradburn N Schwarz (1996) Thinking About Answers: The Application of Cognitive Processes to Survey Methodology Jossey-Bass San Francisco, CA
F Strack LL Martin N Schwarz (1988) ArticleTitlePriming and communication: The social determinants of information use in judgments of life-satisfaction Eur J Soc Psychol 18 429–442
N Schwarz F Strack (1999) Reports of subjective well-being: Judgmental processes and their methodological implications D Kahneman E Diener N Schwarz (Eds) Well-being: The Foundations of Hedonic Psychology Russell-Sage New York 61–84
S Haberstroh D Oyserman N Schwarz U Kühnen LJ Ji (2002) ArticleTitleIs the interdependent self more sensitive to question context than the independent self? Self-construal and the observation of conversational norms J Exp Soc Psychol 38 323–329 Occurrence Handle10.1006/jesp.2001.1513
S Oishi U Schimmack SJ Colcombe (2003) ArticleTitleThe contextual and systematic nature of life-satisfaction judgments J Exp Soc Psychol 39 232–247 Occurrence Handle10.1016/S0022-1031(03)00016-7
N Schwarz F Strack HP Mai (1991) ArticleTitleAssimilation and contrast effects in part-whole question sequences: A conversational logic analysis Public Opin Q 55 3–23 Occurrence Handle10.1086/269239
N Schwarz H Bless (1992) Constructing reality and its alternatives: Assimilation and contrast effects in social judgment LL Martin A Tesser (Eds) The Construction of Social Judgment Erlbaum Hillsdale, NJ 217–245
N Schwarz (1996) Cognition and Communication: Judgmental Biases, Research Methods and the Logic of Conversation Erlbaum Hillsdale, NJ
A Norenzayan N Schwarz (1999) ArticleTitleTelling what they want to know: Participants tailor causal attributions to researchers’ interests Eur J Soc Psychol 29 1011–1020 Occurrence Handle10.1002/(SICI)1099-0992(199912)29:8<1011::AID-EJSP974>3.0.CO;2-A
HH Clark MF Schober (1992) Asking questions and influencing answers JM Tanur (Eds) Questions About Questions Russel Sage New York 15–48
H Schuman S Presser (1981) Questions and Answers in Attitude Surveys Academic Press New York
HP Grice (1975) Logic and conversation P Cole JL Morgan (Eds) Syntax and Semantics, Vol.3: Speech Acts Academic Press New York 41–58
HH Clark SE Haviland (1977) Comprehension and the given-new contract RO Freedle (Eds) Discourse Production and Comprehension Erlbaum Hillsdale, NJ 1–40
H Cantril (1944) Gauging Public Opinion Princeton University Press Princeton, NJ
SL Payne (1951) The Art of Asking Questions Princeton University Press Princeton
RM Groves RB Cialdini MP Couper (1992) ArticleTitleUnderstanding the decision to participate in a survey Public Opin Q 56 475–495 Occurrence Handle10.1086/269338
RM Groves FJ Fowler MP Couper JM Lepkowski E Singer R Tourangeau (2004) Survey Methodology Wiley New York
MA Okun WA Stock MJ Haring RA Witter (1984) ArticleTitleHealth and subjective well-being: A Meta-Analysis Int J Aging Hum Dev 19 111–132 Occurrence Handle1:STN:280:BiqC3czptVQ%3D Occurrence Handle6519817
M Wänke N Schwarz (1997) Reducing question order effects: The operation of buffer items L Lyberg P Biemer M Collins E DeLeeuw C Dippo N Schwarz (Eds) Survey Measurement and Process Quality Wiley Chichester, UK 115–140
N Schwarz B Knäuper (2000) Cognition, aging, and self-reports D Park N Schwarz (Eds) Cognitive Aging. A primer Psychology Press Philadelphia, PA 233–252
D Kahneman AB Krueger D Schkade N Schwarz AA Stone (2004) ArticleTitleA survey method for characterizing daily life experience: The Day Reconstruction Method (DRM) Science 306 1776–1780 Occurrence Handle10.1126/science.1103572 Occurrence Handle1:CAS:528:DC%2BD2cXhtVKitb7K Occurrence Handle15576620
J Riis G Loewenstein J Baron C Jepson A Fagerlin PA Ubel (2005) ArticleTitleIgnorance of hedonic adaptation to Hemo-Dialysis: A␣study using ecological momentary assessment J Exp Psychol Gen 134 IssueID1 3–9 Occurrence Handle10.1037/0096-3445.134.1.3 Occurrence Handle15702959
J Jaccard CK Wan (1996) LISREL Approaches to Interaction Effects in Multiple Regression Sage Publications Thousand Oaks
S Frederick G Loewenstein (1999) Hedonic adaptation D Kahneman E Diener N Schwarz (Eds) Well-being: Foundations of Hedonic Psychology Russell-Sage New York 302–329
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Smith, D.M., Schwarz, N., Roberts, T.R. et al. Why are You Calling Me? How Study Introductions Change Response Patterns. Qual Life Res 15, 621–630 (2006). https://doi.org/10.1007/s11136-005-4529-5
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11136-005-4529-5