Order Effects in Batteries of Questions
- Peter Siminski
- … show all 1 hide
Purchase on Springer.com
$39.95 / €34.95 / £29.95*
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.
Batteries of questions with identical response items are commonly used in survey research. This paper suggests that question order has the potential to cause systematic positive or negative bias on responses to all questions in a battery. Whilst question order effects have been studied for many decades, almost no attention has been given to this topic. The primary aim is to draw attention to this effect, to demonstrate its possible magnitude, and to discuss a range of mechanisms through which it might occur. These include satisficing, anchoring and cooperativeness. The effect seems apparent in the results of a recent survey. This was a survey of Emergency Department patients presenting to Wollongong Hospital (Australia) with apparently less urgent conditions in 2004. Two samples were taken. Question order was fixed in the first sample (n = 104; response rate RR2 = 94%), but randomised in the second sample (n = 46; response rate RR2 = 96%). Respondents were asked to indicate whether each of 18 reasons for presenting to the ED was a ‘very important reason’ a ‘moderately important reason’ or ‘not a reason’ The mean number of very important reasons selected was 56% higher in the first sample as compared to the second sample.
- Alwin D.F., Krosnick J.A. (1991). The reliability of survey attitude measurement: the influence of question and respondent attributes. Sociological Methods and Research 20:139–181 CrossRef
- American Association for Public Opinion Research (2004) Standard Definitions: Final Disposition of Case Codes and Outcome Rates for Surveys, 3rd edn. Lenexa, Kansas: AAPOR. Available online at http://www.aapor.org (accessed July 2005).
- Dillman, D. A., Smyth, J. D., Christian, L. M. & Stern, M. J. (2003). Multiple Answer Questions in Self-Administered Surveys: The Use of Check-All-That-Apply and Forced-Choice Question Formats Paper presented at the annual meeting of the American Statistical Association, San Fransisco, CA.
- Duffy B. (2003). Response order effects-how do people read?. International Journal of Market Research 45: 457–475
- Grice, H. P. (1975). Logic and conversation. In: P. Cole, & J. L. Morgan (eds.), Syntax and Semantics, Vol. 3. Speech Acts. New York: Academic Press, pp. 41–48.
- Knauper B. (1999). The impact of age and education on response order effects in attitude measurement. Public Opinion Quarterly 63:347–370 CrossRef
- Krosnick J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5: 213–236 CrossRef
- Krosnick J.A., Alwin D.F. (1987). An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opinion Quarterly 51: 201–219 CrossRef
- McClendon M. J. (1986). Response order effects for dichotomous questions. Social Science Quarterly 67: 205–211
- Narayan S., Krosnick J.A. (1996). Education moderates some response effects in attitude measurement. Public Opinion Quarterly 51: 201–219
- Schwarz N. (1996). Cognition and Communication: Judgmental Biases, Research Methods, and the Logic of Conversation. Lawrence Erlbaum Associates, New Jersey
- Schwarz, N. & Hippler, H. J. (1991). Response alternatives: the impact of their choice and presentation order. In: P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (eds.), Measurement Error in Surveys. Wiley Series in Probability and Mathematical Statistics New York: Wiley, pp. 41–46.
- Siminski P., Cragg S., Middleton R., Masso M., Lago L., Green J., Eagar K. (2005). Primary care patients’ views on why they present to Emergency Departments—inappropriate attendances or inappropriate policy?. Australian Journal of Primary Health 11: 87–95 CrossRef
- Sudman S., Bradburn N.M., Schwarz N. (1996). Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. Jossey-Bass, San Fransisco
- Tourangeau R. (1984). Cognitive science and survey methods. In: Jabine T., Straf M., Tanur J., Tourangeau R. (eds). Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines. National Academy Press, Washington, DC, pp. 73–100
- Tourangeau R., Rasinski K. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin 103: 299–314 CrossRef
- Tourangeau R., Rips L.J., Rasinski K. (2000). The Psychology of Survey Response. Cambridge University Press, Cambridge
- Wanke M, Schwarz N. (1997). Reducing question order effects: the operation of buffer items. In: Lyberg L.E., Biemer P.P., Collins M., de Leeuw E., Dippo C., Trewin D. (eds). Survey Measurement and Process Quality. Wiley, New York, pp. 115–139
- Order Effects in Batteries of Questions
Quality and Quantity
Volume 42, Issue 4 , pp 477-490
- Cover Date
- Print ISSN
- Online ISSN
- Springer Netherlands
- Additional Links
- question order effects
- survey research
- survey design
- emergency department
- context effects
- Peter Siminski (1)
- Author Affiliations
- 1. Centre for Health Service Development, University of Wollongong, Wollongong, NSW, 2522, Australia