Abstract
When measuring (complex) attitudes within a social survey, researchers often use balanced lists of positive and negative items. The purpose of the present research is to investigate: (a) whether a specific order of measurement scale items can lead to the bipolar (single-dimensional) concept (attitude) being recognised as a dual (bi-dimensional) concept and vice-versa; and (b) whether item order can affect the consistency (metric characteristics) of a measurement scale. An experiment on a group of social science students was conducted: students were randomly split into three subgroups and three different version of a questionnaire (with three differing item orders) were applied. A multi-group confirmatory factor analysis (‘CFA’) and a single group CFA for each item order separately were applied. The final conclusion of the experiment is that there is no general rule about how and when respondents form separate (dual) or unidimensional (continuous) representations of measured concepts. Item-order effects are possible, but they are not as important as one would expect. The results of the experiment also suggest that other factors should be taken into account: the content of the measured concept and the cognitive sophistication of the respondents.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cambré, B., Welkenhuysen-Gybels, J., Billiet, J.: Is it content or style? An evaluation of two measurement models applied to a balanced set of ethnocentrism items. IJCS (2002). doi:10.1177/002071520204300101
Cheung, M.W.-L., Leung, K., Au, K.: Evaluating multilevel models in cross-cultural research: an illustration with social axioms. JCCP (2006). doi:10.1177/0022022106290476
Fishbein M., Ajzen I.: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Addison-Wesley, London (1975)
Hair, M.: Using an individual differences perspective to investigate context effects in surveys: a test of the belief-sampling model. IJPOR (2005). doi:10.1093/ijpor/edho49
Hittner, J.B.: Factorial invariance of the 13-item sense of coherence scale across gender. JHP (2007). doi:10.1177/1359105307074256
Johnson, T.P., O’Rourke, D., Severns, E.: Effects of question context and response order on attitude questions. In: 1998 Proceedings of the Section on Survey Research Methods, pp. 857–860. American Statistical Association, Alexandria, VA (1999). http://www.amstat.org/sections/srms/proceedings/. Accessed 14 September 2009
Krosnick J.A.: The impact of cognitive sophistication and attitude importance on response order and question-order effects. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 203–218. Springer, New York (1992)
Li, T., Wehr, K.: The validity of the factor structure of the general social survey environmentalism scales across gender and ethnicity in the United States. Organ. Environ. (2007). doi:10.1177/1086026607306461
MacIntosh R.: A confirmatory factor analysis of the affect balance scale in 38 nations: a research note. Soc. Psychol. Q. 61, 83–91 (1998)
Marsh H.W., Hocevar D.: Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. Psychol. Bull. 97, 562–582 (1985)
Ostrom T.M.: Bipolar survey items: an information-processing perspective. In: Hippler, H.-J., Schwartz, N., Sudman, S. (eds) Social Information Processing and Survey Methodology, pp. 71–85. Springer, New York (1987)
Ostrom T., Betz A.M., Skowronski J.J.: Cognitive representation of bipolar survey items. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 297–311. Springer, New York (1992)
Schuman H., Presser S.: Questions and Answers in Attitude Surveys: Experiments in Question Form, Wording, and Context. Academic Press, New York (1981)
Smith, T.W.: Ballot position: an analysis of context effects related to rotation design (GSS Methodological Report No. 55). NORC, Chicago (1988)
Smith T.W.: Thoughts on the nature of context effects. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 163–184. Springer, New York (1992)
Spini, D.: Measurement equivalence of 10 value types from the Schwartz value survey across 21 countries. JCCP (2003). doi:10.1177/0022022102239152
Sudman S., Bradbourn N.M., Schwartz N.: Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. Jossey-Bass Publishers, San Francisco (1996)
Tourangeau R., Rasinski K.A.: Cognitive processes underlying context effects in attitude measurement. Psychol. Bull. 103, 299–314 (1988)
Tourangeau, R., Singer, E., Presser, S.: Context effects in attitude surveys. Effects on remote items and impact on predictive validity. SMR (2003). doi:10.1177/0049124103251950
Zaller J., Feldman S.: A simple theory of the survey response: answering questions versus revealing preferences. Am. J. Political Sci. 36, 579–616 (1992)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hafner-Fink, M., Uhan, S. Bipolarity and/or duality of social survey measurement scales and the question-order effect. Qual Quant 47, 839–852 (2013). https://doi.org/10.1007/s11135-011-9569-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11135-011-9569-z