Quality & Quantity

, Volume 47, Issue 2, pp 839–852 | Cite as

Bipolarity and/or duality of social survey measurement scales and the question-order effect

  • Mitja Hafner-FinkEmail author
  • Samo Uhan


When measuring (complex) attitudes within a social survey, researchers often use balanced lists of positive and negative items. The purpose of the present research is to investigate: (a) whether a specific order of measurement scale items can lead to the bipolar (single-dimensional) concept (attitude) being recognised as a dual (bi-dimensional) concept and vice-versa; and (b) whether item order can affect the consistency (metric characteristics) of a measurement scale. An experiment on a group of social science students was conducted: students were randomly split into three subgroups and three different version of a questionnaire (with three differing item orders) were applied. A multi-group confirmatory factor analysis (‘CFA’) and a single group CFA for each item order separately were applied. The final conclusion of the experiment is that there is no general rule about how and when respondents form separate (dual) or unidimensional (continuous) representations of measured concepts. Item-order effects are possible, but they are not as important as one would expect. The results of the experiment also suggest that other factors should be taken into account: the content of the measured concept and the cognitive sophistication of the respondents.


Attitudes Measurement scales Duality Bipolarity Context effects Measurement equivalence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Cambré, B., Welkenhuysen-Gybels, J., Billiet, J.: Is it content or style? An evaluation of two measurement models applied to a balanced set of ethnocentrism items. IJCS (2002). doi: 10.1177/002071520204300101
  2. Cheung, M.W.-L., Leung, K., Au, K.: Evaluating multilevel models in cross-cultural research: an illustration with social axioms. JCCP (2006). doi: 10.1177/0022022106290476
  3. Fishbein M., Ajzen I.: Belief, Attitude, Intention and Behavior: An Introduction to Theory and Research. Addison-Wesley, London (1975)Google Scholar
  4. Hair, M.: Using an individual differences perspective to investigate context effects in surveys: a test of the belief-sampling model. IJPOR (2005). doi: 10.1093/ijpor/edho49
  5. Hittner, J.B.: Factorial invariance of the 13-item sense of coherence scale across gender. JHP (2007). doi: 10.1177/1359105307074256
  6. Johnson, T.P., O’Rourke, D., Severns, E.: Effects of question context and response order on attitude questions. In: 1998 Proceedings of the Section on Survey Research Methods, pp. 857–860. American Statistical Association, Alexandria, VA (1999). Accessed 14 September 2009
  7. Krosnick J.A.: The impact of cognitive sophistication and attitude importance on response order and question-order effects. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 203–218. Springer, New York (1992)CrossRefGoogle Scholar
  8. Li, T., Wehr, K.: The validity of the factor structure of the general social survey environmentalism scales across gender and ethnicity in the United States. Organ. Environ. (2007). doi: 10.1177/1086026607306461
  9. MacIntosh R.: A confirmatory factor analysis of the affect balance scale in 38 nations: a research note. Soc. Psychol. Q. 61, 83–91 (1998)CrossRefGoogle Scholar
  10. Marsh H.W., Hocevar D.: Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. Psychol. Bull. 97, 562–582 (1985)CrossRefGoogle Scholar
  11. Ostrom T.M.: Bipolar survey items: an information-processing perspective. In: Hippler, H.-J., Schwartz, N., Sudman, S. (eds) Social Information Processing and Survey Methodology, pp. 71–85. Springer, New York (1987)CrossRefGoogle Scholar
  12. Ostrom T., Betz A.M., Skowronski J.J.: Cognitive representation of bipolar survey items. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 297–311. Springer, New York (1992)CrossRefGoogle Scholar
  13. Schuman H., Presser S.: Questions and Answers in Attitude Surveys: Experiments in Question Form, Wording, and Context. Academic Press, New York (1981)Google Scholar
  14. Smith, T.W.: Ballot position: an analysis of context effects related to rotation design (GSS Methodological Report No. 55). NORC, Chicago (1988)Google Scholar
  15. Smith T.W.: Thoughts on the nature of context effects. In: Schwartz, N., Sudman, S. (eds) Context Effects in Social and Psychological Research, pp. 163–184. Springer, New York (1992)CrossRefGoogle Scholar
  16. Spini, D.: Measurement equivalence of 10 value types from the Schwartz value survey across 21 countries. JCCP (2003). doi: 10.1177/0022022102239152
  17. Sudman S., Bradbourn N.M., Schwartz N.: Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. Jossey-Bass Publishers, San Francisco (1996)Google Scholar
  18. Tourangeau R., Rasinski K.A.: Cognitive processes underlying context effects in attitude measurement. Psychol. Bull. 103, 299–314 (1988)CrossRefGoogle Scholar
  19. Tourangeau, R., Singer, E., Presser, S.: Context effects in attitude surveys. Effects on remote items and impact on predictive validity. SMR (2003). doi: 10.1177/0049124103251950
  20. Zaller J., Feldman S.: A simple theory of the survey response: answering questions versus revealing preferences. Am. J. Political Sci. 36, 579–616 (1992)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Faculty of Social SciencesUniversity of LjubljanaLjubljanaSlovenia

Personalised recommendations