Quality & Quantity

, Volume 48, Issue 1, pp 127-148

Open Access This content is freely available online to anyone, anywhere at any time.

Item comparability in cross-national surveys: results from asking probing questions in cross-national web surveys about attitudes towards civil disobedience

  • Dorothée BehrAffiliated withGESIS – Leibniz Institute for the Social Sciences Email author 
  • , Michael BraunAffiliated withGESIS – Leibniz Institute for the Social Sciences
  • , Lars KaczmirekAffiliated withGESIS – Leibniz Institute for the Social Sciences
  • , Wolfgang BandillaAffiliated withGESIS – Leibniz Institute for the Social Sciences


This article focuses on assessing item comparability in cross-national surveys by asking probing questions in Web surveys. The “civil disobedience” item from the “rights in a democracy” scale of the International Social Survey Program (ISSP) serves as a substantive case study. Identical Web surveys were fielded in Canada (English-speaking), Denmark, Germany, Hungary, Spain, and the U.S. A category-selection and a comprehension probe, respectively, were incorporated into the Web surveys after the closed-ended “civil disobedience” item. Responses to the category selection-probe reveal that notably in Germany, Hungary, and Spain the detachment of politicians from the people and their lack of responsiveness is deplored. Responses to the comprehension probe show that mainly in the U.S. and Canada violence and/or destruction are associated with civil disobedience. These results suggest reasons for the peculiar statistical results found for the “civil disobedience” item in the ISSP study. On the whole, Web probing proves to be a valuable tool for identifying interpretation differences and potential bias in cross-national survey research.


Probing Web surveys Mixed method Comparability Cross-national survey research