Social Indicators Research

, Volume 109, Issue 3, pp 515–534 | Cite as

Can We Improve the Measurement of Attitudes Towards the Welfare State? A Constructive Critique of Survey Instruments with Evidence from Focus Groups

  • Achim GoerresEmail author
  • Katrin Prinzen


There is a large and growing literature on welfare state attitudes, most of which is built on random-sample population surveys with standardised closed-question items. This article criticises the existing survey instruments, especially those that are used within the International Social Survey Programme, in a novel approach with focus group data from Germany. The article demonstrates: firstly, these instruments underestimate the inconsistency (the degree to which attitudes logically contradict each other), the uncertainty (the degree to which individuals are unsure about what to think), the ambivalence (the simultaneous occurrence of positive and negative reactions) and non-attitudes towards welfare state activities that common people have. Secondly, the meaning of these items to respondents seems to vary to such an extent that inference based on such measures is questionable. Finally, the article concludes by suggesting some survey instruments that alleviate these measurement problems.


Attitudes Welfare state Focus groups Survey Measurement 



This work has been supported by a grant of the Fritz Thyssen Foundation given to Achim Goerres “The Demand for Social Policy in Ageing Democracies”. An earlier version was presented at the Methodology meeting of the German Political Science Association (DVPW) in Friedrichshafen, June 2009. We would like to thank the participants for their comments. Also, we want to say thanks to Hans-Jürgen Andreß, Jason Gainous, Marco Gießelmann, Silja Häusermann, and Vanita Matta.


  1. Alvarez, R. M., & Brehm, J. (2002). Hard choices, easy answers: Values, information, and American public opinion. Princeton, New Jersey: Princeton University Press.Google Scholar
  2. Andreß, H.-J., & Heien, T. (2001). Four worlds of welfare state attitudes? A comparison of Germany, Norway, and the United States. European Sociological Review, 17(4), 337–356.CrossRefGoogle Scholar
  3. Andreß, H.-J., Heien, T., & Hofäcker, D. (2001). Wozu brauchen wir noch den Sozialstaat?. Opladen: Westdeutscher Verlag.Google Scholar
  4. Bassili, J. N. (1996). Meta-judgmental versus operative indexes of psychological attributes: The case of measures of attitude strength. Journal of Personality and Social Psychology, 71(10), 637–653.CrossRefGoogle Scholar
  5. Beck, M., & Opp, K.-D. (2001). Der faktorielle Survey und die Messung von Normen. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 53(2), 283–306.CrossRefGoogle Scholar
  6. Blekesaune, M. (2007). Economic conditions and public attitudes to welfare policies. European Sociological Review, 23(3), 393–403.CrossRefGoogle Scholar
  7. Blekesaune, M., & Quadagno, J. (2003). Public attitudes toward welfare state policies: A comparative analysis of 24 nations. European Sociological Review, 19(5), 415–427.CrossRefGoogle Scholar
  8. Boeri, T., Börsch-Supan, A., & Tabellini, G. (2001). Would you like to shrink the welfare state? A survey of European citizens. Economic Policy, 32, 7–50.CrossRefGoogle Scholar
  9. Brooks, C., & Manza, J. (2006). Why do welfare states persist? Journal of Politics, 68(4), 816–827.Google Scholar
  10. Busemeyer, M. R., Goerres, A., & Weschle, S. (2009). Demands for redistributive policies in an era of demographic aging: The rival pressures from age and income in 14 OECD countries. Journal of European Social Policy, 19(3), 195–212.CrossRefGoogle Scholar
  11. Carson, R. T., Flores, N. E., & Meade, N. F. (2001). Contingent valuation: Controversies and evidence. Environmental and Resource Economics, 19(2), 173–210.CrossRefGoogle Scholar
  12. Chong, D. (1993). How people think, reason, and feel about rights and liberties. American Journal of Political Science, 37(3), 867–899.CrossRefGoogle Scholar
  13. Cnaan, R. A. (1989). Public opinion and the dimensions of the welfare state. Social Indicators Research, 21, 297–314.CrossRefGoogle Scholar
  14. Converse, P. E. (1964). The nature of belief systems in mass politics. In D. E. Apter (Ed.), Ideology and discontent (Vol. 5, pp. 206–261). New York: Free Press.Google Scholar
  15. Coughlin, R. M. (1980). Ideology, public opinion and welfare policy: Attitudes towards taxes and spending in industrialized societies. Berkeley: University of California Press.Google Scholar
  16. Deeming, C., & Keen, J. (2003). A fair deal in older age? Public attitudes towards the funding of long-term care. Policy & Politics, 31(4), 431–446.CrossRefGoogle Scholar
  17. Draisma, S., & Dijkstra, W. (2004). Response latency and (para)linguistic expressions as indicators of response error. In S. Presser, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, J. M. Rothgeb, et al. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 131–148). Hoboken: Wiley.CrossRefGoogle Scholar
  18. Eckstein, H. (1975). Case study and theory in political science. In F. Greenstein & N. Polsby (Eds.), Handbook of political science (Vol. 7). Reading, MA: Addison Wellsley.Google Scholar
  19. Edlund, J. (2009). Attitudes towards state-organized welfare in twenty-two societies: A question of convergence? In M. Haller (Ed.), The international social survey programme, 1984–2009: Charting the globe (pp. 125–146). London: Routledge.Google Scholar
  20. Feldman, S., & Zaller, J. (1992). The political culture of ambivalence: Ideological responses to the welfare state. American Journal of Political Science, 36(1), 268–307.CrossRefGoogle Scholar
  21. Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading: Addison-Wesley.Google Scholar
  22. Flora, P., Alber, J., & Kohl, J. (1977). Zur Entwicklung der westeuropäischen Wohlfahrtsstaaten. Politische Vierteljahresschrift, 18, 705–772.Google Scholar
  23. Gainous, J. (2008). Ambivalence about social welfare: An evaluation of measurement approaches. American Review of Politics, 29, 109–134.Google Scholar
  24. Gainous, J., Craig, S. C., & Martinez, M. D. (2008). Attitudes concerning social welfare and ambivalence concerning the role of government. Politics & Policy, 36(6), 972–1004.CrossRefGoogle Scholar
  25. Gelissen, J. (2000). Popular support for institutionalized solidarity: A Comparison between European welfare states. International Journal of Social Welfare, 9, 285–300.CrossRefGoogle Scholar
  26. Gilljam, M., & Granberg, D. (1993). Should we take don’t know for an answer? Public Opinion Quarterly, 57(3), 348–357.CrossRefGoogle Scholar
  27. Glatzer, W., Becker, J., Bieräugel, R., Hallein-Benze, G., Nüchter, O., & Schmid, A. (2009). Reichtum im Urteil der Bevölkerung. Akzeptanzprobleme und Spannungspotenzial. Opladen/Farmington Hills: Barbara Budrich.Google Scholar
  28. Goerres, A., & Prinzen, K. (2010). Using mixed methods for the analysis of individuals: A review of necessary and sufficient conditions and an application to welfare state attitudes. Quality & Quantity. doi: 10.1007/s11135-010-9379-8.
  29. Groves, R. M. (1989). Survey errors and survey costs. New York: Wiley.CrossRefGoogle Scholar
  30. Hansen, J. M. (1998). Individuals, institutions, and public preferences over public finance. American Political Science Review, 92(3), 513–531.CrossRefGoogle Scholar
  31. Iversen, T., & Soskice, D. (2001). An asset theory of social policy preferences. American Political Science Review, 95(4), 875–893.Google Scholar
  32. Jackman, M. R. (1973). Education and Prejudice or education and response-set? American Sociological Review, 38(3), 327–339.CrossRefGoogle Scholar
  33. Jaeger, M. M. (2006). Welfare regimes and attitudes towards redistribution: The regime hypothesis revisited. European Sociological Review, 22(2), 157–170.CrossRefGoogle Scholar
  34. Jaeger, M. M. (2009). United but divided: Welfare regimes and the level and variance in public support for redistribution. European Sociological Review, 25(6), 723–737. doi: 10.1093/esr/jcn079.CrossRefGoogle Scholar
  35. Kitzinger, J. (1995). Qualitative research: Introducing focus groups. British Medical Journal, 311, 299–302.CrossRefGoogle Scholar
  36. Klein, M. (2006). Der faktorielle Survey: Conjoint- und Vignettenanalyse. In J. Behnke, T. Gschwend, D. Schindler, & K.-U. Schnapp (Eds.), Methoden der Politikwissenschaft. Neuere qualitative und quantitative Analyseverfahren (pp. 133–143). Baden-Baden: Nomos.Google Scholar
  37. Klopfer, F. J., & Madden, T. M. (1980). The middlemost choice on attitude items: Ambivalence, neutrality, or uncertainty? Personality and Social Psychology Bulletin, 6(1), 97–101. doi: 10.1177/014616728061014.CrossRefGoogle Scholar
  38. Kohli, M. (1999). Private and public transfers between generations: Linking the family and the state. European Societies, 1, 81–104.Google Scholar
  39. Krömmelbein, S., Bieräugel, R., Nüchter, O., Glatzer, W., & Schmid, A. (2007). Einstellungen zum Sozialstaat. Repräsentative Querschnittsuntersuchungen zu grundsätzlichen gesundheits- und sozialpolitischen Einstellungen in der Bevölkerung 2005. Opladen/Farmington Hills: Barbara Budrich.Google Scholar
  40. Krosnick, J. A. (1999). Survey research. Annual Review of Psychology, 50(1), 537–567. doi: 10.1146/annurev.psych.50.1.537.CrossRefGoogle Scholar
  41. Lindsay, A. C., & Hubley, A. M. (2006). Conceptual reconstruction through a modified focus group methodology. Social Indicators Research, 79(3), 437–454. doi: 10.1007/s11205-005-5075-9.CrossRefGoogle Scholar
  42. Linos, K., & West, M. (2003). Self-interest, social beliefs, and attitudes to redistribution. European Sociological Review, 19(4), 393–409.CrossRefGoogle Scholar
  43. Louviere, J. J., Hensher, D. A., & Swait, J. D. (2000). Stated choice methods, analysis and applications. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  44. Mangold, W. (1960). Gegenstand und Methode des Gruppendiskussionsverfahrens. Aus der Arbeit des Institut für Sozialforschung. Frankfurt: Europäische Verlagsanstalt.Google Scholar
  45. Merton, R. K., Fiske, M., & Kendall, P. L. (1956). The focused interview: A manual of problems and procedures (2nd revised ed.). Glencoe: The Free Press.Google Scholar
  46. Møller, V., Erstad, I., & Zani, D. (2010). Drinking, smoking, and morality: Do ‘drinkers and smokers’ constitute a stigmatised stereotype or a real tb risk factor in the time of HIV/AIDS? Social Indicators Research, 98(2), 217–238. doi: 10.1007/s11205-009-9546-2.CrossRefGoogle Scholar
  47. Narayan, S., & Krosnick, J. A. (1996). Education moderates some response effects in attitude measurement. Public Opinion Quarterly, 60(1), 58–88.CrossRefGoogle Scholar
  48. Nüchter, O., Bieräugel, R., Glatzer, W., & Schmid, A. (2010). Der Sozialstaat im Urteil der Bevölkerung. Opladen/Farmington Hills: Barbara Budrich.Google Scholar
  49. Nüchter, O., Bieräugel, R., Schipperges, F., Glatzer, W., & Schmid, A. (2008). Einstellungen zum Sozialstaat II. Die Akzeptanz der sozialen Sicherung und der Reform der Renten- und Pflegeversicherung 2006. Opladen/Farmington Hills: Barbara Budrich.Google Scholar
  50. Nüchter, O., Bieräugel, R., Schipperges, F., Glatzer, W., & Schmid, A. (2009). Einstellungen zum Sozialstaat III. Sechs Fragen zur Akzeptanz der sozialen Sicherung in der Bevölkerung. Opladen/Farmington Hills: Barbara Budrich.Google Scholar
  51. Pollock, F. (1955). Gruppenexperiment. Ein Studienbericht. Frankfurt: Europäische Verlagsanstalt.Google Scholar
  52. Roller, E. (1992). Einstellungen der Bürger zum Wohlfahrtssstaat der Bundesrepublik Deutschland. Opladen: Westdeutscher Verlag.CrossRefGoogle Scholar
  53. Sabbagh, C., & Vanhuysse, P. (2006). Exploring attitudes towards the welfare state: Students’ views in eight democracies. Journal of Social Policy, 35, 607–628.CrossRefGoogle Scholar
  54. Schuman, H., & Presser, S. (1980). Public opinion and public ignorance: The fine line between attitudes and nonattitudes. The American Journal of Sociology, 85(5), 1214–1225.CrossRefGoogle Scholar
  55. Schwarz, N., & Sudman, S. E. (1992). Context effects in social and psychological research. New York: Springer.CrossRefGoogle Scholar
  56. Sellke, P., & Mayerl, J. (2004). How long does it take to be ignorant? Measuring nonattitudes with response latencies in surveys. In C. van Dijkum, J. Blasius, & B. van Hilton (Eds.), Recent developments and applications in social research methodology. Proceedings of the sixth international conference on logic and methodology. Amsterdam: SISWO.Google Scholar
  57. Sihvo, T., & Uusitalo, H. (1995). Attitudes towards the welfare states have several dimensions: Evidence from Finland. Scandinavian Journal of Social Welfare, 4, 215–223.CrossRefGoogle Scholar
  58. Slothuus, R. (2007). Framing deservingness to win support for welfare state retrenchment. Scandinavian Political Studies, 30(3), 323–344.CrossRefGoogle Scholar
  59. Smith, T. W. (2004). Developing and evaluating cross-national survey instruments. In S. Presser, M. P. Couper, J. T. Lessler, E. Martin, J. Martin, J. M. Rothgeb, et al. (Eds.), Methods for testing and evaluating survey questionnaires (pp. 431–452). Hoboken: Wiley.CrossRefGoogle Scholar
  60. Svallfors, S. (1997). Worlds of welfare and attitudes to redistribution: A comparison of eight western nations. European Sociological Review, 13(3), 283–304.CrossRefGoogle Scholar
  61. Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  62. Veenhoven, R. (2002). Why social policy needs subjective indicators. Social Indicators Research, 58(1), 33–46. doi: 10.1023/a:1015723614574.CrossRefGoogle Scholar
  63. Wikman, A. (2006). Reliability, validity and true values in surveys. Social Indicators Research, 78(1), 85–110. doi: 10.1007/s11205-005-5372-3.CrossRefGoogle Scholar
  64. Wikman, A. (2007). Context effects as an illustration of response uncertainty—A cautionary tale. Social Indicators Research, 84(1), 27–38. doi: 10.1007/s11205-006-9067-1.CrossRefGoogle Scholar
  65. Wilson, T. D., & Hodges, S. D. (1992). Attitudes as temporary constructions. In L. L. Martin & A. Tesser (Eds.), The construction of social judgments (pp. 37–66). Hillsdale, NJ: Erlbaum.Google Scholar
  66. Zaller, J., & Feldman, S. (1992). A simple theory of the survey response: Answering questions versus revealing preferences. American Journal of Political Science, 36(3), 579.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2011

Authors and Affiliations

  1. 1.Faculty of Management, Economics and Social SciencesUniversity of CologneCologneGermany
  2. 2.Faculty of Management, Economics and Social SciencesUniversity of CologneCologneGermany

Personalised recommendations