Skip to main content

Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey

Abstract

In this study we examine the cross-cultural equivalence of two scales that measure attitudes toward democracy across 36 countries in the World Value Survey (WVS) 2000. We examine the equivalence of these scales in order to explore if we can meaningfully compare democratic attitudes across countries. Multiple group confirmatory factor analyses (MGCFA) is applied to answer this question. The analyses indicate that the scales may be compared but only to a certain extent and not across all the countries. We close this article by discussing the implications of the findings.

This is a preview of subscription content, access via your institution.

Fig. 1

Notes

  1. 1.

    The two scale items include rating of the importance of having a democratic system and whether it is better than any other form of government (WVS 1995–1997).

  2. 2.

    The scale was based on a respondent’s quality assessment (good vs. bad) of the political system as it was in [reference to previous regime], as it is today, and as it is expected to be tomorrow? (WVS 1995–1997).

  3. 3.

    For further details, check the website of the World Values Survey Association. http://www.worldvaluessurvey.org/.

  4. 4.

    Several studies have used multiple group CFA (MGCFA) to assess measurement invariance of scales in such cross-national surveys as the European Social Survey (ESS) and the International Social Survey Program (ISSP) (e.g., for human values, Davidov 2008; Davidov et al. 2008; for national identity, Davidov 2009; for trust, Reeskens and Hooghe 2008).

  5. 5.

    In the current analyses, we firstly ran the models using pairwise deletion to deal with the problem of missing values (Schafer and Graham 2002). As a final test, we have also analyzed the models using the Full Information Maximum Likelihood (FIML) procedure which is preferred when portions larger than 5% of the data are missing (Schafer and Graham 2002).

  6. 6.

    For cut off criteria for global fit measures, see Hu and Bentler (1999) and Marsh et al. (2004).

  7. 7.

    Brown (2006) suggests that standardized factor loadings lower than 0.3–0.4 should indicate that the question is inadequate to measure the latent construct.

  8. 8.

    Although scholars have argued that MGCFA may be an inappropriate tool for testing for invariance of Likert scales (see, e.g., Lubke and Muthén 2004), studies have demonstrated that it works well even when data are not continuous or normally distributed but ordinal (De Beuckelaer 2005; Welkenhuysen-Gybels and Billiet 2002; Welkenhuysen-Gybels 2004).

  9. 9.

    Model outputs, factor loadings and global fit measures may be available from the first author upon request.

References

  1. Adcock, R., & Collier, D. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3), 529–546.

    Article  Google Scholar 

  2. Arbuckle, J. L. (1995–2007). Amos 16.0 user’s guide. Chicago, IL: SPSS.

    Google Scholar 

  3. Billiet, J. (2003). Cross-cultural equivalence with structural equation modeling. In J. A. Harkness, F. J. R. Van de Vijver, & P. P. Mohler (Eds.), Cross-cultural survey methods (pp. 247–264). New York: John Wiley.

    Google Scholar 

  4. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  5. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.

    Google Scholar 

  6. Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. London: Lawrence Erlbaum Associates.

    Google Scholar 

  7. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethods matrix. Psychological Bulletin, 56, 81–105.

    Article  Google Scholar 

  8. Canache, D., Mondak, J. J., & Seligson, M. A. (2001). Meaning and measurement in cross-national research on satisfaction with democracy. Public Opinion Quarterly, 65(4), 506–528.

    Article  Google Scholar 

  9. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.

    Article  Google Scholar 

  10. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

    Article  Google Scholar 

  11. Curtice, J. (2007). Comparative opinion surveys. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 898–909). Oxford: Oxford University Press.

    Google Scholar 

  12. Dalton, R. J. (1999). Political support in advanced industrial democracies. In P. Norris (Ed.), Critical citizens: Global support for democratic governance (pp. 57–77). Oxford: Oxford University Press.

  13. Dalton, J. R., & Ong, N. T. (2005). Authority orientations and democratic attitudes: A test of the ‘Asian values’ hypothesis. Japanese Journal of Political Science, 6(2), 211–231.

    Article  Google Scholar 

  14. Dalton, R., Shin, D. C., & Jou, W. (2007). Understanding democracy: Data from unlikely places. Journal of Democracy, 18(4), 142–156.

    Google Scholar 

  15. Davidov, E. (2008). A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Survey Research Methods, 2(1), 33–46.

    Google Scholar 

  16. Davidov, E. (2009). Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis, 17(1), 64–82.

    Article  Google Scholar 

  17. Davidov, E., Schmidt, P., & Schwartz, S. H. (2008). Bringing values back in: The adequacy of the European Social Survey to measure values in 20 countries. Public Opinion Quarterly, 72(3), 420–445.

    Article  Google Scholar 

  18. Davidov, E., Schmidt, P., & Billiet, J. (Eds.). (2010). Cross-cultural analysis: Methods and applications. New York: Taylor and Francis.

  19. De Beuckelaer, A. (2005). Measurement invariance issues in international management research. Unpublished dissertation, Limburgs University Centrum, Centrum, Limburg, the Netherlands.

  20. Diamond, L. J. (1999). Developing democracy: Toward consolidation. Baltimore: Johns Hopkins University Press.

    Google Scholar 

  21. Dixon, J. C. (2008). A clash of civilizations? examining liberal-democratic values in Turkey and the European Union. British Journal of Sociology, 59(4), 681–708.

    Article  Google Scholar 

  22. Esmer, Y. (2002). Is there an Islamic civilization? Comparative Sociology, 1(3–4), 265–298.

    Article  Google Scholar 

  23. Guerin, D., Pétry, F., & Crete, J. (2004). Tolerance, protest and democratic transition: Survey evidence from 13 Post-Communist countries. European Journal of Political Research, 43(3), 371–395.

    Article  Google Scholar 

  24. Haerpfer, C. W. (2008). Support for democracy and autocracy in Russia and the Commonwealth of independent states, 1992–2002. International Political Science Review, 29(4), 411–431.

    Article  Google Scholar 

  25. Harkness, J. A., Van de Vijver, F. J. R., & Mohler, P. P. (2003). Cross-cultural survey methods Hoboken. New Jersey: Wiley-Interscience.

    Google Scholar 

  26. Heath, A., & Martin, J. (1997). Why are there so few formal measuring instruments in social and political research. In L. E. Fyberg, et al. (Eds.), Survey measurement and process quality (pp. 71–86). New York: Wiley.

    Google Scholar 

  27. Heath, A., Fisher, S., & Smith, S. (2005). The globalization of public opinion research. Annual Review of Political Science, 8, 297–333.

    Article  Google Scholar 

  28. Hofmann, S. R. (2004). Islam and democracy: Micro-level indications of compatibility. Comparative Political Studies, 37(6), 652–676.

    Article  Google Scholar 

  29. Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144.

    Article  Google Scholar 

  30. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(4), 1–55.

    Article  Google Scholar 

  31. Inglehart, R. (2003). How solid is mass support for democracy—and how can we measure it? Political Science and Politics, 36(1), 51–57.

    Google Scholar 

  32. Inglehart, R., & Norris, P. (2003). The true clash of civilizations. Foreign Policy, 135, 63–70.

    Google Scholar 

  33. Inglehart, R., & Welzel, C. (2005). Modernization, cultural change, and democracy. Cambridge, UK: Cambridge University Press.

    Google Scholar 

  34. Johnson, T. P. (1998). Approaches to equivalence in cross-cultural and cross-national survey research. ZUMA Nachrichten Spezial, 31–40.

  35. Jowell, R., Roberts, C., Fitzgerald, R., & Gillian, E. (Eds.). (2007). Measuring attitudes cross-nationally: Lessons from the European social survey. London: Sage Publications.

    Google Scholar 

  36. King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 97(4), 567–583.

    Google Scholar 

  37. Kittilson, M. C. (2006). Research resources in comparative political behavior. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 865–895). Oxford: Oxford University Press.

    Google Scholar 

  38. Klingemann, H.-D (1999). Mapping political support in the 1990s: A global analysis. In P. Norris (Ed.), Critical citizens: Global support for democratic Governance (pp. 151–189). Oxford: Oxford University Press.

    Google Scholar 

  39. Knoppen, D., & Saris, W. E. (2009). Do we have to combine values in the Schwartz’ human values scale? A comment on the Davidov studies. Survey Research Methods, 3(2), 91–103.

    Google Scholar 

  40. Lagos, M. (2003). Support for and satisfaction with democracy. International Journal of Public Opinion Research, 15(4), 471–487.

    Article  Google Scholar 

  41. Linde, J., & Ekman, J. (2003). Satisfaction with democracy: A note on a frequently used indicator in comparative politics. European Journal of Political Research, 42(3), 391–408.

    Article  Google Scholar 

  42. Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.

    Article  Google Scholar 

  43. Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341.

    Article  Google Scholar 

  44. Mishler, W., & Rose, R. (2001). Political support for incomplete democracies: Realist vs. idealist theories and measures. International Political Science Review, 22(4), 303–320.

    Article  Google Scholar 

  45. Norris, P. (Ed.). (1999). Critical citizens: Global support for democratic government. Oxford: Oxford University Press.

    Google Scholar 

  46. Norris, P. (2009). The globalization of comparative public opinion research. In N. Robinson & T. Landman (Eds.), Handbook of comparative politics (pp. 522–539). London: Sage.

    Google Scholar 

  47. Pettersson, T. (2008). Different patterns of compartmentalisation among Muslims and Christians? In J. Haynes (Ed.), Routldge handbook of religion & politics (pp. 246–270). New York: Routledge.

    Google Scholar 

  48. Reeskens, T., & Hooghe, M. (2008). Cross-cultural measurement equivalence of generalized trust. Evidence from the European Social Survey (2002 and 2004). Social Indicators Research, 85(3), 515–532.

    Article  Google Scholar 

  49. Rizzo, H., Abdel-Latif, A., & Meyer, K. (2007). The relationship between gender equality and democracy: A comparison of Arab versus non-Arab Muslim societies. Sociology, 41(6), 1151–1170.

    Article  Google Scholar 

  50. Rowley, C. K., & Smith, N. (2009). Islam’s democracy paradox: Muslims claim to like democracy, so why do they have so little? Public Choice, 139(3), 273–299.

    Article  Google Scholar 

  51. Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research. New Jersey: Wiley.

    Book  Google Scholar 

  52. Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16(4), 561–582.

    Article  Google Scholar 

  53. Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.

    Article  Google Scholar 

  54. Schedler, A., & Sarsfield, R. (2007). Democrats with adjectives: Linking direct and indirect measures of democratic support. European Journal of Political Research, 46(5), 637–659.

    Article  Google Scholar 

  55. Shin, D. C. (2007). Democratization: Perspectives from global citizenries. In J. R. Dalton & H. Klingemann (Eds.), The oxford handbook of political behavior (pp. 259–282). Oxford: Oxford University Press.

    Chapter  Google Scholar 

  56. Smith, T. W. (2003). Developing comparable questions in cross-national surveys. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 69–91). New York: John Wiley & Sons.

    Google Scholar 

  57. Steenbergen, M. R. (2000). Item similarity in scale analysis. Political Analysis, 8(3), 261–283.

    Google Scholar 

  58. Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–107.

    Article  Google Scholar 

  59. Tessler, M. (2002). Islam and democracy in the Middle East: The impact of religious orientations on attitudes toward democracy in four Arab countries. Comparative Politics, 34(3), 337–354.

    Article  Google Scholar 

  60. Tessler, M., Moaddel, M., & Inglehart, R. (2006). What do Iraqis want? Journal of Democracy, 17(1), 38–50.

    Article  Google Scholar 

  61. Toros, E. (2010). The relationship between Islam and democracy in Turkey: Employing political culture as an indicator. Social Indicators Research, 95(2), 253–265.

    Article  Google Scholar 

  62. Tusicisny, A. (2007). Security communities and their values: Taking masses seriously. International Political Science Review, 28(4), 425–449.

    Article  Google Scholar 

  63. Van de Vijver, F. J. R. (2003). Bias and equivalence: Cross-cultural perspective. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 143–155). New York: John Wiley & Sons.

    Google Scholar 

  64. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.

    Article  Google Scholar 

  65. Wang, Z., Dalton, R. J. D., & Shin, D. C. (2006). Political trust, political performance, and support for democracy. In J. R. Dalton & D. C. Shin (Eds.), Citizens, democracy, and markets around the Pacific rim (pp. 135–154). Oxford: Oxford University Press.

    Google Scholar 

  66. Welkenhuysen-Gybels, J. (2004). The performance of some observed and unobserved conditional invariance techniques for the detection of differential item functioning. Quality & Quantity, 38(6), 681–702.

    Article  Google Scholar 

  67. Welkenhuysen-Gybels, J., & Billiet, J. (2002). A comparison of techniques for detecting cross-cultural inequivalence at the item level. Quality & Quantity, 36(3), 197–218.

    Article  Google Scholar 

  68. Welzel, C. (2007). Are levels of democracy affected by mass attitudes? Testing attainment and sustainment effects on democracy. International Political Science Review, 28(4), 397–424.

    Article  Google Scholar 

  69. Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment Research & Evaluation, 12(3), 1–26.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gal Ariely.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Ariely, G., Davidov, E. Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey. Soc Indic Res 104, 271–286 (2011). https://doi.org/10.1007/s11205-010-9693-5

Download citation

Keywords

  • Attitudes toward democracy
  • Measurement invariance
  • Multiple-group confirmatory factor analysis (MGCFA)
  • World Value Survey (WVS)