In this study we examine the cross-cultural equivalence of two scales that measure attitudes toward democracy across 36 countries in the World Value Survey (WVS) 2000. We examine the equivalence of these scales in order to explore if we can meaningfully compare democratic attitudes across countries. Multiple group confirmatory factor analyses (MGCFA) is applied to answer this question. The analyses indicate that the scales may be compared but only to a certain extent and not across all the countries. We close this article by discussing the implications of the findings.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
The two scale items include rating of the importance of having a democratic system and whether it is better than any other form of government (WVS 1995–1997).
The scale was based on a respondent’s quality assessment (good vs. bad) of the political system as it was in [reference to previous regime], as it is today, and as it is expected to be tomorrow? (WVS 1995–1997).
For further details, check the website of the World Values Survey Association. http://www.worldvaluessurvey.org/.
Several studies have used multiple group CFA (MGCFA) to assess measurement invariance of scales in such cross-national surveys as the European Social Survey (ESS) and the International Social Survey Program (ISSP) (e.g., for human values, Davidov 2008; Davidov et al. 2008; for national identity, Davidov 2009; for trust, Reeskens and Hooghe 2008).
In the current analyses, we firstly ran the models using pairwise deletion to deal with the problem of missing values (Schafer and Graham 2002). As a final test, we have also analyzed the models using the Full Information Maximum Likelihood (FIML) procedure which is preferred when portions larger than 5% of the data are missing (Schafer and Graham 2002).
Brown (2006) suggests that standardized factor loadings lower than 0.3–0.4 should indicate that the question is inadequate to measure the latent construct.
Although scholars have argued that MGCFA may be an inappropriate tool for testing for invariance of Likert scales (see, e.g., Lubke and Muthén 2004), studies have demonstrated that it works well even when data are not continuous or normally distributed but ordinal (De Beuckelaer 2005; Welkenhuysen-Gybels and Billiet 2002; Welkenhuysen-Gybels 2004).
Model outputs, factor loadings and global fit measures may be available from the first author upon request.
Adcock, R., & Collier, D. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3), 529–546.
Arbuckle, J. L. (1995–2007). Amos 16.0 user’s guide. Chicago, IL: SPSS.
Billiet, J. (2003). Cross-cultural equivalence with structural equation modeling. In J. A. Harkness, F. J. R. Van de Vijver, & P. P. Mohler (Eds.), Cross-cultural survey methods (pp. 247–264). New York: John Wiley.
Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York: Guilford Press.
Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. London: Lawrence Erlbaum Associates.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethods matrix. Psychological Bulletin, 56, 81–105.
Canache, D., Mondak, J. J., & Seligson, M. A. (2001). Meaning and measurement in cross-national research on satisfaction with democracy. Public Opinion Quarterly, 65(4), 506–528.
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464–504.
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.
Curtice, J. (2007). Comparative opinion surveys. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 898–909). Oxford: Oxford University Press.
Dalton, R. J. (1999). Political support in advanced industrial democracies. In P. Norris (Ed.), Critical citizens: Global support for democratic governance (pp. 57–77). Oxford: Oxford University Press.
Dalton, J. R., & Ong, N. T. (2005). Authority orientations and democratic attitudes: A test of the ‘Asian values’ hypothesis. Japanese Journal of Political Science, 6(2), 211–231.
Dalton, R., Shin, D. C., & Jou, W. (2007). Understanding democracy: Data from unlikely places. Journal of Democracy, 18(4), 142–156.
Davidov, E. (2008). A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Survey Research Methods, 2(1), 33–46.
Davidov, E. (2009). Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis, 17(1), 64–82.
Davidov, E., Schmidt, P., & Schwartz, S. H. (2008). Bringing values back in: The adequacy of the European Social Survey to measure values in 20 countries. Public Opinion Quarterly, 72(3), 420–445.
Davidov, E., Schmidt, P., & Billiet, J. (Eds.). (2010). Cross-cultural analysis: Methods and applications. New York: Taylor and Francis.
De Beuckelaer, A. (2005). Measurement invariance issues in international management research. Unpublished dissertation, Limburgs University Centrum, Centrum, Limburg, the Netherlands.
Diamond, L. J. (1999). Developing democracy: Toward consolidation. Baltimore: Johns Hopkins University Press.
Dixon, J. C. (2008). A clash of civilizations? examining liberal-democratic values in Turkey and the European Union. British Journal of Sociology, 59(4), 681–708.
Esmer, Y. (2002). Is there an Islamic civilization? Comparative Sociology, 1(3–4), 265–298.
Guerin, D., Pétry, F., & Crete, J. (2004). Tolerance, protest and democratic transition: Survey evidence from 13 Post-Communist countries. European Journal of Political Research, 43(3), 371–395.
Haerpfer, C. W. (2008). Support for democracy and autocracy in Russia and the Commonwealth of independent states, 1992–2002. International Political Science Review, 29(4), 411–431.
Harkness, J. A., Van de Vijver, F. J. R., & Mohler, P. P. (2003). Cross-cultural survey methods Hoboken. New Jersey: Wiley-Interscience.
Heath, A., & Martin, J. (1997). Why are there so few formal measuring instruments in social and political research. In L. E. Fyberg, et al. (Eds.), Survey measurement and process quality (pp. 71–86). New York: Wiley.
Heath, A., Fisher, S., & Smith, S. (2005). The globalization of public opinion research. Annual Review of Political Science, 8, 297–333.
Hofmann, S. R. (2004). Islam and democracy: Micro-level indications of compatibility. Comparative Political Studies, 37(6), 652–676.
Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18(3), 117–144.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(4), 1–55.
Inglehart, R. (2003). How solid is mass support for democracy—and how can we measure it? Political Science and Politics, 36(1), 51–57.
Inglehart, R., & Norris, P. (2003). The true clash of civilizations. Foreign Policy, 135, 63–70.
Inglehart, R., & Welzel, C. (2005). Modernization, cultural change, and democracy. Cambridge, UK: Cambridge University Press.
Johnson, T. P. (1998). Approaches to equivalence in cross-cultural and cross-national survey research. ZUMA Nachrichten Spezial, 31–40.
Jowell, R., Roberts, C., Fitzgerald, R., & Gillian, E. (Eds.). (2007). Measuring attitudes cross-nationally: Lessons from the European social survey. London: Sage Publications.
King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 97(4), 567–583.
Kittilson, M. C. (2006). Research resources in comparative political behavior. In J. R. Dalton & H. Klingemann (Eds.), Oxford handbook of political behavior (pp. 865–895). Oxford: Oxford University Press.
Klingemann, H.-D (1999). Mapping political support in the 1990s: A global analysis. In P. Norris (Ed.), Critical citizens: Global support for democratic Governance (pp. 151–189). Oxford: Oxford University Press.
Knoppen, D., & Saris, W. E. (2009). Do we have to combine values in the Schwartz’ human values scale? A comment on the Davidov studies. Survey Research Methods, 3(2), 91–103.
Lagos, M. (2003). Support for and satisfaction with democracy. International Journal of Public Opinion Research, 15(4), 471–487.
Linde, J., & Ekman, J. (2003). Satisfaction with democracy: A note on a frequently used indicator in comparative politics. European Journal of Political Research, 42(3), 391–408.
Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534.
Marsh, H. W., Hau, K.-T., & Wen, Z. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11(3), 320–341.
Mishler, W., & Rose, R. (2001). Political support for incomplete democracies: Realist vs. idealist theories and measures. International Political Science Review, 22(4), 303–320.
Norris, P. (Ed.). (1999). Critical citizens: Global support for democratic government. Oxford: Oxford University Press.
Norris, P. (2009). The globalization of comparative public opinion research. In N. Robinson & T. Landman (Eds.), Handbook of comparative politics (pp. 522–539). London: Sage.
Pettersson, T. (2008). Different patterns of compartmentalisation among Muslims and Christians? In J. Haynes (Ed.), Routldge handbook of religion & politics (pp. 246–270). New York: Routledge.
Reeskens, T., & Hooghe, M. (2008). Cross-cultural measurement equivalence of generalized trust. Evidence from the European Social Survey (2002 and 2004). Social Indicators Research, 85(3), 515–532.
Rizzo, H., Abdel-Latif, A., & Meyer, K. (2007). The relationship between gender equality and democracy: A comparison of Arab versus non-Arab Muslim societies. Sociology, 41(6), 1151–1170.
Rowley, C. K., & Smith, N. (2009). Islam’s democracy paradox: Muslims claim to like democracy, so why do they have so little? Public Choice, 139(3), 273–299.
Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research. New Jersey: Wiley.
Saris, W. E., Satorra, A., & Van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Structural Equation Modeling, 16(4), 561–582.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.
Schedler, A., & Sarsfield, R. (2007). Democrats with adjectives: Linking direct and indirect measures of democratic support. European Journal of Political Research, 46(5), 637–659.
Shin, D. C. (2007). Democratization: Perspectives from global citizenries. In J. R. Dalton & H. Klingemann (Eds.), The oxford handbook of political behavior (pp. 259–282). Oxford: Oxford University Press.
Smith, T. W. (2003). Developing comparable questions in cross-national surveys. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 69–91). New York: John Wiley & Sons.
Steenbergen, M. R. (2000). Item similarity in scale analysis. Political Analysis, 8(3), 261–283.
Steenkamp, J. E. M., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25(1), 78–107.
Tessler, M. (2002). Islam and democracy in the Middle East: The impact of religious orientations on attitudes toward democracy in four Arab countries. Comparative Politics, 34(3), 337–354.
Tessler, M., Moaddel, M., & Inglehart, R. (2006). What do Iraqis want? Journal of Democracy, 17(1), 38–50.
Toros, E. (2010). The relationship between Islam and democracy in Turkey: Employing political culture as an indicator. Social Indicators Research, 95(2), 253–265.
Tusicisny, A. (2007). Security communities and their values: Taking masses seriously. International Political Science Review, 28(4), 425–449.
Van de Vijver, F. J. R. (2003). Bias and equivalence: Cross-cultural perspective. In J. A. Harkness, Van de Vijver, J. R. Fons, & P. H. Mohler (Eds.), Cross-cultural survey methods (pp. 143–155). New York: John Wiley & Sons.
Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.
Wang, Z., Dalton, R. J. D., & Shin, D. C. (2006). Political trust, political performance, and support for democracy. In J. R. Dalton & D. C. Shin (Eds.), Citizens, democracy, and markets around the Pacific rim (pp. 135–154). Oxford: Oxford University Press.
Welkenhuysen-Gybels, J. (2004). The performance of some observed and unobserved conditional invariance techniques for the detection of differential item functioning. Quality & Quantity, 38(6), 681–702.
Welkenhuysen-Gybels, J., & Billiet, J. (2002). A comparison of techniques for detecting cross-cultural inequivalence at the item level. Quality & Quantity, 36(3), 197–218.
Welzel, C. (2007). Are levels of democracy affected by mass attitudes? Testing attainment and sustainment effects on democracy. International Political Science Review, 28(4), 397–424.
Wu, A. D., Li, Z., & Zumbo, B. D. (2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment Research & Evaluation, 12(3), 1–26.
About this article
Cite this article
Ariely, G., Davidov, E. Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey. Soc Indic Res 104, 271–286 (2011). https://doi.org/10.1007/s11205-010-9693-5
- Attitudes toward democracy
- Measurement invariance
- Multiple-group confirmatory factor analysis (MGCFA)
- World Value Survey (WVS)