Examining Measurement Isomorphism of Multilevel Constructs: The Case of Political Trust
There is a growing recognition of the importance of testing for measurement equivalence when comparing latent constructs—such as political trust—across countries and over time. Indeed, equivalence of measurements across countries and time points is a precondition for making meaningful and valid comparisons of means, scores, and relationships between constructs. Until recently, most efforts in this area of comparative research have focused on establishing measurement equivalence across groups, such as countries or cultural regions. In contrast, scarce attention has been paid to examining the measurement equivalence of constructs across levels of data. Rather, most empirical studies that use cross-national survey data assume that the factorial structure of a given construct is isomorphic, in other words, similar in measurement and meaning at the individual and country level. This assumption is not always justified, as the dimensions found at the individual level of data do not always generalize to the country level. In such cases, the results and substantive conclusions based on the assumption of measurement isomorphism may be misleading. In this article, we emphasize the importance of examining measurement isomorphism when working with cross-national survey data and describe a testing procedure using multilevel confirmatory factor analysis. We illustrate the procedure by examining cross-level equivalence of the political trust scale included in the European Social Survey (2008–2009). The results of the analysis indicate that the structure of political trust factor can be considered configurally isomorphic across individual and country levels. In addition, the measurement scale across the two levels is found to be partially isomorphic. These findings could be regarded as an encouraging result for the applied researchers who use the aggregated individual scores of political trust. At the same time, we demonstrate that measurement isomorphism cannot be simply assumed and hence should be examined prior to analysis to ensure valid and meaningful results.
KeywordsPolitical trust European Social Survey Measurement isomorphism Measurement equivalence Multilevel confirmatory factor analysis
This research was part of the Re-InVEST project (Rebuilding an Inclusive, Value-based Europe of Solidarity and Trust through Social Investments), funded by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No 649447.
Compliance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
- Allum, N., Read, S., & Sturgis, P. (2011). Evaluating change in social and political trust in Europe. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 35–53). New York, London: Routledge.Google Scholar
- Byrne, B. M. (2013). Structural equation modeling with Mplus: Basic concepts, applications, and programming. London: Routledge.Google Scholar
- Cronbach, L. J. (with Deken, J.E., & Webb, N.). (1976). Research on classrooms and schools: Formulation of questions, design and analysis. Occasional Paper of the Stanford Evaluation Consortium, Stanford University, Stanford, CA.Google Scholar
- Fontaine, J. (2008). Traditional and multilevel approaches in cross-cultural research: An integration of methodological frameworks. In F. J. R. Van de Vijver, D. A. Van Hemert, & Y. H. Poortinga (Eds.), Multilevel analysis of individuals and cultures (pp. 65–92). Hove: Psychology Press.Google Scholar
- Fontaine, J., & Fischer, R. (2011). Data analytic approaches for investigating isomorphism between the individual-level and the cultural-level internal structure. In D. Matsumoto & F. J. R. van de Vijver (Eds.), Cross-cultural research methods in psychology (pp. 273–298). New York: Cambridge University Press.Google Scholar
- Hox, J., van de Schoot, R., & Matthijsse, S. (2012). How few countries will do? Comparative survey analysis from a Bayesian perspective. Survey Research Methods, 6(2), 87–93.Google Scholar
- Kaplan, D., & Depaoli, S. (2012). Bayesian structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modelling (pp. 650–673). New York, NY: Guilford Press.Google Scholar
- Marien, S. (2011). Measuring political trust across time and space. In M. Hooghe & S. Zmerli (Eds.), Political trust: Why context matters (pp. 13–46). Colchester: ECPR Press.Google Scholar
- Marien, S. (2017). The measurement equivalence of political trust. In T. W. G. Van der Meer & S. Zmerli (Eds.), Handbook of political trust. Cheltenham: Edward Elgar Publishing.Google Scholar
- Meuleman, B., & Billiet, J. (2009). A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? Survey Research Methods, 3(1), 45–58.Google Scholar
- Muthén, B. O. (1990). Mean and covariance structure analysis of hierarchical data. Department of Statistics, UCLA Statistics Series, No 62. Los Angeles: University of California.Google Scholar
- Muthén, B. O., & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups. Unpublished technical report. Accessed June 14, 2017, from http://www.statmodel.com.
- Muthén, B. O., & Muthén, L. K. (2012). Mplus 7 base program. Los Angeles, CA: Muthén & Muthén.Google Scholar
- Muthén, L. K., & Muthén, B. O. (2004). Mplus user’s guide (3rd ed.). Los Angeles, CA: Muthén & Muthén.Google Scholar
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: SAGE.Google Scholar
- Schwartz, S. H. (1994). Beyond individualism/collectivism: New cultural dimensions of values. In U. Kim, H. C. Triandis, C. Kagitcibasi, S. C. Choi, & G. Yoon (Eds.), Individualism and collectivism: Theory, method and applications (pp. 85–119). Thousand Oaks: SAGE.Google Scholar
- Snijders, T., & Bosker, R. (1999). Multilevel modeling: An introduction to basic and advanced multilevel modeling. London: SAGE.Google Scholar
- van de Schoot, R., & Depaoli, S. (2014). Bayesian analyses: Where to start and what to report. European Health Psychologist, 16(2), 75–84.Google Scholar
- Van de Vijver, F. J. R. (1998). Towards a theory of bias and equivalence. Zuma Nachrichten Spezial, Cross-cultural survey equivalence, 3, 41–65.Google Scholar
- Van de Vijver, F. J. R., Van Hemert, D. A., & Poortinga, Y. H. (Eds.). (2008). Individuals and cultures in multilevel analysis. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
- Van der Veld, W. M., & Saris, W. E. (2011). Causes of generalized social trust. In E. Davidov, P. Schmidt, & J. Billiet (Eds.), Cross-cultural analysis: Methods and applications (pp. 207–247). New York, London: Routledge.Google Scholar