How to Obtain Comparable Measures for Cross-National Comparisons

  • Jan Cieciuch
  • Eldad DavidovEmail author
  • Peter Schmidt
  • René Algesheimer


Comparisons of means or associations between theoretical constructs of interest in cross-national comparative research assume measurement invariance, that is, that the same constructs are measured in the same way across the various nations under study. While it is intuitive, this assumption needs to be statistically tested. An increasing number of sociological and social psychological studies have been published in the last decade in which the cross-national comparability of various scales such as human values, national identity, attitudes toward democracy, or religiosity, to name but a few, were tested. Many of these studies did not manage to fully achieve measurement invariance. In this study we review, in a nontechnical manner, the methodological literature on measurement invariance testing. We explain what it is, how to test for it, and what to do when measurement invariance across countries is not given in the data. Several approaches have been recently proposed in the literature on how to deal with measurement noninvariance. We illustrate one of these approaches with a large dataset of seven rounds from the European Social Survey (2002–2015) by estimating the most trustworthy means of human values, even when strict measurement invariance is not given in the data. We conclude with a summary and some critical remarks.


Exact measurement invariance Approximate measurement invariance Alignment Human values European Social Survey 

Wie kann man invariante Messungen in international vergleichender Forschung erhalten?


Vergleiche von Mittelwerten und von Beziehungen zwischen theoretischen Konstrukten, die im Rahmen international vergleichender Forschung untersucht werden, gehen davon aus, dass diese Konstrukte messinvariant sind, d. h., dass sie in den verschiedenen Ländern identisch gemessen werden. Obwohl diese Annahme plausibel sein kann, muss sie jedoch statistisch getestet werden. Im letzten Jahrzehnt wurde eine zunehmende Zahl von soziologischen, politikwissenschaftlichen und sozialpsychologischen Studien veröffentlicht, in denen die internationale Vergleichbarkeit von verschiedenen Skalen zur Messung von z. B. menschlichen Werten, nationaler Identität, Einstellungen zu Demokratie oder Religiosität überprüft wurde. In vielen dieser Studien konnte Messinvarianz nicht völlig nachgewiesen werden. Die folgende Studie bietet in einer nicht technischen Art und Weise einen Überblick über die methodologische Literatur zur Messinvarianz. Es wird erklärt, was Messinvarianz ist, wie man sie überprüft und was man tun kann, wenn sie in den Daten nicht gegeben ist. In der Literatur wurden in der letzten Zeit verschiedene Ansätze vorgeschlagen, wie man fehlende Messinvarianz behandeln kann. Die Autoren illustrieren eine dieser Herangehensweisen (Alignment) mit einem großen Datensatz, der 7 Befragungsrunden des European Social Survey (2002–2015) beinhaltet, und schätzen den vertrauenswürdigsten Durchschnitt menschlicher Werte, auch wenn strikte Messinvarianz in den Daten nicht vorhanden ist. Abschließend folgen eine Zusammenfassung und einige kritische Anmerkungen.


Exakte Messinvarianz Approximative Messinvarianz Alignment Menschliche Werte European Social Survey 



The work of the first, second and fourth authors was supported by the University Research Priority Program Social Networks of the University of Zurich. The work of the third author was supported by the Alexander von Humboldt Polish Honorary Research Fellowship granted by the Foundation for Polish Science for the international cooperation between Peter Schmidt and Jan Cieciuch. The authors would like to thank Lisa Trierweiler and Neil Mussett for the English proof of the manuscript.


  1. Aleman, Jose, and Dwayne Woods. 2016. Value orientations from the World Value Survey: How comparable are they cross-nationally? Comparative Political Studies 49:1039–1067. Google Scholar
  2. Ariely, Gal, and Eldad Davidov. 2010. Can we rate public support for democracy in a comparable way? Cross-national equivalence of democratic attitudes in the World Value Survey. Social Indicators Research 104:271–286.Google Scholar
  3. Asparouhov, Tihomir, and Bengt O. Muthén. 2014. Multi-group factor analysis Alignment. Structural Equation Modeling 21:1–14. Google Scholar
  4. Beierlein, Constanze, Eldad Davidov, Peter Schmidt, Shalom H. Schwartz and Beatrice Rammstedt. 2012. Testing the discriminant validity of Schwartz’ Portrait Value Questionnaire items—A replication and extension of Knoppen and Saris (2009). Survey Research Methods 6:25–36.Google Scholar
  5. Bilsky, Wolfgang, Michael Janik and Shalom H. Schwartz. 2011. The structural organization of human values—Evidence from three rounds of the European Social Survey (ESS). Journal of Cross-Cultural Psychology 42:759–776. Google Scholar
  6. Brown, Timothy A. 2015. Confirmatory factor analysis for applied research. New York: Guilford Press.Google Scholar
  7. Byrne, Barbara M., Richard J. Shavelson and Bengt O. Muthén. 1989. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin 105:456–466. Google Scholar
  8. Chen, Fang F. 2007. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 14:464–504. Google Scholar
  9. Chen, Fang F. 2008. What happens if we compare chopsticks with forks? The impact of making inappropriate comparison in cross-cultural research. Journal of Personality and Social Psychology 95:1005–1018. Google Scholar
  10. Cieciuch, Jan, and Eldad Davidov. 2012. A comparison of the invariance properties of the PVQ-40 and the PVQ-21 to measure human values across German and Polish samples. Survey Research Methods 6:37–48.Google Scholar
  11. Cieciuch, Jan, and Eldad Davidov. 2015. Establishing measurement invariance across online and offline samples. A tutorial with the software packages Amos and Mplus. Studia Psychologica 15: 83–99. Google Scholar
  12. Cieciuch, Jan, and Shalom H. Schwartz. 2012. The number of distinct basic values and their structure assessed by PVQ-40. Journal of Personality Assessment 94:321–328. Google Scholar
  13. Cieciuch, Jan, Eldad Davidov and René Algesheimer. 2016. The stability and change of value structure and priorities in childhood: A longitudinal study. Social Development 25:503–527. Google Scholar
  14. Cieciuch, Jan, Eldad Davidov, Peter Schmidt, René Algesheimer and Shalom H. Schwartz. 2014. Comparing results of an exact versus an approximate (Bayesian) measurement invariance test: A cross-country illustration with a new scale to measure 19 human values. Frontiers in Psychology 982:1–10. Google Scholar
  15. Cieciuch, Jan, Eldad Davidov and Peter Schmidt. 2018. Using alignment optimization in establishing measurement invariance. In Cross-cultural analysis: Methods and applications, 2nd edition, eds. Eldad Davidov, Peter Schmidt, Jaak Billiet and Bart Meuleman. New York: Routledge Taylor & Francis Group.Google Scholar
  16. Cieciuch, Jan, Eldad Davidov, Michele Vecchione, Constanze Beierlein and Shalom H. Schwartz. 2014a. The cross-national invariance properties of a new scale to measure 19 basic human values. A test across eight countries. Journal of Cross-Cultural Psychology 45:764–779. Google Scholar
  17. Cieciuch, Jan, Eldad Davidov, Michele Vecchione and Shalom H. Schwartz. 2014b. A hierarchical structure of basic human values in a third-order confirmatory factor analysis. Swiss Journal of Psychology 73:177–182. Google Scholar
  18. Cieciuch, Jan, Eldad Davidov, René Algesheimer and Peter Schmidt. 2017. Testing for approximate measurement invariance of human values in the European Social Survey. Sociological Methods & Research. Google Scholar
  19. Cieciuch, Jan, Shalom H. Schwartz and Eldad Davidov. 2015. Values, social psychology of. In International Encyclopedia of the Social & Behavioral Sciences, 2nd edition, v. 25, ed. James D. Wright, 41–46. Oxford: Elsevier.Google Scholar
  20. Cieciuch, Jan, Shalom H. Schwartz and Michele Vecchione. 2013. Applying the refined values theory to past data: What can researchers gain? Journal of Cross-Cultural Psychology 44:1215–1234. Google Scholar
  21. Coromina, Lluis, and Eldad Davidov. 2013. Evaluating measurement invariance for social and political trust in Western Europe over four measurement time points (2002–2008). ASK Research & Methods 22:35–52.Google Scholar
  22. Davidov, Eldad. 2008. A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Survey Research Methods 2:33–46.Google Scholar
  23. Davidov, Eldad. 2009. Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis 17:64–82.Google Scholar
  24. Davidov, Eldad. 2010. Testing for comparability of human values across countries and time with the third round of the European Social Survey. International Journal of Comparative Sociology 51:171–191. Google Scholar
  25. Davidov, Eldad, and Pascal Siegers. 2010. Comparing basic human values in East and West Germany. In Komparative empirische Sozialforschung (Comparative empirical social research), eds. Tilo Beckers, Klaus Birkelbach, Jörg Hagenah, and Ulrich Rosar, 43–63. Wiesbaden: VS.Google Scholar
  26. Davidov, Eldad, Jan Cieciuch, Peter Schmidt, Bart Meuleman and René Algesheimer. 2015. The comparability of measurements of attitudes toward immigration in the European Social Survey: Exact versus approximate measurement equivalence. Public Opinion Quarterly 79: 244–266. Google Scholar
  27. Davidov, Eldad, Hermann Dülmer, Jan Cieciuch, Anabel Kuntz, Daniel Seddig and Peter Schmidt 2016. Explaining measurement nonequivalence using multilevel structural equation modeling: The case of attitudes toward citizenship rights. Sociological Methods & Research. Google Scholar
  28. Davidov, Eldad, Hermann Dülmer, Elmar Schlueter, Peter Schmidt and Bart Meuleman. 2012. Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. Journal of Cross-Cultural Psychology 43:558–575. Google Scholar
  29. Davidov, Eldad, Bart Meuleman, Jan Cieciuch, Peter Schmidt and Jaak Billiet. 2014. Measurement equivalence in cross-national research. Annual Review of Sociology 40:55–75. Google Scholar
  30. Davidov, Eldad, Peter Schmidt and Shalom H. Schwartz. 2008. Bringing values back in: The adequacy of the European Social Survey to measure values in 20 countries. Public Opinion Quarterly 72:420–445. Google Scholar
  31. De Beuckelaer, Alain, and Gilbert Swinnen. 2018. Biased latent variable mean comparisons due to measurement noninvariance: A simulation study. In Cross-cultural research: Methods and applications, 2nd edition, eds. Eldad Davidov, Peter Schmidt, Jaak Billiet and Bart Meuleman, 127–156. New York: Routledge Taylor & Francis Group.Google Scholar
  32. Döring, Anna, Shalom H. Schwartz, Jan Cieciuch, Patrick J. F. Groenen, Valentina Glatzel, Justyna Harasimczuk, Nicole Janowicz, Maya Nyagolova, Rebecca E. Scheefer, Matthias Allritz, Taciano L. Milfont and Wolfgang Bilsky. 2015. Cross-cultural evidence of value structures and priorities in childhood. British Journal of Psychology 106:675–699. Google Scholar
  33. Durkheim, Émile. 1897/1964. Suicide. Glencoe, IL: Free Press.Google Scholar
  34. Goerres, Achim, Markus B. Siewert and Claudius Wagemann. 2019. Internationally comparative research designs in the social sciences: Fundamental issues, case selection logics, and research limitations. In Cross-national comparative research – analytical strategies, results and explanations. Sonderheft Kölner Zeitschrift für Soziologie und Sozialpsychologie. Eds. Hans-Jürgen Andreß, Detlef Fetchenhauer and Heiner Meulemann. Wiesbaden: Springer VS.
  35. Guenole, Nigel. 2016. The importance of isomorphism for conclusions about homology: A Bayesian multilevel structural equation modeling approach with ordinal indicators. Frontiers in Psychology 7:289. Google Scholar
  36. Hitlin, Steven, and Allyn Piliavin. 2004. Values: Reviving a dormant concept. Annual Review of Sociology 30:359–393. Google Scholar
  37. Hofstede, Geert. 2000. Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations, 2nd edition. Beverly Hills, CA: Sage.Google Scholar
  38. Horn, John L., and John J. McArdle. 1992. A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research 18:117–144. Google Scholar
  39. Inglehart, Ronald, and Wayne E. Baker. 2000. Modernization, cultural change, and the persistence of traditional values. American Sociological Review 65:19–51.Google Scholar
  40. Jak, Suzanne, Frans J. Oort and Conor V. Dolan. 2013. A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling 20:265–282. Google Scholar
  41. Jöreskog, Karl G. 1971. Simultaneous factor analysis in several populations. Psychometrika 36:409–426. Google Scholar
  42. Kluckhohn, Clyde. 1951. Values and value-orientations in the theory of action: An exploration in definition and classification. In Toward a general theory of action, eds. Talcott Parsons and Edward A. Shils, 388–433. Cambridge, MA: Harvard University Press.Google Scholar
  43. Knoppen, Desirée, and Willem Saris. 2009. Do we have combine values in the Schwartz’ human values scale? A comment on the Davidov studies. Survey Research Methods 3:91–103.Google Scholar
  44. Lomazzi, Vera. 2018. Using alignment optimization to test the measurement invariance of gender role attitudes in 59 countries. Methods, data, analyses: A journal for quantitative methods and survey methodology (mda) 12:77–103. Google Scholar
  45. Magun, Vladimir, Maxim Rudnev and Peter Schmidt. 2016. Within- and between-country value diversity in Europe: A typological approach. European Sociological Review 32:189–202. Google Scholar
  46. Marsh, Herbert W., Jiesi Guo, Philip D. Parker, Benjamin Nagengast, Tihomir Asparouhov, Bengt O. Muthén and Theresa Dicke. 2017. What to do when scalar invariance fails: The extended alignment method to multi-group factor analysis comparison of latent means across many groups. Structural Equation Modeling. Google Scholar
  47. Meitinger, Katharina. 2017. Necessary but insufficient: Why measurement invariance tests need online probing as a complementary tool. Public Opinion Quarterly 81:447–472. Scholar
  48. Merkle, Edgar C., and Yves Rosseel. 2016. blavaan: Bayesian structural equation modelling via parameter expansion. arXiv: 1511.05604v2 [stat.CO]. Retrieved from on June 4, 2018.Google Scholar
  49. Meuleman, Bart. 2019. Multilevel structural equation modeling for cross-national comparative research. In Cross-national comparative research – analytical strategies, results and explanations. Sonderheft Kölner Zeitschrift für Soziologie und Sozialpsychologie. Eds. Hans-Jürgen Andreß, Detlef Fetchenhauer and Heiner Meulemann. Wiesbaden: Springer VS.
  50. Millsap, Roger E. 2011. Statistical approaches to measurement invariance. New York: Routledge.Google Scholar
  51. Munck, Ingrid, Carolyn Barner and Judith Torney-Purta. 2017. Measurement invariance in comparing attitudes toward immigrants among youth across Europe in 1999 and 2009. The alignment method applied to IEA CIVED and ICCS. Sociological Methods & Research. Google Scholar
  52. Muthén Bengt O. 1994. Multilevel covariance structure analysis. Sociological Methods & Research 22:376–398. Google Scholar
  53. Muthén, Bengt O., and Tihomir Asparouhov. 2013. BSEM measurement invariance analysis. Mplus Web Notes 17:1–48.Google Scholar
  54. Muthén, Bengt O., and Tihomir Asparouhov. 2014. IRT studies of many groups: The alignment method. Frontiers in Psychology 978:1–7. Google Scholar
  55. Muthén, Bengt O., and Tihomir Asparouhov. 2017. Recent methods for the study of measurement invariance with many groups. Alignment and random effects. Sociological Methods & Research. Google Scholar
  56. Muthén, Linda K., and Bengt O. Muthén. 1998–2014. Mplus user’s guide. Los Angeles: Muthén & Muthén.Google Scholar
  57. Oberski, Daniel L. 2014. Evaluating sensitivity of parameters of interest to measurement invariance in latent variable models. Political Analysis 22:45–60. Google Scholar
  58. Rokeach, Milton. 1973. The nature of human values. New York, NY: Free Press.Google Scholar
  59. Rudnev, Maksim, Ekaterina Lytkina, Eldad Davidov, Peter Schmidt and Andreas Zick. 2018a. Testing measurement invariance for a second-order factor: A cross-national test of the alienation scale. Methods, data, analyses: A journal for quantitative methods and survey methodology (mda) 12:47–76. Google Scholar
  60. Rudnev, Maxim, Vladimir Magun and Shalom Schwartz. 2018b. Relations among higher order values around the world. Journal of Cross-Cultural Psychology 49(8):1165–1182. Google Scholar
  61. Ruelens, Anna, Bart Meuleman and Ides Nicaise. 2016. Examining measurement isomorphism of multilevel constructs: The case of political trust. Social Indicators Research. Google Scholar
  62. Schafer, Joseph L., and John W. Graham. 2002. Missing values: Our view of the state of the art. Psychological Methods 7:147–177. Google Scholar
  63. Schmidt-Catran, Alexander W., Malcolm Fairbrother and Hans-Jürgen Andreß. 2019. Multilevel models for the analysis of comparative survey data: Common problems and some solutions. In Cross-national comparative research – analytical strategies, results and explanations. Sonderheft Kölner Zeitschrift für Soziologie und Sozialpsychologie. Eds. Hans-Jürgen Andreß, Detlef Fetchenhauer and Heiner Meulemann. Wiesbaden: Springer VS.
  64. Schwartz, Shalom H. 1992. Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. In Advances in experimental social psychology, vol. 25, ed. Mark Zanna, 1–65. London, UK: Academic Press.Google Scholar
  65. Schwartz, Shalom H. 2003. A proposal for measuring value orientations across nations. In Questionnaire development package of the European Social Survey, 259–319. Retrieved from, June 30, 2016.Google Scholar
  66. Schwartz, Shalom H., and Jan Cieciuch. 2016. Values. In The ITC international handbook of testing and assessment, eds. Frederick T. L. Leong, Dave Bartram, Fanny M. Cheung, Kurt F. Geisinger and Dragos Iliescu, 106–119. Oxford: Oxford University Press.Google Scholar
  67. Schwartz, Shalom H., Jan Cieciuch, Michelle Vecchione, Eldad Davidov, Ronald Fischer, Constanze Beierlein, Alice Ramos, Markku Verkasalo, Jan-Erik Lönnqvist, Kursad Demirutku, Ozlem Dirilen-Gumus and Mark Konty. 2012. Refining the theory of basic individual values. Journal of Personality and Social Psychology 103:663–688. Google Scholar
  68. Schwartz, Shalom H., Gila Melech, Arielle Lehmann, Steven Burgess, Mari Harris and Vicki Owens. 2001. Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology 32:519–542.Google Scholar
  69. Sokolov, Boris. 2018. The index of emancipative values: Measurement model Misspecifications. American Political Science Review 112:395–408.Google Scholar
  70. Steenkamp, Jan-Benedict E. M., and Hans Baumgartner. 1998. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25:78–90. Google Scholar
  71. Steinmetz, Holger. 2018. Estimation and comparison of latent means across cultures. In Cross-cultural analysis: Methods and applications, 2nd edition, eds. Eldad Davidov, Peter Schmidt, Jaak Billiet and Bart Meuleman, 95–126. New York: Routledge Taylor & Francis Group.Google Scholar
  72. Steinmetz, Holger Rodrigo Isidor, Naissa Baeuerle. 2012. Testing the circular structure of human values: A meta-analytical structural equation modelling approach. Survey Research Methods 6:61–75Google Scholar
  73. van de Schoot, Rens, Anouck Kluytmans, Lars Tummers, Peter Lugtig, Joop Hox and Bengt O. Muthén. 2013. Facing off with Scylla and Charybdis: A comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Frontiers in Psychology 770:1–15. Google Scholar
  74. Vandenberg, Robert J., and Charles E. Lance. 2000. A review and synthesis of the measurement invariance literature: Suggestions, practices and recommendations for organizational research. Organizational Research Methods 3:4–69. Google Scholar
  75. Weber, Max. 1905/1958. The protestant ethic and the spirit of capitalism. New York: Scribner’s.Google Scholar
  76. Welzel, Christian, and Ronald F. Inglehart. 2016. Misconceptions of measurement equivalence: Time for a paradigm shift. Comparative Political Studies 49:1068–1094.Google Scholar
  77. Zercher, Florian, Peter Schmidt, Jan Cieciuch and Eldad Davidov. 2015. The comparability of the universalism value over time and across countries in the European Social Survey: Exact versus approximate measurement invariance. Frontiers in Psychology 733:1–11. Google Scholar

Copyright information

© Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature 2019

Authors and Affiliations

  • Jan Cieciuch
    • 1
    • 5
  • Eldad Davidov
    • 2
    • 3
    Email author
  • Peter Schmidt
    • 4
  • René Algesheimer
    • 5
  1. 1.Institute of PsychologyCardinal Wyszyński University in WarsawWarsawPoland
  2. 2.Institut für Soziologie und SozialpsychologieUniversität zu KölnCologneGermany
  3. 3.Department of Sociology and University Research Priority Program Social NetworksUniversity of ZurichZurichSwitzerland
  4. 4.Center for International Development and Environmental Research (ZEU)University of GiessenGiessenGermany
  5. 5.Department of Business Administration and University Research Priority Program Social NetworksUniversity of ZurichZurichSwitzerland

Personalised recommendations