Skip to main content
SpringerLink
Log in

Scale reliability of alternative cultural theory survey measures

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Grid-Group Cultural Theory (CT) has attracted significant attention across the social sciences and is increasingly being used in survey research. But the reliability of CT survey scales has mostly only been assessed using Cronbach’s alpha. Unfortunately, researchers frequently misinterpret this measure and often use the wrong or incomplete statistics to assess scale reliability. We compare the scale reliability of CT and its “cultural cognition theory” (CCT) variant as operationalized in surveys. Reviewing 73 survey studies using CT and CCT measures since 1990, we find limited application of appropriate method. CT measures, which are heterogeneous across studies, are far less reliable overall. But scales of roughly equal numbers of items are about equally reliable across CT and CCT studies. We apply recommended statistical measures to our data on both CT and CCT responses from an online national U.S. panel to illustrate a better approach to assessing scale reliability. We find that relative reliability is mixed (one CCT scale is more reliable, and one less, than the specific CT scales we used). But CT measures exhibit much better model fit in confirmatory factor analysis, a prerequisite for conducting valid reliability analyses. Thus, our literature review and survey analysis show that assessing the relative reliability of these scales is more complex than simply identifying the higher Cronbach’s alpha. Better tests of scale reliability could enhance CT survey research, and social science generally.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. As we will discuss, methodologists suggest that values > .90 likely indicate excessive overlap between items, but the language used in many empirical papers implies their authors’ belief in the higher-is-better premise.

  2. Some statisticians reject “internal consistency” as a misnomer for what reliability estimates measure, but do not seem to converge on a single alternative (see e.g., Boyle 1991). The concept itself has been defined in various ways too (Tang et al. 2014).

  3. We note that SPSS 28 has introduced menu-driven calculation of omega reliability, although with restrictions (e.g., omega if an item is removed from the scale is estimated only when at least four items comprise the scale).

  4. Another purported advantage of CCT is its minimization of people appearing supportive of more than one worldview (Kahan 2012). While Kahan (2012) believes that the possibility of endorsement of more than one cultural bias is a defect of CT measures, we do not agree, for various reasons that we review in the Implications section of this paper. Empirically, however, CCT measures do not, in fact, appear to prevent this alleged defect of multiple-bias endorsement (Johnson and Swedlow 2021).

  5. Dake’s (1990, p. 59) dissertation refers to “using the Spearman-Brown formula for construct reliability”, a method assessing how reliability estimates change with the number of items in the scale. Yet nowhere here or in later Dake publications are reliability estimates reported. Nor is it clear whether Dake estimated reliability for the Dake (1990) worldview items as well as for other multi-item measures in his survey instrument.

  6. Johnson and Swedlow (2020), Johnson et al. (2020), and Xue et al. (2016) included both CT and CCT measures in their studies, but count only once in the total here.

  7. 22 CT studies did not report reliability estimates (Anastasopoulou and Kokolakis 2013; Bouyer et al. 2001; Capstick and Pidgeon 2014; Dake 1990, 1991; Dake and Thompson 1999; Dake and Wildavsky 1991; Grendstad 2003a; Grendstad and Selle 1999; Grendstad and Sundback 2003; Johnson and Swedlow 2020; Johnson et al. 2020; Jones 2011; Langford et al. 2000; Leiserowitz 2006; Pendergraft 1998; Peters et al. 2004; Ripberger et al. 2012 [this paper says Cronbach’s alpha was calculated, but reports no estimates]; Simmons 2016; Sjöberg 2004; Slovic et al. 2000; Wildavsky and Dake 1990). One paper reported two CT studies: the first, with three items each for the four cultural biases, plus grid and group dimensions, reported no reliability estimates; the second, using Nordic data from a project of which he was co-author (Grendstad et al. 1999), reported reliability estimates (Sjöberg 2003). Latter results appear in Table 3; the former is excluded from the number of non-reporting articles.

  8. For example, Markle (2019) reported borrowing 16 CT items from Rippl (2002) yet showed only 14, including 3 fatalism items, calculated no alpha for fatalism, and did not mention fatalism in the article; Tikir and Lehmann (2011) reported using 23 CT items from Leiserowitz (2006) without specifying items per scale; Xue et al. (2016) used 22 items in their EFA because their CT and CCT measures included the same two items.

  9. Cultural worldviews assessed in five Nordic countries in 1999 were used by Olli (2012) and Sjöberg (2003, Study 3), although they offer slightly different reliability estimates, particularly for individualism and fatalism. Song (2014) and Song et al. (2014) used the same data on vaccination attitudes, reporting the same reliability estimates. Jones (2013) combined results from two surveys of which Jones and Song (2014) used only one, yielding slightly lower reliability estimates in the former (which also reported fatalist results not appearing in the latter); Song and Conner (2015) might report results that include but are not restricted to these two surveys, but this is unclear. Three papers used different parts of data collected 2014–2016 in Arkansas and Oregon from local policy elites and citizens (Song and Moyer 2017, all; Tumlison et al. 2017, included Jones and Song 2014 data on Arkansas elites; Tumlison and Song 2019 used all but 2014 elite data).

  10. Three CCT survey studies did not report reliability estimates (Kahan and Braman 2003; Johnson and Swedlow 2020; Johnson et al. 2020).

  11. Four grid/group measure studies did not report reliability estimates (Petray et al. 2014; Rippl 2002; Shin et al. 1989; Torsello 2013).

  12. Some confirmatory factor analyses also used CT statement measures from Wave 4, which in those cases restricted the sample size due to attrition between Waves 3 and 4 (Johnson et al. 2020). As there is just one CT statement per cultural bias, we could not use those data in reliability analyses.

References

  • Anastasopoulou, K., Kokolakis, S.: Exploring citizens’ intention to use e–government services: the role of cultural bias. Int. J. Electron. Gov. 6(1), 3–19 (2013)

    Google Scholar 

  • Bellaby, P.: To risk or not to risk? Uses and limitations of Mary Douglas on risk-acceptability for understanding health and safety at work and road accidents. Soc. Rev. 38(3), 465–483 (1990)

    Article  Google Scholar 

  • Bellamy, R., Hulme, M.: Beyond the tipping point: understanding perceptions of abrupt climate change and their implications. Weather Clim. Soc. 3, 48–60 (2011)

    Article  Google Scholar 

  • Boholm, A.: Risk perception and social anthropology: critique of cultural theory. Ethnos 61(1–2), 64–84 (1996)

    Article  Google Scholar 

  • Bonett, D.G.: Varying coefficient meta-analytic methods for alpha reliability. Psychol. Methods 15(4), 368–385 (2010)

    Article  Google Scholar 

  • Bouyer, M., Bagdassarian, S., Chaabanne, S., Mullet, E.: Personality correlates of risk perception. Risk Anal. 21(3), 457–466 (2001)

    Article  Google Scholar 

  • Boyle, G.J.: Does item homogeneity indicate internal consistency or item redundancy in psychometric scales? Personal. Individ. Differ. 12(3), 291–294 (1991)

    Article  Google Scholar 

  • Brcka-Lorenz, A., Chiang, Y.-C., Nelson Laird, T.: Internal consistency. FSSE [Faculty Survey of Student Engagement] Psychometric Portfolio (2013). http://hdl.handle.net/2022/24497

  • Brenot, J., Bonnefous, S., Marris, C.: Testing the cultural theory of risk in France. Risk Anal. 18(6), 729–739 (1998). https://doi.org/10.1111/j.1539-6924.1998.tb01116.x

    Article  Google Scholar 

  • Briggs, S.R., Cheek, J.M.: The role of factor analysis in the evaluation of personality scales. J. Pers. 54(1), 106–148 (1986)

    Article  Google Scholar 

  • Cambardella, C., Fath, B.D., Werdenigg, A., Gulas, C., Katzmair, H.: Assessing the operationalization of cultural theory through surveys investigating the social aspects of climate change policymaking. Weather Clim. Soc. 12(4), 651–665 (2020)

    Article  Google Scholar 

  • Capstick, S.B., Pidgeon, N.F.: Public perception of cold weather events as evidence for and against climate change. Clim. Change 122, 695–708 (2014)

    Article  Google Scholar 

  • Chauvin, B., Chassang, I.: Cultural orientation and risk perception: development of a scale operating in a French context. Risk Anal. 42, 2189–2213 (2022)

    Article  Google Scholar 

  • Clark, L.A., Watson, D.: Constructing validity: basic issues in objective scale development. Psychol. Assess. 7(3), 309–319 (1995)

    Article  Google Scholar 

  • Conner, T.W., Nowlin, M.C., Rabovsky, T., Ripberger, J.T.: Cultural theory and managerial values: examining trust as a motivation for collaboration. Public Adm. 94(4), 915–932 (2016)

    Article  Google Scholar 

  • Coughlin, Lockhart, C.: Grid-group theory and political ideology: a consideration of their relative strengths and weaknesses for explaining the structure of mass belief systems. J. Theor. Polit. 10(1), 33–58 (1998)

    Article  Google Scholar 

  • Coyle, D.J.: This land is your land, this land is my land: cultural conflict in environmental and land-use regulation. In: Coyle, D.J., Ellis, R.J. (eds.) Politics, policy, and culture, pp. 33–50. Westview Press, Boulder (1994)

    Google Scholar 

  • Cronbach, L.: Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951)

    Article  Google Scholar 

  • Dake, K.: Orienting dispositions in the perception of risk: an analysis of contemporary worldviews and cultural biases. J. Cross-Cultural Psychol. 22(1), 61–82 (1991)

    Article  Google Scholar 

  • Dake, K.: Myths of nature: culture and the social construction of risk. J. Soc. Issues 48(4), 21–37 (1992)

    Article  Google Scholar 

  • Dake, K., Thompson, M.: Making ends meet, in the household and on the planet. GeoJournal 47, 417–424 (1999)

    Article  Google Scholar 

  • Dake, K., Wildavsky, A.: Individual differences in risk perception and risk-taking preferences. In: Garrick, B.J., Gekler, W.C. (eds.) The analysis, communication and perception of risk, pp. 15–24. Plenum Press, New York (1991)

    Chapter  Google Scholar 

  • Dake, K.: Technology on trial: Orienting dispositions toward environmental and health standards [Ph.D. dissertation]. University of California, Berkeley (1990)

  • de Vaus, D.: Analyzing social science data: 50 key problems in data analysis. Sage, Thousand Oaks (2002)

    Google Scholar 

  • Douglas, M., Wildavsky, A.: Risk and Culture: an essay on the selection of technological and environmental dangers. University of California Press, Berkeley (1982)

    Google Scholar 

  • Ellis, R.J., Thompson, F.: Culture and the environment in the Pacific Northwest. Am. Polit. Sci. Rev. 91(4), 885–897 (1997)

    Article  Google Scholar 

  • Favre, M., Swedlow, B., Verweij, M.: A cultural theory and model of power relations. J. Polit. Power 12(2), 245–275 (2019)

    Article  Google Scholar 

  • Flora, D.B.: Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Adv. Methods Pract. Psychol. Sci. 3(4), 484–501 (2020)

    Article  Google Scholar 

  • Gastil, J., Braman, D., Kahan, D., Slovic, P.: The cultural orientation of mass political opinion. PS Polit. Sci. Polit. 44(4), 711–714 (2011)

    Article  Google Scholar 

  • Goebbert, K., Jenkins, H.C., Klockow, K., Nowlin, M.C., Silva, C.L.: Weather, climate, and worldviews: the sources and consequences of public perceptions of changes in local weather patterns. Weather Clim. Soc. 4(2), 132–144 (2012)

    Article  Google Scholar 

  • Goodwin, R., Nizhardze, G., Luu, L.A.N., Kosa, E., Emelyanova, T.: Glasnost and the art of conversation: a multilevel analysis of intimate disclosure across three former communist cultures. J. Cross Cult. Psychol. 30(1), 72–90 (1999)

    Article  Google Scholar 

  • Grendstad, G.: Grid-group theory and political orientation: effects of cultural biases in Norway in the 1990s. Scand. Polit. Stud. 23(3), 217–243 (2000)

    Article  Google Scholar 

  • Grendstad, G.: Nordic cultural baselines: accounting for domestic convergence and foreign policy divergence. J. Comp. Policy Anal. 3, 5–29 (2001)

    Google Scholar 

  • Grendstad, G.: Comparing political orientations: grid-group theory versus the left-right dimension in the five Nordic countries. Eur. J. Polit. Res. 42(1), 1–21 (2003a)

    Article  Google Scholar 

  • Grendstad, G.: Reconsidering nordic party space. Scand. Polit. Stud. 26(3), 193–217 (2003b)

    Article  Google Scholar 

  • Grendstad, G., Selle, P.: Cultural theory, postmaterialism, and environmental attitudes. In Ellis, R.J., Thompson, M. (eds.) Culture matters: Essays in honor of Aaron Wildavsky, pp. 151–168. Boulder, CO: Westview Press (1997)

  • Grendstad, G., Selle, P.: The formation and transformation of preferences: cultural theory and postmaterialism compared. In: Thompson, M., Grendstad, G., Selle, P. (eds.) Cultural theory as political science, pp. 43–58. Routledge, London (1999)

    Google Scholar 

  • Grendstad, G., Selle, P.: Cultural myths of human and physical nature: integrated or separated? Risk Anal. 20(1), 27–39 (2000)

    Article  Google Scholar 

  • Grendstad, G., Sundback, S.: Socio-demographic effects on cultural biases: a Nordic study of grid-group theory. Acta Sociologica 46(4), 289–306 (2003)

    Article  Google Scholar 

  • Grendstad, G., Jensen, L., Kristinsson, G.-H., Sjöberg, L., Sundback, S.: Nordic cultures: measurement, consequences and comparisons; study description and frequencies. LOS Senteret, University of Bergen, Norway (1999)

    Google Scholar 

  • Guttman, L.: A basis for analyzing test-retest reliability. Psychometrika 10, 255–282 (1945)

    Article  Google Scholar 

  • Hair, J.F., Jr., Hult, G.T.M., Ringle, C.M., Sarstedt, M., Danks, N.P., Ray, S.: Evaluation of reflective measurement models. In: Partial Least Squares Structural Equation Modeling (PLS-SEM) Using R, pp. 75–90. Springer, Cham (2021)

    Chapter  Google Scholar 

  • Hattie, J.A.: Methodology review: assessing unidimensionality of tests and items. Appl. Psychol. Meas. 9, 139–164 (1985)

    Article  Google Scholar 

  • Herron, K.G., Jenkins-Smith, H.C.: Critical masses and critical choices: evolving public opinion on nuclear weapons, terrorism, and security. University of Pittsburgh Press, Pittsburgh (2006)

    Book  Google Scholar 

  • Hirsch, R., Baxter, J.: Context, cultural bias, and health risk perception: the “everyday” nature of pesticide policy preferences in London, Calgary, and Halifax. Risk Anal. 31(5), 847–865 (2011)

    Article  Google Scholar 

  • Iwaki, Y.K.: The culture of green: the role of cultural worldviews, psychological connectedness, time discounting, and social norms in environmental decisions. Columbia University, New York (2011). https://doi.org/10.7916/D85D8ZSN

    Book  Google Scholar 

  • Jackson, N.M.: A theory of preference formation among ideologues and nonideologues. Soc. Sci. Q. 96, 1–18 (2014)

    Article  Google Scholar 

  • Jenkins-Smith, H.C., Smith, W.K.: Ideology, culture, and risk perception. In: Coyle, D.J., Ellis, R.J. (eds.) Politics, policy, and culture, pp. 17–32. Westview Press, Boulder (1994)

    Google Scholar 

  • Jensen, L.: Images of democracy in Danish social housing. In: Thompson, M., Grendstad, G., Selle, P. (eds.) Cultural theory as political science, pp. 167–180. Routledge, London (1999)

    Google Scholar 

  • Johnson, B.B.: The environmentalist movement and grid/group analysis: a modest critique. In: Johnson, B.B., Covello, V.T. (eds.) The social and cultural construction of risk: essays on risk selection and perception, pp. 147–175. Reidel Publishing Company, Dordrecht (1987)

    Chapter  Google Scholar 

  • Johnson, B.B., Swedlow, B.: Comparing cultural theory and cultural cognition theory to each other and as explanations for judged risk. J. Risk Res. 23(10), 1278–1300 (2020)

    Article  Google Scholar 

  • Johnson, B.B., Swedlow, B.: Cultural theory’s contributions to risk analysis: a thematic review with directions and resources for further research. Risk Anal. 41(3), 429–450 (2021). https://doi.org/10.1111/risa.13299

    Article  Google Scholar 

  • Johnson, B.B., Swedlow, B., Mayorga, M.: Cultural theory and cultural cognition theory survey measures: confirmatory factoring and predictive validity of factor scores for judged risk. J. Risk Res. 23(11), 1467–1490 (2020)

    Article  Google Scholar 

  • Johnson, B., Brandt, M., Swedlow, B., Turner-Zwinkels, F.: How alike are political values measures?: Comparing measures of universal values, moral foundations, cultural theory, and cultural cognition theory. Comparing Measures of Universal Values, Moral Foundations, Cultural Theory, and Cultural Cognition Theory (2022). https://doi.org/10.2139/ssrn.4050134

  • Jones, M.D.: Leading the way to compromise? Cultural theory and climate change opinion. PS Polit. Sci. Polit. 44(4), 720–725 (2011)

    Article  Google Scholar 

  • Jones, M.D.: Cultural characters and climate change: how heroes shape our perception of climate science. Soc. Sci. Q. 95(1), 1–39 (2013)

    Article  Google Scholar 

  • Jones, M.D., Song, G.: Making sense of climate change: how story frames shape cognition. Polit. Psychol. 35(4), 447–476 (2014)

    Article  Google Scholar 

  • Kahan, D.M.: Culture, cognition, and consent: who perceives what, and why, in acquaintance-rape cases. Univ. Pa. Law Rev. 158(3), 729–813 (2010)

    Google Scholar 

  • Kahan, D.M.: Cultural cognition as a conception of the cultural theory of risk. In: Roeser, S., Hillerbrand, R., P. Sandin, P., Petersen, M. (eds.) Handbook of risk theory: Epistemology, decision theory, ethics, and social implications of risk, pp. 725–759. Springer, Netherlands (2012)

    Chapter  Google Scholar 

  • Kahan, D.M., Braman, D.: More statistics, less persuasion: a cultural theory of gun-risk perceptions. Univ. Pa. Law Rev. 151(4), 1291–1327 (2003)

    Article  Google Scholar 

  • Kahan, D.M., Braman, D.: The self-defensive cognition of self-defense. Am. Crim. Law Rev. 45(1), 1–65 (2008)

    Google Scholar 

  • Kahan, D.M., Braman, D., Gastil, J., Slovic, P., Mertz, C.K.: Culture and identity-protective cognition: explaining the white-male effect in risk perception. J. Empir. Leg. Stud. 4(3), 465–505 (2007)

    Article  Google Scholar 

  • Kahan, D.M., Braman, D., Slovic, P., Gastil, J., Cohen, G.: Cultural cognition of the risks and benefits of nanotechnology. Nat. Nanotechnol. 4(2), 87–91 (2009)

    Article  Google Scholar 

  • Kahan, D.M., Braman, D., Cohen, G.L., Gastil, J., Slovic, P.: Who fears the HPV vaccine, who doesn’t, and why? An experimental study of the mechanisms of cultural cognition. Law Hum Behav. 34(6), 501–516 (2010a)

    Article  Google Scholar 

  • Kahan, D.M., Braman, D., Monahan, J., Callahan, L., Peters, E.: Cultural cognition of public policy: the case of outpatient commitment laws. Law Hum Behav. 34(2), 118–140 (2010b)

    Article  Google Scholar 

  • Kahan, D.M., Jenkins-Smith, H., Braman, D.: Cultural cognition of scientific consensus. J. Risk Res. 14(2), 147–174 (2011a)

    Article  Google Scholar 

  • Kahan, D.M., Wittlin, M., Peters, E., Slovic, P., Ouellette, L.L., Braman, D., Mandel, G.: The tragedy of the risk-perception commons: culture conflict, rationality conflict, and climate change (Temple University Legal Studies Research Paper No. 2011b-26. In: Cultural Cognition Project Working Paper No. 89, Yale Law & Economics Research Paper No. 435, Yale Law School, Public Law Working Paper No. 230) (2011b). https://doi.org/10.2139/ssrn.1871503

  • Kahan, D.M., Jamieson, K.H., Landrum, A., Winneg, K.: Culturally antagonistic memes and the Zika virus: an experimental test. J. Risk Res. 20(1), 1–40 (2017)

    Article  Google Scholar 

  • Kiss, S.J., Montpetit, É., Lachapelle, E.: Beyond regions and ideology: using cultural theory to explain risk perception in Canada. Can. J. Polit. Sci. 53(2), 439–460 (2020)

    Article  Google Scholar 

  • Kline, P.: Psychometrics and psychology. Academic Press, New York (1979)

    Google Scholar 

  • Lachapelle, E., Montpetit, É., Gauvin, J.P.: Public perceptions of expert credibility on policy issues: The role of expert framing and political worldviews. Policy. Stud. J. 42(4), 674–697 (2014)

  • Langford, I.H., Georgiou, S., Bateman, I.J., Day, R.J., Turner, R.K.: Public perceptions of health risks from polluted coastal bathing waters: a mixed methodological analysis using cultural theory. Risk Anal. 20(5), 691–704 (2000)

    Article  Google Scholar 

  • Leiserowitz, A.: Climate change risk perception and policy preferences: the role of affect, imagery, and values. Clim. Change 77, 45–72 (2006)

    Article  Google Scholar 

  • Li, M., Zhang, H., Xiao, H., Chen, Y.: A grid-group analysis of tourism motivation. Int. J. Tour. Res. 17(1), 35–44 (2013)

    Article  Google Scholar 

  • Liu, L.-Y.: Irrational decision making: cultural influence on environmental nonprofit organizations’ advocacy strategies. Int. J. Nonprofit Volunt. Sect. Mark. 23(2), e1627 (2018). https://doi.org/10.1002/nvsm.1627

    Article  Google Scholar 

  • Lounsbury, J.W., Gibson, L.W., Saudardas, R.A.: Scale development. In: Leong, F.T.L., Austin, J.T. (eds.) The psychology research handbook: a guide for graduate students and research assistants, 2nd edn., pp. 125–146. Sage, Thousand Oaks (2005)

    Google Scholar 

  • Maleki, A., De Jong, M.: A proposal for clustering the dimensions of national culture. Cross-Cult. Res. 48(2), 107–143 (2014)

    Article  Google Scholar 

  • Maleki, A., Hendriks, F.: Grid, group, and grade: Challenges in operationalizing cultural theory for cross-national research. Cross-Cultu. Res. 49(3), 250–280 (2015)

  • Markle, G.: Understanding pro-environmental behavior in the US: insights from grid-group cultural theory and cognitive sociology. Sustainability 11(2), 532 (2019). https://doi.org/10.3390/su11020532

    Article  Google Scholar 

  • Marris, C., Langford, I.H., O’Riordan, T.: A quantitative test of the cultural theory of risk perceptions: comparison with the psychometric paradigm. Risk Anal. 18(5), 635–647 (1998)

    Article  Google Scholar 

  • McDonald, R.P.: The dimensionality of tests and items. Br. J. Math. Stat. Psychol. 34(1), 100–117 (1981)

    Article  Google Scholar 

  • Meader, N., Uzzell, D., Gatersleben, B.: Cultural theory and quality of life. Eur. Rev. Appl. Psychol. 56(1), 61–69 (2006)

    Article  Google Scholar 

  • Michaud, K.E.H., Carlisle, J.E., Smith, E.R.A.N.: The relationship between cultural values and political ideology, and the role of political knowledge. Polit. Psychol. 30(1), 27–42 (2009)

    Article  Google Scholar 

  • Nowlin, M.C., Rabovsky, T.M.: A cultural theory of partisanship and policy attitudes. Soc. Sci. Q. 101(2), 878–892 (2020)

  • Nunnally, J.C.: Psychometric theory, 2nd edn. McGraw-Hill, New York (1978)

    Google Scholar 

  • Nunnally, J.C., Bernstein, I.H.: Psychometric theory, 3rd edn. McGraw Hill, New York (1994)

    Google Scholar 

  • Olli, E., Swedlow, B.: Cultural theory, rejection of cultural bias, and party preference. Party Polit. (2022). https://doi.org/10.1177/13540688211071065

    Article  Google Scholar 

  • Olli, E.: Rejected cultural biases shape our political views: A migrant household study and two large-scale surveys (Ph.D. dissertation). University of Bergen, Norway (2012)

  • Palmer, C.G.S.: Risk perception: an empirical study of the relationship between worldview and the risk construct. Risk Anal. 16, 717–723 (1996)

    Article  Google Scholar 

  • Panayides, P.: Coefficient alpha: interpret with caution. Eur. J. Psychol. 9(4), 687–696 (2013)

    Article  Google Scholar 

  • Pendergraft, C.A.: Human dimensions of climate change: cultural theory and collective action. Clim. Change 39, 643–666 (1998)

    Article  Google Scholar 

  • Peters, E., Slovic, P.: The role of affect and worldviews as orienting dispositions in the perception and acceptance of nuclear power. J. Appl. Soc. Psychol. 26(16), 1427–1453 (1996)

    Article  Google Scholar 

  • Peters, E.M., Burraston, B., Mertz, C.K.: An emotion-based model of risk perception and stigma susceptibility: cognitive appraisals of emotion, affective reactivity, worldviews, and risk perceptions in the generation of technological stigma. Risk Anal. 24(5), 1349–1367 (2004)

    Article  Google Scholar 

  • Peterson, R.A.: A meta-analysis of Cronbach’s coefficient alpha. J. Consum. Res. 21(2), 381–391 (1994)

    Article  Google Scholar 

  • Petray, L., Moyer, R., Song, G.: Understanding public risk perceptions associated with gay and lesbian parenting in the United States. In: Conference: APPAM 2014 Fall Conference (2014). https://doi.org/10.13140/RG.2.1.4483.6002

  • Price, J.C., Walker, I.A., Boschetti, F.: Measuring cultural values and beliefs about environment to identify their role in climate change responses. J. Environ. Psychol. 37, 8–20 (2014)

    Article  Google Scholar 

  • Raykov, T.: Scale construction and development using structural equation modeling. In: Hoyle, R.H. (ed.) Handbook of structural equation modeling, pp. 472–492. The Guilford Press, New York (2012)

    Google Scholar 

  • Rayner, S.: Cultural theory and risk analysis. In: Krimsky, S., Golding, D. (eds.) Social theories of risk, pp. 83–115. Praeger, Westport (1992)

    Google Scholar 

  • Renn, O., Rohrmann, B. (eds.): Kluwer Publishers, New York (2000)

    Google Scholar 

  • Ripberger, J.T., Jenkins-Smith, H.C., Herron, K.G.: How cultural orientations create shifting national security coalitions on nuclear weapons and terrorist threats in the American public. PS Polit. Sci. Polit. 44(4), 715–719 (2011)

    Article  Google Scholar 

  • Ripberger, J.T., Song, G., Nowlin, M.C., Jones, M.D., Jenkins-Smith, H.C.: Reconsidering the relationship between cultural theory, political ideology, and political knowledge. Soc. Sci. Q. 93(3), 713–731 (2012)

    Article  Google Scholar 

  • Ripberger, J.T., Gupta, K., Silva, C.L., Jenkins-Smith, H.C.: Cultural theory and the measurement of deep core beliefs within the advocacy coalition framework. Policy Stud. J. 42(4), 509–527 (2014)

    Article  Google Scholar 

  • Rippl, S.S.: Cultural theory and risk perception: a proposal for a better measurement. J. Risk Res. 5(2), 147–165 (2002)

    Article  Google Scholar 

  • Rizopoulos, D.: ltm: an R package for latent variable modeling and item response theory analyses. J. Stat. Softw. 17, 1–25 (2006)

    Article  Google Scholar 

  • Robinson, R., Swedlow, B.: Beyond liberal and conservative: advancing the study of judicial behavior with a cultural theory of political values. J. Law Courts 6(2), 263–302 (2018)

    Article  Google Scholar 

  • Santor, D.A., Gregus, M., Welch, A.: Eight decades of measurement in depression. Measurement 4(3), 135–155 (2006)

    Google Scholar 

  • Schmitt, N.: Uses and abuses of coefficient alpha. Psychol. Assess. 8(4), 350–353 (1996)

    Article  Google Scholar 

  • Shin, D.C., Chey, M., Kim, K.-W.: Cultural origins of public support for democracy in Korea: a empirical test of the Douglas-Wildavsky theory of culture. Comp. Pol. Stud. 22(2), 217–238 (1989)

    Article  Google Scholar 

  • Sijtsma, K.: On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika 74(1), 107–120 (2009)

    Article  Google Scholar 

  • Simmons, R.: Improvement and public service relationships: cultural theory and institutional work. Public Adm. 94(4), 933–952 (2016)

    Article  Google Scholar 

  • Sjöberg, L.: World views, political attitudes and risk perception. Risk Health Saf. Environ. 9(2), 137–152 (1998b)

    Google Scholar 

  • Sjöberg, L.: Distal factors in risk perception. J. Risk Res. 6(3), 187–211 (2003)

    Article  Google Scholar 

  • Sjöberg, L.: Principles of risk perception applied to gene technology. EMBO Rep. 5, S47–S51 (2004)

    Article  Google Scholar 

  • Sjöberg, L.: Are received risk perception models alive and well?. Risk Anal. 22(4), S47–S51 (2005a)

    Article  Google Scholar 

  • Sjöberg, L.: The importance of respect for empirical findings. Response to Tansey. J. Risk Res. 8(78), 713–715 (2005b)

    Article  Google Scholar 

  • Sjöberg, L.: Explaining risk perception: An empirical evaluation of cultural theory. In R. E. Löfstedt & L. Frewer (Eds.), The Earthscan reader in risk and modern society (Vol. 2, pp. 115–132). Earthscan. (Republished from “Explaining risk perception: an empirical evaluation of cultural theory,” 1997, Risk Decision and Policy, 2, 113–130 (1998a)

  • Slovic, P., Flynn, J., Mertz, C.K., Poumadere, M., Mays, C.: Nuclear power and the public: a comparative study of risk perception in France and the United States. In: Renn, O., Rohrmann, B. (eds.) Cross-cultural risk perception: a survey of empirical studies, pp. 55–102. Kluwer Academic, Dordrecht (2000)

    Chapter  Google Scholar 

  • Song, G.: Understanding public perceptions of benefits and risks of childhood vaccinations in the United States. Risk Anal. 34(3), 541–555 (2014)

    Article  Google Scholar 

  • Song, G., Conner, T.E.: Understanding environmentalism: the impacts of cultural worldview, party identification, and religious affiliation on environmental concern. SSRN (2015). https://doi.org/10.2139/ssrn.2621626

    Article  Google Scholar 

  • Song, G., Moyer, R.: Core political values? A comparative analysis of left-right political ideology and cultural orientations among policy elites and the general public. Risk Anal. 37(5), 1018–1036 (2017)

    Google Scholar 

  • Song, G., Silva, C.L., Jenkins-Smith, H.C.: Cultural worldview and preference for childhood vaccination policy. Policy Stud. J. 42(4), 528–554 (2014)

    Article  Google Scholar 

  • Steg, L., Sievers, I.: Cultural theory and individual perceptions of environmental risks. Environ. Behav. 332, 250–269 (2000)

    Article  Google Scholar 

  • Stern, P.C., Dietz, T., Abel, T.D., Guagnano, G.A., Kalof, L.: A value-belief-norm theory of support for social movements: the case of environmentalism. Hum. Ecol. Rev. 6(2), 81–97 (1999)

    Google Scholar 

  • Swedlow, B.: Beyond liberal and conservative: two-dimensional conceptions of ideology and the structure of political attitudes and values. J. Polit. Ideol. 13(2), 155–178 (2008)

    Article  Google Scholar 

  • Swedlow, B.: Editor’s introduction: cultural theory’s contributions to political science. PS Polit. Sci. Polit. 44(4), 703–710 (2011)

    Article  Google Scholar 

  • Swedlow, B.: Advancing policy theory with cultural theory: an introduction to the special issue. Policy Stud. J. 42(4), 465–483 (2014)

    Article  Google Scholar 

  • Swedlow, B.: Introduction to the symposium: an institutional theory of cultural biases, public administration and public policy. Public Adm. 94(4), 867–880 (2016)

    Article  Google Scholar 

  • Swedlow, B., Ripberger, J.T., Liu, L.-Y., Silva, C.L., Jenkins-Smith, H., Johnson, B.B.: Construct validity of cultural theory survey measures. Soc. Sci. Q. 101(6), 2332–2383 (2020)

    Article  Google Scholar 

  • Tang, W., Cui, Y., Babenko, O.: Internal consistency: do we really know what it is and how to assess it? J. Psychol. Behav. Sci. 2(2), 205–220 (2014)

    Google Scholar 

  • Thompson, M., Ellis, R., Wildavsky, A.: Cultural theory. Westview Press, Boulder (1990)

    Google Scholar 

  • Tikir, A., Lehmann, B.: Climate change, theory of planned behavior and values: a structural equation model with mediation analysis. Clim. Change 104, 389–402 (2011)

    Article  Google Scholar 

  • Torsello, D.: The perception of corruption as social and institutional pressure: a comparative analysis of cultural biases. Hum. Aff. 23(2), 160–173 (2013)

    Article  Google Scholar 

  • Traub, R.E.: Reliability for the social sciences: theory and applications. Sage, Thousand Oaks (1994)

    Google Scholar 

  • Trizano-Hermosilla, I., Alvarado, J.M.: Best alternatives to Cronbach’s alpha reliability in realistic conditions: congeneric and asymmetrical measurements. Front. Psychol. 7, 769 (2016). https://doi.org/10.3389/fpsyg.2016.00769

    Article  Google Scholar 

  • Trousset, S.S., Gupta, K., Jenkins-Smith, H., Silva, C.L., Herron, K.: Degrees of engagement: using cultural worldviews to explain variations in public preferences for engagement in the policy process. Policy Stud. J. 43(1), 44–69 (2015)

    Article  Google Scholar 

  • Tumlison, C., Song, G.: Cultural values, trust, and benefit-risk perceptions of hydraulic fracturing: a comparative analysis of policy elites and the general public. Risk Anal. 39(3), 511–534 (2019)

    Article  Google Scholar 

  • Tumlison, C., Moyer, R.M., Song, G.: The origin and role of trust in local policy elites’ perceptions of high-voltage power line installations in the state of Arkansas. Risk Anal. 37(5), 1018–1036 (2017)

    Article  Google Scholar 

  • Turner-Zwinkels, F.M., Johnson, B.B., Sibley, C.G., Brandt, M.J.: Conservatives’ moral foundations are more densely connected than liberals’ moral foundations. Personal. Social Psychol. Bull. 47(2), 167–184 (2021)

  • van den Bergh, R.: Results of PATT research in France, Denmark and the Netherlands. In: Coenen-van den Bergh, R. Eed.) Report: PATT Conference, vol. 2 contributions, pp. 41–51. University of Technology (1987)

  • Verweij, M., Luan, S., Nowacki, M.: How to test cultural theory: suggestions for future research. PS Polit. Sci. Polit. 44(4), 745–748 (2011)

    Article  Google Scholar 

  • Wasserstein, R., Lazar, N.: The ASA’s statement on p-values: context, process, and purpose. Am. Stat. 70, 129–133 (2016)

    Article  Google Scholar 

  • Wasserstein, R.L., Schirm, A.L., Lazar, N.A.: Moving to a world beyond “p < .05.”. Am. Stat. 73(S1), 1–19 (2019). https://doi.org/10.1080/00031305.2019.1583913

    Article  Google Scholar 

  • Wildavsky, A.: Cultural analysis: politics, public law, and administration. Transaction Publishers, New Jersey (2006)

    Google Scholar 

  • Wildavsky, A., Dake, K.: Theories of risk perception: who fears what and why? Daedalus 119(4), 41–60 (1990)

    Google Scholar 

  • Wouters, K., Maesschalck, J.: Surveying organizational culture to explore grid-group cultural theory: instrument design and preliminary empirical results. Int. J. Organ. Anal. 22(2), 224–246 (2014). https://doi.org/10.1108/IJOA-11-2011-0529

    Article  Google Scholar 

  • Xue, W., Hine, D.W., Loi, N.M., Thorsteinsson, E.B., Phillips, W.J.: Cultural worldviews and environmental risk perceptions: a meta-analysis. J. Environ. Psychol. 40, 249–258 (2014). https://doi.org/10.1016/j.jenvp.2014.07.002

    Article  Google Scholar 

  • Xue, W., Hine, D.W., Marks, A.D.G., Phillips, W.J., Zhao, S.: Cultural worldviews and climate change: a view from China. Asian J. Soc. Psychol. 19(2), 134–144 (2016). https://doi.org/10.1111/ajsp.12116

    Article  Google Scholar 

  • Yuan, M., Swedlow, B.: Chinese cultural biases, value congruence, and support for and compliance with protective policies during the COVID-19 pandemic. Rev. Policy Res. 00, 1–33 (2022). https://doi.org/10.1111/ropr.12515

    Article  Google Scholar 

  • Zanocco, C.M., Jones, M.D.: Cultural worldviews and political process preferences. Soc. Sci. Q. 99(4), 1377–1389 (2018). https://doi.org/10.1111/ssqu.12500

    Article  Google Scholar 

Download references

Acknowledgements

This material is based upon work supported by the U.S. National Science Foundation under Grant No. 1644853. Literature was collected, and the paper formatted, by Lei "Annie" Tong, Derek Shaw, Levi Ratliff, and Ernest Plange Kwofie of Northern Illinois University, and survey data were collected under the supervision of Marcus Mayorga of Decision Research. We appreciate the suggestions of the editor and two anonymous reviewers.

Funding

Funding was provided by the U.S. National Science Foundation (Grant No. 1644853).

Author information

Authors and Affiliations

  1. Decision Research, P.O. Box 72538, Springfield, OR, 97475, USA

    Branden B. Johnson

  2. Department of Political Science, School of Public and Global Affairs, Faculty Associate, College of Law and Institute for the Study of the Environment, Sustainability, and Energy, Northern Illinois University, 1425 West Lincoln Highway, DeKalb, IL, 60115-2887, USA

    Brendon Swedlow

Authors
  1. Branden B. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brendon Swedlow
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BBJ initiated overall survey conception and design, and supervised material preparation, data collection and analysis, with BS suggesting addition of CT measures to the earlier CCT measures. Both authors contributed to paper conception and design, and BBJ conducted its statistical analyses while BS oversaw literature identification and collection. BBJ wrote the first draft, and both authors commented on previous versions of the manuscript and approved the final manuscript.

Corresponding author

Correspondence to Branden B. Johnson.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Johnson, B.B., Swedlow, B. Scale reliability of alternative cultural theory survey measures. Qual Quant 58, 527–557 (2024). https://doi.org/10.1007/s11135-023-01653-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-023-01653-4

Keywords

Search

Navigation