Abstract
In order to measure ideology, political scientists heavily rely on the so-called left-right scale. Left and right are, however, abstract political concepts and may trigger different associations among respondents. If these associations vary systematically with other variables this may induce bias in the empirical study of ideology. We illustrate this problem using a unique survey that asked respondents open-ended questions regarding the meanings they attribute to the concepts “left” and “right”. We assess and categorize this textual data using topic modeling techniques. Our analysis shows that variation in respondents’ associations is systematically related to their self-placement on the left-right scale and also to variables such as education and respondents’ cultural background (East vs. West Germany). Our findings indicate that the interpersonal comparability of the left-right scale across individuals is impaired. More generally, our study suggests that we need more research on how respondents interpret various abstract concepts that we regularly use in survey questions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
In their seminal work “The Psychology of Survey Response” Tourangeau et al. (2000, p. 45) dedicate a whole chapter to the general problem (see Chap. 2.4.2) and discuss the study by Belson (1981). Belson (1981) reports that only 8 % understood the question as intended. Whereas some respondents associated “children” with “kids 8 years old or younger”, “others understood children as those 19–20 years old or younger” (Tourangeau et al. 2000, p. 45).
Google Ngram Viewer seems to indicate that widespread use really picked up in 1970.
Person A might have a small car in mind whereas person B might think of a Hummer which may affect their evaluation of e.g. the impact of cars on the environment.
It has long been argued that differences between scale points (e.g. the step from 9 to 10) may be interpreted differently by different respondents. Quantifiers on response scales such as “totally agree” also represent vague concepts and might be used differently by different respondents. For instance, Tourangeau et al. (2000, p. 47) and King et al. (2004a) discuss this problem and potential solutions. The problem we discuss here is similar in nature, however, we want to emphasize the role played by more abstract concepts such “democracy” or “left”. Especially, in cross-cultural and cross-linguistic research the error induced by the latter concepts should be more relevant.
We only illustrate the impact on measurement for different associations with “left”. The same problem exists of course exists for “right” which is the other end of the scale.
This might also hold for politically interest or sophistication. More interested and sophisticated individuals are assumed to be aware of the multidimensionality of the policy space and the complexity of political concepts. Therefore they might refer to abstract ideologies rather than to specific policy fields or the position of a specific party. It is also likely that partisans of different parties have different associations with “left” and “right” which might affect the relationship between partisanship and ideology as measured through left-right self-placement (see Inglehart and Klingemann 1976). For instance, supporters of left-wing parties might have more positive associations with “left” than supporters of right-wing parties.
As we will see below a large percentage of respondents did not give any associations with “left” and “right” when being probed in our study.
In terms of overall validity it is clear that a survey question should match a researcher’s conceptual definition (Sturgis and Smith 2010, p. 89). The problems of interpersonal incomparability discussed here may, however, render a seemingly valid measure of a scientific concept invalid.
In the first sample stage municipalities (Gemeinden) in Western Germany and municipalities in Eastern Germany were selected with a probability proportional to their number of adult residents; in the second sample stage individual persons were selected at random from the municipal registers of residents. Targeted individuals who did not have adequate knowledge of German to conduct the interview were treated as systematic unit non-responses. The method of data collection were personal interviews with standardized questionnaire (CAPI—computer assisted personal interviewing) [see http://www.gesis.org/en/allbus/study-profiles/2008/ (12/28/2015)].
A different approach would be to query their associations before letting them locate themselves on the left-right scale. However, here we want to investigate how respondents deal with the left-right scale, and thus we consider this question order to be more adequate for that purpose. Another technique to reveal associations during the answering process could be the think aloud technique. Note, however, that this method comes with certain weaknesses suggested by Tourangeau et al. (2000, pp. 44–45).
In additional analyses (not reported here) we investigated the missings (don’t know, no information) for a set of more concrete closed-ended policy questions in the same survey. The number of missings seems to increase with the difficulty of the question. Morever, don’t knows regarding associations with left/right seem to predict those missings, albeit the effects are not very strong.
As we demonstrate in Figs. 15 and 16 in the Appendix, these results also hold even if we do not rely on this topic modeling technique and use instead simple word counts. This analysis shows that individuals located on different extremes of the left-right scale associate very different words to describe these two concepts. To facilitate reading these figures, in Table 4 in the Appendix we provide the top ten words on each extreme of the x-axis that were mentioned 5 or more times.
Preliminary analyses (without controls) not reported here but included in the replication files seem to indicate that associations with left and right are also linked to party left-right placements.
Importantly, however, our results also matter for questions that ask respondents’ to locate others (such as parties) on the left-right scale.
References
Aldrich, J. H., & McKelvey, R. D. (1977). A method of scaling with applications to the 1968 and 1972 presidential elections. American Political Science Review, 71(1), 111–130.
Alwin, D. F., & Krosnick, J. A. (1991). The reliability of survey attitude measurement the influence of question and respondent attributes. Sociological Methods & Research, 20(1), 139–181.
Bafumi, J., & Herron, M. C. (2010). Leapfrog representation and extremism: A study of American voters and their members in congress. American Political Science Review, 104(3), 519–542.
Bakker, R., Edwards, E., Jolly, S., Polk, J., Rovny, J., & Steenbergen, M. (2014a). Anchoring the experts: Using vignettes to compare party ideology across countries. Research & Politics, 1(3), 2053168014553502.
Bakker, R., Jolly, S., Polk, J., & Poole, K. (2014b). The European common space: Extending the use of anchoring vignettes. The Journal of Politics, 76(4), 1089–1101.
Barberá, P. (2015). Birds of the same feather tweet together: Bayesian ideal point estimation using twitter data. Political Analysis, 23(1), 76–91.
Bauer-Kaase, P. (2001). Politische ideologie im wandel? eine lngsschnittanalyse der inhalte der poitischen richtungsbegriffe “links” und “rechts”. In H. D. Klingemann & M. Kaase (Eds.), Wahlen und Wahler Analysen aus Anlass der Bundestagswahl 1998 (pp. 207–243). Wiesbaden: Westdeutscher Verlag.
Belson, W. A. (1981). The design and understanding of survey questions. Aldershot: Gower.
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. The Journal of Machine Learning Research, 3, 993–1022.
Bonica, A. (2013). Ideology and interests in the political marketplace. American Journal of Political Science, 57(2), 294–311.
Brady, H. E. (1985). The perils of survey research: Inter-personally incomparable responses. Political Methodology, 11(3/4), 269–291.
Bratton, M. (2010). Anchoring the ’d-word’ in comparative survey research. Journal of Democracy, 21(4), 106–113.
Bratton, M., Mattes, R. B., & Gyimah-Boadi, E. (2004). Public opinion, democracy, and market reform in Africa. Cambridge: Cambridge University Press.
Braun, M., Behr, D., & Kaczmirek, L. (2013). Assessing cross-national equivalence of measures of xenophobia: Evidence from probing in web surveys. International Journal of Public Opinion Research, 25(3), 383–395.
Campbell, A., Converse, P. E., Miller, W. E., & Donald, E. (1960). The American voter. New York: Wiley.
Canache, D., Mondak, J. J., & Seligson, M. A. (2001). Meaning and measurement in cross-national research on satisfaction with democracy. Public Opinion Quarterly, 65(4), 506–528.
Conover, P. J., & Feldman, S. (1981). The origins and meaning of liberal/conservative self-identifications. American Journal of Political Science, 25(4), 617–645.
Converse, P. E. (1964). The nature of belief system in mass publics. In D. E. Apter (Ed.), Ideology and discontent (pp. 206–261). New York: The Free Press.
Corbetta, P., Cavazza, N., & Roccato, M. (2009). Between ideology and social representations: Four theses plus (a new) one on the relevance and the meaning of the political left and right. European Journal of Political Research, 48(5), 622–641.
Dalton, R. J., Sin, D. C., & Jou, W. (2007). Understanding democracy: Data from unlikely places. Journal of Democracy, 18(4), 142–156.
Downs, A. (1957). An economic theory of democracy. New York: Harper & Row.
Dunn, K. (2011). Left-right identification and education in Europe: A contingent relationship. Comparative European Politics, 9(3), 292–316.
Eisenstein, J., Ahmed, A., & Xing, E. P. (2011). Sparse additive generative models of text. In Proceedings of the 28th international conference on machine learning (ICML-11) (pp. 1041–1048).
Ekehammar, B., Nilsson, I., & Sidanius, J. (1987). Education and ideology: Basic aspects of education related to adolescents’ sociopolitical attitudes. Political Psychology, 8(3), 395–410.
Everett, J. A. (2013). The 12 item social and economic conservatism scale (secs). PloS One, 8(12), e82131.
Fitzgerald, J. (2013). What does ’political’ mean to you? Political Behavior, 35(3), 453–479.
Fitzgerald, R., Widdop, S., Gray, M., & Collins, D. (2011). Identifying sources of error in cross-national questionnaires: Application of an error source typology to cognitive interview data. Journal of Official Statistics, 27(4), 569–599.
Freire, A. (2006). Bringing social identities back in: The social anchors of left-right orientation in western Europe. International Political Science Review, 27(4), 359–378.
Freire, A., & Belchior, A. M. (2011). What left and right means to Portuguese citizens. Comparative European Politics, 9(2), 145–167.
Fuchs, D., & Klingemann, H. D. (1989). Das links-rechts-schema als politischer code. ein interkultureller vergleich auf inhaltsanalytischer grundlage. In M. Haller, H. J. Hoffmann-Nowotny, & W. Zapf (Eds.), Kultur und Gesellschaft: Verhandlungen des 24. Deutschen Soziologentags, des 11. Österreichischen Soziologentags und des 8. Kongresses der Schweizerischen Gesellschaft für Soziologie in Zürich 1988 (pp. 484–498). Frankfurt am Main: Campus Verlag.
Fuchs, D., & Klingemann, H. D. (1990). The left-right schema. In M. K. Jennings & J. W. van Deth (Eds.), Continuities in political action: A longitudinal study of political orientations in three western democracies (pp. 203–234). Berlin: de Gruyter.
Fuhse, J. A. (2004). Links oder rechts oder ganz woanders? zur konstruktion der politischen landschaft. Österreichische Zeitschrift für Politikwissenschaft, 33(2), 209–226.
Gabennesch, H. (1972). Authoritarianism as world view. American Journal of Sociology, 77(5), 857–875.
Gerber, A. S., Huber, G. A., Doherty, D., Dowling, C. M., & Ha, S. E. (2010). Personality and political attitudes: Relationships across issue domains and political contexts. American Political Science Review, 104(1), 111–133.
GESIS—Leibniz Institute for the Social Sciences. (2011). ALLBUS/GGSS 2008 (Allgemeine Bevölkerungsumfrage der Sozialwissenschaften/German general social survey 2008). GESIS data archive, Cologne. ZA4600 data file version 2.0.
GESIS—Leibniz Institute for the Social Sciences. (2009). ALLBUS/GGSS 2008 (Allgemeine Bevölkerungsumfrage der Sozialwissenschaften/German General social survey 2008): Open answers to the questions on associations with the terms “left” and “right”. GESIS data archive, Cologne. ZA4605 data file version 1.0. doi:10.4232/1.11485.
Groves, R. M. (1989). Survey errors and survey costs. New York: Wiley.
Hare, C., Armstrong, D. A., Bakker, R., Carroll, R., & Poole, K. T. (2015). Using Bayesian Aldrich-McKelvey scaling to study citizens’ ideological preferences and perceptions. American Journal of Political Science, 59(3), 759–774.
Hochschild, J. L. (1986). What’s fair: American beliefs about distributive justice. Cambridge: Harvard University Press.
Hotelling, H. (1929). Stability in competition. The Economic Journal, 39(153), 41–57.
Huber, J. D. (1989). Values and partisanship in left-right orientations: Measuring ideology. European Journal of Political Research, 17(5), 599–621.
Inglehart, R., & Klingemann, H. D. (1976). Party identification, ideological preference and left-right dimension among western mass publics. In I. Budge, I. Crewe, & D. J. Farlie (Eds.), Party identification and beyond: Representations of voting and party competition (pp. 243–273). London: Wiley.
Jaccard, J., & Jacoby, J. (2010). Theory construction and model-building skills: A practical guide for social scientists. New York: Guilford Press.
Jahn, D. (2011). Conceptualizing left and right in comparative politics towards a deductive approach. Party Politics, 17(6), 745–765.
Jessee, S. A. (2009). Spatial voting in the 2004 presidential election. American Political Science Review, 103(01), 59–81.
Kaiser, F. M., & Lilly, J. R. (1975). Political attitudes among students: A small college experience. Adolescence, 10(38), 287–295.
King, G., & Wand, J. (2007). Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes. Political Analysis, 15(1), 46–66.
King, G., Murray, C. J., Salomon, J. A., & Tandon, A. (2004a). Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review, 98(1), 191–207.
King, G., Murray, C. J. L., Salomon, J. A., & Tandon, A. (2004b). Enhancing the validity and cross-cultural comparability of measurement in survey research. The American Political Science Review, 98(1), 191–207.
Kitschelt, H. (2004). Diversification and reconfiguration of party systems in postindustrial democracies. Bonn: Internationale Politikanalyse, Friedrich-Ebert-Stiftung.
Kitschelt, H., & Hellemans, S. (1990). The left-right semantics and the new politics cleavage. Comparative Political Studies, 23(2), 210–238.
Klingemann, H. D. (1972). Testing the left-right continuum on a sample of German voters. Comparative Political Studies, 5(1), 93–106.
Klingemann, H. D. (1979). Measuring ideological conceptualizations. In S. H. Barnes, M. Kaase, K. R. Allerback, H. D. Klingemann, A. Marsh, & L. Rosenmayr (Eds.), Political action: Mass participation in five western democracies (pp. 215–254). Beverly Hills, CA: Sage.
Klingemann, H. D., & Wright, W. E. (1973). Dimensions of political belief systems: Levels of conceptualization as a variable. Some results for USA and FRG 1968/69. Comparative Political Studies, 5, 93–106.
Knutsen, O. (1995). Value orientations, political conflicts and left-right identification: A comparative study. European Journal of Political Research, 28(1), 63–93.
Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences USA, 110(15), 5802–5805.
Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236.
Leonisio, R., & Strijbis, O. (2014). Beyond self-placement: Why nationalism is a better predictor of electoral behaviour in the Basque Country. Revista Española de Investigaciones Sociológicas, 146, 47–68.
McClintock, C. G., & Turner, H. A. (1962). The impact of college upon political knowledge, participation, and values. Human Relations, 15(2), 163–176.
Meisenberg, G., & Williams, A. (2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44(7), 1539–1550.
Morton, R., Tyran, J. R., & Wengström, E. (2011). Income and ideology: How personality traits, cognitive abilities, and education shape political attitudes. University of Copenhagen Department of Economics Discussion Paper.
Neundorf, A. (2009). Growing up on different sides of the wall-a quasi-experimental test: Applying the left-right dimension to the German mass public. German Politics, 18(2), 201–225.
Neundorf, A. (2011). Die links-rechts-dimension auf dem prüfstand: Ideologisches wählen in ost-und westdeutschland 1990–2008. Politische Vierteljahresschrift, 45, 227–250.
Piurko, Y., Schwartz, S. H., & Davidov, E. (2011). Basic personal values and the meaning of left-right political orientations in 20 countries. Political Psychology, 32(4), 537–561.
Poole, K. T. (1998). Recovering a basic space from a set of issue scales. American Journal of Political Science, 42(3), 954–993.
Poole, K. T., & Rosenthal, H. (2007). Ideology and congress. Piscataway, NJ: Transaction Publishers.
Raschke, J. (1998). Die erfindung von links/rechts als politisches richtungsschema. In M. T. Greven, H. Münkler, & R. Schmalz-Bruns (Eds.), Bürgersinn und Kritik (pp. 185–206). Baden-Baden: Nomos.
Roberts, M. E., Stewart, B. M., & Tingley, D. (2014). stm: R package for structural topic models. R package version 0.6.1.
Roberts, M. E., Stewart, B. M., Tingley, D., Lucas, C., Leder-Luis, J., Gadarian, S. K., et al. (2014b). Structural topic models for open-ended survey responses. American Journal of Political Science, 58, 1064–1082.
Rosenberg, M. (1956). Misanthropy and political ideology. American Sociological Review, 21(6), 690–695.
Rudi, T. (2010). Die links-rechts-dimension in mittel-und osteuropa: “super-issue” oder bedeutungslos? In T. Faas, K. Arzheimer & R. Siegried (Eds.), (pp. 169–189). VS Verlag für Sozialwissenschaften: Information-Wahrnehmung-Emotion.
Schmitt, H., & van der Eijk, C. (2009). On the changing and variable meaning of left and right. In Working paper.
Scholz, E., & Züll, C. (2012). Item non-response in open-ended questions: Who does not answer on the meaning of left and right? Social Science Research, 41(6), 1415–1428.
Stokes, D. E. (1963). Spatial models of party competition. The American Political Science Review, 57(2), 368–377.
Sturgis, P., & Smith, P. (2010). Assessing the validity of generalized trust questions: What kind of trust are we measuring? International Journal of Public Opinion Research, 22(1), 74–92.
Taddy, M. (2013). Multinomial inverse regression for text analysis. Journal of the American Statistical Association, 108(503), 755–770.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
Uslaner, E. M. (2002). The moral foundations of trust. Cambridge: Cambridge University Press.
Vries, C. E. D., Hakhverdian, A., & Lancee, B. (2013). The dynamics of voters’ left/right identification: The role of economic and cultural attitudes. Political Science Research and Methods, 1(2), 223–238.
Weber, W. (2011). Testing for measurement equivalence of individuals’ left-right orientation. Survey Research Methods, 5(1), 1–10.
Weil, F. D. (1985). The variable effects of education on liberal attitudes: A comparative-historical analysis of anti-semitism using public opinion survey data. American Sociological Review, 50, 458–474.
Zaller, J. R. (1992). The nature and origins of mass opinion. Cambridge: Cambridge University Press.
Zechmeister, E. (2006). What’s left and who’s right? A q-method study of individual and contextual influences on the meaning of ideological labels. Political Behavior, 28(2), 151–173.
Zechmeister, E., & Corral, M. (2010). The varying economic meaning of ’left’ and ’right’ in Latin America. AmericasBarometer Insights Series, 38, 1–10.
Züll, C., & Scholz, E. (2012). Assoziationen mit den politischen Richtungsbegriff en “links” und “rechts” im internationalen Vergleich: Kategorienschema für die Codierung offener Angaben. GESIS.
Züll, C., Scholz, E., & Schmitt, H. (2010). Kategorienschema für die Vercodung von Assoziationen mit den politischen Richtungsbegriffen“ links” und“ rechts”. GESIS.
Acknowledgments
We thank the participants of the political behavior colloquium at the European University Institute, the participants of the EPSA 2014 Panel ‘Big Data and Political Psychology’, the participants of the EQMC and ESA RN21 conference 2014 and in particular Geoffrey Evans, Neal Beck, Molly Roberts and Matthias Fatke as well as the three anonymous reviewers and David Peterson for valuable comments and suggestions. Reproduction files:10.7910/DVN/ERNXOP. Data available from: http://www.gesis.org/.
Author information
Authors and Affiliations
Corresponding author
Appendix
Appendix
History and Usage of the Left-right Scale
The two concepts, “left” and “right”, are used as description of the political space measured by a scale contrasting liberal or progressive with conservative political positions. They originate from the seating arrangement in the French Parliament (Fuhse 2004; Raschke 1998). Right after the French Revolution the Members of Parliament started to sit next to each other according to their ideological position: the conservatives sat on the right side, the progressives sat on the left side. This is how we began to associate these two simple adjectives of spatial positions and directions with political ideologies. From the very start, distinguishing between “left” and “right” has thus been a means to reduce the complexity of the political space, “which serves primarily to provide an orientation function for individuals and a communications function for the political system” (Fuchs and Klingemann 1990, p. 205).
The first step towards measuring ideologies on a one-dimensional scale was made by the economist Hotelling (1929) who analyzed effects of the distance between the relevant market actors on the market price of a good. Taking up this concept of a spatial market, Downs (1957) developed the idea of a one-dimensional political market in which the whole spectrum of political preferences is “[...] ordered from left to right in a manner agreed upon by all voters” (Downs 1957, p. 115). His political spatial market ranged from 0 to 100, covering the degree to which percentage the government should intervene in economic affairs which made his model the first to be based on a liberal-conservative scale, ranging from left to right. The liberal-conservative scale is the Anglo-American counterpart to the Western European left-right scale and they are theoretically very similar (and practically often treated as the same) (see Fuchs and Klingemann 1990, p. 204; Huber 1989, p. 601; Inglehart and Klingemann 1976, p. 244; Neundorf 2011, p. 233; Poole and Rosenthal 2007; Stokes 1963, p. 368).
In current social science research the left-right scale is widely-used to measure respondents’ ideology as well as to position political actors and parties.Footnote 15 The response scales used in these publications differ widely. While some use three- to eleven-point scales, others forgo a neutral middle point and apply scales with an even number of scale points. Even though the vast majority of these articles trusts in the explanatory power of the scale, we are, of course, not the first social scientists to be suspicious of the left-right scale. There are a number of studies mainly focusing on the variance of interpretations of this scale (Bauer-Kaase 2001; Conover and Feldman 1981; Corbetta et al. 2009; Freire 2006; Freire and Belchior 2011; Fuchs and Klingemann 1990; Inglehart and Klingemann 1976; Jahn 2011; Klingemann 1972, 1979; Knutsen 1995; Leonisio and Strijbis 2014; Neundorf 2009, 2011; Piurko et al. 2011; Rudi 2010; Schmitt and van der Eijk 2009; Vries et al. 2013; Zechmeister 2006). With the exception of Corbetta et al. (2009) and—to some extent—Rudi (2010), all of them report difficulties with the scale. Schmitt and van der Eijk (2009), Jahn (2011) and Vries et al. (2013) for example show that the issue preferences or policy orientations associated with “left” and “right” differ across countries and within countries over time. Differences in the party polarization might be one explanation for different interpretations across countries (Freire 2006). Regarding differences within a country, Freire and Belchior (2011) find that the interpretations of Portuguese citizens concerning “left” and “right” lack clarity and structure. Zechmeister (2006) comes to the same conclusion for Mexico and Argentina. Regarding Germany, Neundorf (2009, 2011) concludes that there has been a considerable increase in the diversity of understandings of “left” and “right” over time that is the concepts lost clarity and became more and more vague. The study of Weber (2011) uses another approach by assessing the measurement equivalence using two different wordings of the question within one survey. According to her results, group means of self-placement on the scale are comparable among different countries, while relationships to other variables are not. We build our study on these former analyses dealing with potential problems of the left-right scale.
Validation Using the Dictionary by Züll et al. (2010)
We relied on topic models to analyze respondents’ answers. A different approach would have consisted in employing the dictionary (based on manual coding) developed by Züll et al. (2010) to automatically code respondents’ answers into different categories based on the their answers. This dictionary currently comprises a total of 7814 phrases, full words or parts of words and can be applied to any raw text data containing associations with “left” and “right”. It was developed with the aim of allowing cross-time and cross-country comparisons of interpretations of the left-right scale. Their general scheme draws on earlier similar work by Fuchs and Klingemann (1989, 1990) and Bauer-Kaase (2001) and comprises eight broad categories into which respondents’ answers can be coded: Ideologies, general social values, specific social values, social change (comprising forms, characteristics and means of social change), social groups, political actors, concrete aspects and affective evaluations (Züll and Scholz 2012, pp. 7–16). However, prior to coding answers into these eight categories, answers are coded into the 270 categories that are derived from an empirical “atheoretical” coding stage. In other words respondents’ answers are coded into about 270 subcategories into which answers or parts of respondents’ answers are sorted (see Züll and Scholz 2012, pp. 7–16 for the subcategories).
Any categorizing of open responses into fewer dimensions be it manually or automatically lumps together respondents. Generally, the fewer the lumping categories the higher the variance within the categories. As a consequence, groups are blurred, as is their distinctiveness and as a consequence there impact of their distinctiveness on e.g. left-right self-placement. While we prefer a model driven approach that avoids human error, we want to ensure that the general conclusions of our empirical analysis are not largely due to our approach of categorizing data with the topic models. Therefore we carry out additional analyses using the dictionary. In particular, besides using the topic model we also analyzed respondents’ answers after they have been coded into the 270 different categories included in the left-right dictionary devised by Züll and Scholz (2012).
Figure 10 displays absolute numbers of respondents in the most common categories for both “left” and “right”. Most respondents associate “left” with either values (solidarity, justice), ideologies (communism, socialism), parties (left party, SPD) or some political figure (politicians). The picture for “right” is similar with many people mentioning ideologies or values (national socialism, right wing radicalism, conservatism, patriotism), parties (NPD, DVU, republicans) but then also descriptions such as xenophobic or radical. Importantly, Fig. 10 only displays the 10 most frequent categories.
Most common categories of associations as coded with the dictionary
In addition we compare means for respondents whose associations belong into one of the most frequent categories. Figure 11 displays the left-right scale means of those respondents whose answers were in the 11 most mentioned subcategories for “left”. Figure 12 is the same but for “right”. The sample mean is indicated by the dashed line. Just as for our previous categorization into 4 topics throughout the study we can see here that left-right measurement values differ for groups of respondents whose answers have been coded into categories using the dictionary. Groups that associate “left” with values such as equality, justice or solidarity display measurement values that lean to the left. In contrast, groups that associate “left” with real socialism, with radicals or with communism display measurement values that lean to the right. Groups that associated “right” with national socialism, xenophobia or violence display measurement values that lean to the left. Importantly, these associations are highly consistent with our findings when we employ the four categories discovered by our topic model. In sum, these additional analyses seem to confirm our main conclusions, namely that there is considerable variation in the associations and that this variation may impact measurement values (Figs. 13, 14, 15, 16).
Left-right scale means for 11 most common categories of associations with “left” (dashed line = sample mean)
Left-right scale means for 11 most common categories of associations with “right” (dashed line = sample mean)
Question wording in Allbus 2008 (see Scholz and Züll 2012, p. 1420)
Words that are associated with “left” and “right” (size \(\propto\) wordcount)
Words that are associated with “left” by individuals of different self-reported left-right position (size \(\propto\) wordcount)
Words that are associated with “right” by individuals of different self-reported left-right position (size \(\propto\) wordcount)
Rights and permissions
About this article
Cite this article
Bauer, P.C., Barberá, P., Ackermann, K. et al. Is the Left-Right Scale a Valid Measure of Ideology?. Polit Behav 39, 553–583 (2017). https://doi.org/10.1007/s11109-016-9368-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11109-016-9368-2