Skip to main content
SpringerLink
Log in

Measuring populist discourse with semantic text analysis: an application on grassroots populist mobilization

  • Published:
Quality & Quantity Aims and scope Submit manuscript

Abstract

Populism is a concept employed to qualify the political behavior of a large number of actors at a worldwide scale, with scientists classifying the latter into populists and non-populists according to dimensions such as ideology, strategy, discourse, economic policy, and even style. This article analyzes existing schools of thought on the nature of populism and argues that conceptualizing populism as a specific type of anti-elite discourse in the name of the People is both conceptually and methodologically the most coherent and useful way to understand the phenomenon. Additionally, it suggests discarding crude, dichotomous classification in favor of a gradated view of populist mobilization by means of quantifying populist discourse and observing its spatial and temporal variation. It adds value to current methods of measurement by demonstrating why and how clause-based semantic text analysis can provide optimal quantitative results while retaining qualitative elements for mixed-methods analysis. Aiming, moreover, at expanding the scope of populism studies by overcoming a narrow view that focuses exclusively at party system developments, it applies semantic text analysis to the study of grassroots mobilization during the Great Recession. Results point to the wide use of populist discourse on the part of movement activists seeking an inclusive language when framing disparate social grievances in a given constituency, a finding with important implications with regards to how populism can facilitate straddling the divide that purportedly distinguishes institutionalized party system behavior from the social movement milieu.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The list is non-exhaustive.

  2. Müller (2016), for instance, argues that the US People’s Party in the 1890s was not populist.

  3. For example, Mair (2002) discusses Tony Blair as a populist leader in the UK.

  4. For an overview of the academic development of the term, see Moffitt (2016), Chap. 2.

  5. Harrison and Bruter (2011) worked along similar lines in their analysis of the European extreme right.

  6. Pauwels (2011) includes absurd, admit, politic*, propaganda, ruling*, tradition*, and truth; (Harrison and Bruter 2011) include nonsense, shambles, chaos, contempt, favouritism, blind, innovation, and regret; Oliver and Rahn (2016) include politician(s), IRS, donors, and CEOs. The validity of such spurious choices is evidently contested.

  7. The numbers reflect both single words and n-grams employed by the authors. Words can be stemmed or remain whole. Oliver and Rahn (2016) only provide examples from their dictionaries rather than their whole content.

  8. Rooduijn and Pauwels (2011) had to include ‘context-specific words’ in their cross-national study, since many idiosyncratic political expressions fail to transcend linguistic barriers. The significance of the word regenten for Flemish and Dutch populism, or katestimeno for Greek populism are telling illustrations. In terms of proportion, the Dutch dictionary in Rooduijn and Pauwels (2011) contains fourteen ‘core’ words and seven ‘context-specific’ words, proving that contextual impact is far from negligible when analyzing text (Krippendorff 2004).

  9. Bonikowski and Gidron (2016) read 40.1% of their total dataset, hand-coding excerpts from 890 speeches, and manually assessing all positive instances identified in a first iteration of dictionary analysis. They report a Cohen’s kappa of 0.70 for intercoder reliability when performing manual coding (footnotes 5–6).

  10. A higher range would require splitting text into syllables or letters (nonsensical in this context).

  11. Their rationale is using this evaluation as a boolean variable for statistical analysis.

  12. Bernhard et al.’s (2015) coding scheme draws from March (2012) with the overall score for each sample unit resulting from the combination of scores in three different dimension.

  13. Hawkins (2009) attributes this moderate performance on the large number of coders, the small set of speeches, and the limited experience of coders in reading political texts.

  14. For a critique against using Cohen’s kappa for content analysis and a way to calculate the reliability of ordinal data, see Hayes and Krippendorff (2007).

  15. Personal communication with authors.

  16. The ‘people index’ incorporates references to any ‘group[s] of people having explicit constant features in common’ (Reungoat 2010: 311).

  17. Reliability cannot be assessed since coding was performed exclusively by the first author (personal communication with first author).

  18. The method is also applied to newspaper articles in Rooduijn (2014), with improved reliability scores. In the same paper, scores given to ‘introductory paragraphs’ are doubled and paragraphs in long manifestos are weighted differently than those in shorter ones, thus employing an interesting non-linear scoring scheme.

  19. Apart from the option of manually distilling semantic triplets, software packages exist (e.g. IDEA, KEDS) to parse text for syntactical elements. With improved tools, we can expect increased reliability and even lower processing costs in the future. See discussion in Franzosi (2010: 62–63).

  20. To compensate for any potential loss in contextualization, coders in CBSTA projects are instructed to draw judgment from larger context units (Krippendorff 2004), or even the entire sample unit, prior to appending codes to individual semantic triplets.

  21. The dataset was coded by two individuals (one of which one was the first author) with no actual overlap in content, and therefore, no reliability test was performed (personal communication with first author).

  22. The J14 movement in Israel did not produce a manifesto and therefore is not part of the case selection.

  23. Preparation of each manifesto took an average of 2 h.

  24. The researcher was not part of the coding team.

  25. The software package was used to streamline the coding process and avoid typing or other unnecessary coding mistakes. Any spreadsheet or word processing application can be used with equal merit.

  26. The use of decision schemes is another significant advantage of this method, enhancing reliability by organizing the coding process along a series of dichotomous decisions, increasing coding speed and minimizing errors owed to coder fatigue (Krippendorff 2004).

  27. The moderate intercoder agreement in identifying the presence of an actor in the clause (NAC) is largely due to disagreement over the eligibility of vague notions such as ‘society’ or ‘humanity’. Further training can disambiguate such observations and lead to highly increased intercoder scores altogether.

  28. The value of factor c, indicating weight, is of course a matter of perspective.

  29. Residual categories receive no score.

  30. The unusually rounded score for the manifesto of the benchmark case was a matter of coincidence.

  31. For instance, a text with 10% full populist frames, 15% people-centrism, and 15% anti-elitism yields PDI = 0.20. A text with no full populist frames would require a combined 60% of partial populist frames to pass this threshold, a rather unlikely scenario for non-populist discourse. Hence, the possibility of a false positive is very slim.

References

  • Albertazzi, D., McDonnell, D.: Introduction: the sceptre and the spectre. In: Albertazzi, D., McDonnell, D. (eds.) Twenty-First Century Populism: The Spectre of Western European Democracy, pp. 1–14. Palgrave Macmillan, Basingstoke (2008)

    Chapter  Google Scholar 

  • Armony, A.C., Armony, V.: Indictments, myths, and citizen mobilization in argentina: a discourse analysis. Lat. Am. Polit. Soc. 47(4), 27–54 (2005)

    Article  Google Scholar 

  • Aslanidis, P.: Is populism an ideology? a refutation and a new perspective. Polit. Stud. 64(IS), 88–104 (2016a)

    Article  Google Scholar 

  • Aslanidis, P.: Populist social movements of the great recession. Mobil. Int. Q. 21 (3), 301–321 (2016b)

    Article  Google Scholar 

  • Bauer, M.W.: Classical content analysis: a review. In: Bauer, M.W., Gaskell, G. (eds.) Qualitative Researching with Text, Image and Sound, pp. 132–152. Sage, London (2000)

    Chapter  Google Scholar 

  • Bell, D.: What is liberalism? Polit. Theory 42(6), 682–715 (2014)

    Article  Google Scholar 

  • Bernard, H.R., Ryan, G.W.: Analyzing Qualitative Data: Systematic Approaches. Sage, Thousand Oaks (2010)

    Google Scholar 

  • Bernhard, L., Kriesi, H., Weber, E.: The populist discourse of the swiss people’s party. In: Kriesi, H., Pappas, T.S. (eds.) European Populism in the Shadow of the Great Recession, pp. 123–138. ECPR Press, Colchester (2015)

    Google Scholar 

  • Bonikowski, B., Gidron, N.: The populist style in American politics: presidential Campaign discourse, 1952–1996. Soc. Forces 94(4), 1593–1621 (2016)

    Article  Google Scholar 

  • Bruhn, K.: “To hell with your corrupt institutions!”: AMLO and populism in Mexico. In: Mudde, C., Kaltwasser, C.R. (eds.) Populism in Europe and the Americas: Threat or Corrective for Democracy?, pp. 88–112. Cambridge University Press, Cambridge (2012)

    Chapter  Google Scholar 

  • Budge, I., Klingemann, H.-D., Volkens, A., Bara, J., Tanenbaum, E.: Mapping Policy Preferences: Estimates for Parties, Electors, and Governments 1945–1998. Oxford University Press, Oxford (2001)

    Google Scholar 

  • Caiani, M., della Porta, D.: The elitist populism of the extreme right: a frame analysis of extreme right-wing discourses in Italy and Germany. Acta Polit. 46(2), 180–202 (2011)

    Article  Google Scholar 

  • Calhoun, C.: Occupy wall street in perspective. Br. J. Sociol. 64(1), 26–38 (2013)

    Article  Google Scholar 

  • Canovan, M.: Populism. Harcourt Brace Jovanovich, New York (1981)

    Google Scholar 

  • Canovan, M.: The People. Polity Press, Cambridge (2005)

    Google Scholar 

  • Caramani, D.: Will vs. Reason: The Populist and Technocratic Forms of Political Representation and Their Critique to Party Government. Am. Polit. Sci. Rev. 111(1), 54–67 (2017)

    Article  Google Scholar 

  • Conniff, M.L. (ed.): Populism in Latin America, 2nd edn. The University of Alabama Press, Tuscaloosa (2012)

    Google Scholar 

  • Deegan-Krause, K., Haughton, T.: Toward a more useful conceptualization of populism: types and degrees of populist appeals in the case of Slovakia. Polit. Policy 37(4), 821–841 (2009)

    Article  Google Scholar 

  • de la Torre, C.: Populist Seduction in Latin America: The Ecuadorian Experience. Ohio University Center for International Studies, Athens (2000)

    Google Scholar 

  • della Porta, D.: Social Movements in Times of Austerity. Polity Press, Cambridge (2015)

    Google Scholar 

  • della Porta, D., Mattoni, A. (eds.): Spreading Protest: Social Movements in Times of Crisis. ECPR Press, Colchester (2014)

    Google Scholar 

  • di Tella, T.: Populism into the twenty-first century. Gov. Oppos. 32, 187–200 (1997)

    Article  Google Scholar 

  • Dornbusch, R., Edwards, S. (eds.): The Macroeconomics of Populism in Latin America. The University of Chicago Press, Chicago (1991)

    Google Scholar 

  • Formisano, R.P.: For the People: American Populist Movements from the Revolution to the 1850s. The University of North Carolina Press, Chapel Hill (2008)

    Google Scholar 

  • Formisano, R.P.: The Tea Party: A Brief History. The Johns Hopkins University Press, Baltimore (2012)

    Google Scholar 

  • Franzosi, R.: The return of the actor. Interaction Networks among social actors during periods of high mobilization (Italy, 1919–1922). Mobil. Int. Q. 4(2), 131–149 (1999)

    Google Scholar 

  • Franzosi, R.: From Words to Numbers: Narrative, Data, and Social Science. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  • Franzosi, R.: Content Analysis. Sage, Thousand Oaks (2008)

    Book  Google Scholar 

  • Franzosi, R.: Quantitative Narrative Analysis. Sage, Thousand Oaks (2010)

    Book  Google Scholar 

  • Franzosi, R., De Fazio, G., Vicari, S.: Ways of measuring agency: an application of quantitative narrative analysis to lynchings in georgia (1875–1930). Sociol. Methodol. 42(1), 1–42 (2012)

    Article  Google Scholar 

  • Gallie, W.B.: Essentially contested concepts. Proc. Aristot. Soc. 56, 167–198 (1956)

    Article  Google Scholar 

  • Gamson, W.A.: Arab Spring, Israeli summer, and the process of cognitive liberation. Swiss Polit. Sci. Rev. 17(4), 463–468 (2011)

    Article  Google Scholar 

  • Gerbaudo, P.: Tweets and the Streets: Social Media and Contemporary Activism. Pluto Press, London (2012)

    Google Scholar 

  • Gerring, J.: Ideology: a definitional analysis. Polit. Res. Q. 50(4), 957–994 (1997)

    Article  Google Scholar 

  • Grimmer, J., Stewart, B.M.: Text as data: the promise and pitfalls of automatic content analysis methods for political texts. Polit. Anal. 21(3), 267–297 (2013)

    Article  Google Scholar 

  • Harrison, S., Bruter, M.: Mapping Extreme Right Ideology: An Empirical Geography of the European Extreme Right. Palgrave Macmillan, New York (2011)

    Book  Google Scholar 

  • Hayes, A.F., Krippendorff, K.: Answering the call for a standard reliability measure for coding data. Commun. Methods Meas. 1(1), 77–89 (2007)

    Article  Google Scholar 

  • Hawkins, K.A.: Is Chavez populist?: Measuring populist discourse in comparative perspective. Comp. Polit. Stud. 42(8), 1040–1067 (2009)

    Article  Google Scholar 

  • Hawkins, K.A.: Venezuela’s Chavismo and Populism in Comparative Perspective. Cambridge University Press, Cambridge (2010)

    Book  Google Scholar 

  • Iglesias, P.: Understanding Podemos. New Left Rev. 93(May–June), 7–22 (2015)

    Google Scholar 

  • Ionescu, G., Gellner, E. (eds.): Populism: Its Meanings and National Characteristics. Weidenfeld and Nicolson, London (1969)

    Google Scholar 

  • Jagers, J., Walgrave, S.: Populism as political communication style: an empirical study of political parties’ discourse in Belgium. Eur. J. Polit. Res. 46(3), 319–345 (2007)

    Article  Google Scholar 

  • Jansen, R.S.: Populist mobilization: a new theoretical approach to populism. Sociol. Theory 29(2), 75–96 (2011)

    Article  Google Scholar 

  • Johnston, H., Alimi, E.Y.: A methodology analyzing for frame dynamics: the grammar of keying battles in palestinian nationalism. Mobil. Int. Q. 18(4), 453–474 (2013)

    Google Scholar 

  • Johnston, H., Oliver, P.E.: Breaking the frame. Mobil. Int. Q. 5(1), 61–64 (2000)

    Google Scholar 

  • Kazin, M.: The Populist Persuasion: An American History. Cornell University Press, Ithaca (1995)

    Google Scholar 

  • Kriesi, H., Pappas, T.S. (eds.): European Populism in the Shadow of the Great Recession. ECPR Press, Colchester (2015)

    Google Scholar 

  • Krippendorff, K.: Content Analysis: An Introduction to Its Methodology, 2nd edn. Sage, Thousand Oaks (2004)

    Google Scholar 

  • Kumar, A.: The Making of a Small State: Populist Social Mobilisation and the Hindi Press in the Uttarakhand Movement. Orient Blackswan, New Delhi (2011)

    Google Scholar 

  • Laclau, E.: Politics and Ideology in Marxist Theory: Capitalism–Fascism–Populism. LNB, London (1977)

    Google Scholar 

  • Laclau, E.: On Populist Reason. Verso, London (2005)

    Google Scholar 

  • Mair, P.: Populist democracy vs party democracy. In: Meny, Y., Surel, Y. (eds.) Democracies and the Populist Challenge, pp. 81–98. Palgrave Macmillan, Basingstoke (2002)

    Chapter  Google Scholar 

  • March, L.: Towards an understanding of contemporary left-wing populism. In: Political Studies Association (PSA) Annual International Conference, Belfast, 3–5 April 2012

  • Moffitt, B.: The Global Rise of Populism: Performance, Political Style, and Representation. Stanford University Press, Stanford (2016)

    Book  Google Scholar 

  • Moffitt, B., Tormey, S.: Rethinking populism: politics, mediatisation and political style. Polit. Stud. 62(2), 381–397 (2014)

    Article  Google Scholar 

  • Mudde, C.: The populist zeitgeist. Gov. Oppos. 39(4), 542–563 (2004)

    Article  Google Scholar 

  • Mudde, C.: Populist Radical Right Parties in Europe. Cambridge University Press, Cambridge (2007)

    Book  Google Scholar 

  • Müller, J.-W.: What Is Populism?. University of Pennsylvania Press, Philadelphia (2016)

    Book  Google Scholar 

  • Munck, G.L., Verkuilen, J.: Conceptualizing and measuring democracy: evaluating alternative indices. Comp. Polit. Stud. 35(1), 5–34 (2002)

    Google Scholar 

  • Neuendorf, K.A.: The Content Analysis Guidebook. Sage, Thousand Oaks (2002)

    Google Scholar 

  • Oliver, J.E., Rahn, W.M.: Rise of the Trumpenvolk: populism in the 2016 election. Ann. Am. Acad. Polit. Soc. Sci. 667(1), 189–206 (2016)

    Article  Google Scholar 

  • Panizza, F.: Introduction: populism and the mirror of democracy. In: Panizza, F. (ed.) Populism and the Mirror of Democracy, pp. 1–31. Verso, London (2005)

    Google Scholar 

  • Pappas, T.S.: The specter haunting Europe: distinguishing liberal democracy’s challengers. J. Democr. 27(4), 22–36 (2016)

    Article  Google Scholar 

  • Pauwels, T.: Measuring populism: a quantitative text analysis of party literature in Belgium. J. Elect. Public Opin. Parties 21(1), 97–119 (2011)

    Article  Google Scholar 

  • Pauwels, T.: Populism in Western Europe: Comparing Belgium, Germany and the Netherlands. Routledge, Oxon (2014)

    Google Scholar 

  • Pfaff, S.: Exit-Voice Dynamics and the Collapse of East Germany: The Crisis of Leninism and the Revolution of 1989. Duke University Press, Durham (2006)

    Book  Google Scholar 

  • Philip, G., Panizza, F.: The Triumph of Politics: The Return of the Left in Venezuela, Bolivia and Ecuador. Polity Press, Cambridge (2011)

    Google Scholar 

  • Popping, R.: Ag09: a computer program for interrater agreement for judgments. Soc. Sci. Comput. Rev. 28(3), 391–396 (2010)

    Article  Google Scholar 

  • Popping, R., Roberts, C.W.: Semantic text analysis and the measurement of ideological developments within fledgling democracies. Soc. Sci. Inf. 54(1), 23–37 (2015)

    Article  Google Scholar 

  • Reungoat, E.: Anti-EU parties and the people: an analysis of populism in french euromanifestos. Perspect. Eur. Polit. Soc. 11(3), 292–312 (2010)

    Article  Google Scholar 

  • Roberts, C.W.: A generic semantic grammar for quantitative text analysis: applications to East and West Berlin radio news content from 1979. Sociol. Methodol. 27(1), 89–129 (1997)

    Article  Google Scholar 

  • Roberts, C.W., Zuell, C., Landmann, J., Wang, Y.: Modality analysis: a semantic grammar for imputations of intentionality in texts. Qual. Quant. 44(2), 239–257 (2010)

    Article  Google Scholar 

  • Roberts, K.M.: Neoliberalism and the transformation of populism in Latin America: the peruvian case. World Polit. 48(1), 82–116 (1995)

    Article  Google Scholar 

  • Rooduijn, M.: The mesmerising message: the diffusion of populism in public debates in Western European media. Polit. Stud. 62(4), 726–744 (2014)

    Article  Google Scholar 

  • Rooduijn, M., Pauwels, T.: Measuring populism: comparing two methods of content analysis. West Eur. Polit. 34(6), 1272–1283 (2011)

    Article  Google Scholar 

  • Rooduijn, M., de Lange, S.L., van der Brug, W.: A populist Zeitgeist? Programmatic contagion by populist parties in Western Europe. Party Polit. 20(4), 563–575 (2014)

    Article  Google Scholar 

  • Ruzza, C., Fella, S.: Populism and the Italian right. Acta Polit. 46(2), 158–179 (2011)

    Article  Google Scholar 

  • Shapiro, G., Markoff, J.: Revolutionary Demands: A Content Analysis of the Cahiers de Doléances of 1789. Stanford University Press, Stanford (1998)

    Google Scholar 

  • Snow, D.A., Benford, R.D.: Ideology, frame resonance, and participant mobilization. Int. Soc. Mov. Res. 1(1), 197–217 (1988)

    Google Scholar 

  • Stenner, K.: The Authoritarian Dynamic. Cambridge University Press, Cambridge (2005)

    Book  Google Scholar 

  • Urbinati, N.: Democracy Disfigured: Opinion, Truth, and the People. Harvard University Press, Cambridge (2014)

    Book  Google Scholar 

  • Van Dyke, N., Meyer, D.S. (eds.): Understanding the Tea Party Movement. Ashgate, Farnham (2014)

    Google Scholar 

  • Vasilopoulou, S., Halikiopoulou, D., Exadaktylos, T.: Greece in crisis: austerity, populism and the politics of blame. J. Common Mark. Stud. 52(2), 388–402 (2014)

    Article  Google Scholar 

  • Walton, J., Seddon, D.: Free Markets & Food Riots: the Politics of Global Adjustment. Blackwell Publishers, Cambridge (1994)

    Book  Google Scholar 

  • Weyland, K.: Clarifying a contested concept: populism in the study of Latin American politics. Comp. Polit. 34(1), 1–22 (2001)

    Article  Google Scholar 

Download references

Acknowledgments

The author would like to thank Nikos Marantzidis and Hanspeter Kriesi for their guidance, the two anonymous reviewers of this journal for their constructive critique, and his team of coders for their patience and perseverance.

Author information

Authors and Affiliations

  1. Department of Political Science and Hellenic Studies Program, Yale University, 34 Hillhouse Avenue, New Haven, CT, 06520-8206, USA

    Paris Aslanidis

Authors
  1. Paris Aslanidis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paris Aslanidis.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aslanidis, P. Measuring populist discourse with semantic text analysis: an application on grassroots populist mobilization. Qual Quant 52, 1241–1263 (2018). https://doi.org/10.1007/s11135-017-0517-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11135-017-0517-4

Keywords

Search

Navigation