The internal fragility of representative democracy: was Schumpeter right?

Abstract

In recent decades, the rapid emergence of new political leaders capturing growing social discontent with populist promises has highlighted elements of the internal fragility of democracy. Schumpeter predicts such fragility in Capitalism, Socialism and Democracy, where, in analyzing thse “competitive struggle for the popular vote,” he makes two assumptions that reverse the classical model of rational decision-making: the limited use of “conscious rationality” by voters and the potential unfairness of political leaders. These two elements introduce systematic distortions into the process of collective decision-making in democracy, suggesting a possible explanation for the rise of populism. Based on delegation, representative democracy must function despite the small amount of intellectual effort that most citizens put into forming their political opinions. Ideologies have historically functioned as implicit heuristics, allowing citizens to evaluate political facts, shape their expectations, and simplify political reasoning; as such, they have been the tools used by parties to give credibility to their programs while at the same time affiliating and polarizing the electorate. While they continue to serve as anchors for defining political identity, today's ideologies are fragmented and therefore less effective in supporting long-term programs and retaining voters; thus, keeping voters close to their political affiliation requires massive use of the media, and for emerging parties lacking strong identitarian values, a short-term political offer becomes less risky than formulating long-term political strategies. At the same time, the decline of classical ideologies makes any political commitment to long-term perspectives hardly credible to the electorate. This leads to an adverse selection process in which populist programs have a better chance of success than long-term policy programs. As a result, the role of the political leader as an entrepreneur is severely weakened. The competition for votes becomes unfair (since it is not based on the quality or plausibility of policy results), while at the same time the process of polarization disrupts the elements of mediation. The result is a process of democratic backsliding that can ultimately lead to a loss of trust in democracy.

1 Introduction – The context and the problem

A preliminary version of this paper has been presented at the Schumpeter Society Conference, Seoul July 2018; The current version has been presented at the Schumpeter Society Conference, Roma July 2021. I am indebted to Richard Langlois, Leonardo Morlino, Richard Nelson, Carlo Trigiglia, and an anonymous referee for their insightful comments.

In recent decades, social dissatisfaction has grown remarkably in several advanced countries, resulting in political polarization. A significant part of the population has moved towards more radical political positions, sometimes to the extreme ends of the left–right axis. The rapid emergence of new political leaders, capturing growing social discontent with populist promises, has highlighted elements of the internal fragility of democracy. The new leaders have achieved consensus through messages that appeal more to emotions than to rationality and, thanks also to the massive use of the media, have brought about an increasing polarization of the electorate.² This change undermines the traditional role of representative democracy as a “mediator” between opposing legitimate interests, and weakens the power (and sometimes the reputation) of the mediators, leading to the prevalence of direct relationships between the new leaders and citizens. In short, in many countries we have seen a transition from Elitist Bureaucracy (Clinton) to Charismatic Authority (Trump).³

This transformation weakens the typical dynamics of a representative democracy, making it more convenient for leaders to compete on the basis of short-term crowd impulses – amplified by emotional elements – rather than long-term perspectives. Building “collective rationality” becomes an increasingly difficult task, and emerging leaders, though able to grasp some of the sources of popular discontent, remain tempted to offer simplistic short-term solutions to increasingly complex problems. In the short term, this process may lead to a formally more “direct” and radical democracy, and sometimes to higher levels of popular participation, but these features may also become the risky preconditions for a shift to a more authoritarian regime.

If this process is sustained, the more the parties move towards extreme positions, the less mediation (negotiation) is possible and the competition between them turns into conflict. The question is whether stabilization will lead to an “illiberal democracy” or whether we are witnessing a transition in which the weakening of democratic features is a temporary effect during the achievement of a more stable configuration of the emerging parties.

Antony Downs’ model of Democracy (Downs 1957b), and the rich literature that followed his article, are the typical references for discussing the threat of polarization to the stability of representative democracies. However, these risks have a deeper origin, namely an intrinsic fragility of democracy as first outlined by Schumpeter (2003) in Capitalism, Socialism, and Democracy (hereafter CSD).

In short, I argue that the fragility highlighted by Schumpeter stems from the imperfections of the implicit contract between citizens and their political representatives (or leaders), a contract that involves trust and delegation, but also leadership, persuasion, and sometimes manipulation. Max Weber’s view of “charismatic democracy” well describes the ambiguity of the role of the political leader, who oscillates between being a “leading strategist” and a “demagogue”. In order to construct shared policy proposals, political leaders need to be in constant dialogue with citizens, to understand people’s expectations and grievances, and to offer convincing solutions. Persuasion is thus essential, and yet persuasion and manipulation remain separated only by the ethic of responsibility, and no one can be sure that a political actor possesses this moral quality.

With the development of the media, the role of information and communication systems becomes increasingly essential for political action but also more prone to manipulation and misuse. On the one hand, it can be a powerful instrument for political subjects who want influence citizens’ opinions through political manipulation and advertising; on the other, it can stimulate a “dialogue” between citizens and their representatives by eliciting critical views and discourse. Schumpeter worried against the manipulative use of communication systems, but pointed out that modern democracy has to be working despite the risks of misuses, hence the origin of the fragility of democracy. The ambiguity of the relation between citizens and their representatives is an essential and integral part of modern democracies, but since nobody can guarantee that political leaders of high profile and moral caliber will emerge from an election, the impact of manipulation may overcome the role of the open critical dialog among the different parties.

Since the 1950s, the pervasive diffusion of televised media worldwide has amplified a party’s ability to increase citizens’ affiliation through political communication and advertising.⁴ The recent rapid growth of “social media” has further promoted the establishment of direct relationships between the new emerging leaders and “the people”, while overcoming the traditional influence of opinion leaders in newspapers and TV channels. While debates in the traditional media should (at least in principle) provide citizens with the necessary elements for critical evaluation of policies, the messages of the new emerging leaders on social media do not appeal to rationality, but rather to approval and confirmation of their views. This process can lead to increased polarization through a self-reinforcing process that weakens the democratic system and makes it more fragile.⁵ Thus, as polarization progresses, it increasingly shrinks the realm of common interests and principles, paving the way for irreconcilable political positions and, at the same time, favoring more direct and unfettered access to political power.

Why does this happen? A starting point for addressing this difficult question is to recognize that in the political context there is no endogenous system for coordinating conflicting interests, as there is in markets through the 'invisible hand'. In particular, there is no reason to believe that the players in the democratic game, as they compete for power, will always pursue the interests of their constituents. Since the Scottish Enlightenment, political philosophers have sought to design democratic institutions in such a way as to prevent participants in the struggle for power from placing their personal advantage above the public interest.

Hume had clearly identified the problem:

“Political writers have established it as a maxim, that, in contriving any system of government, and fixing the several checks and controls of the constitution, every man ought to be supposed a knave, and to have no other end, in all his actions, than private interest. By this interest we must govern him, and, by means of it, make him, notwithstanding his insatiable avarice and ambition, co-operate too public good. Without this, say they, we shall in vain boast of the advantages of any constitution, and shall find, in the end, that we have no security for our liberties or possessions, except the good-will of our rulers; “(Hume 1994, vol. I : 24)

We find a similar vision in the famous metaphor of the “invisible hand”. Adam Smith (2007: 349) claimed that individuals, while aiming only at their own interests, are led by an “invisible hand” to pursue the collective advantage even though this end “is not within their intentions”. In the context of politics, however, competition does not have the same characteristics as Adam Smith’s invisible hand, because there is no internal process that “spontaneously” leads to the alignment of political behavior with common rules.

Hume’s solution to control the “insatiable ambition” of the participant to the political game was division of power, but this solution does not prevent discrepancies and violations, it simply mitigates them. Thus, if we do not attribute to all politicians a high moral caliber, the only element that may induce them to pursue the programs they have promised to their voters is their fear of losing voters’ support. Therefore, the institutional form taken by the competition for votes characterizes the good or bad functioning of the process by which citizens form their political opinions and choose candidates.

In the Olympic representation of the decision-making process, as proposed by classical democratic theory, citizens are supposed to be fully informed and competent. Politicians are supposed to give a clear account of the principles and strategies for the conduct of government and to keep their promises after the election.⁶ But this representation is far from reality; the process of collective decision-making is more complicated, and it may be unfair: politicians, for example, may fail to maintain their promises despite the risk of losing popular support if they perceive that voters are deeply affiliated to their party or polarized to their values.

Thus, affiliation to the ideology of a party and polarization are typical elements that undermine the effectiveness of competition in promoting the accountability of political representatives. This is a clear case of violation of the conditions of fair competition for votes, although there is no striking evidence of violation of fundamental democratic rights. More generally, in order to identify the failures of the political decision-making process, it is necessary to detect the elements that undermine the effectiveness of competition; In fact, according to Schumpeter, competition for votes is the core element of democracy and should therefore ensure the fulfillment of the implicit contract between the electorate and the elected:

“The democratic method is that institutional arrangement for arriving at political decisions in which individuals acquire the power to decide by means of a competitive struggle for the people’s vote” (CSD: 269)

After a long period of success, this definition was challenged by many critics who considered it incomplete because it did not (explicitly) refer to the universal principles of democracy.⁷ I do not share this critical view, because the definition offered by Schumpeter focuses on a central problem of democracy (already emphasized by David Hume): while there is no “invisible hand” in the political arena, the main question is what allows individuals with conflicting interests to cooperate. Therefore, I will not follow the strategy to list a comprehensive catalog of social, political, economic, and religious rights that allow a country to be called democratic. Instead, I will examine the process of individual and collective political decision-making, and its potential failures.

Anyway, it is easy to see that Schumpeter, in chapter XX of CSD, before proceeding to his definition of democracy and together with the critique of Rousseau’s idea of the “Common Will”, ascribes to freedom of thought and speech the role of the founding principle of democracy. Implicitly, he establishes a hierarchy of importance among different principles, some of which are contestable (Baumol 1982) by parties with different political orientations, while others are fundamental and essential to democracy (I will discuss this point in the next section).

I will therefore consider the democratic principles that Schumpeter implicitly regards as fundamental because their violation prevents the “struggle for the vote” from achieving its goals, i.e., the principles that guarantee the autonomous formation of political opinions, such as freedom of thought and freedom of speech. Even when these rights appear to be formally respected, they can be covertly violated when political manipulation, polarization, or ideologization lead citizens to make biased decisions.

In the following arguments, I will consider several potential failures that undermine the Olympic representation of the decision-making process, caused by the limits of rationality in politics, or, to use Schumpeter’s words, by the limits of “conscious rationality” ⁸ :

• Failures in opinion formation: The construction of opinions and preferences is a very complex task that requires full information and specific competence; the information provided by the media or by the parties is not constructed to be fair, and the same happens in public debates (Section 3). Then, citizens may try to simplify the opinion-building process through the cues provided by a political ideology (Section 4), and in this case the opinion-building process is shaped by that ideology.

• Failure of rationality and critical thinking: A strong identification with the values and ideological frames of a political party (routinized thinking) has two effects: Undermines citizens’ willingness to withdraw their support when the party fails to deliver on its promises (Section 4). And reduces citizens’ critical attitudes and exposes them to polarization. (Section 6).

• Failure of autonomous evaluation. Polarized voters prioritize candidates’ partisanship over their accountability. This leads to an adverse selection process in which politicians are renewed even if they have proven to be mediocre or have failed to keep their promises. Importantly, polarized voters with limited ideological frames prefer a political offer based on short-term promises. This leads to an adverse selection process in which populist politicians are preferred to political innovators (Section 5).

Because these failures are systematic, they create a dynamic of competition for votes that reverses the ideal Olympic mechanism of political decision-making processes and the ways in which parties should gain the trust of voters. I will discuss this dynamic in Sections 4, 5, and 6.

My perspective is to interpret the processes of democratic backsliding, briefly outlined here, in light of the “distortions” of the collective decision mechanisms that characterize the political competition for power. Although they do not explicitly and directly violate fundamental democratic principles, these “imperfections” can gradually lead to a loss of trust in democracy.⁹ In the face of persistent failures, citizens may withdraw from the electoral process, or even become hostile to democratic rules and turn to more centralized and authoritarian structures.

2 Freedom of thought and the common core of democratic principles

In his famous lecture “The Profession and Vocation of Politics,” Max Weber views conflict as inherent to society and attributes the right to regulate it to legitimate public institutions. This view is at odds with Rousseau’s vision, which is based on the idea that conflicts can be resolved “naturally” because citizens recognize a “common good” that can be achieved through a “common will”. Chapter XXI of Schumpeter’s CSD is devoted to a critical analysis of this vision:

“It is held, then, that there exists a Common Good, the obvious beacon light of policy, which is always simple to define and which every normal person can be made to see by means of rational argument.[ …. ] There is, first, no such thing as a uniquely determined common good that all people could agree on or be made to agree on by the force of rational argument. This is not primarily due to the fact that some people may want things other than the common good, but rather to the much more fundamental fact that “common good” is an expression bound to mean different things to different individuals and groups. This fact, hidden from the utilitarian by the narrowness of his outlook on the world of human valuations, will introduce rifts on questions of principle which cannot be reconciled by rational argument because ultimate values—our conceptions of what life and what society should be—are beyond the range of mere logic.” (CSD: 251)

Since people have different and sometimes conflicting systems of values, ideas, and motivations, we should recognize that it is impossible to rank such systems, i.e., that there is no universal order among them. Any general ordering principle can be questioned because it would itself be based on general principles. Then the idea that it is possible to identify a “universal common good” as the product of a “common will” encompassing a wide range of principles is a denial of the principles of individual freedom and pluralism. This point can also be seen by referring to Arrow’s Impossibility Theorem (Arrow 1950), which can be interpreted as antithetical to the idea of common will.

This question emerges clearly also if we limit the notion of common good to the notion of social welfare and if we reflect on how to improve the welfare of citizens in a market economy. As is well known, for any given initial allocation of resources there is a Pareto efficient equilibrium, i.e., a price system that allows every individual—after trading—to be better off, without anyone else be worse off. Consequently, under the usual assumptions, once the Pareto equilibrium is reached, any attempt to further improve people’s welfare by modifying the allocation of resources should be advantageous for a part of the population and disadvantageous for the rest. Now, assume that a part of the population are Utilitarians, and the rest are Equalitarians. Utilitarians could consider a Pareto optimal allocation as fair, for the reason just expressed, while Equalitarians should consider it unfair because it does not respect principles of equality. Two different and irreconcilable political and ethical systems, and two different strategies. Equalitarians could propose a reallocation of resources through a taxation for reducing inequalities, a goal that could be realized only through majority voting: the conflict of opinions cannot be cancelled, a majority voting on the issue signals that a common will on this issue does not exist.

Now suppose that a new allocation of resources has been reached. It is a stable achievement, i.e., an allocation that will never be changed in the long run? Of course not: a change of majority between Utilitarians and Equalitarians can put again under attention the question and reverse the choice. The reversal of a majority is a part of the normal dynamic of the competition for power, and in general does not lead to a threat to the democratic order, provided that the conflict on this issue does not encompasses conflicting opinions an all principles (and the related institutions), that remain shared by the competitors.

Thus, we can conclude that social institutions and shared principles of a community manifest different degrees of stability: while the allocation of power among the parties changes over time, some elements remain shared, and others come under discussion becoming contestable. Changes of relative political powers may render convenient to a party to re-negotiate various elements, which then become again the object of political competition.

Then if a community achieves an agreement on a set of shared principles, ideas, and institutions, after a—sometimes very complex—political intermediation among different interests and goals, the “social contract” is not entirely stable. A restricted core of elements remains stable over time, not being subject to political competition, while many elements become debatable and contestable (if a relevant law is routinely changed at every change of majority between two opposite parties, we have a signal of instability and a serious loss of trust by citizens).

Then, since conflicting opinions and systems of values are intrinsic of political action, the political parties can maintain an agreement on a common core of institutions and principles, while competing on all other political issues. The vital role of democracy in fact is to permit a pacific clash among them, allowing for different interests to be mediated and at the same time legitimized.

While, according to Schumpeter, claiming to base the concept of the universal common good on abstract universal principles is inconsistent, the formation and modification of a common core of institutions and principles, dynamically modified by the competition of parties through history, constitutes a central process of how democracy operates.

I argue that this schema is coherent with the picture given by Schumpeter because the conception of democracy is pluralistic by definition, compatible with the idea that principles, laws, and institutions all have different levels of contestability depending upon the breadth of consensus that they receive.

I have noted that, according to Schumpeter, there can be no universal order among principles: at the same time, in defining the characteristics of democracy, Schumpeter implicitly assumes a partial order by giving priority to the principles that guarantee the existence of a liberal democracy:

“There are ultimate ideals and interests which the most ardent democrat will put above democracy, and all he means if he professes uncompromising allegiance to it is that he feels convinced that democracy will guarantee those ideals and interests such as freedom of conscience and speech, justice, decent government and so on. The reason why this is so, is not far to seek. Democracy is a political method, that is to say, a certain type of institutional arrangement for arriving at political—legislative and administrative—decisions and hence incapable of being an end in itself, irrespective of what decisions it will produce under given historical conditions. And this must be the starting point of any attempt at defining it. ” (CSD: 242)

This does not lead to a contradiction with the impossibility of the common good, because by choosing as fundamentals the principles that guarantee the existence (not necessarily the well- functioning) of a democracy, we select a particular political order among many possible ones.

In other words, under the democratic system, different collections of principles can operate: some principles, those whose violation, implicit or explicit, disrupts the functioning of the democratic institution, are essential.¹⁰ Some others, which have developed historically as a result of confrontations between parties, are contestable and can be different in different countries and in the course of history. There is an implicit hierarchy of importance between them: a democracy in which some rights of minorities are not recognized can self-correct; a democracy that does not allow freedom of thought and speech is not a democracy.¹¹

By “common core” I mean a set of social institutions and principles—like the freedom of thought and speech, or the principle that the power to decide must be attributed to representatives through a vote—that constitute the basic rules of the game to which citizens and parties accept to participate and are by and large universally accepted.

Some elements of the common core have a “constitutional” status, meaning that they are hierarchically superordinate to the rest. Their modification should have disruptive effects on all structure of shared principles, laws, and institutions. Thus, their persistence guarantees the long-term stability of a political system and in a given society and a given age they are, for long periods of time, not taken to be politically contestable.

They become unstable at particular historical fault lines, such as the English and French Revolutions in the 17th and 18th centuries, or the ratification of the American Constitution in 1787. Or the rise of Nazi power in the 1930s (Kershaw 2008). In these historical conditions, the struggle for power leads to the dominance of one part over the other. The core becomes contestable and collapses after radical social and political confrontations.

The discussion in this paper aims to identify some of the core principles and the consequences of their violation. As is clear from the discussion of the common will, an essential element of the common core is the principle of freedom of thought and speech, and of course related principles such as pluralism and freedom of choice: a systematic violation of these principles undermines democracy. In the introduction, I have identified several potential elements that undermine the process of collective political choice and the related process of political competition for votes. I will discuss these processes in the following sections.

3 The impact of media and social media on the autonomy of political decision-making

Suppose we abandon one of the basic assumptions of the classical model of democracy, namely the idea that citizens are fully competent to make their political decisions. In this case democratic institutions should be designed to allow for the rational construction of decisions relevant to the functioning of a modern state, despite the limited information and knowledge available to its citizens. To this end, democratic institutions should enable a process through which parties transform the different beliefs, expectations, and interests of citizens into political strategies. Such process must work despite the limited competence of citizens to make political choices, providing to the voter, who typically cannot claim to be professionally competent in the management of public affairs, vast set of different and competing ideas and proposals offered by parties, opinion leaders, newspapers, and other media.

In this case, the debate, and sometimes the clash, between parties should offer the citizens the opportunity to freely choose a personal position among opposing alternatives, with the reasonable expectation that they can trust the democratic process as well as the specific party that they consider more suited to their interests and views. Under these conditions, citizens can be confident that the delegation of power to a particular party will result in policies that are demonstrably and reasonably consistent with their orientation and interests.

Thus, if we drop the assumption of full competence and information of the citizens, the design of the institutions that regulate the information process becomes a crucial element to ensure the proper functioning of the system. In particular, the functioning of the media system (opinion leaders, newspapers, television channels, etc.) and of internal institutions (civil servants and, more generally, people involved in the management of public affairs), which represent and promote different political positions, are crucial elements in the process of individual opinion building and collective decision-making.

The current, rather complex, and sophisticated system of communication is not designed to promote the rationality of citizens, but to persuade them effectively, and this leads to the concrete risk of distorting the vital process of forming autonomous political opinions.

In the competitive struggle for the people’s vote, it is natural that each party will try to persuade the citizens of its proposals and processes by which political and social forces steer new political opinions or even ideologies are the everyday reality of modern societies. Manipulation and persuasion are intrinsic to any process in which one part has more information and knowledge about an issue than another. Information asymmetry and competence asymmetry are intrinsic to the functioning of democracy. In a pluralistic society, a citizen who may have little competence on political issues can have confidence in a system in which different proposals and perspectives are all represented, if the competition between different parties or opinion leaders allows the real terms of a political issue to be brought out. This means that the construction of political opinions, channeled through competing newspapers and media, can stimulate intelligence and autonomous thinking.

This is a critical point, however. No one can guarantee that the debates in the media will have the effect of clarifying the political issues and developing the critical capacities of the citizens. The political communication systems, based on psychological techniques of persuasive communication, are inherently ambiguous and the construction of a rational political model becomes less and less effective as the use of narratives based on emotional factors and political advertising overrides the open debate based on the contrast of ideas.

Traditionally, the counterweight to possible manipulation is pluralism. Various legitimate political positions compete for the votes of citizens through newspapers and political parties. Then, the limits of individual rationality should be compensated by the conceptual elaborations of opinion leaders, columnists, etc. In this case, despite modest levels of competence and knowledge, citizens should make rational decisions in politics. However, the risk of being manipulated by politicians remains because their primary interest is in attracting votes, and to that end, persuasion and manipulation can be more effective than critical debate. ¹²

This was clear in Schumpeter’s analysis. In his view, individuals may have different degrees of conscious rationality, i.e., they may be limited in their ability to exercise full rationality in politics, which requires the possession of specific competence and knowledge with full awareness. When competence and awareness are absent or very modest, he warns, emotional elements may replace logical thinking and individuals are exposed to irrational external messages, forms of persuasion and advertising, both in the economic and political arena.

“…. we may for our purpose speak of genuine and definite volitions that at any moment are given independently of attempts to manufacture them, although we recognize that these genuine volitions themselves are the result of environmental influences in the past, propagandist influences included.” (CSD: 259,260) ¹³

In circumstances where citizens make decisions with low rationality, persuasion and political advertising play a crucial role, and the problem of finding remedies to the limitations of representative democracies becomes crucial. ¹⁴

Schumpeter’s view is sharp on this point:

“The weaker the logical element in the processes of the public mind and the more complete the absence of rational criticism and of the rationalizing influence of personal experience and responsibility, the greater are the opportunities for groups with an ax to grind. These groups may consist of professional politicians or of exponents of an economic interest or of idealists of one kind or another or of people simply interested in staging and managing political shows. The sociology of such groups is immaterial to the argument in hand. The only point that matters here is that Human Nature in Politics being what it is, they are able to fashion and, within very wide limits, even to create the will of the people. What we are confronted with in the analysis of political processes is largely not a genuine but a manufactured will.” (CSD: 263) (italics added).

Formally, democracy has a wider reach because citizens can now respond to and engage with leaders’ messages through social media and freely express their criticism. Unfortunately, much of the debate on social media is flawed by confirmation bias and driven by emotional motivations; the risk of “homologized thinking,” which is normally moderated by competition between different political visions, is amplified by the emergence of a multiplicity of self-produced, often contradictory “narratives” that compound and reinforce people’s prejudices.

In the past, advertising and propaganda were still essential elements of social persuasion and political manipulation, and they were used extensively; however, ideologies were important tools of reasoning, and citizens who embraced an ideology, while subject to the prejudices implicit in that ideology, were involved in the intense clash between opposing political ideologies and were intellectually stimulated.

4 The role of ideologies and political frames

Within a frame of shared social and constitutional principles, parties exhibit neat and stable differentiations based not only on their political programs but also on systems of values and beliefs. Sometimes these systems derive from a broad philosophical vision of society, while at other times they may consist merely of beliefs and values derived from experience. Regardless of their degree of consistency, systems of values and beliefs provide the frames through which individuals represent and interpret social and political facts. Denzau and North call such frames “ideologies”:

“Ideologies are the shared framework of mental models that groups of individuals possess that provide both an interpretation of the environment and a prescription as to how that environment should be structured.” (Denzau and North 2000: 1)

A similar concept is unfolded by Lakoff (2004: xv):

“Frames are mental structures that shape the way we see the world. As a result, they shape the goals we seek, the plans we make, the way we act and what counts as a good or a bad outcome of our actions. In politics our frames shape our social policies and the institutions we form to carry out policies. To change our frames is to change all of this. Reframing is social change.” ¹⁵

It is useful to distinguish between two different types of political frameworks. Following Denzau and North, I will call “ideologies” political frames that are characterized by a large philosophical background and a reasonable degree of consistency, while on the other side of the spectrum we have “fragmented political frames,” collections of beliefs (and sometimes prejudices) that maintain stability thanks to emotional components such as fear and hatred and the mechanism of confirmation bias.

When consistent and diffused, ideologies are the lenses through which citizens evaluate the relevance of political facts, shape their expectations, and simplify political reasoning and decision-making. The “classical” ideologies of the 20th century had a decisive impact on people’s participation in political life and on their positioning in the political struggle.¹⁶

In the mid-1960s, theories of the “end of ideology” by prominent scholars such as Lipset, Aron, and Daniel Bell supported the idea of the death of revolutionary political ideologies. The transformation of ideologies into the more subtle and ephemeral form of political frame occurred in recent years, has not diminished their importance in influencing the construction of individual political beliefs.

Usually, the new political frames emerge as the flags of new parties that are the expression and legitimation of pre-existing social movements and, as such, are the bearers of the values and perspectives of the movements from which they emerged. ¹⁷ Typically, social movements, such as the climate change movement or the social movement for global justice,¹⁸ are characterized by long-term values and goals, often supported by recurrent mass protest. However, the range of issues that characterize their protest is more limited than that of classical ideologies, such as liberalism or socialism (I will consider this issue in Section 6).

Precisely because they are newly emerging fragments of ideologies that do not sufficiently cover the vast area of political problems and social needs, these political frames need a strong emotional cover, which can be negative, i.e., based on fear and hatred, or positive, usually guided by aspirations for a better future. In both cases, the use of the media and social media is necessary to maintain and reinforce affiliation to the new party; but in the case of populist parties, which rely on very limited distinctive political identities, the resort to negative emotions often hides the lack of purposeful political solutions.

The main characteristic of ideologies and ideological frames, however, is that when individuals adopt them, they are guided by affirmative rather than critical criteria. The most glaring difference between an ideology and a theory is that the former is not subject to scrutiny and refutation, nor is it required to be fully consistent. Once individuals have embraced an ideology, they will tend to retain and entertain only those experiences that are consistent with and confirm the ideology.¹⁹

Bacon clearly described the underlying mental mechanism, four centuries ago:

“The human understanding, when any preposition has been once laid down, (either from general admission and belief, or from the pleasure it affords) forces every thing else to add fresh support and confirmation; and although more cogent and abundant instances may exist to the contrary, yet either does not observe or despises them, or gets rid of and rejects them by some distinction, with violent and injurious prejudice, rather than sacrifice the authority of its first conclusions.” (Francis Bacon, Novum Organum, 2001: 46). ²⁰

Second, ideologies are anchors that attract voters’ attention, allow them to position themselves in the landscape of possible political orientations, and give “meaning” to their actions. This happens not only with classical ideologies, but also when parties’ political positions are not based on a coherent ideology, but rather on fragmented political frames, be they characterized by negative emotions such as hate and fear, or by social protest as carried out by social movements. These frames have a less robust anchoring effect than classical ideologies and require a more pervasive process of reinforcement and confirmation through political advertising.

5 Adverse selection of political representatives

In their electoral programs, parties place more emphasis on values and cultural identities than on strategies to be implemented in the future. There are two reasons for this. One is that long-term policy strategies promise benefits, but their outcomes are uncertain, while they typically involve sacrifices in the short term. Another is that values and—if exist—ideologies are stable and allow for persistent identification, whereas most policy strategies imply a commitment to achieving their outcomes and the possibility that they will not be achieved. This is also the case with ideological frames: acritical adherence to an ideological frame polarizes citizens around stable beliefs and values; therefore, we have—contrary to the past—polarization without ideology.

I have already emphasized that polarization may allow for the emergence of an adverse selection process in which politicians are renewed even if they have failed to deliver on their promises: strong political identification or allegiance to a political party weakens citizens’ willingness to withdraw their support from parties or leaders even if they fail to deliver. ²¹

Adverse selection of political representatives is not a new phenomenon. Even when parties were clearly identified by classical ideologies, the ideological loyalty of a political candidate was more important than the quality of his profile. However, “lofty” ideologies also entailed a shared vision of long-term prospects. Thus, loyalty to an ideology also induced the sharing of general principles, values, and perspectives. As a result, the negative effects of adverse selection were partially mitigated.

With the decline of classical ideologies, voters’ identification with a particular party weakens, and a pervasive system of political communication and persuasion becomes essential to maintain voters’ allegiance, but whereas in the past polarization was generated by participation in a particular ideology, today party identity does not coincide with a well-defined system of values and shared perspectives.²² Therefore, it becomes necessary to reinforce citizens’ identification with communicative and persuasive techniques typical of advertising. To some extent voters are considered as consumers that must be retained to a brand.²³ Political campaigns are largely based on advertising techniques, long-term perspective becomes scarcely relevant, and political candidates are not required to possess the moral principles that in the past found their raison d’être in such long-term promises and perspectives.

Some features of this process have been suggested by Frisell (2004):

“In their pursuit of being re-elected, politicians might not choose high-quality policies but just conform to popular wisdom. The larger are the office spoils, and the more precise is an incumbent’s knowledge of voter opinion, the more likely that she will resort to such populism. My main result is that the public’s trust or distrust in politicians’ behavior may be self-fulfilling.”

6 Polarization without ideology and ideologies without scope

It is useful to clarify the reason why it is convenient for politicians to “conform to popular wisdom”: keeping voters close to their political affiliation requires massive use of the media, and for emerging parties lacking strong identitarian values, a short-term political offer becomes less risky than formulating long-term political strategies. In fact, the decline of classical ideologies makes any political commitment to long-term perspectives hardly credible to the electorate: most voters cannot easily evaluate the effectiveness of long-term strategies, while they vividly feel the discomfort of welfare loss. This leads to a second, more relevant process of adverse selection in which short-term populist programs (and their demagogic leaders) have a better chance of success than long-term policy programs (and their leading strategists).

Then the ambiguity of leadership roles (Weber) between “leading strategist” and “demagogue” does not guarantee rational and fair political competition. Populist leaders simplify political problems and convey information, proposals, or slogans to citizens in a way that disregards consistency but appears more credible and feasible than any long-term proposal. They do not appeal to rationality, but rather use communicative and persuasive techniques typical of advertising, based on psychological associations and related emotions. ²⁴

In conditions of increasing dissatisfaction and emerging social protest, political leaders who try to be carriers of social disaffection can be more successful by simplifying the problems and by increasing the emotional aspect of their proposals at the expense of rationality and consistency. They do convey a vision of the political problems, albeit through very fragmented political frames, which lack the consistency and often also the intellectual rigor of classical ideologies. However, the same psychological phenomenon that motivates a voter to comply with an ideology characterizes also the adhesion to the fragmented political frames. We thus observe the emergence of political “echo chambers” ²⁵ in which followers are locked in a fragmented political frame and are seen to comply to such a frame. In this case the reaction of voters to the leaders’ inability to realize their programs is less sharp than we should rationally expect. This again favors demagogues and makes it convenient for the new emerging charismatic leaders to massively use political advertising. Parallel to the backsliding of the role of ideologies, charismatic leaders also end up having more modest profiles.

In a careful empirical study of the relation between charismatic leadership and populism, Takis S. Pappas (2016) reminds us that Weber in his mature works

“conceives of charismatic leaders as creative agents endowed with extraordinary personal attributes who appear during political crises and other emergency situations to provide solutions. However, in his earlier (pre-1913) writings, Weber understood political charisma as the power of leaders (mostly of religious movements) to defy prevailing worldviews, forging instead new collective entities on the basis of discourses of justification against the established order” and by providing a “radical founding of a novel structure of legitimacy”. ”

In the social confrontations of the last two centuries, the birth or development of new ideologies, such as Marxism, was a process that developed along with the emergence of charismatic leaders, where charisma must be intentional in the full sense of Max Weber, but one can have charisma without proposing a radical change in the social order, and this seems to be the case with most populist leaders today.

In fact, in recent decades we have witnessed the birth and development of new parties that, although often born out of conditions of social hardship, were not the bearers of a universal ideology or a project of radical change in the social order. In the absence of new ideologies and value systems, parties still have to anchor voters, using all the techniques of persuasion and manipulation typical of advertising. The result is a “non-ideological polarization” based on leaders with “limited” charisma, who are often great salesmen without political values and visions.

The absence or weakness of ideologies forces leaders to propose perspectives that promise short-term benefits. In this case, however, short-term promises must be kept, and the problem of the accountability of parties and their leaders arises. One possible consequence is that the newly emerging leadership, to the extent that it is built primarily on citizens’ expectations, proves unstable and gives rise to a cyclical process of the rise and fall of new, often populist, parties. ²⁶

But beyond the diffusion and growth of populism, that relies on polarization without ideology an opposite phenomenon is evolving, i.e., the diffusion of ideologies without scope.

Politicians leading a new party born out of an emerging social movement may pursue a strategy opposite to that of populist leaders. They may rely on a “new ideological” affiliation. Their fortunes will depend on the robustness and size of their ideological framework and the success it will have with citizens, which requires more time and effort than the populist approach.

This is particularly evident in the new parties that have emerged from social movements, such as the climate change movement or the movements that have emerged in defense of minorities; these movements do indeed defend principles, values, and perspectives that still allow citizens to identify with them. However, the scope of their distinctive themes is more limited than that of classical ideologies such as liberalism or socialism. As a result, the new parties that emerge from these principles retain the loyalty of voters, but they run the risk of remaining permanent minorities. Therefore, to achieve a broad and growing consensus, also these parties must make intensive use of political communication and persuasion.

One consequence of the limited scope of fragmented ideologies is the progressive polarization of movements into distinct political identities. In “Against Identity Politics,” Francis Fukuyama (2018) explains the radical change in the traditional division of roles between left and right precisely by the emergence and growth of a multiplicity of polarized movements.

In any case, while polarization reduces the space for mediation between parties, it can lead to instability or discontinuity in the legislative process. A simple example illustrates some elements of the process. Suppose that an important law L is proposed to parliament by the ruling party and passed despite strong opposition from the minority; suppose that in the next election period the majority is reversed, and the new ruling party introduces a new law that goes in the opposite direction to L. In this case, a mediation/negotiation process would be necessary. Since mediation/negotiation on the features of L is not accepted, any future reversal of the elected majority will correspond to a cyclical repeal and re-approval of the law. Of course, this would cause citizens to lose faith in the parties. The more polarized the parties are, the less likely it is that their negotiations will be successful, and that the ruling party's program will be implemented in a stable manner. Political action will therefore be unable to expand the common core by adding new common principles or institutions in a stable way.

7 Can people be fooled all the time?

The pluralism connected with the different visions or ideologies is generally stable, in the sense that preferences and expectations of citizens that do not converge to a common view, but rather they remain diversified and shaped by political beliefs and ideologies. A considerable number of citizens remain loyal to a party even when their expectations about its program are not fulfilled. ²⁷ Then, political allegiance reduces the degree of rationality with which individuals evaluate parties’ performances or leads to behaviors that are not consistent with a rational assessment of the situation.

This suggests that citizens’ decision-making should be characterized as boundedly rational and therefore that the formation of political expectations is likely to be flawed. Two questions need to be considered: First, whether the systematic differentiations of expectations and beliefs are likely to be clustered around competing ideological positions, rather than converging to a common will. Second, whether the influence of ideologies and political beliefs can induce systematic distortions in the process of opinion building.

To address the former, I will consider a parallelism between Aumann’s criticism of rational expectations in markets and Schumpeter’s criticism of Rousseau view of “common will” in democracy. Briefly, should speculative traders in securities markets share a common vision of the markets, if their views continue to be common knowledge, even after each trader has updated his priors with heterogenous information, then their views would be equal, and therefore they would want to make the same buying or selling decision, which would render speculative transactions impossible. Therefore, to keep the markets working, traders’ expectations must diverge systematically. ²⁸

As it occurs for securities markets, which exist precisely because opinions, beliefs and expectations of traders are systematically different, the differentiation of political expectations and beliefs maintains the democratic institutions “open”. In the case of markets, a strong convergence to a shared view would lead to their disappearance. I argue that a similar property holds in the political arena when the conditions for the working of democratic institutions are held. Should different citizens have rational political expectations, on average their predictions would be correct, but this can only happen under the hypothesis that all of them share the same principles and goals, i.e., have a “universal common will”. Accordingly, the convergence of individual opinions to a common will and the parallel convergence of expectations is not likely to happen: a process of global convergence should be successful only if a political Pareto equilibrium would be existing for the entire society. However, as we have argued before, a political Pareto optimum does not exist: while in markets exchanges are supposed to lead to Pareto improvements through negotiations, in the world of politics negotiations do not exhaust the conflict of interests.²⁹

Exactly for this reason, the contrast of interests/goals can be solved only by intermediate bodies, that is political parties. ³⁰ In this line of reasoning, Schumpeter’s neat critique of the idea of the common will proves relevant to show that political competition is essential to maintain pluralism and ensure a fair bargaining process among parties.

Now consider the second effect of the assumption of bounded rationality. This assumption contrasts with the idea that people have rational expectations, i.e., that their expectations are, on average, fulfilled. The properties of rational expectations are well exposed by Thomas Sargent (2002):

“The concept of rational expectations asserts that outcomes do not differ systematically (i.e., regularly or predictably) from what people expected them to be. The concept is motivated by the same thinking that led Abraham Lincoln to assert, “You can fool some of the people all of the time, and all of the people some of the time, but you cannot fool all of the people all of the time.” From the viewpoint of the doctrine of rational expectations, Lincoln’s statement gets things right. It does not deny that people often make forecasting errors, but it does suggest that errors will not persistently occur on one side or the other.”

Amazingly, 50 years earlier, Schumpeter had argued exactly the opposite:

“And there is truth in Jefferson’s dictum that in the end the people are wiser than any single individual can be, or in Lincoln’s about the impossibility of “fooling all the people all the time.” But both dicta stress the long-run aspect in a highly significant way. It is no doubt possible to argue that given time the collective psyche will evolve opinions that not infrequently strike us as highly reasonable and even shrewd. History, however, consists of a succession of short-run situations that may alter the course of events for good. If all the people can in the short run be “fooled” step by step into something they do not really want, and if this is not an exceptional case which we could afford to neglect, then no amount of retrospective common sense will alter the fact that in reality they neither raise nor decide issues but that the issues that shape their fate are normally raised and decided for them. More than anyone else the lover of democracy has every reason to accept this fact and to clear his creed from the aspersion that it rests upon make-believe.”(CSD: 264, italics added).

As many scholars have noted, rising inequality can create social conditions in which citizens’ opinions become more radical and polarized. ³¹ Therefore, the task of mediation may become more difficult, and the emergence of new leaders who give a strong emotional and radical stamp to their messages becomes more likely. Under these conditions, there is a real possibility that people will be deceived in the short term and that the effects will be felt in the long term, given the considerable time that usually passes between two political elections. Citizens who have expressed their preferences and will on the basis of misleading, incomplete information cannot correct the errors by quickly changing their decisions, thus creating the opportunity for further manipulation.³² Being deceived in the short run may have unavoidable consequences in the medium or long run, as a path-dependent sequence of biased decisions emerges because people remain at least partially unaware of past mistakes:

“There is much evidence that once people have formed an opinion, they cling to it too tightly and for too long […]. At least two effects appear to be at work. First, people are reluctant to search for evidence that contradicts their beliefs. Second, even if they find such evidence, they treat it with excessive skepticism. Some studies have found an even stronger effect, known as confirmation bias, whereby people misinterpret evidence that goes against their hypothesis as actually being in their favor” (Barberis and Thaler 2003: 1068)

Moreover, when propaganda and political advertising prevail over rational debate, the chances of manipulation and deception seriously increase, while conversely the chances that people will be able to change their minds and correct their views decrease dramatically (Brexit docet). Therefore, confirmation bias keeps different groups of people in their ideological frames. Polarization is a result of this phenomenon.

8 Disintermediation and risk of impoverishment of the public officials’ competences

Another element that can cause citizens’ dissatisfaction with democracy is the imperfect functioning of the bureaucratic apparatus, which is expected to implement the political strategy of the parties in power. In this area of state activity, too, adverse selection may operate, if politicians prefer obedience and political affinity to quality and competence.

The new political leaders may emerge on the wave of emotional/irrational elements, without the need for the mediation (and rationalization) offered by traditional institutional intermediaries. This process is reinforced when the incumbent government has failed to grasp the emerging problems and provide the answers that citizens expect, causing citizens to lose faith in the intermediate institutions.

I have already noted that a defining characteristic of democracy is that it is an institution that can function even when the level of awareness, competence, and analytical effort of citizens is low, provided that it is balanced by responsible behavior on the part of its leaders and its internal institutions.

The balance, which can be reversed by the emergence of “groups with an ax to grind,” is based on two stabilizing forces: competition and competence. By “competence” I mean the essential quality that intermediate institutions (parliamentary offices, civil servants, bureaucrats, columnists, etc.) must have in order to deal with the complexity of public affairs. I argue that both stabilizing forces will be weakened by the effects of digitalization. We have already discussed the condition under which competition may fail in its stabilizing role. Now consider competence. The management of public decisions in a modern state is a complex discipline that requires a broad knowledge of public affairs and, beyond that, of the rules of law, as well as the relevant problem-solving competence. Let me quote Schumpeter again.

“It is not enough that the bureaucracy should be efficient in current administration and competent to give advice. It must also be strong enough to guide and, if need be, to instruct the politicians who head the ministries. In order to be able to do this it must be in a position to evolve principles of its own and sufficiently independent to assert them. It must be a power in its own right.” (CSD: 293) ³³

This view sheds light on the complexity of running the machinery of a modern state, which requires sophisticated skills that need to be constantly renewed; Going beyond Max Weber, Schumpeter suggested that the task of modern mandarins is highly complex and that the expansion of the functions of the modern state requires more and more expertise to manage public affairs. Yet a widespread popular narrative goes in exactly the opposite direction, blaming the bureaucratic apparatus for all the inefficiencies of public power. Moreover, the impact of digitalization has spread the myth that all bureaucratic activities, insofar as they are supposed to be essentially routine, can be easily replaced by automatic procedures guided by algorithms. This issue deserves more attention than we can afford here, but the crux of the matter is that the management of public affairs cannot be fully routinized. There are two reasons for this argument: First, public affairs management cannot be routinized because it is based on a knowledge of the political system that allows one to respond to everyday political problems that present themselves with novel features and characteristics and, as such, rarely (if ever) fit into routinized behavior. Second, because such knowledge, if separated from its traditional bearers, cannot be fully attributed to citizens, who by definition cannot directly and personally govern the political system. ³⁴

Disintermediation is then a misleading goal. To consider the intermediate apparatus of the state as useless and out-of-date is a harmful narrative on which a charismatic leader may gain advantage. By aligning his messages to the fable of the inefficiencies of the mandarins he would implicitly suggest reducing their independence and therefore the continuity of the rules of public management and the pluralism of the system.

9 Few concluding remarks

In our search for an explanation of the failures of the process that governs political competition, we have rediscovered the main features of Schumpeter's theory of democracy and identified the properties of human behavior that are consistent with his approach. To fully understand his theory, we must start from the assumption that bounded rationality is the cornerstone of citizens’ behavior in politics. The idea that all citizens independently and consciously form their opinions on politics and build their preferences on candidates with rational calculation turns out to be utopian. Forming a clear and not superficial political opinion is a very complex task. In general, citizens do not have the time, the opportunity, or sometimes even the will to make a fully rational political calculation. There are two reasons for this: First, because autonomous construction would require a very high and extensive level of expertise, along with free access to all relevant information, all in an extremely complex world such as that of politics. This was clear to Schumpeter:

“… our chief troubles about the classical theory centered in the proposition that “the people” hold a definite and rational opinion about every individual question and that they give effect to this opinion—in a democracy—by choosing “representatives” who will see to it that opinion is carried out. “(CSD: 269)

And second, because the interest of political parties is to influence the construction of political opinions of citizens and to attract their votes and, if possible, their affiliation. This is done through political advertising and a sophisticated use of the media. As a result, citizens are constantly exposed to the threat of biased information and sometimes even manipulation. This crucial element was also absolutely clear to Schumpeter. ³⁵

Thus, the rapid emergence of new populist political leaders in recent decades can be interpreted in the light of Schumpeter's picture. Compared to the historical period to which Schumpeter was referring, the conditions under which citizens make political decisions have changed in some relevant areas. To name a few, there has been an impressive evolution of the media system and, more recently, the emergence of social media. In addition, many socio-political conditions, such as the role of ideologies, have changed. Nevertheless, the structural features of the economic and political landscape have remained the same or very similar. The processes we have considered in the previous sections, which I will briefly summarize, take into account the changes in the landscape. ³⁶

First of all, the impact of social media:

”The diffusion of social media caused a shift of paradigm in the creation and consumption of information. We passed from a mediated (e.g., by journalists) to a more disintermediated selection process. Such a disintermediation elicits the tendencies of the users to a) select information adhering to their system of beliefs—i.e., confirmation bias—and b) to form groups of like-minded people where they polarize their opinion”. ³⁷

This implies the polarization of citizens around belief systems (political frames) that, while fragmented and sometimes inconsistent, still serve as anchors to define political identities. Polarized voters are reluctant to switch parties when their leaders fail to deliver on their promises. This condition paves the way for a process of adverse selection of political representatives, making a political offer based on short-term populist slogans more convenient than verifiable political strategies. The competition for votes becomes unfair since it is not based on the quality or plausibility of political results, while at the same time the process of polarization disrupts the elements of mediation. This, in turn, reduces the common core of shared principles and conventions among the competing parties and limits the space for mediation between them.

A landscape that is strikingly similar to the one vividly described by Schumpeter more than 70 years ago in “Capitalism, Socialism, and Democracy” and that is now re-emerging with disturbing vigor.