- 164 Downloads
How does the process of consensus formation affect the accuracy and reliability of our knowledge? Cognitive and epistemic division of labor creates a problem of trust in the use and application of knowledge. Consequently, the reliability of scientific consensus depends on whether the incentives, which the self-interested members of scientific communities face, are aligned in the right way.
KeywordsExpert Consensus Justify Belief Scientific Consensus Prediction Market Public Deliberation
Consensus is a conventional source of justified beliefs. In modern democratic societies, rational consensus, formed by means of free and open discussion, is a criterion of political legitimacy.
Modern democracy as a form of government is a unique phenomenon, existing for only a short period of time in the history of our civilization. In this contribution, the role of consensus in directing the course of modern democracies is addressed. Public choice scholars have extensively studied the problem of amalgamating individual beliefs into aggregate social estimates for the purposes of legitimizing political authority. The aggregation of individual beliefs, however, is a specific example of a general question about the creation and use of knowledge. How does the process of consensus formation affect the accuracy and reliability of our knowledge? Without understanding how the division of labor brings together scientific communities and without understanding how these communities produce expert consensus, the criteria according to which we assess the accuracy and reliability of our warranted beliefs remain unclear.
According to literature, the reliability and accuracy of expert consensus depend on the nature of scientific institutions. In general, institutions are understood to be the rules of the game that determine the structure of payoffs for the agents involved. Given the cognitive and epistemic division of labor, expert consensus introduces a problem of trust. This problem is documented by empirical evidence that shows gaps between expert and nonexpert beliefs. If expert consensus results from the coordination of self-interested individuals, its reliability is dependent on whether the incentives, which the self-interested members of scientific communities face, are aligned in the right way.
Determining the right way to align incentives is ultimately an empirical problem. Therefore, the study of the economics of scientific knowledge through a comparative institutional analysis should be of interest to anyone who is curious about the role institutions play in the creation and use of knowledge in modern society.
Is There a Consensus Among Economic Experts?
If we want to know how the process of consensus formation influences the accuracy of our beliefs, we should perhaps first look into the state of expert consensus today. Some claim that the scope of consensus among economists is overwhelming; others, however, find this contention questionable. Generally, there seems to be a consensus regarding mainstream economic theory. With regard to economic policy, however, the attitudes among economic experts often differ.
According to Mark Blaug, “nothing like an overwhelming consensus has emerged from … postwar economic methodology. But despite some blurring around the edges, it is possible to discern something like a mainstream view” (1992, p. 110). Today, this mainstream expert agreement can be summarized as follows: “Economists share the view that individuals are utility-maximizers, human wants are unlimited, and that mathematical modeling should be an important part of economic modeling” (May et al. 2013, p. 25). These are the basic building blocks of mainstream economic theory.
The scope of agreement among economists has been examined from the 1970s (Brittan 1973), and the surveys and expert polls seem to have established consensus on a range of economic questions. This consensus can be generally summarized with the following statement: “Price system or market is taken to be an effective and desirable social choice mechanism” (Frey et al. 1984, p. 994). In fact, Gordon and Dahl (2013) found “no detectable systematic differences in views across departments, or across school of PhD.” Moreover, they found “no evidence to support a conservative versus liberal divide” (p. 635). This would mean that regardless of where economists complete their education and regardless of their priors, they tend to agree on the core points of their discipline.
Expert consensus is, and has often been, challenged, however. Attitudes toward economic policy differ among male and female economists (May et al. 2013), among Democrat and Republican economists (Klein et al. 2013), and, in 1984, it was shown that expert opinion differed even among economists based in different countries: “The American, German, and Swiss economists tend to support more strongly the market and competition than their Austrian and French colleagues, who rather tend to view government interventions into the economy more favorably” (Frey et al. 1984, p. 994). These findings suggest that gender, political affiliation, and cultural and historical circumstances influence what kind of questions economists ask and what kind of answers they tend to give and agree on.
But why should we care if economists agree on anything? Presumably, the answer lies in the fact that economists are trained to take things which are not seen, at least not immediately, into account. The qualified point of view is then needed to uncover common fallacies. “What economists think, and whether there is consensus among economists, would not be a matter of concern if beliefs do not have a very strong effect on economic policy decisions and on the state of the economy” (Frey et al. 1984, p. 986 emphasis original). Expert opinion is a consequence of a particular kind of division of labor that takes place in modern democratic societies. As such, expert opinion is a source of warranted beliefs. The division of labor, however, introduces a problem of trust.
Can the Consensus Be Trusted?
A division of cognitive and epistemic labor has fostered the creation of knowledge by means of specialization and expertise. This division of labor, however, introduces problems of trust among the producers and consumers of expert knowledge. If the consumers of expert opinion do not trust its reliability, the expert consensus becomes inconsequential. Evidence shows that there is indeed a gap between expert and nonexpert opinion.
Scientific opinion is an opinion not held by any single human mind, but one which, split into thousands of fragments, is held by a multitude of individuals, each of whom endorses the others’ opinion at second hand, by relying on the consensual chains which link him to all the others through a sequence of overlapping neighbourhoods.
There is a vast division of cognitive labor among scientists. Given his or her specialization, each expert looks at the world from a particular perspective. Each specialist, in turn, observes different aspects of reality. The deeper the specialization, the more diverse the observed aspects become. The essential difficulty consists in making sure these diverse observations are reliable and, above all, reconcilable with observations established by experts in distant neighborhoods of science.
Because of the division of cognitive labor, “nobody knows more than a tiny fragment of science well enough to judge its validity and value at first hand”; scientists have to “rely on views accepted at second hand on the authority of a community of people accredited as scientists” (Polanyi 1974, p. 173). Experts must mutually rely on the accuracy of peer review in scientific communities distant from their own. “If the aim of scientists is to maximize their knowledge of the world,” writes Jesús Zamora Bonilla (2008, p. 4), “they need trust in the word of their colleagues, making science a collective enterprise.”
On the one hand, the cognitive division of labor makes each scientist specialize in his or her area of expertise and build on the knowledge gained from exchanges and interactions with other experts, past and present. On the other hand, there is also epistemic division of labor. Unlike the cognitive division of labor which results in specialization among scientists, epistemic division of labor separates the scientist from a nonscientist. At some point, the scientist had to make a decision to enter into the business of curiosity in the first place; it is a consequence of the epistemic division of labor that some people get to contribute to the production of expert consensus, while others become its mere consumers.
A pragmatist conception of democracy takes “the epistemic division of labor as one of the central features of effective and informed public deliberation”; there is a caveat, however: “Any such division of labor which produces epistemic gains will also produce deep asymmetries in the social distribution of knowledge” (Bohman 1999, p. 591). As a result, the body of scientific knowledge “can be received only when one person places an exceptional degree of confidence in another, the apprentice in the master, the student in the teacher, and popular audiences in distinguished speakers or famous writers” (Polanyi 1974, p. 220). Unless such a degree of confidence takes place, expert agreement loses its effect on public opinion.
The first is self-serving bias. A large literature claims that human beings gravitate toward selfishly convenient beliefs. Since economists have high incomes and secure jobs, perhaps they are biased to believe that whatever benefits them, benefits all … The second doubt about economists’ objectivity is less sordid but equally damaging: ideological bias. … A consensus of fundamentalists hardly inspires confidence. It sounds like an intellectual chain letter: Maybe each batch of graduate students was brainwashed by the previous generation of ideologues.
In his analysis, Caplan found a gap between what economists and laymen believe, suggesting that the exceptional degree of confidence in expert consensus is indeed missing. When identifying the sources of the gap, however, it became clear that the “self-serving and ideological bias combined cannot account for more than 20 % of the lay/expert belief gap. The remaining 80 % should be attributed to the experts’ greater knowledge” (2007, p. 56 emphasis in original). The persistence of the gap must therefore be a consequence of some other cause. As Caplan hypothesizes, we “turn off our rational faculties on subjects where we don’t care about the truth” (2007, p. 2 emphasis in original). Consequently, given the persistent ignorance, the expert consensus – even if generally unbiased and reliable – fails to have a strong impact on public opinion.
Another explanation of the gap suggests that although ignorance may be convenient, “the problem, it seems, is not that members of the public are unexposed or indifferent to what scientists say, but rather that they disagree about what scientists are telling them” (Kahan et al. 2011, p. 148). Even if the expert consensus was perceived as generally reliable, it could still fail to have a strong effect on public opinion as a result of a cultural cognition effect according to which “individuals systematically overestimate the degree of scientific support for positions they are culturally predisposed to accept” (pp. 166–167).
Indeed, as Gordon Gauchat (2012) shows, a “growing distrust in science in the United States has been driven by a group-specific decline among conservatives” (p. 179). Such a decline may reflect particular cultural, political, and ideological characteristics of the research agenda that the conservative group of expert-opinion consumers is not willing to take in. This explanation seems plausible, given the mostly progressive composition of the US expert group (Klein et al. (2013) shows that the proportion of Democrat economists to Republican economists is approximately three to one).
Note, for example, that unlike in the United States, there is a “a general tendency to the right among the Swedish social scientists” (Berggren et al. 2009, p. 2). If the political composition of academia influences the approach to research, the situation Gauchat observed in the United States may actually run in reverse in countries like Sweden, where academia tends to be rather conservative as compared to the general public. In these cases, the progressive public may, in fact, display decreased perceived credibility of the scientific consensus arrived at by a more conservative cohort.
In short, there are two conditions needed for the scientific consensus to make a difference in the process of public deliberation. It must be reliable, and it must be credible. As long as it is reliable, the expert consensus has the potential to improve our knowledge of the world. As long as it is credible, the expert consensus has the power to influence public opinion. These two conditions are implied by the process of consensus formation which is inherently social and which therefore crucially depends on the nature of the institutional framework that supports it.
The Nature of Scientific Institutions
Institutions are the rules of the game; they determine the structure of payoffs for the agents involved in the game. If consensus formation is a social process and if expert consensus results from the coordination of self-interested members of the expert communities, the reliability of scientific consensus will depend on whether the incentives which the self-interested members of scientific communities face are aligned in the right way. It is then the analysis of the nature of scientific institutions that sets the stage for our understanding of the creation and use of knowledge in modern democratic societies.
Keeping in mind the cognition effect, let us assume that the influence of expert consensus is a function of its reliability: the more reliable and accurate the scientific consensus gets, the more credible it becomes. There are two essential notions to consider when determining the reliability of scientific consensus. First, scientists follow their self-interest; in this, they are no different from any other subject of economic analysis. Second, expert consensus, as a heuristic source of knowledge, induces conformity. Taking these assumptions into account implies that the reliability of expert opinion depends on particular properties of the scientific network that produced it. Let us look into these points further.
“Could it turn out,” asks Philip Kitcher (1990, p. 6), “that high-minded inquirers, following principles of individual rationality, should do a poor job of promoting the epistemic projects of the community that they constitute?” The epistemic division of labor presumably sorts out people who are better at scientific research from people who are better equipped, for example to, say, start and run a business. This does not imply, however, that there is a hardwired feature in each and every scientist forcing him or her to pursue the discovery of truth at all costs.
It might not be in the best interest of a scientist to advance the stock of reliable knowledge. Brock and Durlauf (1999) emphasize this point: “Whereas the predominant themes in the philosophy of science … presumed … identical desire to find ‘truth’ … recent trends … have been concerned with the social context in which research is conducted” (p. 114). The literature has discerned that the social context of research produces motivations, which may compete with the presumed scientific urge to find truth.
Payoff structures determine the best course of actions for every scientist. When social context determines the structure of payoffs, it is reputation that scientists value highly. As Paul David (1998) argues, “A scientist working in a collegiate reputational reward system will consider the nearer-term reputational consequences of current actions (including expressions of scientific opinion), as well as considering long-term payoffs possibilities in the form of lasting fame for having gotten it right.” In such a case, conformity may push against refining the body of reliable knowledge.
Personal interests must also be expected to play a role. Many referees will be tempted to judge papers according to whether their own contributions are sufficiently appreciated and their own publications quoted. They carry, for instance, no costs when they advise rejection of a paper they dislike (e.g., because it criticizes their own work), even if they expect that it would be beneficial for economics as a discipline.
Economists may not question the generally followed assumptions of their discipline. In a situation when the problem at hand resembles Newtonian celestial dynamics, conforming to a mainstream consensus saves methodological effort. If, however, the sort of problems economists look into turns out to be better explained by some variant of generalized Darwinism, then following others in following Newton might lead the economist off the cliff – and into the land of the inconsequential.
According to Cass Sunstein, “following others can itself be seen as a heuristic, one that usually works well, but that also misfires in some cases” (2002, p. 38). Given the vast range of modern scientific knowledge, the consumers of expert consensus cannot help but follow expert opinion and trust that beliefs coming from the fields of science they are not acquainted with are well substantiated and properly justified. Expert consensus is a mental shortcut, a heuristic of consolidated opinion which, as J. S. Mill anticipated, “is salutary in the case of true opinions, as it is dangerous and noxious when the opinions are erroneous” (1859).
When conformity constitutes a rational course of action, “society can end up making large mistakes. … Social influences … threaten, much of the time, to lead individuals and institutions in the wrong directions.” In such cases, “dissent can be an important corrective” (Sunstein 2002, p. 40). Dissenters, however, are often ignored and even ostracized, accused of destroying peaceful agreement for their own selfish motives. Furthermore, the dissenting opinion is often embedded within a system of language and reasoning that, from the perspective of the prevailing expert consensus, appears to lack rigor and preciseness. New ideas are substitutes for current prevailing thought. If reputation matters, and if the leading researchers are heavily invested in the prevailing consensus, plain dismissal of any dissent may occur.
When dissent does not pay off, originality does not take place. Dissenters, the individuals questioning the prevailing consensus, provide a valuable service that has a public-good character. By sharing their personal knowledge and pointing out where widely shared expert agreement runs short in terms of reliability, they contribute to the refinement of the body of scientific knowledge, benefiting all. Originality and dissent, however, are not absolutely valuable in themselves. They matter on the margin. Some questions of marginal analysis of dissent seem to follow: How much dissent is too much? And when does consensus become noxious?
The answer to these questions lies in the process of emergent consensus formation. An indispensable condition of producing reliable knowledge is an institutional mechanism, which ensures that the incentives scientists face align their self-interest with the general purpose of the epistemic division of labor. Through discovery and experimentation, an incentive-compatible institutional structure produces an emergent ratio of agreement and dissent. At this point, the production of scientific knowledge becomes a subject of comparative institutional analysis or, more broadly, economic analysis of scientific knowledge. “To the extent that we can make realistic presuppositions about human cognitive capacities and about the social relations found in actual communities of inquirers, we can explain, appraise and in principle improve our collective epistemic performance” (Kitcher 2002, p. 441 emphasis in original).
Given the agreement among economists about the price system being a desirable mechanism of social choice, it should be no surprise that one of the institutional arrangements suggested to improve the accuracy of scientific consensus is the prediction market, in other words, betting on beliefs. Prediction markets “doubtless have their limitations but they may be useful as a supplement to the other relatively primitive mechanisms for predicting the future like opinion surveys, politically appointed panels of experts, hiring consultants or holding committee meetings” (Wolfers and Zitzewitz 2004, p. 125 emphasis added).
Robin Hanson, an advocate of prediction markets as an incentive-compatible institution of knowledge creation, claims that “betting prices … are a robustly accurate public institution estimating policy-relevant topics” (Hanson 2013, p. 156). The system of prediction markets supports dissent by rewarding out-of-favor beliefs, which turn out to be true, more than true beliefs that everybody supports. Such a system also seems to transparently draw the line between desirable and undesirable dissent because it encourages the well-informed to step forward and speak up and the ill-informed or dishonest to stay silent.
The scientific wager of Julian L. Simon and Paul Ehrlich on the implications of price theory is, after all, a well-known affair. The practice of betting on beliefs, however, does not come without problems. Today, prediction markets that aggregate information on questions of science or policy are disparaged in the same way life insurance – another form of betting on beliefs – used to be constrained by repugnance. With the exception of British prediction markets (generating odds on matters such as the likelihood of secession from the Eurozone or on the prognosis of the 2015 UK unemployment rate), betting on beliefs is mostly illegal.
Prediction market is but one of the institutional arrangements aggregating individual estimates into a composite indicator. Such an arrangement may provide some benefits, but does it outperform the current organization of science in producing justified beliefs? “Whether the rules according to which scientists compete for recognition among each other, and the rules that govern their competition for resources, are well aligned and whether they support or inhibit each other in promoting the growth of knowledge is an empirical matter” (Vanberg 2010, p. 43). Eventually, a comparative analysis should provide insights into the effects of diverse mutations of scientific institutions on the process of consensus formation.
The epistemic performance of modern democratic societies depends on the division of labor in the creation of knowledge. At the same time, however, there is a gap between expert and nonexpert opinion, suggesting that the cognitive and epistemic division of labor creates a problem of trust in the use and application of knowledge. There are, in fact, several reasons why expert consensus may fail to impact public opinion: expert consensus may be irrelevant, biased, or simply opposed. In general, expert opinion fails to make a difference in the process of public deliberation when it is unreliable.
The reliability of expert consensus depends on the nature of scientific institutions. In other words, the reliability of scientific consensus depends on whether the incentives, which the self-interested members of scientific communities face, are aligned in the right way. The analysis of scientific institutions sets the stage for our understanding of the creation and use of knowledge in modern democratic societies. It is the empirical assessment of how different institutional arrangements perform in the social process of consensus formation that future research will have to address.
- Berggren N, Jordahl H, Stern C (2009) The political opinions of Swedish social scientists. Finn Econ Pap 22(2):75–88Google Scholar
- Brittan S (1973) Is there an economic consensus?: an attitude survey. Macmillan, LondonGoogle Scholar
- Caplan B (2007) The Myth of the rational voter: why democracies choose bad policies. Princeton University Press, PrincetonGoogle Scholar
- David PA (1998) Communication norms and the collective cognitive performance of “invisible colleges.” In: Creation and transfer of knowledge. Springer, Heidelberg, pp 115–163. Retrieved from http://link.springer.com/chapter/10.1007/978-3-662-03738-6_7
- Frey BS, Pommerehne WW, Schneider F, Gilbert G (1984) Consensus and dissension among economists: an empirical inquiry. Am Econ Rev 74(5):986–994. doi:10.2307/557Google Scholar
- Kitcher P (2002) Contrasting conceptions of social epistemology. In: Brad Wray K (ed) Knowledge and inquiry: readings in epistemology. Broadview Press, PeterboroughGoogle Scholar
- Klein DB, Davis WL, Hedengren D (2013) Economics professors’ voting, policy views, favorite economists, and frequent lack of consensus. Econ J Watch 10(1):116–125Google Scholar
- May A, McGarvey MG, Whaples R (2013) Are disagreements among male and female economists marginal at best?: a survey of AEA members and their views on economic policy. Contemp Econ Policy. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/coep.12004/full
- Mayer T (1993) Truth versus precision in economics. E. Elgar, Aldershot/BrookfieldGoogle Scholar
- Mill JS (1859) On liberty. In: Essays on politics and society, vol XVIII. University of Toronto Press, TorontoGoogle Scholar
- Polanyi M (1974) Personal knowledge: towards a post-critical philosophy. University of Chicago Press, ChicagoGoogle Scholar
- Šťastný D (2010) The economics of economics: why economists aren’t as important as garbagemen (but they might be). Instituto Bruno Leoni/CEVRO Institute and Wolters Kluwer, Turin/PragueGoogle Scholar
- Sunstein CR (2002) Conformity and dissent. Law School, University of Chicago, Chicago. Retrieved from http://www.law.uchicago.edu/files/files/34.crs_.conformity.pdf
- Zamora Bonilla JP (2008) The elementary economics of scientific consensus. Theor Rev Teoría Hist Fundam Cienc 14(3):461–488Google Scholar