Simple and Formal Models in Comparative Politics
- First Online:
- Cite this article as:
- Little, A.T. & Pepinsky, T.B. Chin. Polit. Sci. Rev. (2016) 1: 425. doi:10.1007/s41111-016-0032-0
- 620 Downloads
In this essay, we review the current state of formal theory in comparative politics. Although formal theoretical work has had an important impact of the development of political science, we demonstrate that formal theory remains a relatively small part of the subdiscipline of comparative politics. We then discuss the role of simplification in theory building more generally, arguing that the simplification that formal theory requires is particularly useful when confronting the complexity of the social and political world. We recommend that comparativists explore ways that formal theorists can work together with statistical, historical, and ethnographic scholars as our subdiscipline matures.
KeywordsComparative politics Formal theory Game theory Complexity
Analysis requires simplicity; beware of incoherence that passes itself off as complexity.
Philippe Schmitter’s essay “Comparative politics: Its past, present and future” (2016) is an exciting overview of the subdiscipline of comparative politics. It makes a strong case for field research, foreign language study, and embracing classificatory typologies and “mere description” (Gerring 2008) as essential to comparative politics. Schmitter also joins in the chorus of comparativists who are skeptical that national borders denote the most relevant units of analysis for comparative politics (Franzese and Hays 2008; Gourevitch 1978; Milner 1998; Oatley 2011; Schmitter 2009; Tilly 1995). Comparativists who read Schmitter’s essay should come away with a greater appreciation of the limits of the nation state as a unit of analysis, and also a renewed enthusiasm for the field experience that attracts so many scholars to this subdiscipline in the first place. The message is, “get the local context right, but don’t confine yourself to the local context.”
We find Schmitter’s message to be a welcome one, but his discussion also contains stark criticisms of formal theory and game theory. A central thrust in Schmitter’s essay is a warning that “the real challenge currently facing comparative politics…comes from… ‘formal modeling’ based on rational choice assumptions.” There are two claims here. First, that formal modeling is or could become the dominant methodology in the subfield. And second, that formal models—and, perhaps more importantly, simple models—are not an effective tool for studying comparative politics.
The first part of Schmitter’s claim is plainly false. There is no evidence whatsoever that formal modeling has or could become the dominant approach comparative politics, in the United States or elsewhere. To show this, we examine data on types of articles published in top field and subfield journals, the specializations of PhD students who were on the academic job market in fall 2015, and curricular requirements in prominent political science departments. Our findings parallel those of Munck and Snyder (2007), who reached the same conclusion nearly a decade ago.
Schmitter’s second claim—that formal modeling is not an effective way to study comparative politics—is more provocative. It also has much higher stakes, as it rests on beliefs about the value of simplification and logical coherence that extend far beyond the small corner of the discipline that practices formal theory. Nor is this debate limited to comparative politics or political science as a whole: tellingly (and purposefully), many of the examples we draw on and scholars we quote are from sociology or economics.
Many of Schmitter’s objections to formal theory—e.g., that it is methodologically individualist, focuses primarily on material interests, and assumes perfect maximization of these preferences—are common features of this approach, but are not necessary for formalization. Importantly, a now large body of formal work in political science (and even more so in economics)—broadly lumped into the category of “behavioral” work—loosens if not completely breaks some of the assumptions Schmitter finds so objectionable. The more consequential aspect of formalizing theory is that it makes us be transparent about our assumptions and means of deduction, and in doing so generally requires substantial simplification. Whereas Schmitter objects to simplicity when theorizing about complex phenomena, we argue that this forced simplicity is actually a strength of formal modeling, particularly when in dialog with the other more empirical and complexity-embracing modes of research that make up the bulk of what is being done in comparative politics. In responding to Schmitter’s narrow criticism of formal theory as a threat to comparative politics, we aim to expose and discuss a series of more general issues that are relevant to all social scientists.
Before proceeding, however, we observe that all of the points made by Schmitter and many of our responses to them have already been made multiple times, in highly cited work by prominent authors and in the introductions of nearly every game theory textbook. While some early formal work exhibits a notable degree of revolutionary fervor about the power of simple models of selfish maximizing political actors, its proponents have repeatedly admitted the limits of such assumptions, and defended the simplification their methods required, while also pointing out how their approach can prove fruitful with a wide variety of assumptions about preferences. It is natural that some disagreement lingers, but many of Schmitter’s points have been repeatedly debated and some have been repeatedly debunked in the decades since the introduction of formal theory to our discipline.
No methodology or body of work is perfect, and pointing out that the flaws of various approaches to political inquiry are central to the scientific endeavor and are a necessary part of critical scholarship. However, for these exchanges to be fruitful, critics must make a good faith effort to understand that which they criticize. And it is hard to accept that one who has followed debates over formal theory with an open mind could believe, for example, that formal theory requires making particular assumptions about preferences, or worse that formal theorists believe that these assumptions are literally true. We are thus skeptical that adding yet another defense of formal theory will do much to bridge the gap between its proponents and critics.
Still, perhaps new authors elaborating on these points in a new venue and in a somewhat different manner will have some additional value, and the issues that Schmitter raises should resonate with all comparativists concerned with theory and the complex world of global politics, so we proceed. In the next section, we present our analysis of trends in academic publishing and U.S. university training. We then consider the strengths and weaknesses of formal theory in comparative politics in more depth, highlighting errors in Schmitter’s characterization of this body of scholarship. We also defend simple formal theories as essential to understanding complex phenomena. Our final section concludes by discussing the implications of our response to Schmitter for how comparative politics ought to develop.
2 Don’t Fear the Theorists
Schmitter argues that formal theory has become, or threatens to become, the dominant methodology in comparative politics. To probe this claim, we first examine patterns of publication in top disciplinary and subfield journals. This gives us a picture of the current state of formal theory in comparative politics. We then examine the current state of academic training in comparative politics, which arguably gives us the best possible picture of the future directions that comparative politics will develop in the coming decades.1
The middle panel illustrates the trends in approach across all subfields. While the trends are somewhat volatile, around 10 % of papers utilize only formal modeling, with another 10 % using formal theory and a quantitative empirical test. Again, there is no clear trend upwards for formal theory. The right panel contains a similar plot of submitted papers by field for AJPS. Again, comparative politics represents about 30 % of submissions with a general upward trend. Formal theory and (empirical) methodology papers are never more than 10 % of submission for 1 year, with no upward trend.
So, unless the magnitude or trend in these numbers is drastically different for comparative papers—which would mean that formal theory is becoming more dominant in comparative politics but simultaneously less dominant for other subfields such as American politics or international relations—the data are inconsistent with Schmitter’s claim that formal theory is or risks becoming the dominant approach in comparative politics.
To go beyond APSR and AJPS and separate comparative from other subfields, we analyzed back issues of one additional general journal (Journal of Politics) and three prominent subfield journals (Comparative Political Studies, Comparative Politics, and World Politics). For each of these journals, we examined all articles published in 2005, 2010, and 2015.
Comparative Political Studies published more formal theory articles in 2015 than it had in previous years, but articles with formal models number only six in 2015, and comprise just ten percent of all articles appearing in that year. Comparative Politics has also seen a growth in formal theory articles, from zero formal theory articles in 2005 and 2010 to one in 2015. None of the articles that we identified in any of these three subfield journals primarily used formal theory. In all, results from these prominent subfield journals provide no evidence that formal theory is displacing other forms of comparative politics research, nor is there any threat that it will do so anytime soon.
Our analysis of the journal data excludes books, which remain a central form of scholarly output for most comparativists. How would our conclusions change if we were to include these as well? Our best guess is that this would shift the results even more strongly against formal and game theory in comparative politics. Books that contain formal theory in comparative politics certainly do exist (some prominent examples from recent years include Boix 2003; Mares 2006; Svolik 2012), but their number pales in comparison to books that eschew formal modeling.
2.2 Job Market Candidates
Our findings from a review of recent publications in comparative politics suggest that formal theory is not currently dominant in the most prominent disciplinary or subfield outlets. But perhaps it is dominant among the next generation of comparativists. To investigate if this is true, in fall 2015 we collected data on the substantive and methodological approach of all students on the job market for the 2015–2016 season from the top seven programs US programs in CP and political science in general by the US News and World Report rankings (the sets perfectly overlap), and an additional three programs which have been “hotbeds” of formal theory in recent years.4 All of these programs had websites that listed their current students seeking academic jobs, and nearly all of the candidates had personal websites with information about their research. For each candidate, we coded if one of their main subfields is comparative politics, and whether their research included any use of formal theory or primarily used formal theory (see the “Appendix” for more details of the coding). Our data include 178 job market candidates, 153 from the top seven programs and 25 from hotbed programs.
The right panel looks at how these trends differ among the top seven and hotbed schools, for all subfields. Not surprisingly, the usage of formal theory is more prevalent in the hotbeds, though notably the modal student from these departments mixes formal theory with other methods. Further, students from the hotbed departments are not only smaller in numbers due to our sampling method, but because these departments tend to have far fewer students on the market than the top seven: with an average of twenty two for top seven departments and an average of eight in hotbed departments.
Of course, our analysis of job market candidates from 2015 is just a snapshot from one year. Nevertheless, the message is stark: right now, most graduate students from top departments in comparative politics, and the discipline as a whole, do not use any formal theory in their research, and extremely few are primarily formal theorists.
2.3 Curricular Requirements
Regardless of the share of journal articles occupied by formal theory, and prevalence of formal theorists among current job market candidates, do departments prioritize formal theory in graduate training? To answer this question, we collected data on the formal theory course offerings and requirements in the eighteen “top 15” departments by overall ranking in the US News and World Report (four schools were tied at 15). All of these departments offered an introductory game theory class, with some also offering advanced game theory and other kinds of formal modeling (e.g., social choice theory, agent-based modeling, or “behavioral” models). However, requirements are rare. Importantly, none of the aforementioned top seven programs requires their PhD students to take a class in any kind of formal modeling.6 Among the 18 schools we examined, only five required an introductory game theory class.7
So, perhaps even more so than reflecting in publishing and job market candidates, our analysis of formal theory in graduate curricula reveals that it plays a small role in the way that graduate students in comparative politics (and political science more generally) are trained.
These data largely square with our experiences as a formal theorist and a primarily empirical researcher who occasionally employs but more often draws on insights derived from formal theories of politics and political economy. While formal theorists certainly would like to have a bit more space in top journals formal papers (and, in particular, their own papers), there is no consensus even among formal theorists themselves that formal theory ought to become dominant in comparative politics. Similarly, while we hope that some talented graduate students specialize in formal theory and the rest at least get some exposure to our approach, if anything, we have found that students who enter graduate school with interest in formal theory often end up shifting to a more empirical approach than the other way around. Informal discussions with our colleagues at other institutions indicate that a majority (but far from all) of their comparative students take at least one game theory class, and rarely more than one. Further, several colleagues expressed worry that enrollments in game theory classes are declining, with some speculating that increasing offerings and requirements in quantitative empirical methods are leading some students to take less formal theory.
An important caveat to make is that, as we discuss below, much work which is not explicitly formal still uses a broadly “rationalist” approach to theorizing, or is primarily empirical but draws on formal theory to develop hypotheses or interpret the data. By focusing on the use of formal methods, we certainly are not capturing the full influence of this style of work on articles or scholars. However, when formal theory influences how scholars using other methods work, this reassures us that the formal work that has been done (and, hopefully, the work currently being done) is useful to a wider community.8
Of course, it could be the case that we are misreading or misrepresenting the sentiments of formal theorists, who really do aspire to be hegemonic or at least expansionary. Or, one might think that even the approximately 10–20 % of our discipline’s intellectual energy dedicated to formal modeling distracts or even subtracts from our knowledge of comparative politics. If so, then what we observe in the journals data is the result of the aspirational hegemony of formal theory being confronted by a vigorous non-formal theorist defense. A skeptical reader might conclude, then, that Schmitter’s warnings against the threat of formal theory are still relevant, so the opponents of formal theory must be ever vigilant. But does Schmitter’s characterization of why non-theorists should resist formal theory hold merit? We now turn to this question.
3 Simple Theories, Formal Theories, and “Thick Rationality”
Schmitter objects to approaches to work in comparative politics which (1) is mathematical (or “formal”), (2) relies on assumptions of preferences that are sometimes referred to as “thick rationality” (this terminology appears to originate as a thickening of what Satz and Ferejohn (1994) call “thin rationality”), and (3) emphasizes simplicity. In this section, we first observe that these three properties, while often complementary, are distinct. We then argue that although not every theory should be formal, conceive of political actors as maximizing materialistic goals, or aim to be simple, theories that invoke these properties either singly or together can have important benefits into the study of politics.
3.1 Formalism and “Thick Rationality”
“Roughly speaking, our working definition of rational behavior is behavior consistent with complete and transitive preferences.9 Sometimes we call such behavior ‘thinly’ rational, as [completeness and transitivity] contain little or no substantive content about human desires. Thin rationality contrasts with thick rationality whereby analysts specify concrete goals such as wealth, status, or fame. The thin characterization of rationality is consistent with a very large number of these substantive goals. In principal, thinly rational agents could be motivated by any number of factors including ideology, normative values, or even religion” (McCarty and Meirowitz 2007, p. 6).
“The theory of rational choice is a component of many models in game theory. Briefly, this theory is that a decision-maker chooses the best action according to other preferences, among all the actions available to her. No qualitative restriction is placed on the decision-maker’s preferences; her rationality lies in the consistency of her decisions when faced with different sets of available actions, not in the nature of her likes and dislikes.” (Osborne 2004, p. 4).
We did not cherry-pick these examples: a passage like this can be found near the beginning of any game theory textbook.10 As these quotes clearly indicate, formal theorists do not think that they know exactly what political actors’ preferences actually are (or are “supposed to be”). What formal modeling entails is making tentative, hopefully reasonable, assumptions about some aspects of the preferences held by key political actors to see how trying to attain their goals might result in behavior that illuminates our understanding of real-world outcomes.
To be concrete, one of us has written several papers where individuals decide whether to participate in an anti-government protest. Of course individuals may be motivated to protest (or not) for a wide variety of reasons, and there is certainly great value in research that uses interviews or other methodology to learn more about the distribution and magnitude of these motivations. But what is needed to write productive theories of protest is not a complete and accurate description of everyone’s motivations, but some general statements about when people are more apt to protest, such as when they are more dissatisfied with the government and when they expect others to join as well. Combined with other assumptions about how citizens learn about the preferences of others, these partial statements about when citizens want to protest can lead to non-obvious insights about, for example, when peaceful alternation of power after elections is possible (Little, Tucker, and LaGatta 2016) or whether and how social media affects levels of anti-regime activity (Little 2015).
More generally, formalizing a theory does not mean claiming that assumptions are descriptively true—if it did, then it would indeed be possible to dismiss formal theorists by just observing that the world is complicated. Formal theories, like other theories, are models (Clarke and Primo 2012). Formalizing a theory just means using mathematical tools to determine how preferences, information, strategies, and rules of deduction lead to behavioral predictions.
“it is striking that for roughly a decade the cutting edge of microeconomic theory has been devoted almost entirely to models in which the actors are less than fully rational [and perfectly informed]… Notably, no economic theorist has decried this shift away from models about fully rational agents as a betrayal of the premises of ‘rationalism’ (so far as we know). Rather, the spirit of the research has been, as before, ’what happens in a model of this problem if one makes such-and-such assumptions?”’ (Fearon and Wendt 2002).
Although much early rational choice work in political science was primarily materialist, and many battles were fought over the degree to which we should model political actors as self-interested utility maximizers, those being trained today are the second (or third) generation of formal modelers whose “core commitment is the formalization of arguments to explore the consequences of different assumptions rather than to any particular assumption of actor rationality” (Fearon and Wendt 2002). At this point, to argue otherwise suggests a basic ignorance of the past quarter century of formal theory in political science.
3.2 Formalism and Simplicity
Schmitter’s other main objection to formal theory is that formalization features too much simplification to provide insight into a messy and complex world. In our view, the connection between formalism and simplicity is indeed tighter than the connection between formalism and thick rationality. However, the reason why this is true is itself one of the main reasons why formal theory is so valuable.
In our experience, the reason why formal models tend to be simpler (or have stronger assumptions) is that the process of formalization makes it clear to the modeler what assumptions are required to reach a particular conclusion. Even among those who have spent years or decades writing formal models, it is quite common to think “X logically follows from A, B,” only to realize that that is not true without an assumption of condition C when trying to formalize the argument (see e.g., Elster 2000, p. 690) (excellent examples from international relations include the debates between offensive and defensive realists; see Powell (1999), chapter 5 and Wagner (2007), chapter 1). Sometimes this condition C is something minor or defensible, giving us a better idea of the scope conditions under which the argument works. Other times, it becomes clear that the argument we thought made sense only works under highly restrictive circumstances. For example, the intuition that natural disasters and other events outside of the incumbent control should not affect their re-election probability—which underlies a large literature questioning voter rationality—only works in the knife-edged case where the impact of the politician ability and magnitude of disasters on outcomes are additively separable (Ashworth et al. 2014).
A common response to arguments such as these is that some theories are just too complex to formalize. However, if a theory is too complex to formalize, under what conditions can we trust our verbal intuitions? Theorizing without math frequently leads to small logical inconsistencies, and sometimes to major ones. How can we know which? Theorizing without the rigor of formalization seems appealing in part because it allows us to avoid realizing when our conclusion does not follow from our premises.
One defense against the problem of making logically incoherent claims without formalization is to just make theories simple enough to be “obviously” logical. This may be fine: many arguments about why political actors behave the way they do are straightforward (rich people prefer lower taxes, those who feel a stronger sense of civic duty are more likely to vote), but there is work to do in empirically testing their relative importance. Nevertheless, formalizing these ideas may give us additional theoretical predictions about when to expect the relationships will be strong. Put another way, for scholars who put a high weight on the importance of being sure that theories are logically coherent, the ability to use formal tools can allow for more complicated theories without losing this confidence.
Still, there is considerable variance in the level of complexity within formal models. Models can have more or fewer actors, more or fewer strategies to choose from, payoffs that incorporate more or fewer incentives, and have weaker or stronger assumptions about, say, functional forms. A particular list of features that Schmitter asks for is: “(1) fuzzy and under-specified models about a plurality of types of actors with preferences that are contingent upon differences in political setting, (2) strategic interaction between a large number of players at different levels of aggregation with inconsistent payoffs, (3) constant communication and multiple interdependencies and endogenization not of single discrete variables one at a time, but of patterns of multiple variables within the same time frame” (Schmitter 2016). Other than the fuzzy and underspecified part, all these realistic complications are things that can be incorporated into formal models, though generally at a substantial cost of making the models harder to solve and perhaps less likely to provide clear insights (particularly to readers without training in formal theory).11 Again, the real constraint is the ability to logically show what conclusions follow from the assumptions, and any purported theory that cannot demonstrate these conclusions formally ought to raise the suspicion that it is simply not correct.
One potential alternative to formal theory that Schmitter endorses would draw insights from complexity theory to explain interactions across levels of analysis. We are not opposed to the notion that complexity theory can provide theoretical insights into comparativists, and we strongly support the general arguments that emergent phenomena exist, and that the properties of macrostructures cannot necessarily be inferred from the properties of their constituent parts. However, we disagree that complexity theoretic research in comparative politics will prove to be satisfying to critics of formal theory who object to assumptions or simplification. Laver and Sergenti’s (2012) recent analysis of party competition in multi-dimensional policy spaces, for example, contains extensive discussions of the role and purpose of assumptions and behavioral models (see also the contributions in Kollman, Miller, and Page 2003). Axelrod and Cohen’s (1999) introduction to Complex Adaptive Systems begins with descriptions of model primitives—agents, strategies, types, measures of success—which are essentially the same as that in standard formal theory. Nor do we believe that the distinction between computational and analytical models is particularly strong (Laver and Sergenti (2012: 7) carefully acknowledge that “Curry-Howard isomorphism” implies that computational and analytical models are actually equivalent). Indeed, some of the terms that Schmitter attributes to international relations scholars drawing on a complex systems approach are common in standard formal theory and comparative politics: spiral model, cheap talk, multi-layered governance, among others. In all, an appeal to systemic approaches to global politics and complex dynamics in multi-level systems is welcome, but is not an argument against (nor even an alternative to) formal theory.
Still, some may prefer the more complex among formal models for the reasons Schmitter lays out. We now turn to consider the costs and benefits of simple models.
3.3 The Value of Simplicity
Since the choices to make models simple, formal, and to utilize thick rationality assumptions are all distinct, there are three sets of tradeoffs to think about when determining which ways of theorizing are most likely to prove fruitful. The costs and benefits of formalization have been extensively discussed elsewhere. For example, Walt (1999) and the responses to that article provide a relatively productive discussion, and we highlight some of the main issues above, so do not elaborate on this point here. Suffice it to say that we do not hold that all theories must be formal, but since the position that non-formal or verbal theory has value is more commonly accepted, we direct our energy toward formal theory.
We also avoid further discussion of the value of thick rationality assumptions, as the value of different kinds of assumptions in general is ultimately an empirical question. Schmitter seems to recognize this, in describing the reaction of modelers to some empirical shortcomings of the rational turnout literature as “if the actors do not confirm the initial suspicion that their purpose is to acquire more wealth or material goods opportunistically by optimizing at the margin in each political exchange, the rational choicers will simply substitute another preference and, if necessary, yet another preference until the individual’s rationality has been proven” (Schmitter 2016). Other than the incorrect statement that the goal of formal theory is to “prove” individual rationality, this is simply a description of the scientific process: models with some set of assumptions lead to empirically problematic conclusions, and so we try different models. In our view, thick rationality assumptions have led to successful models in some domains (e.g., party competition, inter- and intra-state conflict, and electoral institutions) and have had less success in others (e.g., voter turnout).
So, we focus our discussion here primarily on the value of simplicity. Schmitter makes a compelling case that the political world that comparativists study is complex. In Schmitter’s view, formal theory makes simplifications that prevent it from grasping complexity. His position misunderstands what makes any theory useful, and the indispensability of simplification for managing complexity.
All theorizing involves simplification. Of course, theorizing looks very different across disciplines, approaches, and ontological positions. Still, to be useful—and by useful, we mean insightful and generative—theory must abstract from the specific details of a particular social situation. The perils of attempting to explain or even just describe any political phenomenon without simplification are succinctly described in Jorge Luis Borges’ “On the Rigor of Science”, where cartographers build a “Map of the Empire whose size was that of the Empire, and which coincided point for point with it.” Although such a map is clearly as realistic and complex as possible, the next generation rightly deems it useless (for further discussion of models as maps (which also use this example) see Clarke and Primo (2012) and Rodrik (2015)).
all theories are fictions. I’m not going to literally describe all actions in any theory, I don’t care if it’s physics, biology, economics, any social science, any physical science, any biological—they’re fictions… Everyone of us here is going to be different. And yet, in some dimensions, we’re the same. So maybe we abstract from these differences for a lot of problems and just look at the similarities…And the question is—when analyzing good and bad theories—which fiction works better for whatever problem you have in mind (quoted in Becker et al. 2012).
Typological theory too requires simplification. When Schmitter invokes ideal types as “the best tool” for identifying the new “building blocks” upon which to build comparative analysis, for the ideal type “invites attention to differences in type, not differences in magnitude,” he is arguing in favor of the clarifying simplifications that make useful social theory possible. Put bluntly: an argument against simplification in the face of complexity is an argument against theory itself.
This does not imply that more simplification is always better. To pick an example on the other extreme as Borges’ cartographers, let us consider the simplest possible (formal) model for why a political actor might choose action X instead of action Y. Assume the actions of all other actors are irrelevant, and the hero of our model believes—for any reason Z—that the expected payoff for choosing X is ux which is greater than the corresponding uy. So, for any X and Y and any reason Z we deem reasonable, we have a theory for why a political actor would choose X.
While obviously silly, it is instructive to ponder why the preceding example is a useless bit of theorizing. It is useless first because it is obvious: the conclusions transparently follow from the assumptions. More importantly, it is useless because it is not generative: we have just pushed the explanation one step back to determining whether Z is a reasonable explanation for preferring X to Y.
There are also other, more substantial costs to making models as simple as possible. Simple models can lead to simplistic conclusions. In fact, many important theoretical contributions take the form “by loosening or changing assumption X in an existing model, the main results change,” introducing a bit of complexity in order to reach richer or more insightful conclusions. Changing an assumption could mean incorporating a new actor, as Ansell and Samuels (2014) do by dividing “the rich” into two groups, the landed and industrial elites, in a model of inequality and democratization, and as Stokes et al. (2013) do by invoking brokers as central to the functioning of electoral clientelism. It could also mean looking at more general preferences, as Lee et al. (2007) do in models of party competition and redistribution in which voters may be racist. It could mean allowing for feedback in processes that unfold over time (e.g., Greif and Laitin 2004). One of the most prominent and cited formal theory papers in political science argues that many existing theories of war (formal or not) no longer can explain the very existence of war if the potential bargainers are given a new (and realistic) strategy: namely, to bargain before conflict begins (Fearon 1995) (And one of our papers shows that some conclusions from this paper do not hold if one complicates this model by endogenizing the preferences of the actors (Little and Zeitzoff 2016)). Still, even in cases where a more complex model casts doubt on the conclusions of a simpler model, starting with a simple—perhaps brutally simple—model before moving to more complex extensions may be clarified, and lead to a richer understanding of what is being studied.
So how complex should our models be? As with most interesting political questions, the answer must be that it depends. It is useful to have theories with more and fewer actors, or more and fewer actions. But this is an unnecessary question. Formal theorists frequently present models of varying levels of complexity, often within the same paper. Typically, this entails starting with the simplest possible theoretical model to illustrate the main mechanism that the authors wish to highlight, followed by more general and complicated versions to demonstrate the robustness of this mechanism and generate further insights.
This discussion also suggests an important role for non-theorists in the process of formal modeling. Those with the richest knowledge of individual cases and understanding about the relationships between variables we care about cross-regionally or cross-nationally are particularly well positioned to push theorists to include important actors, or extend their models to explain important facts. This is not a threat to formal theory, but rather an opportunity for collaboration.
Despite the many gains of simplicity, there can still be great value in some complex models—indeed, of complex models of atomistic, selfish utility maximizers. Even a model which is thickly rationalist, completely apolitical, highly complex and mathematical, and founded on plainly unrealistic assumptions about how people make decisions may provide important insights into real-world problems that comparativists care about. One prominent example that critics of formal theory in political science might find instructive is Kenneth Arrow’s (1963) foundational analysis of the health care industry, which responds to results from general equilibrium analysis in economic theory (see e.g., Arrow and Debreu 1954). Those results established the existence of a competitive equilibrium under certain assumptions about markets which correspond tightly to the view of market participants as egoistic, thickly rational utility maximizers. A simple reading of such results is that they are mathematical defenses of free market exchange; a sophisticated reading is that they are rigorous demonstrations of (1) the conditions under which market exchange produces a competitive equilibrium, and perhaps more importantly, (2) the criterion by which one might consider that equilibrium to be socially valuable. As Arrow observes—and remember, these are comments about some of his own theoretical results—the conditions under which the competitive equilibrium will obtain are highly restrictive,12 and the benefit of such an equilibrium is not abundantly clear anyway. The implication is that a free market for health care services is not guaranteed to produce even an outcome of questionable social merit. This observation has major implications for prominent contemporary policy debates about health care policy in the United States and elsewhere.
Even though we believe that simplicity is a virtue in formal theory, we recognize that Arrow’s own critiques of the applicability of general equilibrium analysis “in the real world” have great intellectual value, and also important implications for comparative politics. It does not follow from this example that every complicated model of an unrealistic social interaction is useful. Rather, our point is that the observation that a model is complicated and unrealistic does not permit the inference that that model is not useful tout court. Other comparativists may point to similar, equally complex, formal theoretical results that have comparably large implications for how we think about politics. These are not results about why certain political phenomena occur, but instead about why some theoretical models do not permit particular conclusions, such as the so-called Gibbard Satterthwaite theorem (Gibbard 1973) or McKelvey’s famous results about the instability of any status quo with multidimensional policies (McKelvey 1976). In sum, our defense of simplicity is practical: “to isolate specific causes and how they work.” When it turns out that only a highly abstract treatment invoking many implausible assumptions can produce a particular theoretical result, then that, too, is a useful contribution. When, in contrast, highly abstract mathematical theories are obfuscatory and unrelated to the verbal intuitions that they are meant to capture—a phenomenon that Paul Romer (2015) terms “mathiness”—then we agree that complexity and abstraction are obstacles to intellectual progress.
3.4 Do Formal Theorists Over-Supply Simplicity?
Now that we have discussed the costs and benefits of varying levels of simplicity in our models from the perspective of generating useful knowledge in comparative politics, we ask a natural “formal modeler”-type question: do disciplinary incentives lead formal theorists to make their models too simple or too complex? While we hesitate to make strong claims here without formalizing an argument or bringing actual data to bear on this question, we speculate that formal modelers generally have incentives to make their work more complex than would be ideal for maximizing the generation of knowledge. Where Schmitter sees excessive simplicity, we see professional incentives that push formal theory in comparative politics to be not simple enough.
“if you want to publish a paper in economic theory, there is a safe approach: make a conceptually minor but mathematically difficult extension to some familiar model because the basic assumptions of the model are already familiar, people will not regard them as strange; because you have done something technically difficult, you will be respected for your demonstration of firepower. Unfortunately, you will not have added much to human knowledge.”
There are several relevant points here. First, referees may be predisposed to give positive evaluations to papers where it is clear that the author has done hard—in the sense of mathematically challenging—work. Coming up with the kinds of assumptions that lead to a simple model is in fact hard too (formal nor not), but this kind of challenge may be less obvious to the casual reader than the challenge of solving a complex model with many moving parts. Second, if the assumptions required to simplify a model look unfamiliar to referees – even though they are no less “problematic” or “unrealistic” than the standard ones – then referees may object to the model simply because of its unfamiliarity.
More perniciously, in our experience, it is often the case that if one were to really present the simplest version of a model possible, either (1) it would then seem obvious or (2) it would clearly rely on strange—if not obviously incorrect—mechanisms. In the first case, presenting the simple version can be extremely valuable for the accumulation of knowledge: many influential arguments seem obvious once one has gone through the logic of the model, but either has not been made previously or is clearer with the accompanying formal exposition. Still, one risks having a referee or editor say “this is obvious” even if the argument has not yet been widely considered or accepted. In the second case, the model should probably be jettisoned, but once one has solved something difficult, career incentives can push researchers to present the complicated and “mathy” version (sometimes accompanied with a not entirely accurate description of how the main results arise; see Romer 2015) and hope that no one notices.
Even outside the journal process, junior scholars (and, even more acutely, graduate students) face strong incentives to signal not only that they can publish, but that they have the technical capabilities to publish in the future.13 Again, this is easier to signal by solving hard models than presenting simple versions where the reader may be unsure if the author was smart enough to know how to make an argument in a simple fashion, or just got lucky. And in fact, the common perception among non-formal theorists that formal models are inevitably complicated and mathematically intensive itself reinforces modelers’ own incentives to focus on mathematically challenging results.
These incentives to demonstrate insight by appealing to complexity apply more generally, across the discipline of political science. Those employing statistical methods may be tempted to signal their chops using whatever difficult technique is currently in vogue. For those for whom deep understanding of individual cases is important, invoking extraneous detail to make this mastery obvious may have career benefits even if doing so obscures the concrete contribution of the case analysis. In all these cases, when not including “all the details,” one is open to criticism of what Healy calls “connoisseurs”. These scholars signal their sophistication by making unproductive calls for more nuance, a practice which “gets its aesthetic bite from the easy insinuation that the person trying to simplify things is a bit less sophisticated a thinker than the person pointing out that things are more complicated” (Healy 2015).
In sum, both as producers and consumers of new research, professional incentives push formal theorists and other comparativists to under-supply simplicity. Calls to embrace complexity by appealing to ever-more complicated theories and ever complex analyses, however well intentioned, will exacerbate these perverse incentives.
The standards by which any formal theory ought to be judged are the same as with informal theory: its simplicity, the plausibility of the assumptions, the provocativeness or novelty of the predictions, the model’s generativity, and the fit with the data. Formal theories will sometimes prove unproductive, which has been arguably been the case with egoistic “rational choice” theories of turnout, as Schmitter notes (see also Green and Shapiro 1994). Others will prove productive, as the Olson’s formal treatment of collective action problems has (Olson 1965). The formal analysis of collective action problems as multiplayer Prisoner’s Dilemmas, in fact, helps to illuminate exactly why and how collective action problems emerge and the conditions under which we should expect to them to hinder cooperation in groups.
Formal theory, of course, is not without drawbacks. It is hard. One needs to make a serious investment of time to learn how to set up and solve models correctly. Formal theorists devote considerable time and energy to working out the details of proofs once 99 % of the model is done. This is time that could be spent on reading history, learning languages, doing fieldwork, or collecting and analyzing data. We agree that if comparative politics jettisoned all of these things and devoted their efforts exclusively to formal theory, we would have a poor subdiscipline indeed. But it seems to us that if Schmitter is right that comparative politics must be ever-attendant to complex interdependence, then there is strong case that comparativists ought to remain methodologically catholic. What this recommends is not rejection of formal theory, but simply a division of labor.
What role would formal theory play in such a division of labor? We contrast two positions on this question. One is that formal theorists should respond to the complexity that Schmitter identifies by writing the most complicated models of politics possible. The other is that formal theorists should attack complex political phenomena by constructing multiple simple models that address complexity from multiple angles. Our preference is for the latter—simple models for complex problems. When combined with other forms of evidence, in particular case studies, they are particularly useful for identifying theoretical mechanisms and clarifying their logic (see Goemans and Spaniel 2016; Lorentzen et al. 2016 for discussions; see Nalepa 2010 as an example). Simplifying and formalizing a complex problem from multiple directions can also help to identify what kinds of data would be needed to distinguish among alternative mechanisms, or to show the different observable consequences of related causal processes (see Lorentzen and Scoggins 2015 for an example). Scholars interested in the “mechanismic” approach to social scientific inquiry, then, should welcome the contributions that formal theory can make.
What would such a division of labor look like in practice? We echo Gehlbach (2015) that the objective is not for comparativists to master formal theory alongside modern statistics and case details from some part of the world, which is one conclusion that a generation of graduate students seems to have drawn (or have been instructed to draw) from Laitin’s (2002) discussion of the “tripartite method.” Likewise, we do not advocate that the objective of every piece of research is to combine all methods and theories. Instead, we think that empirical researchers ought to understand what formal theory actually entails—to learn what words like “equilibrium” and “rational” mean—and should be open to learning from formal theorists who are able to spot those unstated assumptions and inconsistencies in their theoretical claims. We think that those who specialize in formal theory ought to be aware of the types of theoretical and practical questions that occupy country and area specialists, and mindful of alternative modes of theorizing that are not actor centric. To us, these objectives are modest and sensible. They should facilitate collaboration without insisting upon it, heighten the value of middle-range theorizing without ignoring the value of abstraction, and position comparativists to grasp the complexity of the political world without being overwhelmed by that complexity.
Again, we are not arguing against the existence of complexity. It is important to have comparativists who master the details, who understand the context, and who can illuminate the complexities of cases to spur better theorizing and empirical testing. Our challenge is to match details and complexity with the insights that theory—formal theory as well as verbal theory—can provide. We do believe, for example, that formal theory’s contributions to comparative politics are too often obscured due to a focus on technical details, which unfortunately comes from the fact that modelers have incentives to write for other modelers (as they are the manuscript referees) rather than for other comparativists. But we also suspect that outmoded views of what formal theory is stand in the way of the types of collaborative opportunities we think would be most productive for the subdiscipline.
Finally, we observe that if Schmitter wants to worry about the dangers of oversimplification—and in particular, about individualistic analyses that ignore cross-level effects—then his discussion has missed the single most important development in the social sciences over the past two decades: the causal inference and identification revolution and the rise of “empirical microfoundations” (see e.g., Angrist and Pischke 2010; Samii 2016). As one of us has argued elsewhere (Pepinsky 2014), a growing focus on the estimation of treatment effects has clear implications for the study of complex phenomena in the social sciences. The core insight of the identification revolution is a good one: make statistical analyses credible by freeing them from statistical assumptions that imply implausible or unverifiable theoretical claims about the social world. One can do that by rejecting observational data and relying on experiments, or by exploiting clever natural experiments that provide design-based leverage on important causal questions. The challenge is, what to do when comparativists wish to study empirical phenomena that are not causally identifiable without invoking the very assumptions that the identification revolution hopes to avoid? A still larger concern is whether training scholars to work only on identifiable causal effects will discourage them from even perceiving the existence of important but unidentifiable causal relationships. Schmitter’s review of the state of comparative politics is curiously silent about the intellectual trend that is perhaps most directly challenging to his vision for how the subfield ought to develop.14
As discussed later in the essay, the use of formal theory is somewhat distinct from the use of simple theories and theories that assume political actors are self-interested, so we are only directly tracking trends of one aspect of what Schmitter criticizes. We do this primarily because it is more straightforward to code whether papers or scholars use formal theory.
Any article including an equation which was not just a regression to be estimated was classified as at least “some formal.” Articles were classified as primarily formal if the majority of the analysis involved setting up and analyzing models as opposed to empirical testing.
World Politics publishes work both in comparative politics and international relations. We make no effort to distinguish which articles fall properly within each subfield, but do observe that only one of the articles using formal theory that we identified would be counted as an international relation article.
The top seven (by CP ranking) are Harvard, Stanford, Princeton, UC Berkeley, Columbia, Yale, and Michigan. The three “hotbed” departments are NYU, Washington University in St. Louis, and Rochester. In addition to our subjective judgment, the three hot bed departments are the three programs that do not make the initial list but are in the top seven of the US News rankings for “Methodology”.
The relatively high number of students using some formal theory within comparative is mostly driven by the fact that over half of candidates from two of the hotbed departments (NYU and Wash U) are classified as comparative. Still, it is telling that even in the hotbed departments few students primarily employ formal methods.
Colleagues at some of these institutions indicate that taking a game theory class (as well as statistics) is a de facto requirement for working with certain faculty. Still, the fact that none of these departments formally codify such a requirement is telling.
The five with such a requirement are the three “hotbed” departments—Wash U, NYU, and Rochester—plus UC San Diego and Duke. Rochester is the only department to require two classes which appear to mostly cover formal theory. Wash U and NYU also require a class on mathematics.
Similarly, coursework outside of methods sequences usually will include reading some formal theory papers in seminars.
Completeness means that actors can compare any two outcomes X and Y in the sense that they either strictly prefer X to Y, strictly prefer Y to X, or are indifferent between the two. Transitivity implies that if X is preferred to Y and Y is preferred to Z, X is preferred to Z.
The only exception we found looking through ten texts was one aimed primarily at economics undergraduates.
A common reaction to a formal theory paper is that it (1) has too much math but (2) does not capture something that the reader thinks is important. However, incorporating the commenter’s proposed addition will almost certainly increase the amount of technical analysis.
He discusses at length the “marketability of goods and services” in the context of health care as a prerequisite for the existence of a competitive equilibrium.
Our informal sense from colleagues in economics is that this problem is even more acute in that field. To see this, take a look at the job market papers of recent PhDs in economics, which tend to be very long and even more packed with difficult technical material than published work in the field.
We do not wish to overstate this case, as we believe that much as we advocate in favor of simple models to tackle complex interdependence, there is probably a case for breaking up the complexity of the political world into smaller yet causally-identifiable phenomena as well. Moreover, topics such as treatment effect heterogeneity and external validity are areas of active research, and may yet provide new insights on how to bridge the gap between identifiable causal questions and complex political phenomena.