On IGO withdrawal by states vs leaders, and exogenous measures for inference

We are pleased that our article “Hello, goodbye: When do states withdraw from international organizations?” has stimulated interest, and we welcome the opportunity to respond to Seung-Whan Choi’s attempt to challenge its conclusions. In our article, we examine why states withdraw from intergovernmental organizations (IGOs). Using what researchers know about IGO accession as a starting point, we group potential exit reasons into those related to domestic politics, IGO characteristics, and geopolitical factors. In the main analyses of our original dataset of all IGO-member state years from 1945 to 2014, we test these three sets of explanations with nine variables. We find that IGO withdrawal is mainly driven by geopolitical factors such as contagion and preference divergence, along with smaller effects related to IGO characteristics.

One of the nine potential drivers of withdrawal related to nationalism; our statistical analyses show that nationalism of states is not a key driver of state withdrawals across space and time. Given the recent narrative linking nationalism and IGO withdrawal, we probed our null finding in several ways. Our article tests seven¹ different measures of state nationalism in seven separate models. None of these coefficient estimates are statistically significant. We also conducted a textual analysis of original data on states’ public justifications for IGO withdrawals; we find that states reference factors related to nationalism in fewer than 3% of withdrawals. From these results, we concluded that nationalism is not a key driver of state withdrawals from IGOs when we systematically analyze IGO exits across time and space.

Choi takes issue with these findings and proposes an eighth measure of nationalism, which he labels “leader nationalism,” and proposes that this measure is a key driver of IGO withdrawals. Below we address Choi’s primary criticisms, explain how they are based on misrepresentations of our article, and then show that his proposed substitute variable suffers from measurement error and endogeneity. We conclude by noting that Choi’s replication confirms our original finding: historically, international cooperation and nationalism can and have existed side-by-side. While some states have said “hello” and then “goodbye” to an IGO, withdrawing is not primarily a response to state nationalism.²

Criticisms are based on misrepresentations

Choi proposes that nationalist leaders play a “critical role”³ in IGO withdrawals, such as President Trump announcing the World Health Organization (WHO) withdrawal in 2020.⁴ He argues that our “research risks measurement error – the theoretical concept and the nationalism variable do not align well.”⁵ That is wrong. As detailed in the original article, our theory and empirics are both about nationalism of states. Our article’s title and research question are “Under what conditions do states withdraw from intergovernmental organizations?”⁶ Our hypotheses are about states (not leaders),⁷ and our unit of analysis is the IGO-member state-year.⁸ While we refer to “leaders” eight times in the article,⁹ we refer to “states” more than 200 times. In other words, leaders are not central to our theory; states are.

Our article’s argument is about states instead of leaders because the ability of leaders to decide on IGO withdrawals varies across states and across IGOs. State institutions limit leaders’ IGO withdrawal decisions. Leaders may be an intervening variable that is part of the mechanism, but leaders are not a sufficient condition for IGO withdrawal across states. Since we are interested in theorizing and testing the drivers of withdrawal across states, IGOs, and years (rather than just, say, recent US cases where leader nationalism may play a larger role), we focus on the institutions – states – that ultimately execute IGO withdrawals regardless of context. Our article underscores that leaders are often constrained by hurdles that stand between threats and actual exits, including IGO procedures (e.g., waiting periods or rules surrounding payment of dues) and other state institutions (e.g., party politics or decision-making rules in the legislature). For instance, while President Trump’s implicit threat to leave NATO in 2018 suggests that a leader could tweet a withdrawal in the morning and make it happen in the afternoon, the US Congress passed a resolution stating that the US President “shall not withdraw the United States from NATO.”¹⁰ Furthermore, in parliamentary systems, where leaders strongly depend on legislatures, legislatures play a key role in withdrawal (and other policy) decisions. In addition to leaders and parliaments, states can take IGO withdrawal decisions through referendums, as seen in the UK withdrawal from the EU (Brexit) in 2016/2020 or Greenland’s withdrawal from the European Economic Community in 1982/1985.

Therefore, since we are interested in nationalism of the state, we use measures of state nationalism.¹¹ Specifically, we use measures for executive nationalism, government nationalism, and opposition nationalism based on the Database of Political Institutions (DPI).¹² Given our theory about states, if we had only used leader nationalism as a measure, as Choi proposes, then we would have committed measurement error by omitting many components of states that go beyond a single person. Since we align our theory and measure of state nationalism, there is no measurement error as Choi alleges.

We note that some aspects of leadership are included in DPI’s measures of executive nationalism and government nationalism, as leaders reside in government and in the executive. While these DPI measures are not specific to a single individual leader, by using (broader) measures of executive and government nationalism we have tested for the role of states, which includes the leader. In short, some of our measures of state nationalism include leaders as one aspect (among others) of state nationalism.

Choi’s misrepresentation of our theory generates two other criticisms that are not warranted. Choi complains that the DPI coding for executive, government, and opposition nationalism is based on party platforms because he says a leader may be more nationalist than his or her party platform.¹³ Again, our goal is to capture state nationalism (not leader nationalism). Choi also objects to our use of “opposition nationalism.”¹⁴ We include this measure to capture the politics of the state more comprehensively: opposition forces can be an important component of state politics.

In addition to this main misrepresentation of our theory (and what it implies for measurement), we also note three other incorrect claims by Choi.

First, we do not “conflate”¹⁵ the variables for executive, government, and opposition nationalism. As detailed in our article (especially Figure 5),¹⁶ we use one aggregate measure but we also parse out these three different forms of state nationalism (executive, government, opposition) with two operationalizations for each form (a binary and an alternative vote share measure) in six separate models. The various operationalizations of state nationalism thus provide reliability and robustness checks rather than over-relying on one measure, and show a close match between the concept and measure of state nationalism.

Second, Choi’s claim that the DPI nationalism variable only captures the political right¹⁷ is incorrect. The DPI measures of nationalism include all types of government orientation,¹⁸ coding right, left, and center separately.

Third, Choi claims that “no economic variables are included in the empirical models”¹⁹ of our article. That is also wrong. As we note in our article, factors such as unemployment and economic growth could be key, which is why we included them in our article to mitigate potential omitted variable bias (page 356). We went even a step further by using matching techniques to mitigate against the common factor problem that arises when we include economic growth, which could drive both nationalism and IGO withdrawal (page 357).

The proposed substitute variable suffers from measurement error and endogeneity

Aiming to empirically test his argument about the “critical role” of leader nationalism, Choi replaces our measures of state nationalism with the V-Dem variable for government nationalism.²⁰ Ironically, instead of focusing on an individual leader (as he suggests), this variable “includes the chief executive along with the cabinet, ministries, and top civil servants.”²¹ This variable includes dozens of people. It is broader than a single leader (e.g., President Trump) and measures government nationalism more broadly. Thus, Choi has chosen a variable, government nationalism, that does not align well with his own theoretical concept of a leader. This introduces measurement error in his article.²²

Moving past Choi’s measurement error, it is important to ask why the V-Dem variable for government nationalism in Choi’s article produces different results than the DPI variables for government nationalism in our article. In a word, endogeneity. The V-Dem measure used by Choi is endogenous to withdrawals; this makes Choi’s inferences questionable. We unpack this below.

The V-Dem measure is backward looking and based on country experts’ opinions, which can be influenced by state withdrawal from IGOs. For example, coders in 2020 retrospectively estimate values for 2019 taking into account what happened in 2019, including government actions and policies. Coders’ knowledge of government actions and policies (such as withdrawal from international institutions) can therefore influence their coding of government nationalism. In fact, if prevailing sentiment is that nationalism drives IGO withdrawal, and a government withdraws from an IGO, that government is likely to be labeled as nationalist in hindsight. This is also concerning because the V-Dem variable is about legitimation strategies; one way governments can justify their regime is by taking certain actions, like making nationalist claims or leaving IGOs.

Using a measure of nationalism that is endogenous to IGO withdrawal in order to predict withdrawal is problematic precisely because withdrawal may appear (to some degree) on both sides of the equation. This can generate a positive correlation between the “independent” and dependent variable, and is a likely explanation for the positive correlation in Choi’s analysis. Such a measure – like other V-Dem variables – can have many productive uses, but it of course depends on how and for which purpose the measures are employed.

The DPI measures of nationalism that we use are instead forward looking (before IGO withdrawal occurs). These variables are coded based on documents (party platforms) to measure which policies state actors intend to pursue. In other words, the coding of the explanatory variable is exogenous to the dependent variable. We do not claim that the DPI measures are perfect for our purpose (all measures are imperfect), but they are superior to the V-Dem alternative given their exogeneity to the outcome variable. With more research and new data collection, better measures may well be produced.

In sum, the measure Choi proposes suffers from measurement error and endogeneity. Thus, concluding his article by stating that our “contention on party-based nationalism is not” upheld²³ is incorrect.

Our findings stand: Geopolitics drive IGO withdrawals across time and space

To conclude, we do not argue that nationalism plays no role in IGO withdrawals. Rather, our results indicate that while positive, the coefficients on the state nationalism variables are statistically insignificant, and other factors are statistically and substantively significant drivers of IGO exit.

However, it is essential to remind readers what our article does conclude: we find that geopolitics, such as divergence in a state’s preferences from other IGO members, increases the likelihood that a state withdraws from an IGO. Moreover, contagion effects are important:²⁴ other states become more likely to withdraw after key states exit. IGO characteristics also play a (small) role: states are less likely to leave densely democratic IGOs and more democratic countries are more likely to withdraw. These results are confirmed by Choi’s analysis: the average marginal effects are largest for contagion (geopolitics) and the effect of his (flawed) nationalism variable is statistically indistinguishable from that of preference divergence. Reassuringly, then, our article’s main findings stand.