On IGO withdrawal by states vs leaders, and exogenous measures for inference

We are pleased that our article “Hello, goodbye: When do states withdraw from international organizations?” has stimulated interest, and we welcome the opportunity to respond to Seung-Whan Choi’s attempt to challenge its conclusions. In our article, we examine why states withdraw from intergovernmental organizations (IGOs). Using what researchers know about IGO accession as a starting point, we group potential exit reasons into those related to domestic politics, IGO characteristics, and geopolitical factors. In the main analyses of our original dataset of all IGO-member state years from 1945 to 2014, we test these three sets of explanations with nine variables. We find that IGO withdrawal is mainly driven by geopolitical factors such as contagion and preference divergence, along with smaller effects related to IGO characteristics. One of the nine potential drivers of withdrawal related to nationalism; our statistical analyses show that nationalism of states is not a key driver of state withdrawals across space and time. Given the recent narrative linking nationalism and IGO withdrawal, we probed our null finding in several ways. Our article tests seven different measures of state nationalism in seven separate models. None of these coefficient estimates are statistically significant. We also conducted a textual analysis of original data on states’ public justifications for IGO withdrawals; we find that states reference factors related to nationalism in fewer than 3% of withdrawals. From these results, we concluded that https://doi.org/10.1007/s11558-021-09419-z

nationalism is not a key driver of state withdrawals from IGOs when we systematically analyze IGO exits across time and space.
Choi takes issue with these findings and proposes an eighth measure of nationalism, which he labels "leader nationalism," and proposes that this measure is a key driver of IGO withdrawals. Below we address Choi's primary criticisms, explain how they are based on misrepresentations of our article, and then show that his proposed substitute variable suffers from measurement error and endogeneity. We conclude by noting that Choi's replication confirms our original finding: historically, international cooperation and nationalism can and have existed side-by-side. While some states have said "hello" and then "goodbye" to an IGO, withdrawing is not primarily a response to state nationalism. 2 1 Criticisms are based on misrepresentations Choi proposes that nationalist leaders play a "critical role" 3 in IGO withdrawals, such as President Trump announcing the World Health Organization (WHO) withdrawal in 2020. 4 He argues that our "research risks measurement errorthe theoretical concept and the nationalism variable do not align well." 5 That is wrong. As detailed in the original article, our theory and empirics are both about nationalism of states. Our article's title and research question are "Under what conditions do states withdraw from intergovernmental organizations?" 6 Our hypotheses are about states (not leaders), 7 and our unit of analysis is the IGO-member state-year. 8 While we refer to "leaders" eight times in the article, 9 we refer to "states" more than 200 times. In other words, leaders are not central to our theory; states are.
Our article's argument is about states instead of leaders because the ability of leaders to decide on IGO withdrawals varies across states and across IGOs. State institutions limit leaders' IGO withdrawal decisions. Leaders may be an intervening variable that is part of the mechanism, but leaders are not a sufficient condition for IGO withdrawal across states. Since we are interested in theorizing and testing the drivers of withdrawal across states, IGOs, and years (rather than just, say, recent US cases where leader nationalism may play a larger role), we focus on the institutionsstatesthat ultimately execute IGO withdrawals regardless of context. Our article underscores that leaders are often constrained by hurdles that stand between threats and actual exits, including IGO procedures (e.g., waiting periods or rules surrounding payment of dues) and other state institutions (e.g., party politics or decision-making rules in the legislature). For instance, while President Trump's implicit threat to leave NATO in 2018 suggests that a leader could tweet a withdrawal in the morning and make it happen in 2 Choi (2021: Figure 1). 3 Choi (2021: 2). 4 Choi (2021: 4 9 These mentions of leaders (heads of government and state) are in reference to them making a statement on withdrawal or reneging on commitments. We also refer to the term "leaders" as in states leading other states within an international organization four times. the afternoon, the US Congress passed a resolution stating that the US President "shall not withdraw the United States from NATO." 10 Furthermore, in parliamentary systems, where leaders strongly depend on legislatures, legislatures play a key role in withdrawal (and other policy) decisions. In addition to leaders and parliaments, states can take IGO withdrawal decisions through referendums, as seen in the UK withdrawal from the EU (Brexit) in 2016/2020 or Greenland's withdrawal from the European Economic Community in 1982Community in /1985 Therefore, since we are interested in nationalism of the state, we use measures of state nationalism. 11 Specifically, we use measures for executive nationalism, government nationalism, and opposition nationalism based on the Database of Political Institutions (DPI). 12 Given our theory about states, if we had only used leader nationalism as a measure, as Choi proposes, then we would have committed measurement error by omitting many components of states that go beyond a single person. Since we align our theory and measure of state nationalism, there is no measurement error as Choi alleges.
We note that some aspects of leadership are included in DPI's measures of executive nationalism and government nationalism, as leaders reside in government and in the executive. While these DPI measures are not specific to a single individual leader, by using (broader) measures of executive and government nationalism we have tested for the role of states, which includes the leader. In short, some of our measures of state nationalism include leaders as one aspect (among others) of state nationalism.
Choi's misrepresentation of our theory generates two other criticisms that are not warranted. Choi complains that the DPI coding for executive, government, and opposition nationalism is based on party platforms because he says a leader may be more nationalist than his or her party platform. 13 Again, our goal is to capture state nationalism (not leader nationalism). Choi also objects to our use of "opposition nationalism." 14 We include this measure to capture the politics of the state more comprehensively: opposition forces can be an important component of state politics.
In addition to this main misrepresentation of our theory (and what it implies for measurement), we also note three other incorrect claims by Choi.
First, we do not "conflate" 15 the variables for executive, government, and opposition nationalism. As detailed in our article (especially Figure 5), 16 we use one aggregate measure but we also parse out these three different forms of state nationalism (executive, government, opposition) with two operationalizations for each form (a binary and an alternative vote share measure) in six separate models. The various operationalizations of state nationalism thus provide reliability and robustness checks 10 The US Congress passed H.R. 676, the NATO Support Act, on 22 January 2019. Moreover, the NATO treatyas many other treatiesprovides a one year waiting period from withdrawal announcement to execution, a time frame in which state institutions (the courts or legislature, for example) can prevent a leader's threat from becoming reality. The US has not withdrawn from NATO as of 2020. 11 King et al. (1994: 155), "the researcher's purpose must determine the choice that is made." 12 Cruz et al. (2018). 13 Choi (2021: 4). 14 Choi (2021: 4). 15 Choi ( rather than over-relying on one measure, and show a close match between the concept and measure of state nationalism.
Second, Choi's claim that the DPI nationalism variable only captures the political right 17 is incorrect. The DPI measures of nationalism include all types of government orientation, 18 coding right, left, and center separately.
Third, Choi claims that "no economic variables are included in the empirical models" 19 of our article. That is also wrong. As we note in our article, factors such as unemployment and economic growth could be key, which is why we included them in our article to mitigate potential omitted variable bias (page 356). We went even a step further by using matching techniques to mitigate against the common factor problem that arises when we include economic growth, which could drive both nationalism and IGO withdrawal (page 357).

The proposed substitute variable suffers from measurement error and endogeneity
Aiming to empirically test his argument about the "critical role" of leader nationalism, Choi replaces our measures of state nationalism with the V-Dem variable for government nationalism. 20 Ironically, instead of focusing on an individual leader (as he suggests), this variable "includes the chief executive along with the cabinet, ministries, and top civil servants." 21 This variable includes dozens of people. It is broader than a single leader (e.g., President Trump) and measures government nationalism more broadly. Thus, Choi has chosen a variable, government nationalism, that does not align well with his own theoretical concept of a leader. This introduces measurement error in his article. 22 Moving past Choi's measurement error, it is important to ask why the V-Dem variable for government nationalism in Choi's article produces different results than the DPI variables for government nationalism in our article. In a word, endogeneity. The V-Dem measure used by Choi is endogenous to withdrawals; this makes Choi's inferences questionable. We unpack this below.
The V-Dem measure is backward looking and based on country experts' opinions, which can be influenced by state withdrawal from IGOs. For example, coders in 2020 retrospectively estimate values for 2019 taking into account what happened in 2019, including government actions and policies. Coders' knowledge of government actions and policies (such as withdrawal from international institutions) can therefore influence their coding of government nationalism. In fact, if prevailing sentiment is that 17 Choi (2021: 4): "the notion that nationalism is limited to the political right does not reflect reality." 18 For example, DPI also codes nationalist but leftist executive in Angola since 1976, Libya since 1975, and Slovakia in the 1990s. 19 Choi (2021: 9). 20 Choi uses the V-Dem variable "v2exl_legitideolcr_0" which is the survey question: "to what extent does the current government promote a specific ideology [here: nationalism] … in order to justify the regime in place" (Coppedge et al. 2020: 208). This variable was not yet available when our original article was published. But even if it had been available, we would have had reservations about using it to predict state withdrawals for the reasons detailed here. 21 Coppedge et al. (2020: 208). Emphasis added. 22 King et al. (1994); von Borzyskowski and Wahman (2021). nationalism drives IGO withdrawal, and a government withdraws from an IGO, that government is likely to be labeled as nationalist in hindsight. This is also concerning because the V-Dem variable is about legitimation strategies; one way governments can justify their regime is by taking certain actions, like making nationalist claims or leaving IGOs.
Using a measure of nationalism that is endogenous to IGO withdrawal in order to predict withdrawal is problematic precisely because withdrawal may appear (to some degree) on both sides of the equation. This can generate a positive correlation between the "independent" and dependent variable, and is a likely explanation for the positive correlation in Choi's analysis. Such a measurelike other V-Dem variablescan have many productive uses, but it of course depends on how and for which purpose the measures are employed.
The DPI measures of nationalism that we use are instead forward looking (before IGO withdrawal occurs). These variables are coded based on documents (party platforms) to measure which policies state actors intend to pursue. In other words, the coding of the explanatory variable is exogenous to the dependent variable. We do not claim that the DPI measures are perfect for our purpose (all measures are imperfect), but they are superior to the V-Dem alternative given their exogeneity to the outcome variable. With more research and new data collection, better measures may well be produced.
In sum, the measure Choi proposes suffers from measurement error and endogeneity. Thus, concluding his article by stating that our "contention on partybased nationalism is not" upheld 23 is incorrect.

Our findings stand: Geopolitics drive IGO withdrawals across time and space
To conclude, we do not argue that nationalism plays no role in IGO withdrawals. Rather, our results indicate that while positive, the coefficients on the state nationalism variables are statistically insignificant, and other factors are statistically and substantively significant drivers of IGO exit.
However, it is essential to remind readers what our article does conclude: we find that geopolitics, such as divergence in a state's preferences from other IGO members, increases the likelihood that a state withdraws from an IGO. Moreover, contagion effects are important: 24 other states become more likely to withdraw after key states exit. IGO characteristics also play a (small) role: states are less likely to leave densely democratic IGOs and more democratic countries are more likely to withdraw. These results are confirmed by Choi's analysis: the average marginal effects are largest for contagion (geopolitics) and the effect of his (flawed) nationalism variable is statistically indistinguishable from that of preference divergence. Reassuringly, then, our article's main findings stand. 23 Choi (2021: 11). 24 Recent studies further emphasize the importance of contagion risks; see De Vries et al. (2021).
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.