Stability and change in international policy-making: A punctuated equilibrium approach

Abstract

International organizations (IOs) have developed into important policy venues beyond the state. Yet our understanding of the broader dynamics of IO policy-making is limited. This article offers the first comparative analysis of macro patterns in IO policy-making. Theoretically, we draw on punctuated equilibrium theory to develop hypotheses about stability and change in the orientation of IO policy agendas. Empirically, we examine novel data on the policy output of five general-purpose IOs between 1980 and 2015, combining statistical analysis and comparative case illustrations. The analysis yields two central results. First, the policy agendas of all five IOs display patterns of punctuated equilibria, with longer periods of stability interrupted by shorter periods of dramatic change. Second, the level of institutional friction in decision-making contributes to variation in punctuations across IOs and within IOs over time. The results suggest four broader implications: (1) punctuated equilibrium theory applies to a broader empirical domain than previously thought; (2) patterns of change in IOs are more complex than conventionally expected; (3) institutional friction matters for IOs’ responsiveness to societal demands and problem pressures; and (4) deeper integration of punctuated equilibrium theory into the study of IOs can pave the way for a promising IR research agenda.

Over recent decades, international organizations (IOs) have developed into important policy venues beyond the nation state. Not only have IOs grown in numbers, but more importantly, these organizations have been vested with more far-reaching policy-making authority in an increasing range of domains (Hooghe et al. forthcoming). While some of these IOs are task-specific organizations that specialize in particular policy domains, such as the International Atomic Energy Association (IAEA), others are general-purpose organizations that cover the full range of problems confronting a set of member states, such as the European Union (EU) (Lenz et al. 2015). Part of this expansion of international authority has come at the expense of national policy-making, while another part responds to new problems not previously dealt with domestically. Accordingly, it has become common to speak of “global health policy,” “global trade policy,” and “global environmental policy”.

Yet, despite this development, we know little about the broader dynamics of IO policy-making. Are IO policy agendas marked by stability and inertia or by openness to change, as new issues and ideas produce shifts in political attention? When change occurs, is this a smooth and seamless process of adjustment or a disruptive and dramatic break with the status quo? What determines whether IOs are capable of flexibly adapting their policy agendas to new information? These are issues of great importance for the theory and practice of policy-making in IOs. Ultimately, it is a question of how responsive IOs are to changing societal demands and pressing global problems. If IOs’ policy responses are slow, irregular or exaggerated, their effectiveness in mitigating urgent societal problems will suffer. This goes particularly for problems marked by crisis and urgency, such as epidemics, famine, financial instability, and military escalation, where efficient policy responses are essential. Yet, so far, the dynamics of IO policy-making have received limited systematic attention.

To gain traction on these matters, this article engages in the first comparative analysis of macro patterns in IO policy-making. We focus on IO policy agendas, understood as the allocation of policy output across policy topics, and ask two specific questions. First, what dynamics of stability and change characterize the policy agendas of IOs? Second, what explains variation in these dynamics across IOs and over time?

Theoretically, we turn to the study of policy agendas in American and Comparative Public Policy for inspiration. The most powerful expectation in this area originates from punctuated equilibrium theory (PET), originally developed by Baumgartner and Jones (1993) and subsequently refined by others (e.g., Baumgartner et al. 2006; Jones and Baumgartner 2005; Jones et al. 2003). According to PET, political attention is scarce and institutions are marred by friction, leading to policy agendas characterized by periods of stability, when there is little or no change, and periodic punctuations, marked by dramatic and rapid change. Empirical research in American and Comparative Public Policy has consistently shown that policy dynamics tend to follow the distinct pattern predicted by PET. In International Relations (IR), the notion of punctuated equilibrium has informed analyses of institutional change (Colgan et al. 2011; Goertz 2003; Krasner 1984; Young 2010). However, PET’s expectations about stability and change in the orientation of policy-making have only been evaluated in single-IO studies and never comparatively tested on IOs.¹

Empirically, we examine the policy output of five general-purpose IOs between 1980 and 2015, based on newly gathered data on nearly 20,000 individual policy decisions. The IOs are the African Union (AU), EU, Organization of American States (OAS), Organization of Islamic Cooperation (OIC), and United Nations (UN). These IOs all have a multi-issue orientation, produce comparable policy output, and cover a wide geographic scope. The empirical analysis maps the overall patterns in IO policy-making orientation, evaluates the presence of policy punctuations, and uses controlled comparisons both across and within IOs to assess explanations of variation in punctuation.

The analysis reveals two principal findings. First, all five IOs have policy agendas that reflect the pattern of punctuated equilibria predicted by PET. Frequency distributions of year-on-year policy changes exhibit the presence of agenda punctuations: most of the time, agendas remain stable, but occasionally, there are dramatic shifts in the attention awarded a particular policy area. Second, the degree of punctuation in IO policy agendas is conditioned by institutional friction. The pattern of punctuated equilibria is more pronounced when IO decision-making is associated with higher institutional friction, in terms of demanding decision rules and large memberships with heterogeneous preferences. When institutional friction is higher, policy agendas are marked by greater stability combined with more dramatic change when shifts actually occur. Conversely, when institutional friction is lower, policy agendas change more continuously with fewer disruptive large-scale shifts.

These results suggest four broader implications for research on public policy and global governance, further developed in the conclusion. First, they demonstrate that PET’s scope of applicability is broader than previously thought. While repeated studies have shown that PET predicts policy dynamics in diverse national contexts, our findings point to applicability beyond the theory’s traditional home domain. The similarities between national and international policy-making on key dimensions appear more powerful than the differences.

Second, the results suggest a more nuanced understanding of IOs’ propensity for change. While it has been common in IR to emphasize why IOs should be resistant to change, we join a growing group of scholars identifying and explaining varying patterns of change in global governance. The central question should not be whether IOs are amenable to policy change – they clearly are – but what change looks like and under what conditions it is more likely to occur. This article shows that change typically is punctuated and that IOs’ internal organization influences the likelihood and nature of change.

Third, the findings speak to an emerging literature on the performance of IOs, suggesting that IOs are more likely to be responsive to changing societal demands and problem pressures when institutional friction is lower. IOs with laxer decision-making rules, fewer member states, and more homogenous memberships are better positioned to respond efficiently to urgent societal problems. The implication is that member states, through their control of institutional design, can shape IOs’ capacity for efficient policy response and, ultimately, effective problem-solving.

Fourth, the results suggest that deeper integration of PET into the study of IOs can pave the way for a promising IR research agenda. PET offers theoretical and methodological tools that allow us to gain traction on macro patterns of change and stability in international policy-making. While we make some inroads on this agenda, our findings also point to a need for further examination of the scope conditions and mechanisms shaping the punctuated characteristics of IO policy-making.

1 The argument: Punctuated equilibrium theory and international policy-making

We suggest that PET may offer a fruitful approach for understanding the policy dynamics of IOs. In the following, we first outline the general tenets of PET, then explain why this approach may or may not apply to IOs, and finally identify specific hypotheses for empirical testing.

1.1 Punctuated equilibrium theory

Baumgartner and Jones (1993) developed PET to explain the tendency of policy agendas to be stable most of the time with interruptions of periodic abrupt change. PET builds on the assumption that governmental attention is scarce. While there are constantly ongoing social processes “out there” demanding political attention, governments have a limited capacity to attend to all policy problems. Cognitive limitations to information processing and “institutional friction” in decision-making constrain governments’ ability to respond smoothly to continually changing social processes (Jones and Baumgartner 2005). Over time, agendas will therefore change in a punctuated pattern as policy-makers respond irregularly to new information and infrequently overcome institutional hurdles. Long periods of inattention will be supplanted by short periods of attentiveness in a pattern of infrequent and disproportionate response. The statistical signature of such a pattern is a leptokurtic distribution of aggregated policy changes over an extended period of time: more cases of no or little change, combined with more cases of extreme changes, compared to a normal distribution of changes (Baumgartner et al. 2009).

This pattern of punctuated equilibria contrasts with the image of a hypothetical smoothly functioning organization in which the policy agenda changes efficiently as new information about the state of the world is seamlessly integrated. It has been demonstrated that if such an organization responds to multiple, independent social processes over longer periods of time, the distribution of its cumulated agenda changes would be normal (Jones and Baumgartner 2005). Societal problems requiring policy will not all occur simultaneously nor all necessitate the same degree of change in policy. Rather, the timing and degree of required change will be random and diverse. Many problems will require moderate adaptations to policy, while few will demand large-scale or minor policy change. Consequently, a hypothetical organization that can smoothly adapt policy to real world dynamics of diverse and multiple problems will exhibit a distribution of agenda changes that is normally distributed. Importantly, as per the Central Limit Theorem, the normal distribution of aggregate policy changes would result even if individual social processes, such as armed conflicts or epidemics, are non-normal or badly skewed (Jones and Baumgartner 2005, 127ff; see Padgett 1980 for a mathematical proof).

Hence, while individual punctuations may result from exogenous events (e.g., the sweeping policy changes that followed the 9/11 attacks in the United States), such events cannot explain the aggregate pattern of long-term changes in organizations exposed to multiple and independent problems. Historical evidence demonstrates that there is not a one-to-one relationship between exogenous events and policy responses: Even strong external forces, on their own, frequently fail to trigger immediate reactions (e.g., the delayed response to climate change), and dramatic policy changes can follow from endogenous mobilization that occurs in the absence of clear external events (e.g., the British government’s decision to hold a referendum on EU membership). Because external events cannot explain aggregate patterns in policy punctuations, PET places emphasis on how the internal characteristics of an organization contribute to the characteristic pattern of underreaction mixed with occasional overreaction (Jones and Baumgartner 2005).

More specifically, PET suggests that an organization’s internal institutional friction is central to explaining varying levels of punctuation across settings (Baumgartner et al. 2009; Jones and Baumgartner 2005). In abstract terms, institutional friction can be understood as the costs involved in the translation of policy inputs into policy outputs (Jones et al. 2003). In concrete terms, institutional friction consists of the hurdles built into policy-making through divisions of power and procedural thresholds for policy adoption. The degree of friction in a system matters because “friction causes the linkage between inputs and outputs of the system to be disproportionate – underresponse because of friction, then overresponse in response to built-up pressures” (Baumgartner et al. 2009: 607). As a consequence, even under similarly complex social pressures and policy problems, organizations with higher institutional friction are associated with more punctuated policy-making than organizations with lower institutional friction.

1.2 Applicability to international policy-making

This dynamic of punctuated equilibria has been established in several studies on American public policy (Baumgartner and Jones 1993; Jones and Baumgartner 2005), comparative public policy (Baumgartner et al. 2009), and more recently EU public policy (Alexandrova et al. 2012). In addition, punctuated equilibria have been observed in a range of different policy fields, including public health policy (Baumgartner and Jones 1993), environmental policy (Repetto 2006), and welfare policy (Jensen 2009).

Yet, until now, policy punctuations have not been systematically and comparatively studied in the international domain. There may be good reasons for this. After all, there are distinct differences between national and international policy-making. We cannot assume PET automatically extends to the international domain. Compared to national policy-making, international policy-making is usually (a) more decentralized and less hierarchical, as decisions are taken by a collective principal composed of member governments with their internal processes of preference formation and decision-making (Lyne et al. 2006); (b) less exposed to public pressure, as international issues normally are less politically salient and international decision-makers are not accountable to electorates in the same way (Held and Koenig-Archibugi 2005); and (c) less subject to clear divisions of responsibility, as IOs have more fluid and overlapping mandates (Alter and Meunier 2009).

At the same time, there are multiple similarities between national and international policy-making, suggesting that PET may have explanatory power in the latter setting as well. Importantly, the central conditions contributing to punctuated equilibria in the national context are present in the international domain as well. First, IOs are policy venues, just like the national political systems for which PET was originally formulated. Over recent decades, IOs have gained growing decision-making authority in an ever-increasing range of domains (Hooghe et al. forthcoming). As Barnett and Finnemore state, IOs nowadays “make authoritative decisions that reach every corner of the globe and affect areas as public as governmental spending and as private as reproductive rights” (2004, 1).

Second, international policy-making, similar to domestic contexts, is subject to constant demands for policy in response to ongoing social processes. Wars break out, famines occur, epidemics spread, financial systems tumble, migration flows intensify, technological shifts take place, environmental disasters happen, and demographics change. Oftentimes, the problems that concern national and international policy-making are the same (cf. Baumgartner et al. 2006, 605–606). Information on societal problems and pressures for action reach IOs through multiple channels: member governments representing domestic constituencies (Moravcsik 1997); supranational bureaucracies with interests in international policy response (Johnson 2014; Barnett and Finnemore 2004); and transnational actors, such as NGOs, leveraging IOs for international action (Keck and Sikkink 1998).

Third, much like national political systems, IOs are prone to capacity limitations that reduce their efficiency in responding to changing conditions (Barnett and Finnemore 2004). IO policy-makers confront cognitive limitations in terms of bounded rationality (Jupille et al. 2013; Poulsen 2014) and institutional friction in the shape of demanding thresholds for policy adoption (Scharpf 1988; Blake and Payton 2015). As a result, IOs typically experience attention scarcity as the number of issues deserving political attention exceeds the capacity of policy-makers to develop new policy. This goes particularly for general-purpose IOs, which share the encompassing policy scope of national political systems (Lenz et al. 2015).

Whether PET is applicable to international policy-making is an empirical issue. Assessing this will help us better understand the specific dynamics of international policy-making and yield important information about the possible scope conditions of PET.

1.3 Theoretical expectations

We build on PET to formulate two core hypotheses about international policy-making – one descriptive and one explanatory. First, PET posits that one of the key dynamics of policy agendas is that they follow patterns of punctuation where stability alternates with radical change. Policy-makers’ cognitive limitations and institutional hurdles to decision-making cause policy agendas to tend toward relative stability. Stability is occasionally broken when attention shifts from one issue to another, for instance, because a large-scale event occurs or new information from member governments, supranational bureaucracies, and transnational actors reaches policy-makers. Focusing on the overall pattern of policy-making, rather than the causes of individual punctuations, PET leads to an expectation of long periods of stability in policy agendas punctuated by rapid change. Since international policy-making shares the central conditions contributing to punctuated equilibria, as described above, we hypothesize:

• H1: The policy agendas of IOs display patterns of punctuation, combining longer periods of stability with shorter periods of radical change.

Second, PET claims that institutional friction conditions the degree of punctuation across organizational contexts. What is institutional friction in the context of international policy-making? We suggest that three factors comprise institutional friction for international policy-making: decision rules, membership size, and preference heterogeneity. This expectation is firmly anchored in rational institutionalist scholarship on national (Scharpf 1988; Tsebelis 2002) and international (Lake and Powell 1999) policy processes.

To begin with, IOs vary in their institutional rules on decision-making (Blake and Payton 2015). Decision rules can contribute to friction, affecting the ability to arrive at policy decisions (Scharpf 1988; Tsebelis and Yataganas 2002). IOs with unanimity rules confront greater hurdles to decision-making, compared to those with majority voting and weighted voting. For example, both the UN Security Council and the WTO are IOs where demanding decision rules raise barriers to efficient decision-making.

In addition, membership size serves as a source of institutional friction. The number of actors is widely understood as shaping the likelihood and nature of international cooperation (Axelrod and Keohane 1985; Koremenos et al. 2001). For IOs, the number of member states can likewise affect the ease of decision-making. All else equal, the transaction costs for arriving at a decision in a large multilateral setting with more than 50 member states, like the Organization of Islamic Cooperation, are larger than those in a small minilateral setting with few member states, such as the Nordic Council of Ministers.

Finally, preference heterogeneity among member states contributes to institutional friction. Previous research has shown that the distribution of state preferences affects institutional design, institutional change, and delegation in international politics (Axelrod and Keohane 1985; Hawkins et al. 2006; Koremenos et al. 2001). When preferences diverge, agreement on new or revised policies will be more difficult to achieve, all else equal. Examples include the UN Security Council, where rapprochement between the permanent members after the end of the Cold War led to a noticeable change in policy output (Malone 2004), and the EU, where the relative preference heterogeneity among its member states has affected the speed of decision-making over time (König 2007).

In line with the expectation that institutional friction conditions the degree of punctuation across organizational contexts, we hypothesize:

• H2: Policy agendas of IOs subject to more demanding decision-making rules, larger memberships or more heterogeneous preferences will be characterized by a higher degree of punctuation.

Our hypotheses draw attention to descriptive (H1) and explanatory (H2) patterns in IO policy dynamics. If PET applies to IOs, as per H1, we would expect to see patterns of policy punctuations among IOs. Empirically, policy punctuations are observed by studying period-to-period policy change distributions. Our confidence in H1 would be increased if we observe IO policy change distributions with a leptokurtic pattern of “tall peaks” and “long tails,” deviating from the normal distribution expected under the null hypothesis of a perfectly responsive IO. If PET can explain variation in punctuation, as per H2, we would expect to see that IOs with higher institutional friction are systematically different from IOs with lower institutional friction, and that shifts in IOs’ institutional friction over time produce corresponding changes in punctuation patterns. Empirically, high-friction IOs are then expected to exhibit period-to-period change distributions that are more peaked and have longer tails than low-friction IOs.

2 Data on IO policy agendas

To test these hypotheses, we gathered separate datasets on the orientation of policy output of five IOs: AU, EU, OIC, OAS, and UN. This sample of IOs provides a suitable starting point for the comparative analysis of international policy-making for three reasons. First, it exclusively includes general-purpose IOs with a broad policy agenda, as opposed to task-specific IOs that are focused on narrow set of issues (Lenz et al. 2015). General-purpose IOs are the most appropriate for a first evaluation of PET in the international setting, partly because a broad policy scope is required to study shifts in political attention across policy topics, and partly because these IOs are those that most approximate the all-purpose orientation of national political systems. Second, while these IOs exhibit varying institutional design and friction, their policy output has comparable format and style: all adopt resolutions (or resolution-like decisions) separated by issue and sector, as compared with IOs that adopt decisions in an aggregate manner and present them to the world in the form of communiqués.² Third, the sample has a wide geographic scope, including the paramount global IO, the UN, and key regional IOs on all major continents. In short, this is a geographically inclusive sample of general-purpose IOs that produce policy output sufficiently similar to engage in systematic comparison. This selection of IOs enables controlled comparison because these IOs are similar along many dimensions yet vary in terms of key factors comprising institutional friction and (as we will show) the degree of policy punctuation.

For each IO, we gathered data on key policy outputs (acts) from the principal intergovernmental decision-making body with IO-wide membership in the period from 1980 to 2015, as summarized in Table 1.³ Our choice of bodies ensures that the policy output emerges at a comparable stage of the policy process (from input to output), diminishing the risk that H2 is falsely accepted because of stage-dependent variation in decision-making costs (see Baumgartner et al. 2009). In total, the dataset contains 19,061 policy acts with a total length exceeding 100,000 pages.

Table 1 Overview of data sources

To identify the allocation of attention across each IO’s policy agenda, we first assigned a topic code to each policy act based on computer-assisted content analysis. We employed a keyword topic classifier (Weber 1990) and a list of topic codes for the main policy areas in global governance (see Appendix 1). Similar to the approach of the Comparative Agendas Project (CAP) (Bevan 2014), we divide the policy space into a set of general categories that are subsequently divided into more specialized policy domains, each with its own sub-code.⁴ The general categories cover the key policy concerns in global governance, including economic development, human rights, and security, but also certain domestic concerns that are subject to frequent international regulation and coordination, such as health policy or labor policy.

We assigned topics to individual policy acts based on the relative frequency of key words drawn from newly constructed topic-specific dictionaries for each of our 16 main topic codes.⁵ In constructing the dictionaries, we analyzed existing policy research dictionaries (e.g., CAP), policy-specific literature in global governance, and IO policy documentation. To increase specificity, the dictionary for each topic contains both 1- and 2-grams. For example, the dictionary for the “Security and defense” category contains both “conflict” and “conflict prevention,” and the dictionary for the “Environment and natural resource management” lists both “climate” and “climate change”.

In accordance with standard practice, we prepared documents for quantitative analysis by carrying out a series of text transformations, including stemming (the removal of word ends), tokenization (turning the text into a “bag of words”), and the removal of punctuation, capitalization, and very common words (so-called “stop words”). For each of the five IOs, we then classified the processed documents by topic, assigning each document the topic code that best corresponded to the content based on keyword frequency. Aggregating across years, we then calculated the distribution of agenda space devoted to a single policy topic, defined as the proportion of policy acts assigned to that topic in relation to the aggregate policy production in the given year.

We evaluated the reliability of the classification algorithm by comparing it to hand-coding for the EU, OAS, OIC, and AU (OAU). For each of these, we randomly selected five years and hand-coded all policy output for those years (between 247 and 400 documents per IO), assigning topic codes based on manual semantic analysis. When compared with the hand-coding, the keyword classifier yielded an average precision of 78% (proportion estimates within 5 percentage points), varying between 69 and 88% depending on IO.⁶

3 Results

We begin by reporting overall patterns in IO agendas, before assessing the presence of policy punctuations (H1) and explanations for variation in punctuation across IOs (H2). We support our analysis of H2 by reporting findings from an analysis of how IO-specific shifts in institutional friction affect the degree of punctuation, including three case illustrations.

3.1 IO agenda patterns

Data on overall agenda patterns presented in Table 2 and Fig. 1 provide some first insights into the macro patterns of IO policy-making. First, all IOs have fairly concentrated agendas, where a few topics dominate the policy output. Across the IOs, the top five topics account for between 67% (EU) and 83% (UN) of the agenda. Second, there is a significant amount of overlap in agenda orientation across IOs. The top concerns of global governance – economic development, human rights and security – are found among the major topics on all IO agendas except that of the EU. Reflecting the deeper integration of its member states, the EU has an agenda oriented slightly more toward national policy areas, such as energy and transport, science, and labor.

Table 2 Distribution of policy topics on IO agendas, 1980–2015

Fig. 1

Agenda allocation in five general-purpose IOs, 1980–2015

Third, within each IO, there is considerable temporal variation in the share of attention devoted to individual topics (as well as the volume of policy output) (Fig. 1). There is evidence of both stability and change in the agendas of each of the five IOs. All IOs display periods of attention stability, in which the allocation of output across topics remains proportional, even though the overall volume of the agenda expands or contracts. These periods of relative agenda stability include, for example, the UN in the 2000s and the EU between 1995 and 2005. Change, however, is frequent. The data contain both signs of gradual change, such as the steady rise in attention awarded by the OAS to the topic of human rights, and of radical change, such as the dramatic increase in AU’s attention to labor issues after 2005, and the near disappearance of security from the agenda of the OAS.

In sum, it is evident that all IOs have agendas that are subject to a considerable amount of fluctuation over time in terms of attention to specific topics. This pattern points to the possible presence of policy punctuations. To ascertain whether this is the case, we carry out further tests.

3.2 Policy punctuations in IO agendas

The conventional method to identify the presence of agenda punctuations is stochastic process analysis of population-level data. Rather than engaging in point prediction of individual punctuations, this approach recognizes punctuations based on the characteristics of frequency plots of annual changes in agenda attention (Jones 2005; Jones et al. 2009). To generate year-to-year change distributions, we calculated the percentage-percentage shift in policy-making attention for each year between 1980 and 2015.⁷ To diminish the risk that data processing induced artificial patterns of stability or dramatic change, which may lead us to falsely accept our first hypothesis, we made two conservative modifications to the raw data. First, we pooled topics, including only the ten most frequent topic codes for each IO, grouping outstanding items under an “Other” category.⁸ Second, we excluded observations where agenda attention remained stable at zero, i.e., topics that remained “off the radar” for longer periods of time (cf. John and Bevan 2012).

We evaluate the presence of policy punctuations via frequency distribution plots and distributional shape statistics. Figure 2 presents histograms of year-on-year changes, with the magnitude of change on the x-axis and the count of observations on the y-axis. For purposes of comparison, we superimpose a normal curve with a mean and standard deviation identical to the given distribution. In a policy process without friction, we would expect a distribution of period-to-period changes to overlap with the normal curve, as policy responds flexibly to changes in the state of the world. In contrast, for a policy process characterized by institutional friction and punctuation, we would expect a distribution that is leptokurtic; that is, a distribution with a tall peak and thicker tails.

Fig. 2

Year-on-year percentage agenda changes, 5 IOs, 1980–2015

The change distributions in the data on the five IOs suggest general leptokurtic tendencies. For all IOs, we observe a high number of observations around zero, which corresponds to periods in which the attention awarded to a topic remained stable. These observations reflect an IO policy agenda in equilibrium. However, the distributions also contain a higher than expected (under the normal distribution) frequency of observations at either extreme, including some observations that are very far off on the right tail. These observations signify dramatic changes in the allocation of attention to a particular issue: punctuations of the equilibrium. For example, a score of +200% indicates a tripling of attention whereas a score of −100% indicates that a topic has been dropped from the agenda completely.

The observed patterns of slender peaks and extended, thicker tails suggest that the distributions are leptokurtic. This finding is further borne out by distribution shape statistics, presented in Table 3. We include statistics for both kurtosis and L-kurtosis, as both are commonly used in the literature, but we attach more weight to the L-kurtosis scores, which are robust to smaller sample sizes (Royston 1992).

Table 3 Shape statistics for frequency distributions of year-on-year changes in policy attention, 1980–2015

A normal distribution has a kurtosis of 3 and a L-kurtosis of 0.123. Higher values mean that a distribution is leptokurtic (high peak and thick tails); lower values that it is platykurtic (low peak and thin tails). All of the five IOs exhibit kurtosis and L-kurtosis scores well above what would be expected had the data been generated from a normal distribution. The smallest L-kurtosis value, 0.26 for the EU, is already a significant deviation from 0.123, and the largest values, 0.31 for the OIC, suggest marked leptokurtic properties. Non-normality is further confirmed by Shapiro-Wilk and Kolmogorov-Smirnov tests for the five IOs, all of which comfortably reject the null hypothesis of normal distributions.

The EU’s lower kurtosis is interpretable visually in Fig. 2. While the EU’s distribution appears jagged, its share of observations in the area between zero and either extreme is higher than any of the other IOs, which are more clearly dominated by observations at the middle and extremes—the telltale sign of the leptokurtic distributions associated with punctuated agendas.

To place our results in the context of previous scholarship, we compare the L-kurtosis statistics with Baumgartner et al. (2009), who report that the change distributions for parliamentary bills, laws, and statutes in Denmark, Belgium, and the United States have L-kurtosis scores of 0.26, 0.29, and 0.25, respectively. These are national policy outputs that emerge at a stage in the policy process that is comparable to the adoption of resolutions and directives by IOs. We note that the magnitude of our kurtosis scores is somewhat higher, on average, but overall in line with the previous results for comparable output distributions at the national level.

In sum, the evidence presented above is congruent with what we would expect from the logic of policy punctuations. We observe annual change distributions with tall peaks and thick tails, as expected for agendas that change little for long periods of time but occasionally are subject to dramatic change. Hence, the data are strongly indicative of the presence of punctuated equilibria in these IOs (H1).

3.3 Comparative analysis

The data suggest that there is variation across the five IOs with respect to the degree of punctuation. The distribution of the EU exhibited lower leptokurtosis than any of the other IOs, whereas the OIC had the highest score. What could explain this variation? Previous research has demonstrated a correlation between institutional friction and leptokurtic distributions in outputs (Baumgartner et al. 2009; Jones et al. 2009). In line with H2, we proceed to test if the same association can be found in our data. While institutional friction conceptually includes cognitive limitations and institutional barriers (Jones and Baumgartner 2005), actual tests have focused on the latter (Jones et al. 2009). We follow that practice.

Above, we identified three sources of institutional friction in IOs: decision rules, membership size, and preference heterogeneity. We do not claim to have identified a final and exhaustive list of factors that drive institutional friction in IOs, but we think that these factors provide a reasonable starting point. In the quantitative analysis, we follow a procedure used by Jones et al. (2009) to measure and rank each IO on the three variables, which yields an additive institutional friction index (Table 4). When we turn to case illustrations, we are able to go beyond this narrow operationalization and integrate other aspects of institutional friction, such as member state rivalries, which allows us to examine further implications of the theory.

Table 4 Indicators of institutional friction in IOs

Decision rules is a measure of the degree to which member state preferences must align before a decision can be adopted, as manifested in IO voting rules. The lowest barriers are raised in IOs that employ simple majority voting, whereas the most restrictive rules require unanimity, that is, all member states must agree. Our measure thus reflects the proportion of votes required for adopting a decision, and the extent to which more flexible decision-making processes (e.g., shifting from a requirement of unanimity to majority voting) could be invoked (Tsebelis and Yataganas 2002).

Membership size is the count of IO member states. All else equal, a higher number of member states means higher transaction costs in decision-making. We sourced observations on membership size from the Correlates of War IGO dataset (Pevehouse et al. 2004) and, where necessary, IO-specific research.

Preference heterogeneity is an ordinal composite measure derived via a deductive method, seeing preferences as determined by a basket of observable member state characteristics (see Frieden 1999: 61ff), including income level, economic size, political regime, population, and cultural composition.⁹ This approach rests on the assumption that countries that vary along these factors are likely to hold different views on core matters discussed within IOs, such as economic development (e.g., redistribution between rich and poor countries), security (e.g., the relative contribution of small and large countries in the provision of security), and normative concerns (e.g., negotiations on human rights between liberal democracies and authoritarian regimes). The greater the heterogeneity among member states along these five observable dimensions, the greater the deducted preference heterogeneity.¹⁰

The results suggest that there is a significant amount of variation in institutional friction across IOs. Of the five organizations, the EU has the lowest institutional friction, primarily a function of a smaller and comparatively homogeneous membership.¹¹ The OIC and the UN, two large and heterogeneous organizations, are associated with the highest institutional friction.

To evaluate the presence of a correlation between institutional friction and leptokurtosis, we plot the two variables in Fig. 3. With only five data points at this aggregate level of analysis,¹² our sample size is limited, warranting caution in interpretation. Yet, the plot suggests a correlation between institutional friction, as approximated by our ordinal ranking, and L-kurtosis statistics. The best-fitting line has a positive slope.¹³ The conclusion is that institutional arrangements that inject friction into decision-making processes appear to be associated with a higher likelihood of punctuations, lending support to H2.

Fig. 3

The relationship between institutional friction and L-kurtosis

It is possible that our additive measure obscures underlying variation. To evaluate the impact of the different factors included in the additive friction index, we plot the values of the three variables decision rules, membership size, and preference heterogeneity against the leptokurtosis values in Fig. 4. This more fine-grained analysis reveals that the correlation is most distinct (r = 0.66) with respect to decision rules, the factor that most immediately captures institutional friction. The correlation is less clear with respect to membership size and least noticeable with respect to preference heterogeneity. In both of the latter cases, the UN, with its uniquely large and heterogeneous membership, is an outlier. If the UN is removed from the sample, the relationship between membership size and leptokurtosis becomes strongly linear (r = 0.97), and that between preference heterogeneity and leptokurtosis more distinct (r = 0.43).

Fig. 4

The relationship between individual measures of institutional friction and L-kurtosis

Overall, the data is thus congruent with the hypothesized association between the factors comprising institutional friction and policy punctuations. Both visual inspection of cross-plots and regression statistics confirm that IOs with more institutional friction tend to have higher degrees of punctuation, as measured by leptokurtosis in change distributions.

3.4 Longitudinal analysis

To further evaluate the link between institutional friction and agenda patterns, we engage in longitudinal within-case comparisons of how IO-specific shifts in institutional friction translate into changes in punctuation. Such longitudinal comparisons allow us to explore how a change in one of the factors comprising institutional friction affects the policy agendas, while holding the organizational context constant.

Table 5 presents an overview of this analysis, detailing the year and relevant context of the shift in institutional friction and the resulting change in L-kurtosis. These results are interesting in two main respects. First, while the cutoffs vary between the IOs, it is clear that there is no common trend in IO policy-making: some IOs exhibit accentuating punctuation levels, while others have moved toward lower levels. In other words, viewed as a group, IOs have not changed consistently in terms of policy punctuations, suggesting that variation is due to IO-specific factors and not an exogenous factor that leads IOs to develop smoother and more efficient policy-making over time.

Table 5 Longitudinal analysis

Second, shifts in agenda patterns are intelligible in light of changes in institutional friction. The general tendency for IOs is to exhibit changes in L-kurtosis that match what would be predicted based on information about institutional friction. Perhaps the clearest example is the UN, which exhibits a severely disjointed agenda during the Cold War, but a considerably lower rate of punctuation in the years after 1991, suggesting that greater preference alignment ushered in an era of more efficient policy-making. Similarly, the 2002 transition of the failing OAU into the new AU is associated with a significant reduction of L-kurtosis, indicating a smoother translation of inputs into outputs after the institutional transformation.

In the following, we look in detail at the other three IOs in our sample, investigating how changes in decision rules (EU), preference heterogeneity (OAS), and membership size (OIC) affect the degree of punctuation, while also highlighting interactions with other components of institutional friction.

In the case of the EU, the decreasing level of leptokurtosis over time is consistent with developments in its institutional friction. While the EU changed in opposite directions with regard to two of the three components of institutional friction during our observation period, it appears that changes in decision-making rules had greater influence. Beginning with the entry into force of the Single European Act in 1987, the EU gradually shifted from unanimity to qualified majority voting as the decision-making rule in the Council. With every new treaty since then (Maastricht Treaty 1993, Amsterdam Treaty 1999, Nice Treaty 2003, and Lisbon Treaty 2009), the EU brought more policy domains into the remit of qualified majority voting, making that the standard principle of present-day decision-making. Research demonstrates that this change in the decision rule had a clear positive effect on decision-making efficiency in the EU (Golub 1999; Schulz and König 2000; Golub and Steunenberg 2007).

However, during the same time period, the EU’s membership more than tripled, from 9 in 1980 to 28 in 2013. This development affected the institutional friction in the opposite direction. In fact, the reason why the EU shifted toward qualified majority voting is partly to counteract a potential negative impact on its decision-making efficiency from an ever-growing number of member states. Based on our data, it appears that this modification of the institutional design allowed the EU to retain a comparatively high decision-making efficiency, despite an expanding membership.

While the third component of institutional friction, preference heterogeneity, remained relatively constant, a development exogenous to our measure of institutional friction – the growing involvement of the European Parliament in EU policy-making – is known to have affected decision-making capacity in a negative direction as well (Golub 1999; Golub and Steunenberg 2007; Schulz and König 2000). We use the Maastricht Treaty as cut-off point in Table 5, but given the gradual nature of these changes, we also repeated the analysis with the other treaties as cut-off points. The results remained the same: a decreasing level of leptokurtosis over time (see online appendix).

In the case of the OAS, we observe a higher rate of punctuations after 1994 (Table 5). Although two components of institutional friction remained relatively constant during the observation period, with a minor increase in member states and no major change in the decision rules, we find evidence that changes in the third component, preference heterogeneity, can be linked to the shift in L-kurtosis. Four out of five indicators of preference heterogeneity (see Appendix 2) exhibit higher scores after the cutoff point, suggesting that the accentuating punctuation of the OAS agenda may be due to the political consequences of growing dissimilarity among its membership.

Somewhat counter-intuitively, increasing similarity in the fifth indicator, democracy, may have compounded this divergence. Processes of democratization in member states such as Brazil (1985) and Chile (1989) transformed the OAS from an IO with an almost equal share of authoritarian and democratic members in 1980 into a predominantly democratic one by the end of the Cold War. While greater regime similarity may lead us to expect a lower rate of dramatic and disruptive changes on the OAS agenda, a closer look tells another story: democratization in Latin America opened up new political fault lines, undermining the traditional dominance of the United States in OAS policy-making and leading to an agenda that evolved in a more disjointed manner. Several authoritarian regimes in Latin America were supported by the United States during the 1980s (Forsythe 1995), so the collapse of these regimes did not lead to greater alignment with the United States. This cleavage grew during the 1990s, when countries like Venezuela, Bolivia, Brazil and others veered to the left, raising the political barriers to policy adoption even further (Castañeda 2006).

Another development beyond the limits of our narrower operationalization of institutional friction also contributed to greater preference heterogeneity among member states. The first ‘Summit of the Americas’ in 1994 marked a new orientation of the OAS toward issues of regional economic integration and the Free Trade Area of the Americas (FTAA) (Feinberg 2006; Horwitz 2011, 29). The issue of trade liberalization led to greater tensions within the OAS, as manifested in strong opposition against the FTAA and the failure of negotiations in 2003 (Kennedy 2003; Prevost 2005).

The OIC represents a second case of growing policy punctuation over time. As Table 5 shows, L-kurtosis scores were higher after the end of the Cold War than before, a shift that corresponds to underlying changes in institutional friction.¹⁴ While the decision-making rule of qualified majority voting in the Council of Ministers remained unchanged throughout the studied period, the other two components of institutional friction, membership size and preference heterogeneity, increased in the early and mid-1990s. These changes resulted in policy-making with greater swings: out of seven punctuations (attention changes larger than 200%; see John and Bevan 2012) of the OIC agenda since 1980, six have taken place after the end of the Cold War, and five since 2000.

Higher institutional friction resulted in part from an expanding membership. The OIC membership grew by almost one third between 1980 and 2000, with a peak in the years immediately following the end of the Cold War. The accession during the 1990s of seven post-Communist countries, two Latin American countries, and two countries from sub-Saharan Africa not only meant that additional players became involved in OIC policy-making; it also contributed to greater heterogeneity of the OIC membership. Our data show that four out five indicators of preference heterogeneity increased from the 1980s to the 1990s, reflecting accentuated differences with regard to wealth, economic orientation, and geopolitical interests. Whereas the Cold War years witnessed tensions between US and Soviet aligned member states, political polarization after the 1990s was driven by intra-regional crises, such as the Iraqi invasion of Kuwait in 1990 (Akbarzadeh and Connor 2005), and accentuated rivalries. Tension between OIC leader states such as Iran, Turkey, Egypt, and Pakistan, coupled with a declining role for Saudi Arabia, which previously dominated and financed the organization, led to a more complicated, and hence disjointed, policymaking process (Akbarzadeh and Connor 2005; Kayaoglu 2015, 41).

Taken together, these three case illustrations substantiate the general conclusion from the longitudinal comparison: when IOs witness significant shifts in institutional friction, due to changes in decision rules, memberships, or preferences, this translates into predictable effects in punctuation patterns.

4 Conclusion

This article has offered the first systematic comparison of stability and change in IO policy-making. Theoretically, we have drawn on PET as originally developed in the study of policy agendas in American and Comparative Public Policy. Empirically, we have mapped and explained agenda dynamics in five general-purpose IOs between 1980 and 2015. The article serves as a first important step in the development of a new research agenda on the dynamics of IO policy-making.

Two principal findings emerge from our analysis. First, IO policy agendas exhibit patterns consistent with punctuated equilibria. Of the five IOs examined, none displays a normal distribution of policy changes, which is the expectation if agendas adapt flexibly to shifts in societal demands and problem pressures. Rather, they are all characterized by longer periods of stability, interrupted by shorter periods of dramatic departure from the equilibrium.

Second, institutional friction, as expressed through decision rules, membership size and preference heterogeneity, conditions the degree of punctuation in IO policy-making. Comparing across organizations, the IO with the highest level of institutional friction – the OIC – has the most punctuated policy agenda, while the IO with the lowest institutional friction – the EU – has an agenda characterized by less punctuation. Moreover, longitudinal analysis further demonstrates how changes in the components of institutional friction within IOs, holding other organizational characteristics constant, translate into the expected effects on patterns of punctuation. Disaggregated analysis suggests that decision-making rules are the component of institutional friction that most immediately impacts the degree of punctuation.

Our findings have four broader implications for the study of public policy and global governance. First, the predictive power of PET is broader than previously thought. With few exceptions (Alexandrova et al. 2012; Goertz 2003), previous scholarship in this tradition has focused on the national level, identifying punctuations in the agendas of national and local government. This article represents the first comparative application of PET to IO policy-making and demonstrates that policy punctuations are prominent in global governance. Our findings are based on observations from five IOs, but these findings are likely to be generalizable to a much broader set of IOs, especially multi-purpose IOs. IOs generally face limitations in their capacity to respond to all policy issues all the time. For this reason, we expect punctuated equilibria to be observed in a host of IOs. In addition, we expect decision-rules, membership size and preference heterogeneity to contribute to the degree of punctuation in IO policy agendas more generally. These findings also suggest that the similarities between national and international policy-making shape outcomes more profoundly than the differences. Despite international policy-making being less hierarchical, less exposed to public pressure, and less clear in divisions of responsibility, IOs produce policy dynamics similar to those at the national level. Both levels of governance involve constant demands for policy adaptation coupled with institutional hurdles to efficient policy response, leading to a pattern of punctuated policy-making.

Second, IOs are more amenable to change than conventionally expected in global governance research. Existing research tends to emphasize the reasons why IOs are resistant to change, citing demanding decision hurdles (Scharpf 1988), stabilizing organizational cultures (Barnett and Finnemore 2004), and feedback effects that lock in the status quo (Fioretos 2011). Our results suggest a more complex picture. It appears that barriers to change do not only contribute to periods of stasis but also contribute to a build-up of pressure that generates disproportionate change when it eventually comes. This pattern is reflected in the policy agendas of the five IOs analyzed for this article, but also in tendencies for IO policy development generally to occur irregularly through larger-scale initiatives and programs.

Third, institutional friction matters for IOs’ capacity to respond efficiently and effectively to global problems. Recent research suggests that IOs’ performance in addressing known societal problems is contingent on multiple sequential steps, from developing policy in a timely fashion, to securing compliance with policy, and to evaluating the impact of policy (Lall 2017; Gutner and Thompson 2010; Hafner-Burton et al. 2008; Tallberg et al. 2016). Our findings speak to the first step in this sequence by highlighting how institutional friction likely reduces IOs’ capacity to respond efficiently to shifting societal demands and problem pressures. This has implications for the performance of IOs in general, but particularly for IOs in areas where a swift response is of the essence, such as financial lending (McDowell 2017), peace operations (Hardt 2014), and food security (Agné 2016). It does not mean that there is a quick fix to IO performance, since states sometimes design IOs to be inefficient and producing policy is only a first step toward impact. But it suggests that one important condition shaping the responsiveness of IOs is within the scope of member states to affect through reforms that lower institutional hurdles.

Finally, this article’s application of PET suggests promising avenues for future research on international policy-making. To deepen our understanding of the macro patterns of IO policy-making, further examination of the theory’s scope conditions in the international domain is merited. Does PET apply equally well to task-specific IOs as it does to general-purpose ones? Further research is also needed to explore the forces that allow IOs to overcome institutional friction in international policy-making. When and how do policy information and public pressure channeled through member governments, supranational bureaucracies, and transnational actors lead IOs to overcome blockages? Overall, this article suggests that PET offers a productive starting point for further exploring both broad and specific dynamics of IO policy-making.

Appendices

Appendix 1

Table 6 Topic codes

Appendix 2: Preference heterogeneity data

We calculated preference heterogeneity scores based on variance statistics on five dimensions: level of development (GDP per capita in the year 2000), level of democracy (Marshall et al. 2015; ranging from −10 to 10, with higher values signifying a higher level of democracy), size of the economy (GDP in the year 2000), population (in millions in the year 2000), and level of ethno-linguistic fractionalization (ELF) (Fearon 2003; ranging from 0 to 1, where higher values signify greater cultural heterogeneity). The rationale for these variables is described in the main text. We scored each IO’s value based on its ordinal ranking (1–5) and aggregated the ordinal rankings across the five dimensions to reach a heterogeneity score, which was itself ranked. Since population and size of economy are strongly correlated, they are weighted at 50% in the aggregation.

Table 7 Preference heterogeneity scores