We explore how “international organizations” have been conceptualized and operationalized in the field of International Relations (IR), identify an important gap between the two, and demonstrate how this shapes our understanding of world politics. Traditionally, we show, IR has embraced a broad conception of international organizations (IOs) that appreciates variation in design. However, the literature has largely coalesced around a measurement standard that reflects the characteristics of major postwar IOs. Prevailing measures, therefore, mainly count formal IOs—bodies founded with legally binding agreements—and omit informal IOs, which are founded with non-binding instruments. We argue that this produces a disconnect between theory and empirical evidence used in the field, since scholars frequently make arguments about IOs in general but draw inferences from formal IOs only. After reviewing how this disconnect has emerged, we use an original dataset on state membership in 260 informal IOs to reanalyze a number of important studies, showing heterogeneous effects for subtypes of IOs that conflict with existing theories to varying degrees. These differences imply that formal and informal IOs have different effects and that existing findings in the field are partly artifacts of the specific way IO variables have been operationalized by scholars. Based on this, we offer recommendations for how to improve research practices moving forward.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
According to Google Scholar, the dataset has been cited in over 500 papers, making it by far the most widely cited dataset of IOs.
The COW dataset was recently extended from 2005 to 2014 (Pevehouse et al., 2020).
Readers who are interested in the conceptualization of IOs will find more information in section 1 of the online appendix of our article. There, we walk through three important cases, and explain how IOs are related to other frequently discussed varieties of cross-border governance.
Sometimes the term “international organization” is applied to international non-governmental organizations. See Wallace and Singer’s extensive discussion of international non-governmental organizations (NGOs) in relation to COW (Wallace & Singer, 1970, 247, 240–241). In this article, we are only concerned with public international organizations that involve agreements among states.
Ironically, all are included in the COW dataset, despite violating the third criterion for inclusion. In total, we find that over 30 institutions are “miscoded” as formal IOs in a similar manner.
Note that, as such, the definition used to generate the underlying dataset is somewhat wider than the one employed by Vabulas and Snidal (2013). In their work, for instance, the absence of an independent secretariat is regarded as a defining feature of informal IOs. In this dataset, by contrast, greater emphasis is placed on an institution’s legal nature under international public law. As a result, IOs may be regarded as informal even if they possess secretariats, provided their constitutive agreements are indeed non-binding. For further discussion of the basic dataset, see the appendix.
For further details on the Yearbook, including its advantages and underlying biases, see Saunier (2019).
A related situation is a study estimating the effect of IO membership that finds a null statistical result using the COW dataset, but where properly operationalizing IO membership to include informal IOs would actually recover the effect. Unfortunately, due to well-known bias against publication of null findings, it is not possible for us to explore this possibility systematically here, though the effect of omitting informal IOs may be theoretically important in such cases.
Of course, it also possible that other venues, like public-private partnerships, may constitute such forums for socialization. If so, an even more expansive measure may be appropriate. However, our reanalysis is a relatively conservative one that hews closely to Greenhill’s original analysis by focusing on organizations that most plausibly fall within the scope conditions of his theory. We take the same approach in the other cases we examine. Future researchers may find it useful to relax this constraint.
Regression tables in the main text have been abbreviated. Full tables that include coefficients for control variables are available in the appendix.
Abbott, K.W., & Faude, B. (2020). Choosing low-cost institutions in global governance. International Theory, 1–30. FirstView.
Abbott, K.W., Green, J.F., & Keohane, R.O. (2016). Organizational ecology and institutional change in global governance. International Organization, 70(2), 247–277.
Abbott, K.W., & Snidal, D. (2000). Hard and soft law in international governance. International Organization, 54(3), 421–456.
Andonova, L.B. (2017). Governance entrepreneurs: international organizations and the rise of global public–private partnerships. Cambridge: Cambridge University Press.
Archer, C. (2015). International organizations, 4th edn. London: Routledge.
Bach, D., & Newman, A.L. (2010). Transgovernmental networks and domestic policy convergence: Evidence from insider trading regulation. International Organization, 64(3), 505–528.
Bernauer, T., Kalbhenn, A., Koubi, V., & Spilker, G. (2010). A comparison of international and domestic sources of global governance dynamics. British Journal of Political Science, 40(3), 509–538.
Boehmer, C., Gartzke, E., & Nordstrom, T. (2004). Do intergovernmental organizations promote peace?. World Politics, 57(1), 1–38.
Brummer, C. (2014). Minilateralism: how trade alliances, soft law and financial engineering are redefining economic statecraft. Cambridge: Cambridge University Press.
Bulkeley, H., Andonova, L., Betsill, M., Compagnon, D., Hale, T., Hoffmann, M.J, Newell, P., Paterson, M., Roger, C., & VanDeveer, S. (2014). Transnational climate change governance. Cambridge: Cambridge University Press.
Carlson, M., & Koremenos, B. (2021). Cooperation failure or secret collusion? Absolute monarchs and informal cooperation. Review of International Organizations, 16(1), 95–135.
Checkel, J.T. (2001). Why comply? Social learning and European identity change. International Organization, 55(3), 553–588.
Clinton, C., & Sridhar, D. (2017). Governing global health: who runs the world and why?. Oxford: Oxford University Press.
Cox, R.W., & Jacobson, H.K. (1973). The anatomy of influence: decision making in international organization. New Haven: Yale University Press.
Dixon, W.J. (1977). Research on research revisited: Another half decade of quantitative and field research on international organizations. International Organization, 31(1), 65–82.
Eilstrup-Sangiovanni, M. (2020). Death of international organizations: The organizational ecology of intergovernmental organizations, 1815–2015. The Review of International Organizations, 15, 339–370.
Farrell, H., & Newman, A.L. (2019). Of privacy and power: the transatlantic struggle over freedom and security. Princeton: Princeton University Press.
Gartzke, E., & Schneider, C. (2013). Data sets and quantitative research in the study of intergovernmental organizations. In Routledge Handbook of International Organization (pp. 67–79). Routledge.
Gleditsch, N.P., Wallensteen, P., Eriksson, M., Sollenberg, M., & Strand, H. (2002). Armed conflict 1946-2001: A new dataset. Journal of Peace Research, 39(5), 615–637.
Government of Canada. (1991). Foreign missions and international organizations act.
Gray, J. (2018). Life, death, or zombie? The vitality of international organizations. International Studies Quarterly, 62(1), 1–13.
Gray, J. (2020). Life, death, inertia, change: The hidden lives of international organizations. Ethics & International Affairs, 34(1), 33–42.
Greenhill, B. (2010). The company you keep: International socialization and the diffusion of human rights norms. International Studies Quarterly, 54(1), 127–145.
Hafner-Burton, E.M., Von Stein, J., & Gartzke, E. (2008). International organizations count. Journal of Conflict Resolution, 52(2), 175–188.
Haftel, Y.Z., & Thompson, A. (2006). The independence of international organizations: Concept and applications. Journal of Conflict Resolution, 50(2), 253–275.
Hagebölling, D. (2019). Explaining the formation of strategic partnership agreements. Paper presented at the American Political Science Association annual conference, Washington, DC, August 29 – September 1.
Hooghe, L., Lenz, T., & Marks, G. (2019). A theory of international organization. Oxford: Oxford University Press.
Johnson, T. (2014). Organizational progeny: why governments are losing control over the proliferating structures of global governance. Oxford: Oxford University Press.
Kahler, M. (2000). Legalization as strategy: the Asia-Pacific case. International Organization, 54(3), 549–571.
Klabbers, J. (2001). Institutional ambivalence by design: Soft organizations in international law. Nordic Journal of International Law, 70, 403–421.
Klabbers, J. (2015). An introduction to international organizations law, 3rd edn. Cambridge: Cambridge University Press.
Kleine, M. (2013a). Informal governance in the European union: how governments make international organizations work. Ithaca: Cornell University Press.
Kleine, M. (2013b). Knowing your limits: Informal governance and judgment in the EU. Review of International Organizations, 8(2), 245–264.
Koremenos, B. (2013). What’s left out and why? Informal provisions in formal international law. The Review of International Organizations, 8(2), 137–162.
Libman, A., & Obydenkova, A. (2013). Informal governance and participation in non-democratic international organizations. Review of International Organizations, 8(2), 221–243.
Lipson, C. (1991). Why are some international agreements informal?. International Organization, 45(4), 495–538.
Mansfield, E.D., & Pevehouse, J.C. (2006). Democratization and international organizations. International Organization, 60(1), 137–167.
Martin, L. (2021). Formality, typologies, and institutional design. Review of International Organizations, 16(1), 175–182.
Newman, A.L., & Posner, E. (2018). Voluntary disruptions: international soft law, finance, and power. Oxford: Oxford University Press.
Paxton, P. (2000). Women’s suffrage in the measurement of democracy: Problems of operationalization. Studies in Comparative International Development, 35(3), 92–111.
Pevehouse, J.C., Nordstrom, T., McManus, R.W., & Jamison, A.S. (2020). Tracking organizations in the world: The Correlates of War IGO Version 3.0 datasets. Journal of Peace Reserarch, 57(3), 492–503.
Pevehouse, J.C., Nordstrom, T., & Warnke, K. (2004). The Correlates of War 2 International Governmental Organizations data version 2.0. Conflict Management and Peace Science, 21(2), 101–119.
Poast, P., & Urpelainen, J. (2018). Organizing democracy: how international organizations assist new democracies. Chicago: University of Chicago Press.
Potter, P.B. (1935). The classification of international organizations, I. American Political Science Review, 29(2), 212–224.
Puetter, U. (2006). The Eurogroup: how a secretive circle of finance ministers shape european economic governance. Manchester: Manchester University Press.
Raustiala, K. (2002). The architecture of international cooperation: Transgovernmental networks and the future of international law. Virginia Journal International Law, 43, 1–92.
Reinisch, A. (2013). The privileges and immunities of international organizations in domestic courts. Oxford: Oxford University Press.
Roger, C.B. (2020). The origins of informality: why the legal foundations of global governance are shifting, and why it matters. Oxford: Oxford University Press.
Rowan, S.S. (2021). Does institutional proliferation undermine cooperation? Theory and evidence from climate change. International Studies Quarterly, 65(2), 461–475.
Russett, B., & Oneal, J. (2001). Triangulating peace: democracy, interdependence, and international organizations. New York: W.W. Norton.
Sauer, T. (2019). The role of informal international organizations in resolving the Iranian nuclear crisis (2003–15). JCMS: Journal of Common Market Studies, 57, 939–955.
Saunier, P.-Y. (2019). Everything one wants to know about international organizations? A biography of the Yearbook of International Organizations, 1909-2016. In D. Laqua, W. van Acker, & C. Verbruggen (Eds.) International organizations and global civil society: histories of the union of international associations (pp. 171–203). Bloomsbury Publishing.
Seyersted, F. (2008). Common law of international organizations. Leiden: Martinus Nijhoff Publishers.
Shanks, C., Jacobson, H.K., & Kaplan, J.H. (1996). Inertia and change in the constellation of international governmental organizations, 1981-1992. International Organization, 50(4), 593–627.
Slaughter, A.-M. (2004). A new world order. Princeton: Princeton University Press.
Stone, R. (2013). Informal governance in international organizations: Introduction to the special issue. Review of International Organizations, 8(2), 121–136.
Tallberg, J., Sommerer, T., Squatrito, T., & Jönsson, C. (2013). The opening up of international organizations. Cambridge: Cambridge University Press.
United Nations. (2009). Report of the International Law Commission, Sixty-First Session (4 May–5 June and 6 July–7 August 2009).
Vabulas, F., & Snidal, D. (2013). Organization without delegation: Informal intergovernmental organizations (IIGOs) and the spectrum of intergovernmental arrangements. Review of International Organizations, 8(2), 193–220.
Vabulas, F., & Snidal, D. (2020). Cooperation under autonomy: Building and analyzing the informal intergovernmental organizations 2.0 data set. Journal of Peace Research. OnlineFirst.
Volgy, T.J., Sabic, Z., Roter, P., & Gerlak, A.K. (2009). Mapping the new world order. Hoboken: Wiley-Blackwell.
Wallace, M., & Singer, J.D. (1970). Intergovernmental organization in the global system, 1815–1964: A quantitative description. International Organization, 24(2), 239–287.
Westerwinter, O. (2021). Transnational public–private governance initiatives in world politics: Introducing a new dataset. Review of International Organizations, 16(1), 137–174.
Westerwinter, O., Abbott, K.W., & Biersteker, T. (2021). Informal governance in world politics. Review of International Organizations, 16(1), 1–27.
White, N.D. (2005). The Law of International Organisations, 2nd edn. Manchester: Manchester University Press.
Zaring, D. (2019). The globalized governance of finance. Cambridge: Cambridge University Press.
We would like to acknowledge helpful comments on earlier drafts from Sarah Bush, Mette Eilstrup-Sangiovanni, Ezequiel Gonzales-Ocantos, Yoram Haftel, Thomas Hale, Ranjit Lall, Yonatan Lupu, Tyler Pratt, Duncan Snidal, Alexandra Zeitz, as well as audiences at IBEI’s research seminar, and the International Studies Association and Political Economy of International Organization annual conferences. Charles Roger would also like to acknowledge the generous support for this project received from the Social Science and Humanities Research Council of Canada (SSHRC).
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Author contributions to research design and conceptualization: C.R. (50%), S.R. (50%); statistical analysis: C.R. (50%), S.R. (50%); writing: C.R. (50%), S.R. (50%). The order of authors is chosen alphabetically.
Responsible editor: Axel Dreher
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Roger, C.B., Rowan, S.S. Analyzing international organizations: How the concepts we use affect the answers we get. Rev Int Organ (2021). https://doi.org/10.1007/s11558-021-09432-2
- International cooperation
- International organizations
- Institutional design
- Global environmental politics