Analyzing international organizations: How the concepts we use affect the answers we get

Abstract

We explore how “international organizations” have been conceptualized and operationalized in the field of International Relations (IR), identify an important gap between the two, and demonstrate how this shapes our understanding of world politics. Traditionally, we show, IR has embraced a broad conception of international organizations (IOs) that appreciates variation in design. However, the literature has largely coalesced around a measurement standard that reflects the characteristics of major postwar IOs. Prevailing measures, therefore, mainly count formal IOs—bodies founded with legally binding agreements—and omit informal IOs, which are founded with non-binding instruments. We argue that this produces a disconnect between theory and empirical evidence used in the field, since scholars frequently make arguments about IOs in general but draw inferences from formal IOs only. After reviewing how this disconnect has emerged, we use an original dataset on state membership in 260 informal IOs to reanalyze a number of important studies, showing heterogeneous effects for subtypes of IOs that conflict with existing theories to varying degrees. These differences imply that formal and informal IOs have different effects and that existing findings in the field are partly artifacts of the specific way IO variables have been operationalized by scholars. Based on this, we offer recommendations for how to improve research practices moving forward.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    According to Google Scholar, the dataset has been cited in over 500 papers, making it by far the most widely cited dataset of IOs.

  2. 2.

    The COW dataset was recently extended from 2005 to 2014 (Pevehouse et al., 2020).

  3. 3.

    Readers who are interested in the conceptualization of IOs will find more information in section 1 of the online appendix of our article. There, we walk through three important cases, and explain how IOs are related to other frequently discussed varieties of cross-border governance.

  4. 4.

    Sometimes the term “international organization” is applied to international non-governmental organizations. See Wallace and Singer’s extensive discussion of international non-governmental organizations (NGOs) in relation to COW (Wallace & Singer, 1970, 247, 240–241). In this article, we are only concerned with public international organizations that involve agreements among states.

  5. 5.

    Ironically, all are included in the COW dataset, despite violating the third criterion for inclusion. In total, we find that over 30 institutions are “miscoded” as formal IOs in a similar manner.

  6. 6.

    Note that, as such, the definition used to generate the underlying dataset is somewhat wider than the one employed by Vabulas and Snidal (2013). In their work, for instance, the absence of an independent secretariat is regarded as a defining feature of informal IOs. In this dataset, by contrast, greater emphasis is placed on an institution’s legal nature under international public law. As a result, IOs may be regarded as informal even if they possess secretariats, provided their constitutive agreements are indeed non-binding. For further discussion of the basic dataset, see the appendix.

  7. 7.

    For further details on the Yearbook, including its advantages and underlying biases, see Saunier (2019).

  8. 8.

    A related situation is a study estimating the effect of IO membership that finds a null statistical result using the COW dataset, but where properly operationalizing IO membership to include informal IOs would actually recover the effect. Unfortunately, due to well-known bias against publication of null findings, it is not possible for us to explore this possibility systematically here, though the effect of omitting informal IOs may be theoretically important in such cases.

  9. 9.

    Of course, it also possible that other venues, like public-private partnerships, may constitute such forums for socialization. If so, an even more expansive measure may be appropriate. However, our reanalysis is a relatively conservative one that hews closely to Greenhill’s original analysis by focusing on organizations that most plausibly fall within the scope conditions of his theory. We take the same approach in the other cases we examine. Future researchers may find it useful to relax this constraint.

  10. 10.

    Regression tables in the main text have been abbreviated. Full tables that include coefficients for control variables are available in the appendix.

References

  1. Abbott, K.W., & Faude, B. (2020). Choosing low-cost institutions in global governance. International Theory, 1–30. FirstView.

  2. Abbott, K.W., Green, J.F., & Keohane, R.O. (2016). Organizational ecology and institutional change in global governance. International Organization, 70(2), 247–277.

    Article  Google Scholar 

  3. Abbott, K.W., & Snidal, D. (2000). Hard and soft law in international governance. International Organization, 54(3), 421–456.

    Article  Google Scholar 

  4. Andonova, L.B. (2017). Governance entrepreneurs: international organizations and the rise of global public–private partnerships. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  5. Archer, C. (2015). International organizations, 4th edn. London: Routledge.

    Google Scholar 

  6. Bach, D., & Newman, A.L. (2010). Transgovernmental networks and domestic policy convergence: Evidence from insider trading regulation. International Organization, 64(3), 505–528.

    Article  Google Scholar 

  7. Bernauer, T., Kalbhenn, A., Koubi, V., & Spilker, G. (2010). A comparison of international and domestic sources of global governance dynamics. British Journal of Political Science, 40(3), 509–538.

    Article  Google Scholar 

  8. Boehmer, C., Gartzke, E., & Nordstrom, T. (2004). Do intergovernmental organizations promote peace?. World Politics, 57(1), 1–38.

    Article  Google Scholar 

  9. Brummer, C. (2014). Minilateralism: how trade alliances, soft law and financial engineering are redefining economic statecraft. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  10. Bulkeley, H., Andonova, L., Betsill, M., Compagnon, D., Hale, T., Hoffmann, M.J, Newell, P., Paterson, M., Roger, C., & VanDeveer, S. (2014). Transnational climate change governance. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  11. Carlson, M., & Koremenos, B. (2021). Cooperation failure or secret collusion? Absolute monarchs and informal cooperation. Review of International Organizations, 16(1), 95–135.

    Article  Google Scholar 

  12. Checkel, J.T. (2001). Why comply? Social learning and European identity change. International Organization, 55(3), 553–588.

    Article  Google Scholar 

  13. Clinton, C., & Sridhar, D. (2017). Governing global health: who runs the world and why?. Oxford: Oxford University Press.

    Google Scholar 

  14. Cox, R.W., & Jacobson, H.K. (1973). The anatomy of influence: decision making in international organization. New Haven: Yale University Press.

    Google Scholar 

  15. Dixon, W.J. (1977). Research on research revisited: Another half decade of quantitative and field research on international organizations. International Organization, 31(1), 65–82.

    Article  Google Scholar 

  16. Eilstrup-Sangiovanni, M. (2020). Death of international organizations: The organizational ecology of intergovernmental organizations, 1815–2015. The Review of International Organizations, 15, 339–370.

    Article  Google Scholar 

  17. Farrell, H., & Newman, A.L. (2019). Of privacy and power: the transatlantic struggle over freedom and security. Princeton: Princeton University Press.

    Book  Google Scholar 

  18. Gartzke, E., & Schneider, C. (2013). Data sets and quantitative research in the study of intergovernmental organizations. In Routledge Handbook of International Organization (pp. 67–79). Routledge.

  19. Gleditsch, N.P., Wallensteen, P., Eriksson, M., Sollenberg, M., & Strand, H. (2002). Armed conflict 1946-2001: A new dataset. Journal of Peace Research, 39(5), 615–637.

    Article  Google Scholar 

  20. Government of Canada. (1991). Foreign missions and international organizations act.

  21. Gray, J. (2018). Life, death, or zombie? The vitality of international organizations. International Studies Quarterly, 62(1), 1–13.

    Article  Google Scholar 

  22. Gray, J. (2020). Life, death, inertia, change: The hidden lives of international organizations. Ethics & International Affairs, 34(1), 33–42.

    Article  Google Scholar 

  23. Greenhill, B. (2010). The company you keep: International socialization and the diffusion of human rights norms. International Studies Quarterly, 54(1), 127–145.

    Article  Google Scholar 

  24. Hafner-Burton, E.M., Von Stein, J., & Gartzke, E. (2008). International organizations count. Journal of Conflict Resolution, 52(2), 175–188.

    Article  Google Scholar 

  25. Haftel, Y.Z., & Thompson, A. (2006). The independence of international organizations: Concept and applications. Journal of Conflict Resolution, 50(2), 253–275.

    Article  Google Scholar 

  26. Hagebölling, D. (2019). Explaining the formation of strategic partnership agreements. Paper presented at the American Political Science Association annual conference, Washington, DC, August 29 – September 1.

  27. Hooghe, L., Lenz, T., & Marks, G. (2019). A theory of international organization. Oxford: Oxford University Press.

    Book  Google Scholar 

  28. Johnson, T. (2014). Organizational progeny: why governments are losing control over the proliferating structures of global governance. Oxford: Oxford University Press.

    Book  Google Scholar 

  29. Kahler, M. (2000). Legalization as strategy: the Asia-Pacific case. International Organization, 54(3), 549–571.

    Article  Google Scholar 

  30. Klabbers, J. (2001). Institutional ambivalence by design: Soft organizations in international law. Nordic Journal of International Law, 70, 403–421.

    Google Scholar 

  31. Klabbers, J. (2015). An introduction to international organizations law, 3rd edn. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  32. Kleine, M. (2013a). Informal governance in the European union: how governments make international organizations work. Ithaca: Cornell University Press.

    Book  Google Scholar 

  33. Kleine, M. (2013b). Knowing your limits: Informal governance and judgment in the EU. Review of International Organizations, 8(2), 245–264.

    Article  Google Scholar 

  34. Koremenos, B. (2013). What’s left out and why? Informal provisions in formal international law. The Review of International Organizations, 8(2), 137–162.

    Article  Google Scholar 

  35. Libman, A., & Obydenkova, A. (2013). Informal governance and participation in non-democratic international organizations. Review of International Organizations, 8(2), 221–243.

    Article  Google Scholar 

  36. Lipson, C. (1991). Why are some international agreements informal?. International Organization, 45(4), 495–538.

    Article  Google Scholar 

  37. Mansfield, E.D., & Pevehouse, J.C. (2006). Democratization and international organizations. International Organization, 60(1), 137–167.

    Article  Google Scholar 

  38. Martin, L. (2021). Formality, typologies, and institutional design. Review of International Organizations, 16(1), 175–182.

    Article  Google Scholar 

  39. Newman, A.L., & Posner, E. (2018). Voluntary disruptions: international soft law, finance, and power. Oxford: Oxford University Press.

    Book  Google Scholar 

  40. Paxton, P. (2000). Women’s suffrage in the measurement of democracy: Problems of operationalization. Studies in Comparative International Development, 35(3), 92–111.

    Article  Google Scholar 

  41. Pevehouse, J.C., Nordstrom, T., McManus, R.W., & Jamison, A.S. (2020). Tracking organizations in the world: The Correlates of War IGO Version 3.0 datasets. Journal of Peace Reserarch, 57(3), 492–503.

    Article  Google Scholar 

  42. Pevehouse, J.C., Nordstrom, T., & Warnke, K. (2004). The Correlates of War 2 International Governmental Organizations data version 2.0. Conflict Management and Peace Science, 21(2), 101–119.

    Article  Google Scholar 

  43. Poast, P., & Urpelainen, J. (2018). Organizing democracy: how international organizations assist new democracies. Chicago: University of Chicago Press.

    Book  Google Scholar 

  44. Potter, P.B. (1935). The classification of international organizations, I. American Political Science Review, 29(2), 212–224.

    Article  Google Scholar 

  45. Puetter, U. (2006). The Eurogroup: how a secretive circle of finance ministers shape european economic governance. Manchester: Manchester University Press.

    Book  Google Scholar 

  46. Raustiala, K. (2002). The architecture of international cooperation: Transgovernmental networks and the future of international law. Virginia Journal International Law, 43, 1–92.

    Google Scholar 

  47. Reinisch, A. (2013). The privileges and immunities of international organizations in domestic courts. Oxford: Oxford University Press.

    Book  Google Scholar 

  48. Roger, C.B. (2020). The origins of informality: why the legal foundations of global governance are shifting, and why it matters. Oxford: Oxford University Press.

    Book  Google Scholar 

  49. Rowan, S.S. (2021). Does institutional proliferation undermine cooperation? Theory and evidence from climate change. International Studies Quarterly, 65(2), 461–475.

    Article  Google Scholar 

  50. Russett, B., & Oneal, J. (2001). Triangulating peace: democracy, interdependence, and international organizations. New York: W.W. Norton.

    Google Scholar 

  51. Sauer, T. (2019). The role of informal international organizations in resolving the Iranian nuclear crisis (2003–15). JCMS: Journal of Common Market Studies, 57, 939–955.

    Google Scholar 

  52. Saunier, P.-Y. (2019). Everything one wants to know about international organizations? A biography of the Yearbook of International Organizations, 1909-2016. In D. Laqua, W. van Acker, & C. Verbruggen (Eds.) International organizations and global civil society: histories of the union of international associations (pp. 171–203). Bloomsbury Publishing.

  53. Seyersted, F. (2008). Common law of international organizations. Leiden: Martinus Nijhoff Publishers.

    Book  Google Scholar 

  54. Shanks, C., Jacobson, H.K., & Kaplan, J.H. (1996). Inertia and change in the constellation of international governmental organizations, 1981-1992. International Organization, 50(4), 593–627.

    Article  Google Scholar 

  55. Slaughter, A.-M. (2004). A new world order. Princeton: Princeton University Press.

    Google Scholar 

  56. Stone, R. (2013). Informal governance in international organizations: Introduction to the special issue. Review of International Organizations, 8(2), 121–136.

    Article  Google Scholar 

  57. Tallberg, J., Sommerer, T., Squatrito, T., & Jönsson, C. (2013). The opening up of international organizations. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  58. United Nations. (2009). Report of the International Law Commission, Sixty-First Session (4 May–5 June and 6 July–7 August 2009).

  59. Vabulas, F., & Snidal, D. (2013). Organization without delegation: Informal intergovernmental organizations (IIGOs) and the spectrum of intergovernmental arrangements. Review of International Organizations, 8(2), 193–220.

    Article  Google Scholar 

  60. Vabulas, F., & Snidal, D. (2020). Cooperation under autonomy: Building and analyzing the informal intergovernmental organizations 2.0 data set. Journal of Peace Research. OnlineFirst.

  61. Volgy, T.J., Sabic, Z., Roter, P., & Gerlak, A.K. (2009). Mapping the new world order. Hoboken: Wiley-Blackwell.

    Book  Google Scholar 

  62. Wallace, M., & Singer, J.D. (1970). Intergovernmental organization in the global system, 1815–1964: A quantitative description. International Organization, 24(2), 239–287.

    Article  Google Scholar 

  63. Westerwinter, O. (2021). Transnational public–private governance initiatives in world politics: Introducing a new dataset. Review of International Organizations, 16(1), 137–174.

    Article  Google Scholar 

  64. Westerwinter, O., Abbott, K.W., & Biersteker, T. (2021). Informal governance in world politics. Review of International Organizations, 16(1), 1–27.

    Article  Google Scholar 

  65. White, N.D. (2005). The Law of International Organisations, 2nd edn. Manchester: Manchester University Press.

    Google Scholar 

  66. Zaring, D. (2019). The globalized governance of finance. Cambridge: Cambridge University Press.

    Book  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge helpful comments on earlier drafts from Sarah Bush, Mette Eilstrup-Sangiovanni, Ezequiel Gonzales-Ocantos, Yoram Haftel, Thomas Hale, Ranjit Lall, Yonatan Lupu, Tyler Pratt, Duncan Snidal, Alexandra Zeitz, as well as audiences at IBEI’s research seminar, and the International Studies Association and Political Economy of International Organization annual conferences. Charles Roger would also like to acknowledge the generous support for this project received from the Social Science and Humanities Research Council of Canada (SSHRC).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Charles B. Roger.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author contributions to research design and conceptualization: C.R. (50%), S.R. (50%); statistical analysis: C.R. (50%), S.R. (50%); writing: C.R. (50%), S.R. (50%). The order of authors is chosen alphabetically.

Responsible editor: Axel Dreher

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 238 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Roger, C.B., Rowan, S.S. Analyzing international organizations: How the concepts we use affect the answers we get. Rev Int Organ (2021). https://doi.org/10.1007/s11558-021-09432-2

Download citation

Keywords

  • International cooperation
  • International organizations
  • Institutional design
  • Legalization
  • Formality
  • Informality
  • Socialization
  • Democratization
  • Global environmental politics

JEL Classification

  • F50
  • F53
  • F55
  • K33