Skip to main content
SpringerLink
Log in

What do fragility indices measure?

Assessing measurement procedures and statistical proximity

Was messen Fragilitätsindizes?

Bewertung der Messverfahren und der Proximität

  • Aufsätze
  • Published:
Zeitschrift für Vergleichende Politikwissenschaft Aims and scope Submit manuscript

Abstract

This article examines nine fragility indices. Their content validity is assessed by reviewing conceptualization, measurement and aggregation methods. Their convergent/divergent validity is assessed via principal component analysis and multidimensional scaling. These techniques are capable of determing the dimensionality of and the statistical proximity within the examined sample of indices. Both the conceptual and the statistical analysis support the hypothesis that there is a group of “holistic” fragility indices which are of little use for investigating the causes and consequences of fragility. The remaining indices address more specific aspects of fragility and produce empirically distinguishable results.

Zusammenfassung

Dieser Artikel untersucht neun Fragilitätsindizes. Die Inhaltsvalidität der Indizes wird durch die Überprüfung von Konzeptualisierung, Messung und Aggregationsmethoden bewertet. Die konvergente/diskriminante Validität wird mittels Hauptkomponentenanalyse und multidimensionaler Skalierung bewertet. Diese Verfahren erlauben die Untersuchung von Dimensionalität und statistischer Ähnlichkeit innerhalb der Gruppe von Indizes. Sowohl die konzeptionelle als auch die statistische Analyse stützen die Hypothese, dass es eine Gruppe „holistischer“ Fragilitätsindizes gibt, die für die Erforschung der Ursachen und Folgen von Fragilität von geringem Nutzen sind. Die restlichen Indizes befassen sich mit spezifischen Aspekten von Fragilität und produzieren empirisch unterscheidbare Ergebnisse.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Conceptual critiques of fragility indices can be found in Marshall 2008; Wulf and Debiel 2009; Call 2010; and Wennmann 2010.

  2. Some of the most prominent critiques are Arndt and Oman 2006; Kurtz and Schrank 2007; Thomas 2007; Langbein and Knack 2010.

  3. Some earlier parts of this article and more descriptive information on the indices can be found in the Users Guide on Measuring Fragility (Fabra Mata and Ziaja 2009).

  4. See Bollen 1993 for such an approach applied to democracy indices.

  5. See OECD 2008. For discussions of various terminologies, see Gros 1996 and Rotberg 2004.

  6. An exception is the CIFP Fragility Index (Carment et al. 2009, pp. 84–89).

  7. This article will not deal in greater detail with the issue of legitimacy; for the relationship between legitimacy and fragility, see Bellina et al. 2009; Call 2010.

  8. The idea of relative capacity is, of course, not new; see, for example, Kugler and Domke (1986) who measure “relative political capacity” as actual over expected extraction of resources.

  9. One could also start with Tilly’s (1985) concept of the state as a protection racket and arrive at a similar operationalization.

  10. Fabra Mata and Ziaja (2009, pp. 109–111) provide a list of all indices reviewed for potentially being fragility indices.

  11. For an assessment of the overall BTI index, see Müller and Pickel 2007.

  12. The index is used for a map in a brochure and mentioned in the documentation of the methodology (Bertelsmann Stiftung 2007, 2008). The producers say, however, that they would construct a “proper index of state fragility” differently (personal communication).

  13. It is, for example, used in the upcoming World Development Report “Conflict, Security and Development” (http://wdr2011.worldbank.org/) and by the International Monetary Fund (IMF 2010).

  14. See Arndt 2008 on the organizational motivations for publishing governance indices.

  15. This is the major reason why the World Bank uses the IDA Resource Allocation Index to measure fragility: for internal purposes, World Bank staff can draw on a time-series starting in the 1970s.

  16. “Measurement” thus refers to measurement in the narrower sense, limited to the quantification of single components, while “measurement procedure” refers to measurement in the broader sense, including previous conceptualization and subsequent aggregation.

  17. The literature on measuring democracy provides examples on how multi-dimensional concepts can be convincingly operationalized by building on a mature corpus of theory (e.g. Bollen 1993; Munck 2009).

  18. For a detailed treatment on visualizing data in network graphs, see Krempel 2005.

  19. With the exception of the IDA Resource Allocation Index and the Country Policy and Institutional Assessments, which have been moved closer to the other World Bank sources to group them together.

  20. An exception is the WGI Political Stability Index.

  21. A rather low bivariate correlation of 0.53 supports the MDS result. The advantage of MDS, however, is that all the different relationships can be grasped at once via the graphical display, dispensing of the need to comb through the correlation table row by row.

  22. See Fabra Mata and Ziaja (2009, p. 32) for an overview of categorization methods applied by fragility indices. An excellent discussion on methodological choices, including a convincing standardization method, can be found in the Index of African Governance (Rotberg and Gisselquist 2008).

  23. On nomological validation, see Adcock and Collier (2001, p. 542).

  24. Collier and Levitsky (2009) use this term with respect to democracy concepts.

  25. Coppedge (2002, p. 39) arrives at a similar conclusion with regard to democracy: “The highest priority for improving the measurement of democracy is therefore improving the measurement of disaggregated attributes of democracy”.

  26. Call (2010) tries to apply this approach to less broadly defined “capacity”, “security” and “legitimacy gaps”.

References

  • Adcock, R., and D. Collier. 2001. Measurement validity: A shared standard for qualitative and quantitative research. The American Political Science Review 95 (3): 529–546.

    Article  Google Scholar 

  • Arndt, C. 2008. The politics of governance ratings. International Public Management Journal 11 (3): 275–297.

    Article  Google Scholar 

  • Arndt, C., and C. Oman. 2006. Uses and abuses of governance indicators. Paris: OECD Development Centre.

    Google Scholar 

  • Bellina, S., D. Darbon, S. S. Eriksen, and O. J. Sending. 2009. The legitimacy of the state in fragile situations. Oslo: Norwegian Agency for Development Cooperation (Norad Report 20/2009).

    Google Scholar 

  • Bertelsmann Stiftung, ed. 2007. Bertelsmann transformation index 2008: Political management in international comparison. Gütersloh: Verlag Bertelsmann Stiftung. Brochure. http://bti2008.bertelsmann-transformation-index.de/fileadmin/pdf/Anlagen_BTI_2008/BTI_2008_Brochure_EN.pdf. Accessed 1 Feb 2011.

    Google Scholar 

  • Bertelsmann Stiftung, ed. 2008. Bertelsmann transformation index 2008: Political management in international comparison. Gütersloh: Verlag Bertelsmann Stiftung.

    Google Scholar 

  • Bertelsmann Stiftung, ed. 2009. About BTI, ranking and country reports. http://www.bertelsmann-transformation-index.de/11.0.html?&L=1. Accessed 1 Feb 2011.

  • Bollen, K. 1993. Liberal democracy. American Journal of Political Science 37 (4): 1207–1230.

    Article  Google Scholar 

  • Borg, I., and P. J. F. Groenen. 2005. Modern multidimensional scaling: Theory and applications. 2nd ed. New York: Springer.

    Google Scholar 

  • Bratton, M., and E. C. C. Chang. 2006. State building and democratization in sub-Saharan Africa: Forwards, backwards, or together? Comparative Political Studies 39 (9): 1059–1083.

    Article  Google Scholar 

  • Brinkerhoff, D. W. 2007. Governance in post-conflict societies: Rebuilding fragile states. New York: Routledge.

    Google Scholar 

  • Call, C. T. 2010. Beyond the “Failed State”: Toward conceptual alternatives. European Journal of International Relations. Online first. doi:10.1177/1354066109353137.

  • Carment, D., S. Prest, and Y. Samy. 2009. Security, development and the fragile state: Bridging the gap between theory and policy. Abingdon: Routledge Studies in Intervention and Statebuilding.

    Google Scholar 

  • Collier, D., and S. Levitsky. 2009. Democracy: Conceptual hierarchies in comparative research. In Concepts and method in the social science: The tradition of giovanni sartori, eds. D. Collier and J. Gerring, 269–288. London: Routledge.

    Google Scholar 

  • Coppedge, M. 2002. Democracy and dimensions. Comparative Political Studies 35 (1): 35–39.

    Google Scholar 

  • Country Indicators for Foreign Policy (CIFP). 2009. Country ranking table 2007. http://www.carleton.ca/cifp/app/ffs_ranking.php. Accessed 1 Feb 2011.

  • Dahl, R. 1971. Polyarchy: Participation and opposition. New Haven: Yale University Press.

    Google Scholar 

  • Department for International Development (DFID). 2005. Why we need to work more effectively in fragile states. London: Department for International Development.

    Google Scholar 

  • Desch, M. C. 1996. War and strong states, peace and weak states? International Organization 50 (2): 237–268.

    Article  Google Scholar 

  • Economist Intelligence Unit (EIU). 2009a. Manning the barricades: Who’s at risk as deepening economic distress foments social unrest. London: EIU.

    Google Scholar 

  • Economist Intelligence Unit (EIU). 2009b. Social unrest. http://viewswire.eiu.com/site_info.asp?info_name=social_unrest_table. Accessed 1 Feb 2011.

  • Economist Intelligence Unit (EIU). 2009c. Political instability index: Vulnerability to social and political unrest. http://viewswire.eiu.com/index.asp?layout=VWArticleVW3&article_id=874361472.ent&task=view&id=102&Itemid=891. Accessed 1 Feb 2011.

  • European Report on Development (ERD). 2009. Overcoming fragility in Africa. San Domenico di Fiesole: European University Institute.

    Google Scholar 

  • Evans, P. B., D. Rueschemeyer, and T. Skocpol. 1985. Bringing the state back in. Cambridge: Cambridge University Press.

    Book  Google Scholar 

  • Fabra Mata, J., and S. Ziaja. 2009. Users’ guide on measuring fragility. Oslo: United Nations Development Programme.

    Google Scholar 

  • Foreign Policy, and Fund for Peace. 2008. The failed states index 2008. Foreign Policy 167: 64–68.

    Google Scholar 

  • Fukuyama, F. 2004. State-building. Ithaca: Cornell University Press.

    Google Scholar 

  • Fund for Peace. 2009a. Failed states index scores 2008. http://www.fundforpeace.org/web/index.php?option=com_content&task=view&id=292&Itemid=452. Accessed 1 Feb 2011.

  • Fund for Peace. 2009b. Failed states index FAQ. http://www.fundforpeace.org/web/index.php?option=com_cont. Accessed 1 Feb 2011.

  • Goertz, G. 2006. Social science concepts. A user’s guide. Princeton: Princeton University Press.

    Google Scholar 

  • Goertz, G. 2008. Concepts, theories and numbers. In The Oxford handbook of political methodology, eds. Robert E. Goodin, Janet M. Box-Steffensmeier, Henry E. Brady and D. Collier, 97–119. Oxford: Oxford University Press.

    Chapter  Google Scholar 

  • Goldstone, J. A., R. H. Bates, D. L. Epstein, T. R. Gurr, M. B. Lustik, M. G. Marshall, J. Ulfelder, and M. Woodward. 2010. A global model for forecasting political instability. American Journal of Political Science 54 (1): 190–208.

    Article  Google Scholar 

  • Gros, J. G. 1996. Towards a taxonomy of failed states in the New World Order: decaying Somalia, Liberia, Rwanda and Haiti. Third World Quarterly 17 (3): 455–472.

    Article  Google Scholar 

  • Gutiérrez Sanín, F. 2009. The quandaries of coding and ranking: Evaluating poor state performance indexes. London: London School of Economics (Crisis States Research Centre Working Paper 58).

    Google Scholar 

  • Harttgen, K., and S. Klasen. 2010. Fragility and MDG progress: How useful is the fragility concept? San Domenico di Fiesole: European University Institute (Robert Schuman Centre for Advanced Studies Working Paper 2010/20).

    Google Scholar 

  • Helman, G. B., and S. R. Ratner. 1992–1993. Saving failed states. Foreign Policy 89 (Winter): 3–20.

    Article  Google Scholar 

  • Herrera, Y. M., and D. Kapur. 2007. Improving data quality. Political Analysis Advance Access 15 (4): 365–386.

    Google Scholar 

  • Hewitt, J. J. 2007a. Peace and conflict instability ledger: Description of data and model estimation. College Park: Center for International Development and Conflict Management, University of Maryland.

    Google Scholar 

  • Hewitt, J. J., J. Wilkenfeld, and T. R. Gurr. 2008. Peace and conflict 2008: Executive summary. Baltimore: Center for International Development and Conflict Management, University of Maryland.

  • Hewitt, J. J. 2007b. The peace and conflict instability ledger: Ranking states on future risks. In Peace and Conflict 2008, eds. J. J. Hewitt, J. Wilkenfeld and T. R. Gurr, 5–20. College Park: Center for International Development and Conflict Management, University of Maryland.

    Google Scholar 

  • Hewitt, J. J., J. Wilkenfeld, and T. R. Gurr, eds. 2009. Peace and conflict 2010. College Park: Center for International Development and Conflict Management, University of Maryland.

    Google Scholar 

  • Holsti, K. J. 1996. The state, war, and the state of war. Cambridge: Cambridge University Press.

    Google Scholar 

  • Independent Evaluation Group (IEG). 2006. Engaging with fragile States: An IEG review of world bank support to low-income countries under stress. Washington D.C.: The World Bank.

    Google Scholar 

  • International Monetary Fund (IMF). 2010. Regional economic outlook: Sub-saharan Africa: Back to high growth? Washington D.C.: International Monetary Fund.

    Google Scholar 

  • Jackman, S. 2008. Measurement. In The Oxford handbook of political methodology, eds. R. E. Goodin, J. M. Box-Steffensmeier, H. E. Brady and D. Collier, 119–152. Oxford: Oxford University Press.

    Chapter  Google Scholar 

  • Jackson, R. H., and C. G. Rosberg. 1982. Why Africa’s weak states persist: The empirical and the juridical in statehood. World Politics 35 (1): 1–24.

    Article  Google Scholar 

  • Kaufmann, D., and A. Kraay. 2008. Governance indicators: Where are we, where should we be going? The World Bank Research Observer 23 (1): 1–30.

    Article  Google Scholar 

  • Kaufmann, D., A. Kraay, and M. Mastruzzi. 2007. Worldwide governance indicators project: Answering the critics. World Bank Policy Research Working Paper 4149. Washington D.C.: World Bank.

    Book  Google Scholar 

  • Kaufmann, D., A. Kraay, and M. Mastruzzi. 2008. Governance matters VII: Aggregate and individual governance indicators 1996–2007. World Bank Policy Research Working Paper No. 4654. Washington D.C.: The World Bank.

    Book  Google Scholar 

  • Kaufmann, D., A. Kraay, and M. Mastruzzi. 2009. Governance matters VIII: Aggregate and individual governance indicators, 1996–2008. World Bank Policy Research Working Paper No. 4978. Washington D.C.: The World Bank.

    Book  Google Scholar 

  • Krempel, L. 2005. Visualisierung komplexer Strukturen: Grundlagen der Darstellung mehrdimensionaler Netzwerke. Frankfurt a. M.: Campus.

    Google Scholar 

  • Kugler, J., and W. Domke. 1986. Comparing the strength of nations. Comparative Political Studies 19 (1): 39–69.

    Article  Google Scholar 

  • Kurtz, M. J., and A. Schrank. 2007. Growth and governance: Models, measures, and mechanisms. The Journal of Politics 69 (2): 538–554.

    Google Scholar 

  • Langbein, L., and S. Knack. 2010. The worldwide governance indicators: Six, one, or none? Journal of Development Studies 46 (2): 350–370.

    Article  Google Scholar 

  • Marshall, M. G. 2008. Fragility, instability, and the failure of states. New York: Council on Foreign Relations.

    Google Scholar 

  • Marshall, M. G., and B. R. Cole. 2008. Global report on conflict, governance and state fragility 2008. Foreign Policy Bulletin 18:3–21.

    Article  Google Scholar 

  • Marshall, M. G., and J. A. Goldstone. 2007. Global report on conflict, governance and state fragility 2007. Foreign Policy Bulletin 17:3–21.

    Google Scholar 

  • Migdal, J. S. 1988. Strong societies and weak states: State-society relations and state capabilities in the third world. Princeton: Princeton University Press.

    Google Scholar 

  • Müller, T., and S. Pickel. 2007. Wie lässt sich Demokratie am besten messen? Zur Konzeptualität von Demokratie-Indizes. Politische Vierteljahresschrift 48 (3): 511–539.

    Article  Google Scholar 

  • Munck, G. L. 2009. Measuring democracy. Baltimore: The John Hopkins University Press.

    Google Scholar 

  • Munck, G. L., and J. Verkuilen. 2002. Conceptualizing and measuring democracy: Evaluating alternative indices. Comparative Political Studies 35 (5): 5–34.

    Google Scholar 

  • Munck, G. L., and J. Verkuilen. 2009. Conceptualizing and measuring democracy. An evaluation of alternative indices. In Measuring democracy, ed. Gerardo L. Munck, 13–37. Baltimore: The John Hopkins University Press.

    Google Scholar 

  • Nyheim, D. 2009. Preventing violence, war, and state collapse: The future of conflict early warning and response. Paris: Organisation for Economic Co-operation and Development.

    Google Scholar 

  • Organisation for Economic Co-operation and Development (OECD). 2008. Concepts and dilemmas of state building in fragile situations. Paris: Organisation for Economic Co-operation and Development.

    Google Scholar 

  • Rice, S. E., and S. Patrick. 2008. Index of state weakness in the developing world. Washington D.C.: Brookings Institution.

    Google Scholar 

  • Rotberg, R. I. 2004. The failure and collapse of nation-states. In When states fail: Causes and consequence, ed. Robert I. Rotberg, 1–50. Princeton: Princeton University Press.

    Google Scholar 

  • Rotberg, R. I., and R. M. Gisselquist. 2008. Strengthening african governance: Index of african governance, results and rankings. Cambridge: Kennedy School of Government’s Program on Intrastate Conflict and Conflict Resolution.

    Google Scholar 

  • The White House. 2002. The national security strategy of the United States of America. Washington D.C.: The White House.

    Google Scholar 

  • Thomas, M. A. 2007. What do the worldwide governance indicators measure? The Paul H. Nitze School of Advanced International Studies.

  • Tilly, C. 1985. War making and state making as organized crime. In Bringing the state back in, eds. P. B. Evans, D. Rueschemeyer and T. Skocpol, 169–187. Cambridge: Cambridge University Press.

    Chapter  Google Scholar 

  • Treier, S., and S. Jackman. 2008. Democracy as a latent variable. American Journal of Political Science 52 (1): 201–217.

    Article  Google Scholar 

  • United States Agency for International Development (USAID). 2006. Measuring fragility indicators and methods for rating state preformance. Silver Spring: USAID.

    Google Scholar 

  • Wennmann, A. 2010. Grasping the strengths of fragile states: Aid effectiveness between “Top-down” and “Bottom-up” statebuilding. CCDP Working Paper 6. Geneva: The Centre on Conflict, Development and Peacebuilding.

    Google Scholar 

  • World Bank. 2009a. IDA Resource Allocation Index (IRAI)—2008. http://go.worldbank.org/S2THWI1X60. Accessed 1 Feb 2011.

  • World Bank. 2009b. Governance matters 2009: Data and research papers. http://info.worldbank.org/governance/wgi/resources.htm. Accessed 1 Feb 2011.

  • World Bank. 2009c. Governance matters 2009: Frequently asked questions. http://info.worldbank.org/governance/wgi/faq.htm. Accessed 1 Feb 2011.

  • World Bank. 2010. Fragile and conflict-affected countries: Definitions of fragility and conflict. http://go.worldbank.org/6B4932MAV0. Accessed 1 Feb 2011.

  • Wulf, H., and T. Debiel. 2009. Conflict early warning and response mechanisms: Tools for enhancing the effectiveness of regional organisations? A comparative study of the AU, ECOWAS, IGAD, ASEAN/ARF and PIF. Crisis States Working Papers Series 49 (2).

  • You, D., T. Wardlaw, P. Salama, and G. Jones. 2010. Levels and trends in under-5 mortality, 1990–2008. The Lancet 375 (9709): 100–103.

    Article  Google Scholar 

  • Ziaja, S., and J. Fabra Mata. 2010. State fragility indices: Potentials, messages and limitations. Briefing Paper 10/2010. Bonn: Deutsches Institut für Entwicklungspolitik.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. German Development Institute/Deutsches Institut für Entwicklungspolitik (DIE), Tulpenfeld 6, 53113, Bonn, Germany

    Sebastian Ziaja

  2. Department of Government, University of Essex, Essex, Colchester, UK

    Sebastian Ziaja

Authors
  1. Sebastian Ziaja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Ziaja.

Additional information

For comments and advice, I would like to thank Jörn Grävingholt, Javier Fabra Mata, Jörg Faust, Christian von Haldenwang, participants of the ZfVP Workshop “Indizes in der Vergleichenden Politikwissenschaft” in May 2009 in Leipzig and two anonymous referees.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ziaja, S. What do fragility indices measure?. Z Vgl Polit Wiss 6 (Suppl 1), 39–64 (2012). https://doi.org/10.1007/s12286-012-0123-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12286-012-0123-8

Keywords

Schlüsselwörter

Search

Navigation