The ‘Failed State Index’ Offers More than Just a Simple Ranking
- First Online:
- Cite this article as:
- Carlsen, L. & Bruggemann, R. Soc Indic Res (2014) 115: 525. doi:10.1007/s11205-012-9999-6
The role of indicators to measure trends in every area of interest is increasing. Especially in the field of politics and sociology, where modeling based on multiple indicators typically is difficult, multi-indicator systems call for attention. Multi-indicator systems are most often the first step to derive a ranking indicator. Correspondingly there is a high interest in how to transform multi-indicator systems into a one-dimensional metric scale. The scientific discipline of decision support systems provides many well-known techniques, classical examples are PROMETHEE, ELECTRE, DEA or the simple TOPSIS. The mathematical technique is pretty sophisticated, therefore the simplest variant, namely the weighted sum of indicators plays its role too, just because of its simplicity and transparency. Although the need of a derivation of a one-dimensional scale is evident, we argue that there is an interim step, between the multi-indicator system and the ranking index, provided by simple elements of partial order theory. This interim step bears useful information too and in this paper we show how and which useful additional information can be derived. We derive for example a bias-free sensitivity study, where the indicator “chronic and sustained human flight” turns out to be the most important indicator within the multi-indicator system of 12 indicators, which are the basis of the Failed State Index.
KeywordsFailed States Index Partial order ranking Hasse diagram technique Indicator analyses Multicriteria analyses Decision support tools PyHasse
1 Introduction: The Multi-indicator System on Which FSI is Based
Once a year the organization Foreign Policy (FP 2011) publishes the so-called ‘Failed State Index’ (FSI) (FSI 2011) that is an annual ranking of world’s most vulnerable states prepared by the Fund of Peace (FFP 2011a). For 2011, 177 states have been included in the study. Hence, the FSI for 2011 has been derived based data originating from 2010 comprising around 130,000 publicly available sources.
The data are collected into 12 so-called conflict assessment indicators or indicators of pressure (FFP, 2011b) comprising social indicators [(1) mounting demographic pressures, (2) massive movement of refugees or internally displaced persons, (3) legacy of vengeance-seeking group grievance or group paranoia, (4) chronic and sustained human flight)], economic indicators [(5) uneven economic development along group lines, (6) sharp and/or severe economic decline)], and political/military indicators [(7) criminalization and/or delegitimization of the state, (8) progressive deterioration of public services, (9) suspension of the rule of law and widespread violation of human rights, (10) security apparatus operates as a “state within a state”, (11) rise of factionalized elites, (12) intervention of other states or external political actors)] (FFP 2006; Baker 2006). Eventually the FSI is based on a composite indicator, or super indicator that is simply the sum of the 12 individual indicators for each country thus leading to single values and as such a simple index, i.e., linear order of the 177 countries.
2 The Inherent Problem in Deriving a Composite Indicator
2.1 Effect of Compensation
Obviously, having the simple scale based on the additive index construction, ranking is easy and straight forward. However, what is the price for this simplicity? The price is in the best case that a significant amount of valuable information is lost. More unfortunate is that such a simple addition of the indicator values may lead to quite erroneous conclusions as high score(s) in certain indicator(s) may be leveled off by low scores in other indicator(s), without taking into account that these indicators point towards quite different topics. In plain words such a simple addition is adding apples and oranges, the eventual result being bananas ranked according to length!
In the present paper we give an example how more information can be retrieved from the FSI data applying to total set of data, i.e., having a data table (technical: data matrix) of 177 × 12, using partial order methodology (Brüggemann and Carlsen 2006; Brüggemann and Patil 2011). Hence, the present paper will unveil some features of partial order ranking. A full paper on an elaborate analysis of the ‘FSI’ by partial order methodology will be published elsewhere. Thus, the reader is recommended to consult two recent text books for details on partial order methodology and its applications (Brüggemann and Carlsen 2006; Brüggemann and Patil 2011).
2.2 Comparability Versus Incomparability
The central concept in partial order is the ‘concept of comparison’. Thus, the 177 states are mutually compared by comparing the single indicator values. Consequently, some states are comparable, because the respective indicator values do not show any conflict, whereas others cannot be compared, because conflicts are displayed between indicator values. To exemplify this we can look at states #1 (Somalia) and #3 (Sudan) where the 12 indicator values are 9.7, 10, 9.5, 8.2, 8.4, 9.3, 9.8, 9.4, 9.7, 10, 9.8, 9.7 and 8.5, 9.6, 9.9, 8.2, 9.1, 6.4, 9.4, 9.0, 9.7, 9.6, 9.9, 9.5, respectively. These two states cannot be directly compared as, e.g., the first two indicator values for #1 are higher than the corresponding values for #3, whereas the third indicator values for #1 is lower than the corresponding value for #3. On the other hand #1 (Somalia) is comparable to #10 (Cote d’Ivoire), with the indicator values 8.1, 8.5, 8.7, 7.9, 8.0, 7.7, 9.5, 8.4, 8.6, 8.6, 9.1, 9.7, as all indicator values for #1 are higher than the corresponding values for #10.
2.3 Order Theoretical Navigation in the Multi-indicator System
Due to the high number of objects (states) and indicators the diagram may at first sight appear rather confusing. In short it can be noted that states being comparable, i.e., the 12 indicator values do not show any conflicts are connected with lines in contrast to states where conflicting indicators are present.
Level population of the Hasse diagram based on the FSI data with 177 states and 12 indicators (Fig. 2)
1 2 3 4 5 6 7 8 9 11 12 14 15 18 22 41
10 13 16 17 19 20 21 23 24 25 26 27 28 29 30 32 33 35 36 37 40 43 44 52 53 54 55 57 59 61 71 72 75 77 90 100 103 109 120
31 34 38 39 42 45 46 47 48 49 50 51 56 58 60 62 63 64 65 66 67 68 69 70 73 74 76 78 79 81 82 84 85 87 89 91 92 93 94 96 99 101 104 116 117 122 123
80 83 86 88 95 97 98 102 105 106 107 108 110 111 113 114 115 118 119 124 125 127 131 132
112 121 126 128 129 130 133 134 135 136 138 140 146 148 165
137 139 141 142 143 144 145 147 149 150 151 152 153 154 155 156 158 159 160 161 162 163 166
157 164 167 168 169 170 171 172 173 174 175 176 177
Scrutinizing Table 1 it becomes clear that although we do see an overall tendency that mimics that of the FSI ranking, pronounced differences are immediately noted. Thus states that are ranked rather low at the FSI scale like #120 (Grenada) and #123 (Brazil) can be found at rather high ranked level 6 and 5, respectively. Further it can, e.g., be noted (Level 7) that the 9 states being ranked according to the FSI as the most vulnerable states (#1–9) are not comparable.
Comparing to Fig. 1 and Table 1, we note that state #107, located at level 4, upwards is comparable to some, but not all states both at levels 5, 6, and 7, respectively (Fig. 2a), similarly downwards #107 is comparable to several of the states located at levels 2 and 1, respectively, but not to any of the states located at level 3. In other words, partial order methodology allows “navigating” through a Hasse diagram in order to reveal peculiarities of each state of interest.
2.4 Indicator Importance
A major count of controversy in the FSI ranking seems to be the fact that all indicators are given an equal weight. However, looking at the single indicators it is hard to believe that they actually should be equally important for rating the countries according to their stability or most at-risk of collapse and violence, respectively. A further important issue would be to disclose priority areas where the single states, possibly with outside help should focus in order to increase the stability of the state.
Partial order methodology offers a unique possibility to disclose the relative importance of the single indicators both on a global and a local scale. Details about the analysis of indicator importance can be found in Brüggemann et al. (2001). The theoretical development of order theory provides different measures to quantify the impact of different indicators on the partial order: A generalized distance concept, where the role of each indicator versus all remaining others is quantified, and an analysis of the role of data if the indicator values are considered as uncertain. This second aspect still is not published in detail, however, see Bruggemann and Patil (2011). For both concepts a unifying analysis was recently published. The theoretical approach is a variance - based sensitivity study that can be found in Annoni et al. (2011).
From Fig. 3 it is seen, maybe not surprising, but not directly intelligible from the original FSI that on the global scale the absolute top rating among the indicators are indicator 4, which describes the ‘chronic and sustained human flight’, i.e., brain drain, followed by indicators 3 (‘legacy of vengeance-seeking group grievance or group paranoia’) and indicator 6 (‘sharp and/or severe economic decline’).
Based on these estimations it may be concluded that initiating programs that will secure not only the generation of human capital but of equally high priority that the human capital generated in the single countries is retained there for the benefit of the country. Further focus areas appear to be activities to decrease group paranoia and reduce or even stop the economic decline.
If we look at the relative importance of the 12 FS indicators on a local scale, i.e., referring to single states, the same picture develop. For the most vulnerable states the picture is clear as the above. Thus, for, e.g. #1 (Somalia) i4 ≫ than i3, whereas for most stable states, like e.g. #173 (Denmark), the single indicators appear to be virtually of equal importance.
The present paper elucidates that the extreme elaborate data material available in the ‘FSI’ offers much more than just a simple ranking based on a composite indicator where all single indicators are assigned equal weight.
First of all we demonstrate that many comparisons of the single states are not possible using the FSI as a substantial number of incomparisons prevail which is unequivocally disclosed applying the full data material. However, we also demonstrate that comparisons between states without any additional assumptions (such as weights) can be deduced.
Secondly we demonstrate that an analysis of the relative importance of the 12 FI indicators provides important information in relation to most appropriate focus areas in an attempt to help and support the most vulnerable states.
A full record on the possibilities applying partial order methodology to decode the encapsulated information in the FSI data is currently in the process of being published. It may further be of interest to the reader to see another paper evaluating nations, based on the Eurobarometer, also applying partial order methodology (Annoni and Brüggemann 2009). The concept of relative importance has received considerable interest, thus it may be of value for the reader to see recent developments of sensitivity analysis (Annoni et al. 2011).