Skip to main content

Composite Index Ranking of Economic Well-Being in U.S. Metropolitan Areas: How Prevalent are Rank Anomalies?


Composite indicators have the advantage of summarizing complex multi-dimensional concepts in a single measurement. They also suffer from disadvantages such as subjectivity in choice of indicators, weighting, and aggregation methods. In this paper, we update Medcalfe’s (Social Indicators Research 139(3), 1147–1167. 2018) Economic Well-Being (EWB) index of US Metropolitan Statistical Areas with the latest (2017) available data. Using this index of EWB, we investigate two social choice violations that have been understudied in the composite indicators literature. We provide theoretical and empirical evidence of cycles and violations of independence of irrelevant alternatives. Depending on the number of cities and ranking components, incidence of these social choice violations can be large, creating ambiguity in a set of rankings. In general, having more ranking components reduces the expected and, for the most part, realized, incidence of social choice violations. Further, the results suggest that the EWB index rankings should potentially be interpreted in terms of rank tiers rather than in terms of individual rankings.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Data Availability

An excel spreadsheet of the EWB index data is available at the link in the appendix.

Code Availability

The appendix includes a link to the code and a description is included in the appendix.


  1. In the pairwise comparison of A and B, A scores 14 and B scores 16. In the pairwise comparison of B and C, B scores 13 and C scores 17. In the pairwise comparison of A and C, A scores 17 and C scores 13.


Download references


No funding was received for this research.

Author information

Authors and Affiliations



All authors contributed to the study conception and design. Medcalfe and Sanders wrote the introduction. Sanders wrote the theoretical background and Medcalfe performed the data collection, analysis, and writing for the EWB index 2017. Sanders wrote the theoretical and empirical implications of social choice violations under EWB. Ehrlich and Sanders wrote the results, visualizations and discussion section. Ehrlich did the computations and visualizations in the figures. Medcalfe and Sanders wrote the conclusion. All authors reviewed first drafts and made changes.

Corresponding author

Correspondence to Simon Medcalfe.

Ethics declarations

Conflict of interest

There are no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Essential Code Discussion

Appendix 1: Essential Code Discussion

All of the code for both the simulation and empirical analysis is freely available online at the following Github Repository:, and was written in R 3.6.1. This Appendix 1 summarizes the key functions that is provided within the lib/rankSum.R script, which is the analytical library that both the simulation script (R/rankSumSimulationTripletsOnlyParallel.R) and the empirical analysis script (R/rankSumFullAnalysisParallelTripletsOnly.R) extensively use for calculating IIA violations and cycles. These individual scripts make use of the doSNOW package to parallelize the code across 24 cores, which is required to compute the results in days instead of months.

The simulation script simulates every possible ranking for n number of columns (categories) for three hypothetical cities. The empirical script selects every possible triplet among the 40 cities, and then selects every n possible columns out of a total 10 empirical columns. The script runs both the simulation and empirical analysis from 3 to 10 columns.

For each of the empirical or simulated cities and each of the 3–10 possible columns, the checkForIndependenceViolationsAndCycles function is used to determine if an IIA violation or a cycle exists. The code for this function is listed in Fig. 

Fig. 5
figure 5

Function to find a cycle or an IIA violation

5. This function returns a vector with containing both a IIA violation indicator and a cycle indicator. This computations for finding IIA violations are located in the independenceViolationWithinSet (Fig. 

Fig. 6
figure 6

Function to find an IIA violations

6) function and the computations for detecting cycles are found in the cycleViolation (Fig. 

Fig. 7
figure 7

Function to find a cycle

7) function.

The independenceViolationWithinSet function (Fig. 6) computes the presence of a IIA violation. This is accomplished by first finding the rank of each city globally by calling the createIndependentRankColumn function on the entire triplet. Then each possible pair is ranked independently by using the createIndependentRankColumn function on each pair. The rank function is called on each pair’s global ranking to find the relative global rankings, which is then compared to the independent ranking, which only uses the pair. For strong violations, which is the subject of this paper, the ranking order needs to be reversed for when comparing a pair’s independent ordering with the pair’s global ordering. The containsTies function is used to determine if a tie exists, which implies a strong IIA violation did not occur. For weak IIA violations, ties are considered and if a tie exists in the independent ordering but not the global ordering (and the reverse), a weak violation is reported.

The cycleViolation function (A-3) computes the presence of a cycle. To do this each of the cities within the triplet are compared to each other and the number of wins and losses are calculated. The paricipant1BeatsParticipant2 function determines if a team beats another team by comparing the sum of the ranks. A cycle occurs if and only if all three cities records losses. If any of the cities have zero losses, then a cycle is impossible, otherwise a cycle occurred.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ehrlich, J., Medcalfe, S. & Sanders, S. Composite Index Ranking of Economic Well-Being in U.S. Metropolitan Areas: How Prevalent are Rank Anomalies?. Soc Indic Res 157, 543–562 (2021).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI:


  • Economic well-being
  • Quality of life
  • Community quality of life
  • Social choice violations
  • Composite index