Difference of Means Formulations for Selected Indices

Abstract

In this chapter I review the implementation of the difference of means framework for calculating indices of uneven distribution for five indices: the delta or dissimilarity index (D), the gini index (G), the separation index (S), the Theil entropy index (H), and the Hutchens square root index (R). For each index I introduce the relevant scoring systems for residential outcomes (y) that makes it possible to obtain the index scores as simple differences of group means on individual residential outcomes (y). To facilitate discussion, I replace the abstract terms “Group 1” and “Group 2” with the more concrete example of Whites and Blacks which has been investigated in hundreds of empirical analyses of uneven distribution in U.S. cities, urban areas, and metropolitan areas. As I move from index to index, I provide comments on the nature of the scaling function that maps scores for contact and exposure based on pairwise area proportion White (p) onto index-specific residential outcome scores (y). In addition, I sometimes offer commentary on the index. Note, however, that I do not provide a comprehensive review of the five indices because this task has been addressed previously in the existing literature and does not need to be repeated here.

In this chapter I review the implementation of the difference of means framework for calculating indices of uneven distribution for five indices: the delta or dissimilarity index (D), the gini index (G), the separation index (S), the Theil entropy index (H), and the Hutchens square root index (R). For each index I introduce the relevant scoring systems for residential outcomes (y) that makes it possible to obtain the index scores as simple differences of group means on individual residential outcomes (y). To facilitate discussion, I replace the abstract terms “Group 1” and “Group 2” with the more concrete example of Whites and Blacks which has been investigated in hundreds of empirical analyses of uneven distribution in U.S. cities, urban areas, and metropolitan areas. As I move from index to index, I provide comments on the nature of the scaling function that maps scores for contact and exposure based on pairwise area proportion White (p) onto index-specific residential outcome scores (y). In addition, I sometimes offer commentary on the index. Note, however, that I do not provide a comprehensive review of the five indices because this task has been addressed previously in the existing literature and does not need to be repeated here.

4.1 Scoring Residential Outcomes (y) for the Delta or Dissimilarity Index (D)

I begin with the delta or dissimilarity index (D) because it is by far the most widely used index of uneven distribution. I review two scoring schemes for the function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for the delta index (D). One is based on interpreting D as a crude variant of the gini index (G). I discuss this below after I have introduced and reviewed the scoring scheme for G. First, however, I review a scoring scheme for D that is especially simple, easy to explain, and attractive on substantive grounds. In this scheme D is obtained as a difference of group means (Y₁–Y₂) based on assigning residential outcomes (y) for individuals a value of either 0 or 1 based on whether area proportion White (p) for their area of residence equals or exceeds proportion White for the city (P).¹ Thus, the relevant scaling function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for D is a monotonic, binary step function where $\mathrm{y}=1$ when $\mathrm{p}\ge \mathrm{P}$ and 0 otherwise (i.e., when p < P), where $\mathrm{p}={\mathrm{n}}_1/\left({\mathrm{n}}_1+{\mathrm{n}}_2\right)$ and $\mathrm{P}={\mathrm{N}}_1/\left({\mathrm{N}}_1+{\mathrm{N}}_2\right)$ per expressions introduced earlier with counts for Whites being used for Group 1 (the reference group) and counts for Blacks being used for Group 2 (the comparison group).

I review the underlying formal basis for this scoring of residential outcomes in Appendices. The material also provides detailed discussions establishing the formal basis for scoring function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for all indices considered in the body of this paper. The discussions are mostly dry and tedious. But I encourage interested readers to review the discussions to verify the basis for the scoring functions and to gain additional insights into the underlying nature of different indices.

The scoring of residential outcomes as either 0 or 1 based on whether area proportion White (p) equals or exceeds the city mean (P) supports a simple, straightforward substantive interpretation of D in terms of group differences in “exposure” and “contact”. Specifically, D can be understood as the White-Black difference in the proportion in each group that experiences “parity” in (pairwise) contact with Whites. Parity here is equated to attaining at least the level of (pairwise) proportion White seen for the city overall. Noting this substantive interpretation for D introduces a theme that will recur throughout this chapter. It is that:

D and all other popular indices of uneven distribution can be interpreted as measures of simple group differences on residential outcomes of scaled group “exposure” or “contact” for individuals.

The contact interpretation of D is simple and easy to grasp. In light of this, it is surprising that it is so infrequently discussed in the broader literature. Instead, it is much more common for the substantive interpretation of D to be framed in terms of the extensiveness of group displacement from even distribution based on “volume of group movement”. In this interpretation D indicates “the minimum proportion of one group that would have to move to a new area to bring about even distribution.”² This interpretation of D is useful for some purposes and it is often seen as an interpretation that is easy to convey to broad audiences. For example, it is relevant for policy analysis assessing consequences of segregation in terms of the “disruption” in residential patterns (or school attendance patterns) that would result if policies promoting integration were implemented.³ But the group difference of means interpretation of D also is useful, very easy to compute, and very easy to convey to broad audiences. So these are not decisive factors for the neglect of this straightforward contact interpretation of D.

Regardless of what factor(s) account for it, the lack of attention given to the difference of mean contact interpretation for D has an unwelcome consequence. It has led researchers to be less familiar with an important property of D; namely, that D is inherently insensitive to, and conveys very little information about, the quantitative magnitude of group differences on residential outcomes. For example, a high value of D means that a higher proportion of Whites than Blacks live in areas that attain parity with the city-wide level of proportion White. But the high value does not, and it inherently cannot, signal whether the kinds of areas that Whites and Blacks typically live in are relatively similar on proportion White or fundamentally different. Relatedly, while the value of D signals the minimum proportion of one group that will need to move to eliminate uneven distribution, it provides little insight into the changes in neighborhood outcomes that would result for the two groups in the comparison.

The value of D does not indicate whether movement to bring about even distribution will lead to substantively important changes in the residential outcomes for the individuals in either group. Specifically, it does not indicate whether movement will bring about sociologically meaningful changes in neighborhood racial composition.

This is not a trivial concern. Accordingly I give it extended attention at several other points in this monograph as well as here. The reason it is not trivial can be put in simple terms. It is logically possible for D to take high values when Whites and Blacks live in areas that are fundamentally similar on area proportion White. In this circumstance, residential redistribution leading to integration would indeed require a high proportion of one group to move, but the movement will not lead to important changes in their residential outcomes or in their comparison on these outcomes with the other group. This important possibility appears not to be widely recognized and appreciated by segregation researchers. It is safe to say it is almost never recognized by broader consumers of segregation research.

In my experience non-specialists and researchers alike overwhelmingly interpret D in a way that is oblivious to this quality and as a result leads them to be prone to make mistaken inferences about the nature of segregation. Specifically, researchers as well as non-specialists are prone to assume that high values of D necessarily indicate that most members of both of the groups in question live apart from each other in areas where their group predominates and as a result elimination of uneven distribution will lead to important changes in racial mix and associated residential outcomes for at least one group. This, of course, is sometimes the case. But it is important to recognize that it is not necessarily the case. Furthermore, this latter outcome is not an esoteric or unusual hypothetical possibility that can be safely ignored. To the contrary, as I show in empirical analyses I review in Chap. 6, instances where high values of D occur but both groups on average live in neighborhoods that are fundamentally similar on neighborhood outcomes can be found with surprising frequency when one systematically examines group differences in residential distribution in detail.

It may be helpful to make the issue more concrete by considering an example where inattention to this issue can lead to an incomplete and potentially misleading understanding of segregation patterns. A relevant case is the comparison of White-Black segregation and White-Asian segregation. Studies generally find that, on average, D for White-Black segregation is higher than D for White-Asian segregation. I find a similar result based on analysis of block-level data for core-based statistical areas (CBSAs) in 1990, 2000, and 2010 with the median value of D being 71.8 for White-Black segregation and 62.8 for White-Asian segregation.⁴ The difference of 9.0 points is relatively modest and suggests that, while White-Asian segregation is appreciably lower than White-Black segregation, White-Asian segregation still should be seen as fairly high.

With values of D being so high for both comparisons, one might assume that Black and Asian residential outcomes would be relatively similar and that most members of both groups would tend to reside in areas where their group predominates and not with Whites. But this is not the case at all. Blacks consistently reside in areas where Blacks predominate; across the CBSAs in the full data set the median for Black (pairwise) contact with Whites is 43.1 % and the median for Black (pairwise) contact with Blacks is 56.9 %. In contrast, Asians rarely reside in areas where Asians predominate; across CBSAs the median for Asian (pairwise) contact with Whites is 83.4 % and the median for Asian (pairwise) contact with Asians is 16.6 %. These results indicate that White-Black segregation is quantitatively fundamentally different from White-Asian segregation even when they have similar values on D. The results for D do not suggest this, but results for the separation index (S) – an alternative measure of uneven distribution I discuss in more detail below – do signal that the group comparisons are very different. In the same analysis I found the median value for the separation index was 48.3 for White-Black segregation and 13.8 for White-Asian segregation. The difference in the two segregation comparisons is much more dramatic using S; the typical level of S for White-Black segregation is three times the typical level of S for White-Asian segregation. This example highlights that D is insensitive to an important aspect of segregation; namely, group residential separation and neighborhood racial polarization. I review this issue in more detail in the next chapter.

4.2 Scoring Residential Outcomes (y) for the Gini Index (G)

For the gini index (G), the relevant function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for scoring residential outcomes (y) so G can be obtained from the difference of group means (Y₁–Y₂) is based on relative rank position – that is, quantile or percentile standing – on area group proportion (p). Specifically, y = percentile scores based on p. This makes y an ever-rising, monotonic, nonlinear function of p. The percentile scores can be obtained as follows. Rank areas from low to high on p. Assign the first area (i.e., $\mathrm{i}=1$) the percentile score $\mathrm{y}=100\cdot \left(\frac{1}{2}\ {\mathrm{t}}_{\mathrm{i}}\right)/\mathrm{T}$. Assign the remaining areas (i.e., areas $\mathrm{i}=2,3,4\dots \mathrm{I}$) percentile scores based on $\mathrm{y}=100\cdot \left(\varSigma {\mathrm{t}}_{\mathrm{i}-1}+\frac{1}{2}{\mathrm{t}}_{\mathrm{i}}\right)/\mathrm{T}$. Under this scoring system, G/2 can be obtained from ${\mathrm{Y}}_1-{\mathrm{Y}}_2$. Alternatively, G can be obtained from $2\cdot \left({\mathrm{Y}}_1-{\mathrm{Y}}_2\right)$. I review the basis for these expressions in Appendices.

The fact that G can be obtained from percentile scoring of area group proportions results because G, when applied using its formulation as a segregation index, is a measure of ordinal or “rank order” inequality between groups. I have previously described this property of G in Fossett and Siebert (1997) and earlier, albeit less directly, in Fossett and South (1983). In Fossett and Siebert (1997: Appendix A) I note that G is equivalent to familiar indices of ordinal inequality and ordinal association. Specifically, G is mathematically equivalent to Lieberson’s (1976) index of net difference (ND), a measure of ordinal inequality between groups and G also is equivalent to Somers’ (1962) d_yx, an index of ordinal association. Based on this Fossett and Siebert (1997) show that G can be given as

$$ \mathrm{G}=100\cdot \varSigma \varSigma {\mathrm{y}}_{\mathrm{ij}}\cdot \left({\mathrm{n}}_{1\mathrm{i}}/{\mathrm{N}}_1\right)\cdot \left({\mathrm{n}}_{2\mathrm{j}}/{\mathrm{N}}_2\right) $$

or, more compactly,

$$ \mathrm{G}=100\cdot \varSigma \varSigma {\mathrm{y}}_{\mathrm{ij}}\cdot \left({\mathrm{s}}_{1\mathrm{i}}\cdot {\mathrm{s}}_{2\mathrm{j}}\right) $$

where i and j index areas ranked for low to high on area proportion White (p) and y_ij is scored −1 if $\mathrm{i}<\mathrm{j}$, 0 if $\mathrm{i}=\mathrm{j}$, and 1 if $\mathrm{i}>\mathrm{j}$ and s_1i and s_2j are group share scores given by ${\mathrm{s}}_{1\mathrm{i}}=\left({\mathrm{n}}_{1\mathrm{i}}/{\mathrm{N}}_1\right)$ ${\mathrm{s}}_{2\mathrm{j}}=\left({\mathrm{n}}_{2\mathrm{j}}/{\mathrm{N}}_2\right)$. This formula reveals that G registers only rank position on p and does not register the size of the quantitative differences involved.

The difference of means formulation of G supports and clarifies the interpretation of G as a measure of group difference on scaled “exposure” and “contact”. In the case of White-Black segregation, G is the White-Black difference in average relative rank position on contact with Whites (p).⁵ Alternatively, G is the White-Black difference in exposure to area percentile rank on proportion White. G’s equivalence to the index of net difference supports a related interpretation. G indicates the difference between two probabilities for rank order comparisons of individual Whites and Blacks on area proportion White. The first, P(A), is the probability that a randomly selected White will live in an area where proportion White is higher than that for a randomly selected Black. The second, P(B), is the probability that a randomly selected White will live in an area where proportion White is lower than that for a randomly selected Black.⁶ The value of G is given by P(A) − P(B) where $\mathrm{P}\left(\mathrm{A}\right)=\varSigma {\mathrm{X}}_{\mathrm{i}-1}{\mathrm{Y}}_{\mathrm{i}}$ and $\mathrm{P}\left(\mathrm{B}\right)=\varSigma {\mathrm{X}}_{\mathrm{i}}{\mathrm{Y}}_{\mathrm{i}-1}$ with X and Y denoting cumulative group proportions over areas ranked from low to high on area proportion White. This expands to $100\cdot \left(\varSigma {\mathrm{X}}_{\mathrm{i}-1}{\mathrm{Y}}_{\mathrm{i}}-\varSigma {\mathrm{X}}_{\mathrm{i}}{\mathrm{Y}}_{\mathrm{i}-1}\right)$, the formula for G from Duncan and Duncan (1955) given in Fig. 2.1. The main value of the net difference interpretation originally introduced by Lieberson (1976) is to drive home the point that G is a measure of inter-group rank order inequality on the residential outcome of area proportion White (p).

It is useful to briefly contrast G with D. Unlike D, G satisfies the principle of transfers and on this basis is technically superior to D.⁷ But G is similar to D in being unable to give a reliable signal about group separation and residential polarization. The reason for this is that G can take high values when the two groups in the comparison have similar distributions on area group composition. This is possible because G registers rank-order differences on area proportion White and will register such differences equally regardless of whether the quantitative differences on area proportion White are small or large. As a result, when on sees a high value on G, it is impossible to know whether the underlying pattern of segregation involves extensive group separation and extreme neighborhood racial polarization such as that observed for White-Black segregation in Chicago or involves a more benign pattern with minimal group separation and fundamentally similar neighborhood fate.

4.3 The Delta or Dissimilarity Index (D) as a Crude Version of G

The index of dissimilarity (D) can be understood as a special case of the gini index (G). Specifically, D is equivalent to G when areas are ranked using a two-category scheme based on whether ${\mathrm{p}}_{\mathrm{i}}<\mathrm{P}$ or ${\mathrm{p}}_{\mathrm{i}}\ge \mathrm{P}$, rather than being ranked on the full range of scores on p_i as would be the case with G. Thus, D is a version of G computed when areas are grouped into two categories based on whether or not they are at or above average on proportion White. Accordingly, D can be obtained using the formula for G after ranking areas on the basis of a two-value recoding of p_i as either 1, when ${\mathrm{p}}_{\mathrm{i}}\ge \mathrm{P}$, or 0 otherwise. These recoded values of p_i are then used to score y in terms of relative rank position on area group proportion as described above for G. Accordingly, the value of D/2 is given by ${\mathrm{Y}}_1-{\mathrm{Y}}_2$, or alternatively, D is given by $2\cdot \left({\mathrm{Y}}_1-{\mathrm{Y}}_2\right)$. If one were graphing the segregation curves associated with D and G for a given comparison, G would produce a conventional segregation curve the data for D would produce a segregation curve in the form of a triangle. The two curves would share three points. The two end points of the curve at (0,0) and (1,1) and one point along the curve at (X,Y) where the values of X and Y are equal to the proportion of Blacks and Whites, respectively, living in areas where $\mathrm{p}<\mathrm{P}$.

This provides insight into why D and G are highly correlated and why scores for D never exceed scores for G (i.e., $\mathrm{D}\le \mathrm{G}$). Both measures register White-Black differences in relative rank position on p_i. However, G registers all rank differences on p_i while D registers only rank differences where group comparisons on p_i are on opposite sides of P. This accounts for the difference between D and G in how they respond to population transfers or exchanges. G will register any transfer or exchange that affects at least one household’s rank position on area proportion White (p). D will register a transfer and exchange only if it causes the value of p for at least one household to shift from $\mathrm{p}<\mathrm{P}$ to $\mathrm{p}\ge \mathrm{P}$ or vice versa.

4.4 Scoring Residential Outcomes (y) for the Separation Index (S)

I use the term Separation Index (S) to refer to a measure that has been known by many names over the decades. A partial list of past names includes: the correlation ratio and eta squared (${\eta}^{{}^2}$) (Duncan and Duncan 1955; Stearns and Logan 1986; Iceland et al. 2002), r or r_ij (Coleman et al. 1975, 1982), the variance ratio (V) (James and Taeuber 1985), and segregation index (S) (Coleman et al. 1966; Zoloth 1976).⁸ I term this measure the separation index because a high value on this index gives a clear and reliable signal that the two groups in the comparison are residentially separated and generally do not reside in the same areas.⁹ That is, it indicates whether the two groups live apart from each other due to being concentrated in areas that are racially polarized in a pattern of “prototypical” segregation wherein, in the example of White-Black segregation, Whites live in predominantly White areas and Blacks live in predominantly Black areas. I clarify the basis for this claim in more detail shortly.

For the separation index (S), the relevant function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for scoring residential outcomes (y) so S can be obtained from (${\mathrm{Y}}_1-{\mathrm{Y}}_2$) is quite simple; it is the identify function $\mathrm{y}=\mathrm{p}$. I review the formal basis for this scoring of residential outcomes for S in Appendices.¹⁰ The scaling function used for S is distinct from those that are used for other popular indices of uneven distribution. It maps the contact score (p) directly onto residential outcome scores (y) based on a one-to-one linear relationship. In contrast, the scaling functions for all other indices map the contact score (p) onto residential outcome scores (y) based on some form of positive, monotonic, nonlinear relationship.

The separation index supports a clear and appealing interpretation based on pairwise “exposure” and “contact”. In the case of White-Black segregation, S is the White-Black difference on average contact with Whites (p). From this vantage point, it becomes clear why it is appropriate to refer to this measure as the “separation index”. The White-Black difference in contact with Whites can be large only if Whites live separately from – that is, apart from, not with – Blacks in neighborhoods that are predominantly White and Blacks live separately from Whites in areas that are predominantly Black. To clarify, in most applications indices of uneven distribution are implemented as “aspatial” measures. In this application, the notion of separation implies only that the groups live in different areas. It does not imply that the different areas are necessarily spatially distant from each other. This would be the case when segregation involves large-scale clustering. But the index score would be the same if Whites and Blacks lived separately from each other in different areas forming a checker board pattern.

The separation index also could be aptly termed the “contact difference” index, but that is a bit cumbersome. Alternatively, it could be named the “concentration” index following Stearns and Logan (1986), but Massey and Denton (1988) popularized the term “concentration” in association with another distinct dimension of segregation. So I adopt the term “separation index” (S) which emphasizes that the measure is sensitive to whether groups live apart from each other and are separated into different areas that differ fundamentally on group composition.

The notion of group separation is closely connected with the notion of area or neighborhood racial polarization discussed by Stearns and Logan (1986).¹¹ As they used it, polarization is high when the areas in which the two groups live fall primarily into two types. In the case of White-Black segregation that would be either predominantly White or predominantly Black with few areas in between. Their usage of the term polarization directs attention to a neighborhood outcome. But polarization of neighborhood racial composition has obvious implications for group differences on residential outcomes for individuals. When areas are racially polarized, individuals in both groups primarily live in neighborhoods where members of their group predominate. In the example of White-Black segregation, Whites live primarily in White neighborhoods and Blacks live primarily in Black neighborhoods. This resonates with the idea that groups live separate and apart from each other, a necessary, but not sufficient, precondition for experiencing disparities on neighborhood residential outcomes other than racial composition per se (e.g., crime, social disorder, inferior amenities, poor schools, and poor government services, etc.). Given this close similarity of group separation and neighborhood polarization, it would not be unreasonable to call the separation index the “polarization” index. But I reserve that term for an alternative measure which I will introduce and discuss shortly.

As noted earlier, I endorse Stearns and Logan’s view that the separation index (S) taps an aspect of uneven distribution that is sociologically important and is not consistently captured by other measures. In particular, the presence or absence of group separation is not captured well by the more widely used delta or dissimilarity index (D). It is interesting then to note that D is used much more widely than S. To be sure, the separation index has been used in segregation studies for many decades – for example, it was given close attention in Duncan and Duncan’s (1955) landmark article on segregation indices.¹² Moreover, it consistently receives high marks in technical reviews of indices (e.g., Zoloth 1976; White 1986; Reardon and Firebaugh 2002). Additionally it has been shown to be far less susceptible than D to the vexing problem of index bias (Winship 1977).¹³ Nonetheless, S is not used nearly as widely as D in empirical studies and its attractive qualities appear not to be widely appreciated. What could explain this?

At least three factors appear to be relevant. One is that the measure has never been consistently used under the same name and interpreted in a consistent way. This alone is likely to lead many people to underestimate both the frequency of its usage and the extent to which different researchers have endorsed its value for assessing segregation.

Another factor is that much of the usage of the index has involved terminology and interpretations that do not highlight what I view as the separation index’s strongest feature for substantive interpretation. For example, the measure has been used most widely under the names “variance ratio”, “correlation ratio”, and “eta squared” in the literature. These names are not technically incorrect or inappropriate. But they also do not call attention to the measure’s most attractive characteristic – its ability to signal when group residential distributions are polarized such that the two groups live separately from each other with members of both groups living primarily in areas where their group predominates. Instead, the names used in the past attention toward substantive interpretations relating to the strength of the individual-level, statistical association between the binary variable of race (coded 0–1) and the categorical variable of area of residence. These “statistical” interpretations are mathematically defensible, but they do not resonate with broad audiences and researchers perhaps because their substantive relevance for group differences in residential outcomes is neither obvious nor easy to convey.¹⁴

The difference of group means formulation of the separation index can potentially address these two points and enhance the attractiveness of S to researchers and broader audiences. The computation of S under the difference of means formulation is simple and easy to implement. In addition, this formulation of S has an appealing substantive interpretation that is easy to convey to both broad and technical audiences; it signals that uneven distribution involves groups residing separately from each other with both groups being disproportionately concentrated in racially polarized neighborhoods such that the two groups experience fundamentally different residential outcomes on area racial composition. Importantly, D, the most widely used index of uneven distribution does not provide a reliable signal for whether or not this pattern of segregation is present.

4.5 A Side Comment on the Separation Index (S) and Uneven Distribution

A third factor that may help explain why the separation index has not been used more widely requires a longer discussion. It is that S is occasionally viewed as a measure of group isolation and exposure rather than a measure of uneven distribution. At one level I view the controversy as minor because most technical reviews correctly characterize the separation index as a measure of uneven distribution (e.g., Zoloth 1976; James and Taeuber 1985; White 1986; Reardon and Firebaugh 2002). But there are contrasting descriptions of S in the literature so the issue warrants a brief side discussion.

Massey and Denton (1988) categorize S (which they refer to as V and eta squared) as an “exposure” measure rather than a measure of uneven distribution. One reason they offer for doing so is that, unlike D and G, S does not have a definite relationship to the segregation curve. This concern should be set aside for two reasons. The first reason is that Massey and Denton themselves do not apply this criterion in a consistent way. For example, they classify the Theil entropy index (H) as an index of uneven distribution but, like S, H also does not have a definite relationship with the segregation curve.

The second reason to set aside this concern is that many authoritative reviews of measures of uneven distribution disregard the segregation curve when evaluating indices (e.g., Zoloth 1976; Stearns and Logan 1986; Reardon and Firebaugh 2002). Some important statements explicitly and forcefully dismiss the relevance of the segregation curve altogether (White 1986; Coleman et al. 1982). I endorse these views. I recognize that the segregation curve is a visually appealing and is potentially a useful graphical tool for depicting group differences in residential distribution. But it does it not embody “the” definitive definition of uneven distribution and it also has clear limitations and deficiencies. For example, the segregation curve does not, and it logically cannot, signal whether the two groups in the comparison live apart from each other in areas that differ in substantively important ways on area racial composition. Accordingly, most authoritative methodological reviews classify both H and S as valid measures of uneven distribution often noting features of these indices that make them attractive for many purposes (e.g., Zoloth 1976; James and Taeuber 1985; White 1986; Reardon and Firebaugh 2002).

Another possible basis for Massey and Denton’s characterization of S as an exposure index is that, under certain circumstances, particular computing formula for S contain terms that are similar to terms found in formulas for exposure indexes. For example, many have noted that S has similarities to Bell’s (1954) revised index of isolation which involves terms that have exposure interpretations (e.g., Duncan and Duncan 1955; Becker, McPartland, and Thomas 1978; Iceland et al. 2002; James and Taeuber 1985: footnote 4; Stearns and Logan 1986). In the final analysis, however, it is clear that there are fundamental logical differences between S and exposure and isolation indices.

The first important logical difference is the population comparison involved. Exposure and isolation indices are calculated by comparing group counts to counts for the full population, not just the two groups in the segregation comparison. In contrast, S, like other indices of uneven distribution, is calculated from “pairwise” counts; that is, it is calculated using only the counts for the two groups in the comparison and is unaffected by the counts for other groups. The distinction can be crucially important in empirical applications because scores and substantive implications of “overall” and “pairwise” isolation can be and often are quite different.

The second important logical difference that distinguishes S from pairwise isolation indices is that the pairwise isolation term incorporated in some computing formulas for S is modified by a “normalizing” calculation. This calculation is crucially important to the issue at hand because it can and often does radically change its value. Equally importantly, the normalizing calculation also fundamentally changes the substantive interpretation of S. Specifically, S does not register the level of pair-wise isolation. It registers something distinctly different; S registers the relative extent to which pair-wise isolation exceeds its expected value. This is fundamentally different from pair-wise isolation itself. The normalizing calculation in this particular formulation of S has a crucially important consequence; it eliminates the mathematical correspondence between isolation and group composition. Thus, while group composition has important implications for the value of pair-wise isolation scores, it has no necessary or mathematically inherent implication for S. As a result, S can take any value over its logical range of 0 to 1 under any arrangement on group composition for the city. This is not the case for measures of isolation. They must take high values when the group in question is large in relative terms.

In sum, S is fundamentally distinct from standard indices of isolation and exposure. Isolation terms found in some formulas for computing S are based on pairwise counts, not overall counts, and they are subject to a normalizing transformation that radically changes their value and eliminates any mathematical correspondence between city ethnic composition and the value of S. Consequently, one cannot reliably infer the value of either overall or pairwise isolation from knowledge of the score of S or vice versa.

Given the confusion in the literature on this issue, it may be useful to consider the hypothetical example of a population with three groups – Whites, Blacks, and Latinos. Then assume that Whites live apart from Blacks and Latinos but that Blacks and Latinos live together. S will register the pattern of uneven distribution as high for both White-Black and White-Latino segregation and low for Black-Latino segregation. Importantly, this result will be the same regardless of the city racial composition. In contrast, both overall isolation and also Bell’s revised index of isolation will vary depending on city racial mix. The revised index of isolation for Blacks will be higher in a city where Whites outnumber Latinos (e.g., Detroit and Cleveland) and it will be low in a city where Latinos outnumber Whites (e.g., El Paso or San Antonio). This issue is not narrowly academic; it can have important practical consequences for index scores and substantive conclusions. This takes on increasingly relevance in recent decades as the growth of the Latino and Asian populations has resulted in more complex racial demography in many cities.

Finally, I close this discussion by stressing that S is not unusual among measures of uneven distribution in having linkages and interpretations relating to exposure and contact. To the contrary, one of the valuable insights gained from the difference of means formulations of indices of uneven distribution set forth in this monograph is that all popular indices of uneven distribution have direct and definite linkages to pairwise contact and exposure. Thus, for the example of White-Black segregation, both S and D can be obtained as simple group differences on exposure to Whites. In the case of S exposure is assessed directly by area proportion White (p). The only difference for D is that exposure is rescaled to either 0 or 1 depending on whether p equals or exceeds P. Thus, the key differences between indices of uneven distribution are found not in whether the indices register contact and exposure – all popular indices do this. The key differences are found in how the specific indices scale contact and exposure differently based on the way segregation-relevant residential outcomes (y) are scored from area group proportion (p).

4.6 Scoring Residential Outcomes (y) for the Theil Index (H)

For the Theil index (H), the relevant function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for scoring of residential outcomes (y) is a continuous function of p in the manner of S, but the form of the function is more complex. Specifically, the function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ is the following continuous, ever-rising, nonlinear expression

$$ \mathrm{y}=\mathrm{Q}+\left[\left(\mathrm{E}-{\mathrm{e}}_{\mathrm{i}}\right)/\mathrm{E}\right]/\left({\mathrm{p}}_{\mathrm{i}}/\mathrm{P}-{\mathrm{q}}_{\mathrm{i}}/\mathrm{Q}\right) $$

where e_i is the entropy score for area i and E is the entropy score for the city as a whole. These are given by the calculations ${\mathrm{e}}_{\mathrm{i}}={\mathrm{p}}_{\mathrm{i}}\cdot \ln \left(1/{\mathrm{p}}_{\mathrm{i}}\right)+{\mathrm{q}}_{\mathrm{i}}\cdot \ln \left(1/{\mathrm{q}}_{\mathrm{i}}\right)$ and $\mathrm{E}=\mathrm{P}\cdot \ln \left(1/\mathrm{P}\right)+\mathrm{Q}\cdot \ln \left(1/\mathrm{Q}\right)$. I owe special thanks to Warner Henson, III for helping me identify the form of this function.¹⁵ I review the formal basis for this scoring of residential outcomes in Appendices.

Theil and Finizza (1971) and Theil (1972) argue that information theory provides an attractive conceptual grounding for using entropy calculations to assess segregation. But most researchers who use H adopt it on a more narrow and practical basis. In particular, H is often used because it is mathematically tractable in ways that facilitate decomposition analysis.¹⁶ The substantive relevance of area (e_i) and city-level (E) entropy scores are seen narrowly as quantifying two-group racial diversity with the expression (E ‐ e_i)/E thus registering uneven distribution as departure of area racial diversity from that which would occur under even distribution given the racial mix of the city population.

I show below that the nonlinear relationship between y and p is visually simpler and more intuitively appealing than the mathematical expression introduced above might suggest. In its essence, the function maps p into y based on an ever-rising, backwards “S-curve”. The undulations of the S-curve and its symmetry, or lack thereof, vary with the relative sizes of the two groups in the comparison.¹⁷ When the groups are identical in size, the undulations in the S-curve are moderate and the resulting curve is symmetrical. In this situation results for H and S tend to track each other very closely. When the two groups are unequal in size, the undulations in the S-curve for y−p relationship for H are asymmetrical and larger in amplitude and the resulting curve departs from linearity in greater degree. In this situation results for H and S may differ.

Under this system for scoring segregation-relevant residential outcomes (y), H can be obtained from (${\mathrm{Y}}_1-{\mathrm{Y}}_2$) and thus fits in the framework for measuring segregation set forth in this paper. As in the previous examples considered, the difference of means formulation of H shows that it can be interpreted in terms of scaled contact and exposure. In the case of White-Black segregation, H is the White-Black difference in average contact with Whites (p) scored on the basis of the nonlinear function described above. When P and Q are balanced (i.e., $\mathrm{P}=\mathrm{Q}=50$), the function is a symmetrical backwards “S”. As a result, the measure responds less to differences in p in the middle of its range (i.e., 25–75) and more to differences in the lower and higher ranges of p. When P and Q are imbalanced, one must study the y−p relationship to understand specifically how H responds differentially to contact over different ranges of p. I discuss this in more detail below.

4.7 Scoring Residential Outcomes (y) for the Hutchens Square Root Index (R)

At this point, only one measure of residential segregation that receives regular attention in methodological studies of indices of uneven distribution has yet to be considered. This is Atkinson’s index (A). While rarely used in empirical studies, it nevertheless has been discussed in several methodological studies of segregation indices. For example, James and Taeuber (1985) praise A for involving a user-specified parameter (δ) which they argue can be used to “tune” the index to be sensitive to particular regions of the segregation curve. Massey and Denton (1988) also comment that this is a potentially interesting quality of A. In contrast, White (1986) and Hutchens (2001, 2004) view this characteristic of A as undesirable. Indeed, they characterize it as a fundamental flaw. They point out that A is “asymmetric” when δ is set to any value other than 0.5 and argue that the property of asymmetry introduces conceptual complications most would view as impractical if not fatal altogether for general use of A in segregation research. For example, the property of asymmetry implies that White-Black segregation can be logically and quantitatively different from Black-White segregation. No one has endorsed this as a desirable quality of segregation indices. I follow White and Hutchens in endorsing the principle of symmetry for segregation indices and therefore limit my consideration of A to only its symmetric implementation A_(0.5) – the special case where δ is set to 0.5. Hereafter, my references to A are to this version so I drop the subscript.

I have not been able to discover a way to express the value of A as a simple difference of means on scores of y based on area group proportion scores (p). However, I have found a difference of means solution for an index that is a close conceptual and mathematical surrogate for A. The index I refer to is the Hutchens (2001, 2004) square root index (R). This index has gained currency in the study of occupational sex segregation, but has not yet gained wide usage in studies of residential segregation. Since it may be unfamiliar to some readers, I briefly note three equivalent formulas for R.

$$ \mathrm{R}=100\cdot \left(1-\varSigma \sqrt{\left({w}_i/\mathrm{W}\right)\cdot \left({\mathrm{b}}_i/\mathrm{B}\right)}\right)=100\cdot \left(1-\varSigma \sqrt{s_{Wi}\cdot {\mathrm{s}}_{Bi}}\right) $$

$$ \mathrm{R}=100\cdot \left[1-\varSigma \left({\mathrm{t}}_{\mathrm{i}}/\mathrm{T}\right)\ \sqrt{\left({\mathrm{p}}_i/\mathrm{P}\right)\cdot \left({\mathrm{q}}_i/\mathrm{Q}\right)}\right] $$

$$ \mathrm{R}=100\cdot \left[1-\varSigma \left({\mathrm{t}}_{\mathrm{i}}/\mathrm{T}\right)\sqrt{{\mathrm{p}}_i{\mathrm{q}}_i/\mathrm{PQ}}\right] $$

The similarity to Atkinson’s A can be seen by comparing the last formula with the following expression for A which obtains when the tuning parameter δ is set to 0.5.

$$ {\mathrm{A}}_{(0.5)}=100\cdot \left[1-{\left\{\varSigma \left({\mathrm{t}}_{\mathrm{i}}/\mathrm{T}\right)\cdot \sqrt{{\mathrm{p}}_i{\mathrm{q}}_i}\right\}}^2/\mathrm{PQ}\ \right]. $$

The close relationship of R and A also can be seen in the fact that the two map onto each other based on the following exact nonlinear relationships

$$ \mathrm{A}=2\cdot \mathrm{R}-{\mathrm{R}}^{{}^2}\kern0.50em \mathrm{and} $$

$$ \mathrm{R}=1-\sqrt{1-A}. $$

Values of R are numerically lower than values of A. But since the relationship of their scores is exact and continuous, the two indices yield identical rank-orderings of segregation comparisons. Hutchens (2001, 2004) argues that R is an attractive measure of segregation in its own right. I include R in the discussion here on that basis as well as because it is a close surrogate for A. Additionally, values of R have a very strong relationship with values of D in empirical studies and R fares much better than D in technical reviews.¹⁸

For Hutchens’ square root index (R), the relevant scoring of residential outcomes (y) is a continuous, ever-rising, nonlinear function of p. Specifically, the function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ is

$$ {\mathrm{y}}_{\mathrm{i}}=\mathrm{Q}+\left(1-\sqrt{{\mathrm{p}}_i{\mathrm{q}}_i/\mathrm{P}\mathrm{Q}}\ \right)/\left({\mathrm{p}}_{\mathrm{i}}/\mathrm{P}-{\mathrm{q}}_{\mathrm{i}}/\mathrm{Q}\right) $$

where p_i, q_i, P, and Q are as introduced earlier. Under this system for scoring residential outcomes (y), R can be obtained from (${\mathrm{Y}}_1-{\mathrm{Y}}_2$) and thus fits in the framework for measuring segregation set forth in this paper. I establish the formal basis for this scoring of residential outcomes for R in Appendices.

As with the other indices, this supports an interpretation of R in terms of scaled contact and exposure. In the case of White-Black segregation, R is the White-Black difference in average contact with Whites (p) scored on the basis of the nonlinear function shown above. Like H, the function produces a continuous, ever-rising, nonlinear curve that forms a backwards “S”. Also like H, the undulations in the nonlinear curve vary with the relative sizes of the groups in the comparison. When the groups are identical in size, the undulations in the S-curve are modest and symmetrical and the resulting curve is relatively close to linear. When the two groups are unequal in size, the undulations in the S-curve are asymmetrical and larger in amplitude and the resulting curve departs from linearity in greater degree. One must study the particular y−p relationship in each case to understand how R registers p over different ranges of p.

Hutchens (2001, 2004) argues R is an attractive index in part because it orders aggregate segregation scores in a manner consistent with the principle of segregation curve dominance advocated by James and Taeuber (1985). The Atkinson index (A), the Gini index (G), and the dissimilarity index (D) all also satisfy this principle. Accordingly, scores for R tend to correlate closely with scores for G and D and especially with scores for A.¹⁹ As I noted earlier, however, the principle of segregation curve dominance is controversial and only a few methodological reviews endorse it.²⁰ One reason for this mentioned earlier is that defining segregation in relation to the segregation curve eliminates two popular indices– the Theil entropy index (H) and the Separation Index (S) – that both have attractive features to recommend them and that both fare well in technical reviews.

In essence, the principle of segregation curve dominance requires that indices place segregation comparisons involving non-crossing segregation curves in the same order as would result from segregation curve analysis.²¹ Some methodological reviews explicitly reject the principle. Most reviews are less direct but, while not explicitly taking a position on the issue, they implicitly reject the principle by giving favorable evaluations of H and S which do not have the property.²² I view the principle of segregation curve dominance as undesirable because it assigns priority to segregation indices that are necessarily insensitive to group residential separation and neighborhood polarization. I emphasize the word “necessarily” because segregation curves register rank order differences between groups on area group proportion (p) without regard to whether group differences on p are large or small in magnitude. As a result, segregation curves can signal high levels of uneven distribution when group residential separation and neighborhood polarization are low. I view this with great concern and accordingly review the issue in more detail in the next chapter.

For now I conclude this discussion by arguing that it is important for researchers to at least have the option of focusing on uneven distribution that involves group separation and neighborhood polarization. In my view segregation that separates groups into residing apart from each other in different neighborhoods that differ fundamentally on racial composition is substantively compelling. Separation conceived in this way is a logical prerequisite for group disparities on neighborhood residential outcomes such as quality of schools, exposure to crime and social problems, availability and quality of services, etc. In contrast, uneven distribution that does not involve group separation and polarization does not necessarily create the logical potential for group differences on these kinds of stratification-related neighborhood outcomes.

To summarize, in this chapter I reviewed how five indices of uneven distribution – G, D, R, H, and S – all can be specified as differences of group means on residential outcomes (y) scored from area group proportion (p). These five indices represent the most popular, widely used, and carefully studied indices of uneven distribution in the literature on segregation measurement. Consequently, I conclude that all popular indices of uneven distribution have ready interpretations as measures of group differences in contact and exposure. All of the indices indicate that groups experience the maximum possible average difference on contact outcomes when uneven distribution is complete and groups live completely apart. Similarly, all of the indices indicate that groups experience identical contact outcomes under conditions of even distribution. From this vantage point, the substantive differences between the indices ultimately trace to one thing; the differences among them in how they register group differences on individual residential contact outcomes (y) in the intermediate ranges. The scaling function $\mathrm{y}=\mathrm{f}\left(\mathrm{p}\right)$ for each index provides insight into this. Accordingly, I review how this function varies across indices in the next chapter.