An open-source framework for non-spatial and spatial segregation measures: the PySAL segregation module

Abstract

In human geography and the urban social sciences, the segregation literature typically engages with five conceptual dimensions along which a given society may be considered segregated: evenness, isolation, clustering, concentration and centralization (all of which can incorporate or omit spatial context). Over the last several decades, dozens of segregation indices have been proposed and studied in the literature, each of which is designed to focus on the nuances of a particular dimension, or correct an oversight in earlier work. Despite their increasing proliferation, however, few of these indices remain used in practice beyond their original conception, due in part to complex formulae and data requirements, particularly for indices that incorporate spatial context. Furthermore, existing segregation software typically fails to provide inferential frameworks for either single-value or comparative hypothesis testing. To fill this gap, we develop an open-source Python package designed as a submodule for the Python Spatial Analysis Library, PySAL. This new module tackles the problem of segregation point estimation for a wide variety of spatial and aspatial segregation indices, while providing a computationally based hypothesis testing framework that relies on simulations under the null hypothesis. We illustrate the use of this new library using tract-level census data in two American cities.

Appendices

A: Point estimation details

Here, we present and explain each formula for the segregation measures presented in Table 1 of Section 2.1. The respective literature used for each measure can be found in Table 4³⁶$^,$³⁷ in addition with the respective dimension.

For consistency of notation, we assume that $n_{ij}$ is the population of unit $i \in \{1,\ldots , I\}$ of group $j \in \{x, y\}$, also $\sum _{j}n_{ij} = n_{i.}$, $\sum _{i}n_{ij} = n_{.j}$, $\sum _{i}\sum _{j}n_{ij} = n_{..}$, ${\tilde{s}}_{ij} = \frac{n_{ij}}{n_{i.}}$, ${\hat{s}}_{ij} = \frac{n_{ij}}{n_{.j}}$. The segregation indices can be build for any group j of the data.

The Dissimilarity Index (D) is given by:

$$\begin{aligned} D=\sum _{i=1}^{I}\frac{n_{i.}\mid {\tilde{s}}_{ij}-\frac{n_{.j}}{n_{..}}\mid }{2n_{..}\frac{n_{.j}}{n_{..}}\left( 1-\frac{n_{.j}}{n_{..}} \right) }. \end{aligned}$$

(2)

The spatial D (SD) is given by:

$$\begin{aligned} SD = D-\frac{\sum _{i_1=1}^{I}\sum _{i_2=1}^{I}\left| {\tilde{s}}_{ij}^{i_1}-{\tilde{s}}_{ij}^{i_2} \right| c_{i_1i_2}}{\sum _{i_1=1}^{I}\sum _{i_2=1}^{I}c_{i_1i_2}}, \end{aligned}$$

(3)

where ${\tilde{s}}_{ij}^{i_1}$ and ${\tilde{s}}_{ij}^{i_2}$ are the proportions of the minority population in the units $i_1$ and $i_2$, respectively and where $c_{i_1i_2}$ denotes an element at $(i_1,i_2)$ in a matrix C, which becomes one only if $i_1$ and $i_2$ are considered neighbors.

The boundary spatial D (BSD) is given by:

$$\begin{aligned} BSD = D - \frac{1}{2}{\sum _{i_1=1}^{I}\sum _{i_2=1}^{I}w_{i_1i_2} \left| {\tilde{s}}_{ij}^{i_1} - {\tilde{s}}_{ij}^{i_2} \right| }, \end{aligned}$$

(4)

where

$$\begin{aligned} w_{i_1i_2} = \frac{cb_{i_1i_2}}{\sum _{i_2=1}^{I}d_{i_1i_2}}, \end{aligned}$$

where ${\tilde{s}}_{ij}^{i_1}$ and ${\tilde{s}}_{ij}^{i_2}$ are the proportions of the minority population in the units $i_1$ and $i_2$, respectively, and $cb_{i_1i_2}$ is the length of the common boundary of areal units $i_1$ and $i_2$.

The perimeter/area ratio spatial D (PARD) is a Spatial Dissimilarity Index that takes into consideration the perimeter and the area of each unit by adding a specific multiplicative term in the second term of BSD (the spatial effect):

$$\begin{aligned} \frac{\frac{1}{2}\left[ \left( \frac{P_i}{A_i} \right) +\left( \frac{P_j}{A_j} \right) \right] }{\mathrm{MAX}\left( \frac{P}{A} \right) }, \end{aligned}$$

(5)

where $P_i$ and $A_i$ are the perimeter and area of unit i, respectively and $\mathrm{MAX}(P{/}A)$ is the maximum perimeter–area ratio or the minimum compactness of an areal unit found in the study region.

The Gini coefficient (G) is given by:

$$\begin{aligned} G=\sum _{i_1=1}^{I}\sum _{i_2=1}^{I}\frac{n_{i_1.}n_{i_2.}\mid {\tilde{s}}_{ij}^{i_1}-{\tilde{s}}_{ij}^{i_2}\mid }{2n_{..}^2\frac{n_{.j}}{n_{..}}\left( 1-\frac{n_{.j}}{n_{..}} \right) }. \end{aligned}$$

(6)

The global entropy (E) is given by:

$$\begin{aligned} E = \frac{n_{.j}}{n_{..}} \ \mathrm{log}\left( \frac{1}{\frac{n_{.j}}{n_{..}}} \right) +\left( 1-\frac{n_{.j}}{n_{..}} \right) {\text {log}}\left( \frac{1}{1-\frac{n_{.j}}{n_{..}}} \right) , \end{aligned}$$

(7)

while the unit’s entropy is analogously:

$$\begin{aligned} E_i = {\tilde{s}}_{ij} \ {\text{ log }}\left( \frac{1}{{\tilde{s}}_{ij}} \right) +\left( 1-{\tilde{s}}_{ij} \right) {\text{ log }}\left( \frac{1}{1-{\tilde{s}}_{ij}} \right) . \end{aligned}$$

(8)

Therefore, the Entropy Index (H) is given by:

$$\begin{aligned} H = \sum _{i=1}^{I}\frac{n_{i.}\left( E-E_i \right) }{En_{..}} \end{aligned}$$

(9)

The Atkinson Index (A) is given by:

$$\begin{aligned} A = 1 - \frac{\frac{n_{.j}}{n_{..}}}{1-\frac{n_{.j}}{n_{..}}}\left| \sum _{i=1}^{I}\left[ \frac{\left( 1-{\tilde{s}}_{ij} \right) ^{1-b}{\tilde{s}}_{ij}^bt_i}{\frac{n_{.j}}{n_{..}}n_{..}} \right] \right| ^{\frac{1}{1-b}}, \end{aligned}$$

(10)

where b is a shape parameter that determines how to weight the increments to segregation contributed by different portions of the Lorenz curve.

The Concentration Profile (R) measure is discussed in Ref. [16] and tries to inspect the evenness aspect of segregation. The threshold proportion t is given by:

$$\begin{aligned} \upsilon _t = \frac{\sum _{i=1}^{I}n_{ij}g(t,i)}{\sum _{i=1}^{I}n_{ij}}. \end{aligned}$$

(11)

In the equation, g(t, i) is a logical function that is defined as:

$$\begin{aligned} g(t,i) = {\left\{ \begin{array}{ll} 1 &{} \text {if }\frac{n_{ij}}{n_{i.}} \ge t \\ 0 &{} \text {otherwise} \end{array}\right. }. \end{aligned}$$

(12)

The Concentration Profile (R) is given by:

$$\begin{aligned} R=\frac{\frac{n_{.j}}{n_{..}}-\left( \int _{t=0}^{\frac{n_{.j}}{n_{..}}}\upsilon _t {\text{ d }}t - \int _{t=\frac{n_{.j}}{n_{..}}}^{1}\upsilon _t{\text{ d }}t \right) }{1-\frac{n_{.j}}{n_{..}}}. \end{aligned}$$

(13)

The SPP is similar to the Concentration Profile, but with the addition of the spatial component in the connecting function:

$$\begin{aligned} \eta _t = \frac{k^2-k}{\sum _{i_1}\sum _{1_2}\delta _{i_1i_2}}, \end{aligned}$$

(14)

where k refers to the sum of g(t, i) for a given t and $\delta _{ij}$ is the distance between $i_1$ and $i_2$. One way of determining $\delta _{i_1i_2}$ would be to use a spatial structure matrix, W. The matrix W present ones if $i_1$ and $i_2$ are contiguous and zero, otherwise. The distance $\delta _{i_1i_2}$ between $i_1$ and $i_2$ is given by is the order of how neighbors is needed to reach from $i_1$ to $i_2$. For example, two census tracts, $x_1$ and $x_2$, that do not have a common boundary but both are adjacent to the same unit, $x_3$, are second-order neighbors, so $\delta _{12}$ becomes 2. Like the Concentration Profile, if the number of thresholds used is large enough, a smooth curve, or a SPP, can be constructed by plotting and connecting $\eta _t$.

Isolation (xPx) assess how much a minority group is only exposed to the same group. In other words, how much they only interact the members of the group that they belong. Assuming $j = x$ as the minority group, the isolation of x is giving by:

$$\begin{aligned} \mathrm{xP}x=\sum _{i=1}^{I}\left( {\hat{s}}_{ix} \right) \left( {\tilde{s}}_{ix} \right) . \end{aligned}$$

(15)

The Exposure (xPy) of x is giving by

$$\begin{aligned} \mathrm{xP}y=\sum _{i=1}^{I}\left( {\hat{s}}_{iy} \right) \left( {\tilde{s}}_{iy} \right) . \end{aligned}$$

(16)

The correlation ratio (V or $\mathrm{Eta}^2$) is given by

$$\begin{aligned} V = \mathrm{Eta}^2 = \frac{\mathrm{xP}x - \frac{n_{.x}}{n_{..}}}{1 - \frac{n_{.x}}{n_{..}}}. \end{aligned}$$

(17)

The SP Index is given by:

$$\begin{aligned} \mathrm{SP} = \frac{{XP}_{xx} + {YP}_{yy}}{{TP}_{tt}}, \end{aligned}$$

(18)

where

$$\begin{aligned}&P_{xx} = \sum _{i_1=1}^{I}\sum _{i_2=1}^{I}\frac{n_{i_1x}n_{i_2x}\zeta _{i_1i_2}}{n_{.x}^2}\\&P_{yy} = \sum _{i_1=1}^{I}\sum _{i_2=1}^{I}\frac{n_{i_1y}n_{i_2y}\zeta _{i_1i_2}}{n_{.y}^2}\\&P_{tt} = \sum _{i_1=1}^{I}\sum _{i_2=1}^{I}\frac{n_{i_1.}n_{i_2.}\zeta _{i_1i_2}}{n_{..}^2}\\&\zeta _{i_1i_2} = \mathrm{exp}(-d_{i_1i_2}), \end{aligned}$$

$d_{i_1i_2}$ is a pairwise distance measure between area $i_1$ and $i_2$ and $d_{ii}$ is estimated as $d_{ii} = (\alpha a_i)^{\beta }$ where $a_i$ is the area of unit i. The default is $\alpha = 0.6$ and $\beta = 0.5$ and for the distance measure, we first extract the centroid of each unit and calculate the euclidean distance.

The RCL measure is given by:

$$\begin{aligned} \mathrm{RCL} = \frac{P_{xx}}{P_{yy}} - 1. \end{aligned}$$

(19)

The Distance Decay Isolation (DDxPx) is given by:

$$\begin{aligned} \mathrm{DDxP}x=\sum _{i_1=1}^{I}\left( {\hat{s}}_{i_1x} \right) \left( \sum _{i_2=1}^{I}P_{i_1i_2} \left( {\tilde{s}}_{i_1x}\right) \right) , \end{aligned}$$

(20)

where

$$\begin{aligned} P_{i_1i_2} = \frac{\zeta _{i_1i_2}n_{i_2.}}{\sum _{i_2=1}^{I}\zeta _{i_1i_2}n_{i_2.}} \end{aligned}$$

such that

$$\begin{aligned} \sum _{i_2=1}^{I}P_{i_1i_2} = 1, \end{aligned}$$

where $\zeta _{i_1i_2}$ is defined as before. This also could be seen as the probability of contact of members of group x to each other weighted by the inverse of distance.

The Distance Decay Exposure (DDxPy) is given by:

$$\begin{aligned} \mathrm{DDxP}y=\sum _{i_1=1}^{I}\left( {\hat{s}}_{i_1x} \right) \left( \sum _{i_2=1}^{I}P_{i_1i_2} \left( {\tilde{s}}_{i_1y}\right) \right) \end{aligned}$$

(21)

where $P_{i_1i_2}$ is defined as before.

The DEL measure is given by the following equation:

$$\begin{aligned} \mathrm{DEL} = \frac{1}{2}\sum _{i=1}^{I}\left| {\hat{s}}_{ij} - \frac{a_i}{A} \right| , \end{aligned}$$

(22)

where $a_i$ is the area of unit i and A is the total area of the given region $A = \sum _{i=1}^{I}a_i$.

The ACO Index is given by:

$$\begin{aligned} \mathrm{ACO} = 1-\frac{ \sum _{i=1}^{I}\left( \frac{n_{ij}a_i}{n_{.j}} \right) - \sum _{i=1}^{n_1}\left( \frac{n_{i.}a_i}{T_1} \right) }{ \sum _{i=n_2}^{I}\left( \frac{n_{i.}a_i}{T_2} \right) - \sum _{i=1}^{n_1}\left( \frac{n_{i.}a_i}{T_1} \right) }, \end{aligned}$$

(23)

where the units are ordered from smallest to largest in areal size. In this formula, $n_1$ is the rank of the unit where the cumulative total population equal the total minority population, $n_2$ is the rank of the unit where cumulative total population equal equal the total minority population from the largest unit down. In addition,

$$\begin{aligned} T_1 = \sum _{i=1}^{n_1}n_{i}, \end{aligned}$$

and

$$\begin{aligned} T_2 = \sum _{i=n_2}^{n}n_{i}. \end{aligned}$$

Another measure of concentration is the RCO Index:

$$\begin{aligned} \mathrm{RCO} = \frac{\frac{\sum _{i=1}^{I}\left( \frac{n_{ix}a_i}{n_{.x}} \right) }{\sum _{i=1}^{I}\left( \frac{n_{iy}a_i}{n_{.y}} \right) }-1}{\frac{\sum _{i=1}^{n_1}\left( \frac{n_{i.}a_i}{T_1} \right) }{\sum _{i=n_2}^{I}\left( \frac{n_{i.}a_i}{T_2} \right) }-1}, \end{aligned}$$

(24)

where $n_1$, $n_2$, $T_1$ and $T_2$ are defined as before.

The degree of centralization can be evaluated through the Absolute Centralization Index (ACE) or through the RCE:

$$\begin{aligned} \mathrm{ACE}= & {} \left( \sum _{i=2}^{I}X_{i-1}A_i \right) - \left( \sum _{i=2}^{I}X_{i}A_{i-1} \right) , \end{aligned}$$

(25)

$$\begin{aligned} \mathrm{RCE}= & {} \left( \sum _{i=2}^{I}X_{i-1}Y_i \right) - \left( \sum _{i=2}^{I}X_{i}Y_{i-1} \right) , \end{aligned}$$

(26)

where $A_i$ is the cumulative area proportion through unit i, $X_i$ is the cumulative frequency proportion through unit i of group x and $Y_i$ is the analogous for group y. In this measure, the area units are ordered by increasing distances from the central business district, which we assume being located in the average latitude and average longitude among all centroid.

The Dct Index based on [7] evaluates the deviation from simulated evenness. This measure is estimated by taking the mean of the classical D under several simulations under evenness from the global minority proportion.

Let $D^*$ be the average of the classical D under simulations draw assuming evenness from the global minority proportion. The value of Dct can be evaluated with the following equation:

$$\begin{aligned} \mathrm{Dct} = {\left\{ \begin{array}{ll} \frac{D-D^*}{1-D^*} &{} \text {if }D \ge D^*\vspace{6pt} \\ \frac{D-D^*}{D^*} &{} \text {if }D < D^*\end{array}\right. }. \end{aligned}$$

(27)

Similarly, the Gct based also on Ref. [7] evaluates the deviation from simulated evenness. This measure is estimated by taking the mean of the classical G under several simulations under evenness from the global minority proportion.

Let $G^*$ be the average of G under simulations draw assuming evenness from the global minority proportion. The value of Gct can be evaluated with the following equation:

$$\begin{aligned} \mathrm{Gct} = {\left\{ \begin{array}{ll} \frac{G-G^*}{1-G^*} &{} \text {if }G \ge G^* \vspace{6pt}\\ \frac{G-G^*}{G^*} &{} \text {if }G < G^* \end{array}\right. }. \end{aligned}$$

(28)

Lastly, the Bias-Corrected (Dbc) and Density-Corrected (Ddc) Dissimilarities indices are presented in Ref. [2]. The Dbc is given by:

$$\begin{aligned} D_\mathrm{bc} = 2D - {\bar{D}}_\mathrm{b}, \end{aligned}$$

(29)

where ${\bar{D}}_b$ is the average of B resampling using the observed conditional probabilities for a multinomial distribution for each group independently.

The Ddc measure is given by:

$$\begin{aligned} D_\mathrm{dc} = \frac{1}{2}\sum _{i=1}^{I}{\hat{\sigma }}_in\left( {\hat{\theta }}_i \right) , \end{aligned}$$

(30)

where

$$\begin{aligned} {\hat{\sigma }}^2_i = \frac{{\hat{s}}_{ix} (1-{\hat{s}}_{ix})}{n_{.x}} + \frac{{\hat{s}}_{iy} (1-{\hat{s}}_{iy})}{n_{.y}}, \end{aligned}$$

and $n\left( {\hat{\theta }}_i \right)$ is the $\theta _i$ that maximizes the folded normal distribution $\phi ({\hat{\theta }}_i-\theta _i) + \phi ({\hat{\theta }}_i+\theta _i)$ where

$$\begin{aligned} \hat{\theta _i} = \frac{\left| {\hat{s}}_{ix}-{\hat{s}}_{iy} \right| }{\hat{\sigma _i}}, \end{aligned}$$

and $\phi$ is the standard normal density.

Table 4 Segregation measures-related literature for PySAL segregation module point estimations

B: Counterfactual composition details

Following the same notation of A and assuming building counterfactual values fro two different cities, we form the cumulative distribution functions (CDF) for these values taken over all the tracts in City 1: $F^{(1)}({\tilde{s}}_{i,j}^{1,t})$, and City 2: $F^{(2)}({\tilde{s}}_{i,j}^{2,t})$. To create a counterfactual distribution that imposes the attribute distribution of City 2 on the spatial structure of City 1 we take $p_{i,j}^{1,t} = F^{(1)}({\tilde{s}}_{i,j}^{1,t})$ and then generate $n_{i,j}^{1,t} |_{attr = 2} = {F^{(2)}}^{-1}(p_{i,j}^{1,t}) n_{i,.}^{1,t}$, where $attr = 2$ means that this population is calculated given the attributes of City 2. This entire process is done for all tracts of a group in City 1 and the majority group population is given by the difference $n_{i,.}^{1,t} - n_{i,j}^{1,t} |_{attr = 2}$. The populations for City 2 are generated analogously.