Introduction

While the long-form decennial Census was officially replaced by the American Community Survey in the USA in 2010, the nationwide ACS data were first made available in 2006. Given the relative currency (1-year, 3-year, and 5-year estimates) of ACS data, they are now serving as an imperative dataset for planning and decision-making in funding allocation, transportation forecasting, poverty evaluation, and segregation pattern analysis (Bazuin & Fraser, 2013; Landis, 2019; Logan et al., 2018; Macdonald, 2006). Spielman et al. (2014) and Jurjevich et al. (2018) suggested that the ACS is now the primary source of high-resolution geographic data about the U.S. population for planners and decision-makers.

The tradeoff of data currency provided by the ACS is the much smaller sample size compared with the previous decennial long-form census due to limited time and budget. The ACS annual estimates are derived from a sample of approximately 1:40 households, whereas the decennial long-form census sampled about 1:6 households (USCB, 2020). While a sampling rate of approximately 12% could be theoretically achieved by aggregating five-year samples, the average sampling rate for ACS 5-year aggregate data ranges from 8 to 10% at the tract level. As a result although the U.S. Census Bureau (USCB) initially expected a 33% more statistical uncertainty than the decennial census survey due to the smaller sample size, studies suggested that sampling errors of ACS estimates are generally 75% larger than those of the decennial long form at the tract level (Navarro, 2012; Spielman et al., 2014). In order to make these errors more transparent to ACS data users, the USCB publishes a margin of error (MOE) at the 90% confidence level for each ACS estimate. Many studies have shown that the data uncertainty of ACS can lead to inaccurate analysis and/or biased decision-making, and it is essential to assess the impacts of such uncertainty (Bazuin & Fraser, 2013; Jung et al., 2019; Logan et al., 2018; Napierala & Denton, 2017; Reardon et al., 2018; Spielman et al., 2014; Wei et al., 2021).

In this paper, we focus on the implications of ACS data uncertainty for measuring residential segregation. Among the oldest pursuits in quantitative social science, residential segregation continues to be a foundational driver of spatial inequality in the USA, exerting influence on outcomes ranging from education to health, to earnings. It is increasingly common to use ACS data to measure residential segregation in the past decade due to its data currency (Anacker et al., 2017; Landis, 2019; Lichter et al., 2012; Logan et al., 2018; Reardon et al., 2018). However, only a few efforts explicitly account for uncertainty associated with ACS data. Napierala and Denton (2017) presented a simulation-based method to derive the confidence interval for the dissimilarity index of two racial/ethnic groups given the published ACS estimates and associated MOE. Logan et al. (2018) and Reardon et al. (2018) proposed approaches to correct the bias toward segregation measures when using ACS income data. Despite the utility of these approaches, ultimately, they rely upon several assumptions regarding the population distribution, each of which remains potentially problematic. For example, the simulation method in Napierala and Denton (2017) assumes that each group’s population in each census tract is independently normally distributed; the bias-corrected estimators from Reardon et al. (2018) may not perform well when the population is unevenly distributed across geographical units. In addition, some approaches are only applicable to certain segregation indices or require access to microdata.

To address these issues, our goal is to develop a generally applicable framework to integrate ACS data uncertainty into segregation measures as well as a method of statistical inference for comparative analysis. The framework we present relies upon variance replicate estimate tables for 5-year ACS estimates to understand how sampling errors propagate through the computations of segregation measures. Given the replicate estimates created using the same estimation method as the original ACS estimate, we can derive the sampling distributions of segregation measures while accounting for the covariance between the ACS estimates used in the calculation. This framework can greatly facilitate ACS data users to incorporate data uncertainty into segregation analysis, leading to more reliable segregation pattern identification and better policy development.

Specifically, our work here makes three distinct contributions. First, we develop a novel method for quantifying the uncertainty associated with segregation indices when measured with Census ACS data and we describe how it differs from other approaches. Second, we make a simple extension to our framework, allowing for statistical inference in the context of comparative analysis (e.g., whether a place has become more or less segregated over time). Finally, we provide an empirical examination of changing racial segregation in California metropolitan regions over the last decade, demonstrating the utility of our framework for practical analysis. In the next section, we provide a review of ACS data uncertainty and existing approaches to tackle data uncertainty in segregation analysis. This is followed by details of our methodological framework. Finally, the proposed framework is applied to the analysis of racial segregation patterns of 29 metropolitan statistical areas in California, USA using the 2010–2014 and 2015–2019 ACS data.

ACS data uncertainty

Data from the ACS, like all survey estimates, are subject to both sampling and nonsampling errors. Sampling error refers to the difference between the estimate derived from a sample and the actual value obtained from an entire population. As mentioned earlier, the sampling error in ACS data is larger than estimates from the census long form due to the smaller sample size. To measure the magnitude of sampling error, the USCB provides the MOE with all published ACS estimates. The MOE is calculated based on a variance estimate that is derived using a successive differences replication (SDR) methodology, rather than an unbiased design-based variance estimate (which cannot exist due to complexity associated with ACS sampling design and weighting adjustments) (USCB 2014). Specifically, the same estimation procedure is repeated independently 80 times and each time a different set of weights is applied to the sampled records. The details on how replicate weights are determined can be found at USCB (2014). This results in 80 separate “replicate” estimates. Using these 80 sets of values, the variance is estimated based on the variability between replicate estimates and the full sample estimate as follows (USCB 2020):

$$V= \frac{4}{80}{\sum_{i=1}^{80}({R}_{i}-F)^2},$$
(1)

where V is the variance, \({R}_{i}\) is the replicate estimate, and \(F\) is the full sample estimate. The factor of 4/80 is an artifact of using the SDR methodology (USCB 2020). The published MOE at the 90% confidence level is determined as follows:

$$MOE=1.645*\sqrt{V}.$$
(2)

Given the published MOE, the ACS data users can determine statistical significance when comparing two ACS estimates. In addition, the USCB provides approximation formulas to calculate the MOE for user-derived estimates, such as aggregated counts, proportions, percentages, ratios, and percent change (USCB 2020). However, these approximation formulas assume that ACS estimates are independent of each other and do not account for covariance between the used estimates. As a result, the USCB first releases the Variance Replicate Tables (VRT) for the 2010–2014 ACS 5-year estimates in July 2016. The VRT includes the eighty replicate estimates (\({R}_{i}\)) that are used to calculate the published MOE. This information allows ACS data users to compute variances for their own measures using a methodology similar to the one employed by the ACS during its production. This leads to an exact variance and MOE for user-derived measures because it accounts for the covariance between variables in the ACS sample. The VRT are available for selected 5-year Detailed Tables at various geographic levels, including the nation, states, counties, census tracts, and block groups. These tables are released on an annual basis, shortly after the release of the standard 5-year data products.

Nonsampling error in ACS data is commonly attributable to sample overcoverage or undercoverage, unit nonresponse, item nonresponse, response error, and processing error (USCB 2020). While the sampling error of ACS data is much larger than decennial census long form, the nonsampling error of ACS data is likely to be less due to lower nonresponse rates and similar completeness rates (Napierala & Denton, 2017; USCB 2020). In addition, there does not exist any direct measure of nonsampling error in ACS data, and indirect measures, such as sample size, coverage rates, and nonresponse rates, are only available at the state and national level. As a result, even though nonsampling error could also bias ACS estimates and user-derived measures of residential segregation, we focus on the impacts of sampling error in the measurement of residential segregation.

Addressing ACS Data Uncertainty in Residential Segregation Estimates

The adoption of the American Community Survey launched a wave of segregation research throughout the 2010s, since, compared with prior decennial datasets, there appeared to be a large spike in income segregation. Understanding whether this spike was due to a change in residential preferences, housing markets, or simply sampling error from the new data has quickly become a focal topic of study in sociology (Logan et al., 2018; Napierala & Denton, 2017; Reardon et al., 2018). Since prior work demonstrated that segregation indices based on random samples are generally biased upward compared with the values that would be computed from full population data, (Reardon & Bischoff, 2011) a series of important papers began to examine how ACS sampling error could impact the measurement of segregation indices and how to develop methods that overcome such bias.

One approach presented by Napierala and Denton (2017) uses a simulation-based method to derive the confidence interval for dissimilarity index given the published ACS estimates and associated MOE. Specifically, it assumes that each group’s population in each census tract is independently normally distributed with the mean set to the published estimate and standard error set to published MOE/1.645. Then a population value is randomly drawn for 50,000 times and each time a dissimilarity index is computed. The mean and variance of the 50,000 simulated dissimilarity indices are computed, and statistical inference is facilitated simply from the derived confidence interval. Following, Logan et al. (2018) and Reardon et al. (2018) proposed approaches to correct the bias toward income segregation measures. Specifically, Logan et al. (2018) used census microdata and sample correction to derive an unbaised rank-order variance ratio index, which is referred to as “sparse-sampling variance decomposition (SSVD)” approach. In addition they derived a formula to approximate bias in the rank-order information theory index. While unbiased rank-order variance ratio index or approximately unbiased rank-order information theory index estimates can be generated using these approaches, they both require access to census microdata. When microdata are not available, individual samples need to be simulated based on aggregated income distribution and then the microdata-based approached can be applied. However, such approach has not been extensively validated and may not work when samples are small or there are wide variations in sample sizes across units (Reardon et al., 2018) To avoid the reliance on access to restricted or simulated microdata, Reardon et al. (2018) developed new formulas to approximate the bias in sample-based estimates of rank-order variance ratio index and rank-order information theory. While these bias-corrected estimators only rely on publicly available data, the bias may not be reduced when the population is unevenly distributed across geographical units and the average sample sizes are small. Moreover, the estimators developed in Logan et al. (2018) and Reardon et al. (2018) are only applicable to variance ratio index and information theory index. No formulas are developed for other widely used segregation indices, like dissimilarity index.

In summary, while ACS data uncertainty has been examined in some literature, there is a lack of generally applicable framework to integrate data uncertainty into segregation measures and its statistical inference. In the following sections, we examine how sampling error in the ACS may propagate through inferential analyses of residential segregation. We use the Variance Replicate Tables for each ACS release to develop analytical margins of error for segregation indices in each metropolitan region. Following, we use these estimates to perform comparative inference on the measurements from each time period, asking whether the observed change in segregation is distinct from a random process. Using these results we describe both how segregation has changed in California over the last decade as well as how the ACS sampling procedures may impact researchers’ inferences regarding the statistical significance of these changes.

Method

To understand how bias from the ACS complicates measures of changing segregation, we propose a generally applicable framework for statistical inferences that integrates ACS data uncertainty at its core. While this framework is applicable to any segregation measure, we use two of the most widely used indices (the Dissimilarity index and Entropy index) to demonstrate how our framework is used to derive the sampling distributions of segregation measures facilitating both single-value and comparative statistical inferences.

Segregation Measures

As the most widely used measure of residential segregation, the dissimilarity index represents the percentage of the minority group that would have to change their residential areas for the two social groups to be evenly distributed across the entire study region (Duncan & Duncan, 1955; Massey & Denton, 1988). Consider the following notation:

$${t}_{j}={\text{The}} \,{\text{total}}\,{\text{population}}\, {\text{at}} \,{\text{areal}} \,{\text{unit}} \; j,$$
$${p}_{j}={\text{The}} \,{\text{minority}} \,{\text{percentage}}\, {\text{at}}\, {\text{areal}} \,{\text{unit}} \; j,$$
$$T={\text{The}} \,{\text{total}}\, {\text{population}}\, {\text{in}}\,{\text{the}}\, {\text{study}} \,{\text{region}},$$
$$P={\text{The}}\, {\text{total}} \,{\text{minority}}\, {\text{population}}\, {\text{percentage}}\, {\text{in}} \,{\text{the}}\, {\text{study}} \,{\text{region}},$$
$$n={\text{The}}\, {\text{number}}\, {\text{of}}\, {\text{areal}}\, {\text{units}}\, {\text{in}} \,{\text{the}}\, {\text{study}}\,{\text{region}}.$$

The dissimilarity index, \(D\), can be defined as follows:

$$D=\sum_{j=1}^{n}\frac{{t}_{j}|{p}_{j}-P|}{2TP(1-P)}.$$
(3)

\(D\) varies from 0.0 to 1.0, with larger values indicating higher levels of segregation. The dissimilarity index is easy to interpret and compute. However, many studies have found that the dissimilarity index is biased upward especially when unit sizes or minority proportions are small. Several extensions to correct such bias have been suggested (Allen et al., 2015; Davidson, 2009; Ransom, 2000). Napierala and Denton (2017) are the first to address ACS data uncertainty in dissimilarity index, but their approach assumes each ACS estimate is independently normally distributed and fails to account for covariance between ACS sample estimates.

Another segregation measure demonstrated here is the entropy index, which is also referred to as the information index. The entropy index measures segregation by estimating each areal unit’s departure from the racial entropy of the entire study region. The racial entropy of the entire study region represents the study region’s extent of racial diversity and can be defined as follows:

$$E=-\left[P\mathrm{ln}P+\left(1-P\right)\mathrm{ln}\left(1-P\right)\right].$$
(4)

Each areal unit’s entropy is analogously defined as follows:

$${E}_{j}=-\left[{p}_{j}\mathrm{ln}{p}_{j}+\left(1-{p}_{j}\right)\mathrm{ln}\left(1-{p}_{j}\right)\right].$$
(5)

The entropy index is defined as the weighted average deviation of each unit’s entropy from the region wide entropy:

$$H=\sum_{j=1}^{n}\frac{{t}_{j}(E-{E}_{j})}{ET}.$$
(6)

\(H\) also varies from 0.0 to 1.0, with larger values suggesting higher levels of unevenness. While it is not as easy to interpret and compute as the dissimilarity index, it satisfies some important properties like organizational equivalence and size invariance and is especially useful for multigroup segregation measures. The binary entropy index can be easily extended into the multigroup version by integrating multiple groups into the definition of \(E\) and \({E}_{j}\) as follows:

$$E=-\sum_{i=1}^{m}{P}_{i}\mathrm{ln}{P}_{i},$$
(7)
$${E}_{j}=-\sum_{i=1}^{m}{p}_{ij}\mathrm{ln}{p}_{ij},$$
(8)

where \(i={\text{the}}\; {\text{index}}\; {\text{of}} \;{\text{race}} \;{\text{groups}}, m={\text{The}}\; {\text{total}} \;{\text{number}}\; {\text{of}} \;{\text{race}}\; {\text{groups}}, {P}_{i}={\text{The}}\;{\text{percentage}} \;{\text{of}}\; {\text{race}} \;{\text{group}}\; i\; {\text{in}}\; {\text{the}} \;{\text{study}} \;{\text{region}}\) and \({p}_{ij}={\text{The}}\; {\text{percentage}}\; {\text{of}} \;{\text{race}}\; {\text{group}}\; i \;{\text{at}} \;{\text{areal}}\; {\text{unit}} j.\)

Both Logan et al. (2018) and Reardon et al. (2018) proposed approaches to correct the bias of entropy index when ACS data are highly uncertain. However, the approaches proposed by Logan et al. (2018) rely on a cadre of assumptions regarding the population distribution to approximate the bias and both of their approaches cannot be extended to other segregation measures.

Measuring Uncertainty

To account for the weaknesses of published MOEs in the ACS described above, the USCB began publishing Variance Replicate Tables (VRT), beginning with the 2010–2014 ACS 5-year estimates in July 2016. The VRT includes the eighty replicate estimates (Ri) that are used to calculate the published MOE. This information allows ACS data users to compute variances for their own measures using a methodology similar to the one employed by the ACS during its production. This leads to an exact variance and MOE for user-derived measures by taking into account the covariance between ACS estimates. The VRT are available for selected 5-year Detailed Tables at various geographic levels, including the nation, states, counties, census tracts, and block groups. These tables are released on an annual basis, shortly after the release of the standard 5-year data products.

Thus, to address the limitations of previously described methods, we rely upon the VRT to incorporate ACS data uncertainty into segregation measures. Specifically, we compute the segregation measure for each replicate estimate in addition to the published full sample estimate. Then we compute the variance of segregation measures using the same method as the one employed by the ACS during its production. Taking the dissimilarity index as an example, its variance can be derived as follows:

$${\text{Var}}(D)= \frac{4}{80}{\sum_{i=1}^{80}({D}_{{R}_{i}}-{D}_{F})^2},$$
(9)

where \({D}_{{R}_{i}}\) and \({D}_{F}\) are the dissimilarity index computed using replicate estimate \({R}_{i}\) and using published full sample estimate \(F\). The USCB (2020) suggests that each derived measure tends to be normally distributed given the production of replicate estimates. That is, the dissimilarity index is no longer a fixed value but a normally distributed variable with the mean as \({D}_{F}\) and variance as \({\text{Var}}(D)\). The variance can also be converted to \({\text{MOE}}(D)\) following Eq. (2). This method can be applied to any segregation measure.

Given the derived distribution of the segregation measure, we can use a two-sample t test to test whether segregation levels differ across two entities–generally either two separate locations or a single location at two points in time. The test statistic is

$$t=\frac{{D}_{F}^{1}-{D}_{F}^{2}}{\sqrt{{\text{Var}}\left({D}^{1}\right)-{\text{Var}}({D}^{2})}},$$

where \({D}_{F}^{1}\) and \({\text{Var}}\left({D}^{1}\right)\) are the mean and variance of the dissimilarity index for period 1, respectively, whereas \({D}_{F}^{2}\) and \(Var\left({D}^{2}\right)\) are the mean and variance of the dissimilarity index for period 2, respectively. With this test statistic, we can derive a p-value to determine whether the segregation levels differ with statistical significance.

We used the first three California census tracts in 2010–2014 VRT table as an example to demonstrate how this method works and how it is different from the simulation method used in Napierala and Denton (2017). Table 1 reports the number of white population and Table 2 reports the number of black population. Column “ACS Estimate” is the published estimate for the white or black population; column “MOE” is the published margin of error associated with the estimate; column “Var_Rep1”, “Var_Rep2”, …, and “Var_Rep80” are the replicate estimates used by the Census Bureau. Because each set of replicate estimates are estimated using a consistent set of sample weights as mentioned earlier, the correlation or covariance among the same set of replicate estimates is preserved. For example, if the first tract’s white population is positively correlated with the second tract’s white population, this relationship will remain in the replicate estimates even though the individual estimates vary across the 80 sets of replicate estimates. Using our proposed method, we will calculate the \(D\) index for whites and blacks for the published ACS estimate, which is \({D}_{F}\), and then we will calculate the index for each set of replicate estimates, which are \({D}_{{R}_{i}}\). Given these calculations, we can derive the variance of the \(D\) index using Eq. (7) (Table 3).

Table 1 VRT for white population
Full size table
Table 2 VRT for black population
Full size table
Table 3 D index
Full size table

The simulation approach used in Napierala and Denton (2017) did not use the replicate estimates, rather it assumes that each group’s population in each census tract is independently normally distributed with the mean set to the published estimate and standard error set to published MOE/1.645. Then a population value is randomly drawn for a certain times and each time a \(D\) index is computed. For example, it assumes the first tract’s white population is normally distributed with a mean of 2235 and standard error of 200/1.645. Then it randomly draw a value from this normal distribution and consider this as a potential population value. This process is repeated for each tract for a certain times. Given these simulated dissimilarity indices, the variance of the \(D\) index is estimated using classic variance formula. Clearly this approach cannot take into account any correlation or covariance among component estimates.

Results

We highlight the utility of this methodology framework for handling elements of uncertainty in the ACS data by examining racial segregation in 29 metropolitan statistical areas (MSAs) in the state of California. The distribution of the dissimilarity index and entropy index for whites and blacks and whites and Asians are computed using 2010–2014 and 2015–2019 ACS data at the census tract level. As mentioned earlier, the 2010–2014 ACS data are the first ACS dataset that publishes the VRTs and the 2015–2019 data are the latest. The 2010–2014 ACS dataset also does not have an overlapping sampling frame with the 2015–2019 dataset, making comparative inference valid.

Descriptive statistics for each of the 29 MSAs are shown in Table 4. The number of census tracts varies from 23 to 2346 in these 29 MSAs, along with a diverse set of racial characteristics. In addition to the published estimates for the average population and minority percentage, we also report the average coefficient of variation (CV) for minority population estimates. The CV is defined as the ratio of the standard error to the reported estimate, representing the relative amount of sampling error associated with a sample estimate. Compared with the MOE, the CV provides a scale-independent measure for comparing reliability among multiple ACS estimates (Sun and Wong 2010; Wei and Grubesic 2017; USCB 2020). The percentage of the minority population is negatively correlated with the average CV. For example, the correlation coefficient between the black population percentage and the average CV for 2014–2019 is − 0.86 and the correlation coefficient between the Asian population percentage and the average CV for 2014–2019 is − 0.83. A small minority population share is usually associated with a large average CV.

Table 4 Descriptive statistics
Full size table

The \(D\) index for whites and blacks and whites and Asians in 2010–2014 and 2015–2019 are reported in Table 5. The “\({D}^{{\text{WB}}_{2014}}\)” represents the dissimilarity index calculated using the published estimates of white and black population in 2010–2014, and the “\({D}^{{\text{WA}}_{2019}}\)” represents that calculated using the white and Asian population in 2014–2019. The “\(MOE({D}^{{\text{WB}}_{2014}}, R)\)” is the margin of error of \({D}^{{\text{WB}}_{2014}}\) estimated using the replicate estimates approach, and the “\({\text{MOE}}({D}^{{\text{WB}}_{2014}}, S)\)” is the margin of error of \({D}^{{\text{WB}}_{2014}}\) estimated using the simulation approach in Napierala and Denton (2017). We also computed the ratio of MOEs using two different approaches to examine their differences. One clear result we find is that the MOEs derived using the ratio replicate estimates are much larger than that using the simulated approach when the number of census tracts is large and the share of minority residents is low, whereas they are similar to or smaller than that using the simulated approach when the number of census tracts is small and the minority share is high.

Table 5 Dissimilarity index
Full size table

For instance, given the same share of black population (3%) in the Anaheim-Santa Ana-Irvine MD and the Napa MSA, the ratio of \({\text{MOE}}({D}^{{\text{WB}}_{2014}}, R)\) to \({\text{MOE}}({D}^{{\text{WB}}_{2014}}, S)\) is 3.61 and 1.23, respectively, due to the large number of census tracts (583) at Anaheim-Santa Ana-Irvine MD but the small number (40) at NAPA MSA. Another interesting example can be found at San Jose-Sunnyvale-Santa Clara MSA where the ratio of \({\text{MOE}}({D}^{{\text{WB}}_{2019}}, R)\) to \({\text{MOE}}({D}^{{\text{WB}}_{2019}}, S)\) is 2.55, whereas the ratio of \({\text{MOE}}({D}^{{\text{WA}}_{2019}}, R)\) to \({\text{MOE}}({D}^{{\text{WA}}_{2019}}, S)\) is 0.66 given the low percentage of the black population but the high percentage of the Asian population. If incorporating VRT is the best method for computing variance estimators using ACS data, as proposed by the Census, this suggests that the simulation approach might underestimate the variance of D for large metropolitan areas with small minority populations but overestimate it for small metropolitan areas with large minority populations. Because the replicate estimates approach accounts for covariance among ACS estimates, it is likely to provide a more accurate estimate of the variance of D. To validate this finding, we also construct a linear regression model, using the ratio of MOEs as the dependent variable, and the number of census tracts and the percentage of minority population as independent variables. The results show that the number of census tracts and the percentage of minority population are strongly significant across all four ratio measures. The number of census tracts is positively correlated with the ratio, whereas the percentage of minority population is negatively correlated with the ratio.

Then we compare the 2010–2014 and 2015–2019 dissimilarity index distribution using the comparative inference framework presented above. The p-values are reported in Table 6. While the dissimilarity index based on published population estimates of each MSA increases or decreases from 2010–2014 to 2015–2019, only a few are statistically significant after accounting for its variance. The segregation between white and black residents decreases significantly in Los Angeles-Long Beach-Glendale MD, Salinas MSA, and Stockton-Lodi MSA but increases significantly in Visalia-Porterville MSA (p-value < 0.05). The segregation between white and Asian residents decreases significantly in San Francisco-Redwood City-South San Francisco MD.

Table 6 Comparative inference of dissimilarity index
Full size table

We also perform the same analysis for the H index with similar results. We find, for example, that the ratio of \({\text{MOE}}({H}^{\text{WB}}, R)\) to \({\text{MOE}}({H}^{\text{WB}}, S)\) increases as the number of census tracts increases and the minority percentage decreases. The comparative inference has slightly different results although as shown in Table 7. Comparisons based on the entropy index show that segregation between white and black residents decreases in Fresno MSA and Los Angeles-Long Beach-Glendale MD but increases in Visalia-Porterville MSA. The segregation between white and Asian residents decreases significantly in San Francisco-Redwood City-South San Francisco MD and Fresno MSA but increases significantly in Santa Cruz-Watsonville MSA and Santa Maria-Santa Barbara MSA.

Table 7 Comparative inference of entropy index
Full size table

Finally, we compare the 2010–2014 and 2015–2019 multigroup entropy index (\({H}^{\text{WBA}}\)) by integrating white, black, and Asian population at census tracts. The results are presented in Table 8. After integrating both black and Asian population, only San Francisco-Redwood City-South San Francisco MD and Fresno MSA are showing statistically significant decrease in the segregation level. None of the MSAs are showing statistically significant increase in the segregation level.

Table 8 Comparative inference of multigroup entropy index
Full size table

Discussion

Focusing first on the substantive results of our comparative analysis, we find clear heterogeneity in residential segregation dynamics for different racial groups in California over the last decade. Specifically, while we find that in many places, segregation has not changed dramatically, those places in which it has increased or decreased greater than we would expect at random differ with respect to location, racial group, and segregation index. In two of California’s largest metro regions, Los Angeles and San Francisco, residential segregation appears to be decreasing, albeit nonuniformly. San Francisco has borne witness to falling Asian segregation, as measured by both the Dissimilarity index and the Entropy index. For Asian Americans, this appears to be a clear trend in a socially positive direction, as two different measures of unevenness are decreasing in agreement (Massey & Denton, 1988). In Los Angeles, a similar story is apparent for Black Americans, with both Dissimilarity and Entropy indices showing statistical evidence of a decrease over the 2010–2019 period. Given the magnitude of the Dissimilarity index for Black Americans in Los Angeles, this decrease seems particularly notable, given the discussion of statistical versus substantive significance provided earlier. Here we have a statistically significant decrease in what Massey (1978) might describe as a substantively interesting segregation measure.

What remains unclear is why these dynamics diverge for different racial groups in their respective metro regions. That is, if racial residential segregation is decreasing in San Francisco, why is it only apparent for Asian Americans, and conversely, why is Black segregation falling in Los Angeles but not any other group? And what might these results suggest about larger urban and neighborhood dynamics, like gentrification and decline, and their unique manifestations in different metro regions? The repeated analyses of VRT rule out issues of sampling bias from the ACS, leaving structural differences in the housing, employment, and social systems between each city ripe for further investigation.

Another important question for further analysis is why indices that measure the same dimension of segregation may differ in their inferential results. In the Fresno metropolitan region, our analyses of the Entropy index show evidence of a statistically significant decrease for both Black Americans and Asian Americans, but the same does not hold true for the Dissimilarity index. Apart from these substantive findings, our results here demonstrate a new technique for incorporating information about sampling error present in the ACS and correcting for the bias it can induce in segregation indices and secondary analyses thereof. Unlike methods proposed elsewhere, our technique avoids the need for microdata, relying only on published Variance Replicate Tables, and can be applied easily using existing open-source software. Further, our methods yield different results than the simulation technique described by Napierala and Denton (2017), in that our error margins are larger than those using the simulated approach when the number of census tracts is large and minority percentage is low and the converse is true when the number of census tracts is small and minority percentage is high. And, as we show, ensuring accurate estimates of error margins is especially important in the context of comparative segregation inference.

It is also worth noting that the proposed method will likely underestimate the variance of the segregation measures when a count estimate is zero because all variance replicates estimates are also published as zero when the published estimate is zero even though the sampling error still exists (USCB 2020). Cautions should be given until the Census Bureau improve its approach for measuring sampling errors on zero estimates.

Conclusion

In this paper, we develop a novel technique for incorporating estimates of uncertainty from the U.S. Census American Community Survey (ACS) into analyses of residential segregation. Whereas, recent scholarship has shown that ACS sampling errors can artificially inflate segregation indices and proposes a simulation technique to address the issue, we advocate the use of ACS Variance Replicate Tables as an alternative method for estimating margins of error associated with segregation indices. Using the lenses of statistical inference and comparative segregation dynamics, we demonstrate first that our method generates different results than simulation-based techniques and second that these differences can result in large, substantive differences when constructing and interpreting a measure of statistical significance.

Following, we use the state of California as a laboratory for examining the utility of our technique by examining ACS data from the 2011–2014 and 2015–2019 samples. We find that the vast majority of metropolitan regions in the state have not witnessed dramatic changes in their levels of residential segregation, but that some places have indeed changed in ways that are analytically clear, statistically distinct from a random process, and substantively interesting to social scientists. Nevertheless, this change is heterogeneous with respect to both location and demographic groups. Aside from illuminating differences in the evolution of California cities over the last decade, our analysis also raises important questions about why certain cities are changing more than others, and why they may do so for certain groups but not others. Although it is beyond the scope of this paper, we argue this finding underscores the importance of calculating and testing several segregation indices in each study location to get a comprehensive picture that explores diverging findings among groups and across different dimensions (Massey et al., 1996). Although we examine only the Dissimilarity and Entropy indices in this paper, the methods we develop and present are applicable to any index and we hope others will apply these methods to other indices and other locations throughout the U.S.A.

Understanding how segregation changes over time remains a critical endeavor for both social science research and public policy. In the U.S.A., the changing data landscape complicates this pursuit substantially because ACS sampling errors can inflate the observed differences between successive observations, making it appear as though segregation has increased or decreased in ways that appear meaningful but result from statistical artifacts. While other scholars have proposed methods to account for ACS sampling error, we find that those methods are ineffective because they fail to account for covariance in the characteristics of ACS respondents. Using the variance replicate tables published by the Census Bureau, we show that when this covariance is taken into account properly, the estimated margins of error for a given segregation statistic differ systematically from those generated by simulation-based methods. As a result, we argue that incorporating additional information from the VRT is the best method for understanding how segregation has changed in the U.S.A. over time, and as an additional benefit we show how these improved variance estimates can facilitate statistical inference with respect to these changes. By adopting the methods we develop in this paper, we argue that policy-makers and researchers will be better equipped to understand the shifting demographic and housing characteristics in American metropolitan areas, because they can be assured that observed differences are not magnified by sampling error.