, Volume 53, Issue 5, pp 1535–1554 | Cite as

Spatial Variation in the Quality of American Community Survey Estimates

  • David C. Folch
  • Daniel Arribas-Bel
  • Julia Koschinsky
  • Seth E. Spielman


Social science research, public and private sector decisions, and allocations of federal resources often rely on data from the American Community Survey (ACS). However, this critical data source has high uncertainty in some of its most frequently used estimates. Using 2006–2010 ACS median household income estimates at the census tract scale as a test case, we explore spatial and nonspatial patterns in ACS estimate quality. We find that spatial patterns of uncertainty in the northern United States differ from those in the southern United States, and they are also different in suburbs than in urban cores. In both cases, uncertainty is lower in the former than the latter. In addition, uncertainty is higher in areas with lower incomes. We use a series of multivariate spatial regression models to describe the patterns of association between uncertainty in estimates and economic, demographic, and geographic factors, controlling for the number of responses. We find that these demographic and geographic patterns in estimate quality persist even after we account for the number of responses. Our results indicate that data quality varies across places, making cross-sectional analysis both within and across regions less reliable. Finally, we present advice for data users and potential solutions to the challenges identified.


American Community Survey Data uncertainty Income estimates Margin of error Spatial analysis 



David C. Folch and Seth E. Spielman acknowledge financial support from the National Science Foundation (Grant No. 1132008). Julia Koschinsky acknowledges funding from the National Institutes of Health (Grant 2-R01CA126858). The authors are solely responsible for the accuracy of the statements and interpretations contained in this publication. Such interpretations do not necessarily reflect the views of any government. This work used the Python Spatial Analysis Library (Rey and Anselin 2007;


  1. Anselin, L. (1988). Spatial econometrics. Dordrecht, The Netherlands: Kluwer Academic Publishers.Google Scholar
  2. Anselin, L. (1990). Spatial dependence and spatial structural instability in applied regression analysis. Journal of Regional Science, 30, 185–207.CrossRefGoogle Scholar
  3. Anselin, L. (1995). Local indicators of spatial association—LISA. Geographical Analysis, 27, 93–115.CrossRefGoogle Scholar
  4. Anselin, L., & Lozano, N. (2008). Errors in variables and spatial effects in hedonic house price models of ambient air quality. Empirical Economics, 34(5), 5–34.CrossRefGoogle Scholar
  5. Arraiz, I., Drukker, D., Kelejian, H., & Prucha, I. (2010). A spatial Cliff-Ord-type model with heteroskedastic innovations: Small and large sample results. Journal of Regional Science, 50, 592–614.CrossRefGoogle Scholar
  6. Bazuin, J. T., & Fraser, J. C. (2013). How the ACS gets it wrong: The story of the American Community Survey and a small, inner city neighborhood. Applied Geography, 45, 292–302.CrossRefGoogle Scholar
  7. Bound, J., Brown, C., & Mathiowetz, N. (2001). Measurement error in survey data. In J. J. Heckman & E. Leamer (Eds.), Handbook of econometrics (Vol. 5, pp. 3705–3843). Amsterdam, The Netherlands: Elsevier Science.Google Scholar
  8. Bruce, A., & Robinson, J. G. (2009). Tract level planning database with census 2000 data (Technical report). Washington, DC: U.S. Census Bureau.Google Scholar
  9. Castro, E. C., Jr., & Hefter, S. P. (2008). Redesigning the American Community Survey computer assisted personal interview sample. In Proceedings of the Survey Research Methods Section, American Statistical Association. Retrieved from
  10. Citro, C. F., & Kalton, G. (2007). Using the American Community Survey: Benefits and challenges. Washington, DC: National Academies Press.Google Scholar
  11. ESRI. (2011). The American Community Survey (Technical report). Redlands, CA: ESRI.Google Scholar
  12. Greene, W. (2003). Econometric analysis. Upper Saddle River, NJ: Prentice Hall.Google Scholar
  13. MacDonald, H. (2006). The American Community Survey: Warmer (more current), but fuzzier (less precise) than the decennial census. Journal of the American Planning Association, 72, 491–503.CrossRefGoogle Scholar
  14. MacEachren, A. M. (1992). Visualizing uncertain information. Cartographic Perspectives, 1992(13), 10–19.Google Scholar
  15. MacEachren, A. M., Robinson, A., Hopper, S., Gardner, S., Murray, R., Gahegan, M., & Hetzler, E. (2005). Visualizing geospatial information uncertainty: What we know and what we need to know. Cartography and Geographic Information Science, 32, 139–160.CrossRefGoogle Scholar
  16. Muchinsky, P. M. (1996). The correction for attenuation. Educational and Psychological Measurement, 56, 63–75.CrossRefGoogle Scholar
  17. Rey, S. J., & Anselin, L. (2007). PySAL: A python library of spatial analytical methods. Review of Regional Studies, 37, 5–27.Google Scholar
  18. Salvo, J. J., & Lobo, A. P. (2006). Moving from a decennial census to a continuous measurement survey: Factors affecting nonresponse at the neighborhood level. Population Research and Policy Review, 25, 225–241.CrossRefGoogle Scholar
  19. Sommers, D., & Hefter, S. P. (2010). American Community Survey sample stratification—Current and new methodology (Technical report). Washington, DC: U.S. Census Bureau.Google Scholar
  20. Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15, 72–101.CrossRefGoogle Scholar
  21. Spielman, S. E., & Folch, D. C. (2015). Reducing uncertainty in the American Community Survey through data-driven regionalization. PLoS ONE, 10(2). doi: 10.1371/journal.pone.0115626
  22. Spielman, S. E., Folch, D. C., & Nagle, N. N. (2014). Patterns and causes of uncertainty in the American Community Survey. Applied Geography, 46, 147–157.CrossRefGoogle Scholar
  23. Sun, M., & Wong, D. W. S. (2010). Incorporating data quality information in mapping American Community Survey data. Cartography and Geographic Information Science, 37, 285–299.CrossRefGoogle Scholar
  24. U.S. Census Bureau. (1994). Geographic areas reference manual (Technical report). Washington, DC: U.S. Census Bureau.Google Scholar
  25. U.S. Census Bureau. (2009a). A compass for understanding and using American Community Survey Data: What researchers need to know. Washington, DC: U.S. Government Printing Office.Google Scholar
  26. U.S. Census Bureau. (2009b). Design and methodology: American Community Survey. Washington, DC: U.S. Government Printing Office.Google Scholar
  27. Wong, D. W., & Sun, M. (2013). Handling data quality information of survey data in GIS: A case of using the American Community Survey data. Spatial Demography, 1, 3–16.CrossRefGoogle Scholar

Copyright information

© Population Association of America 2016

Authors and Affiliations

  • David C. Folch
    • 1
  • Daniel Arribas-Bel
    • 2
  • Julia Koschinsky
    • 3
  • Seth E. Spielman
    • 4
  1. 1.Department of GeographyFlorida State UniversityTallahasseeUSA
  2. 2.Department of Geography and PlanningUniversity of LiverpoolLiverpoolUK
  3. 3.Center for Spatial Data ScienceUniversity of ChicagoChicagoUSA
  4. 4.Department of GeographyUniversity of Colorado at BoulderBoulderUSA

Personalised recommendations