Skip to main content

Advertisement

SpringerLink
Log in

Evaluating the potential of a natural experiment to provide unbiased evidence of neighborhood effects on health

  • Published:
Health Services and Outcomes Research Methodology Aims and scope Submit manuscript

Abstract

In this paper we advance a dual-prong method for evaluating whether data from a natural experiment can be leveraged to draw convincing implications about causal effects of neighborhood context on health. In particular, we probe a natural experiment’s ability to surmount potentially confounding measurement issues arising from geographic selection bias. We offer methods for assessing the degree of this potential bias both immediately after households are assigned to neighborhoods and over longer durations. We then illustrate these methods using data from a natural experiment involving scattered-site public housing in Denver and test whether neighborhood context affects the hazard of children’s asthma diagnoses. We find that, conditioned on ethnicity, the tenant allocation process used by the Denver Housing Authority produced a quasi-random initial assignment across neighborhood characteristics and that the range of bias arising subsequently through selective out-migration can be narrowly delineated by replicating analyses for alternative samples defined by period of residence in public housing. We further find support for the proposition that observing no correlation between neighborhood and observed household characteristics is tantamount to observing no correlation between neighborhood and (typically) unobserved household characteristics. There appears to be little bias reintroduced through selective out-migration and staying in initially assigned neighborhood. These findings suggest that, after employing the evaluation strategy suggested here, natural experiments can indeed be a powerful tool for assessing neighborhood effects on health.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The direction of the bias has been the subject of debate, with Jencks and Mayer (1990) and Tienda (1991) arguing that measures of neighborhood impacts are biased upwards, and Brooks-Gunn et al (1997) arguing the opposite. Gennetian et al. (2011) show that these biases can be substantial enough to seriously distort conclusions about the magnitude and direction of neighborhood effects.

  2. There were no significant ethnic differences in these percentages.

  3. The excluded reference category for African American caregivers is Latinos since our study involves only these two ethnic groups.

  4. We think that this finding may be explicable on the grounds that African American applicants expressed preferences for DHA dwellings in neighborhoods with higher percentages of African Americans and that these preferences were systematically granted by DHA staff. There are at least three plausible reasons why African American applicants’ expressed preferences could have yielded offers in neighborhoods with somewhat higher percentages of African American residents. The first is the desire to maintain close ties to kin, friends, and ethnically distinctive institutions. For example, second-generation DHA applicants may have desired to return to the same neighborhoods from which they came to maintain close proximity with networks providing bonding social capital. Second, African American applicants may have perceived a more welcoming, familiar environment in neighborhoods with higher percentages of residents from their same ethnic group. The converse of the same point is that African American applicants may feel less comfortable or welcome in neighborhoods with higher percentages of Latino residents. Third, the relationship may be partially spurious because many of the neighborhoods with higher percentages of African American residents are located along the major residential and commercial growth corridor of Denver, which stretches east-northeast from the former Stapleton Airport redevelopment towards the current airport. These areas are likely attractive for non-racial reasons. Many DHA African American applicants likely had previously established intra-ethnic social networks in this corridor that informed them of the attractive prospects for employment and quality of life there. Most of both African American and Latino study participants who expressed a geographic preference to DHA identified areas where their ethnic group was disproportionately represented, with the former focusing on the east and northeast parts of Denver and the latter on the west and southwest. Maps showing the ethnic residential geography of Denver are available from the authors.

  5. The programming and execution of these simulations was conducted by Dr. Albert Anderson of PDQ Inc., whose contribution we gratefully acknowledge.

  6. Moreover, an unknown number of these transfers were involuntary, required by regulations after changes in family size or composition.

  7. We control for ethnicity of the child, a crucial condition for achieving quasi-random initial assignments, as indicated by our prior analysis. These models employ robust standard errors to account for clustering of children in households; these and other details about covariates are available upon request from the authors.

  8. These variables were measured at the time of diagnosis onset, or at time of survey or age 18 (whichever younger) if never a diagnosis.

References

  • Aaronson, D.: Using sibling data to estimate the impact of neighborhoods on children’s educational outcomes. J. Hum. Res. 33, 915–946 (1998)

    Article  Google Scholar 

  • Bayer, P., Ross, S., Topa, G.: Place of work and place of residence: informal hiring networks and labor market outcomes. J. Polit. Econ. 116, 1150–1196 (2008)

    Article  Google Scholar 

  • Bolster, A., Burgess, S., Johnston, R., Jones, K., Propper, C., Sarker, R.: Neighborhoods, households and income dynamics: a semi-parametric investigation of neighborhood effects. J. Econ. Geogr. 7, 1–38 (2007)

    Article  Google Scholar 

  • Briggs, X., Ferryman, K., Popkin, S., Rendon, M.: Why did the Moving to Opportunity experiment not get young people into better schools? Hous. Policy Debate 19(1), 53–91 (2008)

    Article  Google Scholar 

  • Briggs, X., Popkin, S., Goering, J.: Moving to Opportunity. Oxford University Press, Oxford (2010)

    Book  Google Scholar 

  • Brock, W., Durlauf, S.: Interactions-based models. In: Heckman, J., Learner, E. (eds.) Handbook of Econometrics, vol. 5, pp. 3297–3380. North-Holland Publishers, Amsterdam (2001)

    Google Scholar 

  • Brooks-Gunn, J., Duncan, G., Aber, J.: Neighborhood Poverty: Vol. 1 Context and Consequences for Children. Russell Sage Foundation, New York (1997)

    Google Scholar 

  • Burdick-Will, J., Ludwig, J., Raudenbush, S., Sampson, R., Sanbonmatsu, L., Sharkey, P.: Converging Evidence for Neighborhood Effects on Children’s Test Scores: An Experimental, Quasi-Experimental, and Observational Comparison. Brookings Institution, Washington (2010)

    Google Scholar 

  • Catalono, R., Pickett, K.: A taxonomy of research concerned with place and health. In: Scrimshaw, S.C. (ed.) The Handbook of Social Studies in Health and Medicine, pp. 64–83. Sage, Thousand Oaks (2000)

    Chapter  Google Scholar 

  • Clampet-Lundquist, S.: No more ‘Bois ball: the impact of relocation from public housing on adolescents. J. Adolesc. Res. 22(3), 298–323 (2007)

    Article  Google Scholar 

  • Clampet-Lundquist, S., Massey, D.: Neighborhood effects on economic self-sufficiency: a reconsideration of the moving to opportunity experiment. Am. J. Sociol. 114(1), 107–143 (2008)

    Article  Google Scholar 

  • Cutler, D., Glaeser, E., Vigdor, J.: When are ghettos bad? Lessons from immigrant segregation in the United States. J. Urban Econ. 63, 759–774 (2008)

    Article  Google Scholar 

  • DeLuca, S., Dayton, E.: Switching contexts: the effects of housing mobility and school choice programs on youth outcomes. Annu. Rev. Sociol. 35, 457–491 (2009)

    Article  Google Scholar 

  • DeLuca, S., Duncan, G., Mendenhall, R., Keels, M.: Gautreaux mothers and their children. Hous. Policy Debate 20(1), 7–25 (2010)

    Article  Google Scholar 

  • Dietz, R.: The estimation of neighborhood effects in the social sciences. Soc. Sci. Res. 31, 539–575 (2002)

    Article  Google Scholar 

  • Duncan, G., Connell, J., Klebanov, P.: Conceptual and methodological issues in estimating causal effects of neighborhoods and family conditions on individual development. In: Brooks-Gunn, J., Duncan, G., Aber, J. (eds.) Neighborhood Poverty. Context and Consequences for Children, vol. 1, pp. 219–250. Russell Sage Foundation, New York (1997)

    Google Scholar 

  • Duncan, G., Raudenbush, S.: Assessing the effect of context in studies of child and youth development. Educ. Psychol. 34, 29–41 (1999)

    Article  Google Scholar 

  • Edin, P., Fredricksson, P., Åslund, O.: Ethnic enclaves and the economic success of immigrants: evidence from a natural experiment. Q. J. Econ. 113, 329–357 (2003)

    Article  Google Scholar 

  • Ellen, I., Turner, M.: Do neighborhoods matter and why? In: Goering, J., Feins, J. (eds.) Choosing a Better Life? Evaluating the Moving to Opportunity Experiment, pp. 313–338. Urban Institute Press, Washington (2003)

    Google Scholar 

  • Fitzpatrick, K., Lagory, M.: Unhealthy Cities: Poverty, Race, and Place in America. Routledge, NY (2011)

    Google Scholar 

  • Foster, H., Brooks-Gunn, J.: Neighborhood influences on antisocial behavior during childhood and adolescence. In: Gibson, C.L., Krohn, M.D. (eds.) Handbook of Life-Course Criminology: Emerging Trends and Directions for Future Research, pp. 69–90. Springer, New York (2013)

    Chapter  Google Scholar 

  • Friedrichs, J.: Do poor neighbourhoods make their residents poorer? Context effects of poverty neighbourhoods on their residents. In: Andress, H. (ed.) Empirical Poverty Research in a Comparative Perspective, pp. 77–99. Ashgate Publishers, Aldershot (1998)

    Google Scholar 

  • Friedrichs, J., Galster, G., Musterd, S.: Neighborhood effects on social opportunities: the European and American research and policy context. Hous. Stud. 18, 797–806 (2003)

    Article  Google Scholar 

  • Galster, G.: Quantifying the effect of neighbourhood on individuals: challenges, alternative approaches and promising directions. J. Appl. Soc. Sci. Stud. 128, 7–48 (2008)

    Google Scholar 

  • Galster, G.: The mechanism(s) of neighbourhood effects: theory, evidence, and policy implications. In: van Ham, M., Manley, D., Bailey, N., Simpson, L., Maclennan, D. (eds.) Neighbourhood Effects Research: New Perspectives, pp. 23–56. Springer, Dordrecht (2012)

  • Galster, G.: Investigating behavioral impacts of poor neighbourhoods: towards new data and analytic strategies. Hous. Stud. 18(3), 893–914 (2003)

    Article  Google Scholar 

  • Galster, G., Andersson, R., Musterd, S.: Who is affected by neighbourhood income mix? Gender, age, family, employment and income sifferences. Urban Stud. 47(14), 2915–2944 (2010)

    Article  Google Scholar 

  • Galster, G., Andersson, R., Musterd, S., Kauppinen, T.: Does neighborhood income mix affect earnings of adults? New evidence from Sweden. J. Urban Econ. 63, 858–870 (2008)

    Article  Google Scholar 

  • Galster, G., Hedman, L.: “Measuring neighbourhood effects non-experimentally: how much do alternative methods matter? Hous. Stud. 28(3), 473–498 (2013)

    Article  Google Scholar 

  • Galster, G., Marcotte, D., Mandell, M., Wolman, H., Augustine, N.: The impact of childhood neighborhood poverty on young adult outcomes. Hous. Stud. 22(5), 723–752 (2007)

    Article  Google Scholar 

  • Gennetian, L., Ludwig, J., Sanbonmatsu, L.: An overview of moving to opportunity. In: Newburger, H., Birch, E., Wachter, S. (eds.) Neighborhood and Life Chances: How Place Matters in Modern America, pp. 163–178. University of Pennsylvania Press, Philadelphia (2011)

    Google Scholar 

  • Gephart, M.: Neighborhoods and communities as contexts for development. In: Brooks-Gunn, J., Duncan, G., Aber, J. (eds.) Neighborhood poverty. Context and Consequences for Children, vol. I, pp. 1–43. Russell Sage Foundation, New York (1997)

    Google Scholar 

  • Ginther, D., Haveman, R., Wolfe, B.: Neighborhood attributes as determinants of children’s outcomes. J. Hum. Res. 35, 603–642 (2000)

    Article  Google Scholar 

  • Harding, D.: Counterfactual models of neighborhood effects: the effect of neighborhood poverty on high school dropout and teenage pregnancy. Am. J. Sociol. 109(3), 676–719 (2003)

    Article  Google Scholar 

  • Harding, D., Gennetian, L., Winship, C., Sanbonmatsu, L., Kling, J.: Unpacking neighborhood influences on education outcomes: setting the stage for future research. In: Duncan, G., Murnane, R. (eds.) Social inequality and educational disadvantage. Russell Sage, NY (2011)

    Google Scholar 

  • Hedman, L., Galster, G.: Neighbourhood income sorting and the effects of neighbourhood income mix on income: a holistic empirical exploration. Urban Stud. 50(1), 107–127 (2013)

    Article  Google Scholar 

  • Jacob, B.: Public housing, housing vouchers, and student achievement: evidence from public housing demolitions in Chicago. Am. Econ. Rev. 94, 233–258 (2004)

    Article  Google Scholar 

  • Jargowsky, P., El Komi, M.: Before or after the bell? School context and neighborhood effects on student achievement. In: Newburger, H., Birch, E., Wachter, S. (eds.) Neighborhood and Life Chances: How Place Matters in Modern America, pp. 50–72. University of Pennsylvania Press, Philadelphia (2011)

    Google Scholar 

  • Jencks, C., Mayer, S.: The social consequences of growing up in a poor neighborhood. In: Lynn, L., McGeary, M. (eds.) Inner-City Poverty in the United States, pp. 111–186. National Academy Press, Washington (1990)

    Google Scholar 

  • Kling, J., Liebman, J., Katz, L.: Experimental analysis of neighborhood effects. Econometrica 75(1), 83–119 (2007)

    Article  Google Scholar 

  • Leventhal, T., Brooks-Gunn, J.: The neighborhoods they live in. Psychol. Bull. 126, 309–337 (2000)

    Article  CAS  PubMed  Google Scholar 

  • Leventhal, T., Dupere, V., Brooks-Gunn, J.: Neighborhood influences on adolescent development. In: Lerner, R.M., Steinberg, L. (eds.) Handbook of Adolescent Psychology, pp. 411–443. Wiley, New York (2009)

    Google Scholar 

  • Lyle, D.: Estimating and interpreting peer and role model effects from randomly assigned social groups at West Point. Rev. Econ. Stat. 89(2), 289–299 (2007)

    Article  Google Scholar 

  • Ludwig, J., Ladd, H., Duncan, G.: Urban poverty and educational outcomes. In: Gale, W., Pack, J.R. (eds.) Brookings Wharton Papers in Urban Affairs 2001, pp. 147–201. Brookings Institution Press, Washington (2001)

    Google Scholar 

  • Labre, M., Herman, E., Dumitru, G., Valenzuela, K., Cechman, C.: Public health interventions for asthma: an umbrella review, 1990–2010. Am. J. Prev. Med. 42(4), 403–410 (2012)

    Article  PubMed  Google Scholar 

  • Ludwig, J., et al.: What can we learn about neighborhood effects from the Moving to Opportunity Experiment? Am. J. Sociol. 114, 144–188 (2008)

    Article  Google Scholar 

  • Manski, C.: Identification of endogenous social effects: the reflection problem. Rev. Econ. Stud. 60, 531–542 (1993)

    Article  Google Scholar 

  • Manski, C.: Identification Problems in the Social Sciences. Harvard University Press, Cambridge (1995)

    Google Scholar 

  • Manski, C.: Economic analysis of social interactions. J. Econ. Perspect. 14, 115–136 (2000)

    Article  Google Scholar 

  • Musterd, S., Andersson, R., Galster, G., Kauppinen, T.: Are immigrants’ earnings influenced by the characteristics of their neighbours? Environ. Plan. A 40, 785–805 (2008)

    Article  Google Scholar 

  • Musterd, S., Galster, G., Andersson, R.: Temporal dimensions and the measurement of neighbourhood effects. Environ. Plan. A 44(3), 605–627 (2012)

    Article  Google Scholar 

  • Oakes, J.: The (mis)estimation of neighborhood effects: causal inference for a practicable social epidemiology. Soc. Sci. Med. 58, 1929–1952 (2004)

    Article  PubMed  Google Scholar 

  • Oreopoulos, P.: The long-run consequences of living in a poor neighborhood. Q. J. Econ. 118, 1533–1575 (2003)

    Article  Google Scholar 

  • Orr, L., Feins, J., Jacob, R., Beecroft, E., Sanbonmatsu, L., Katz, L., Liebman, J., Kling, J.: Moving to Opportunity: Interim Impacts Evaluation. U.S. Department of Housing and Urban Development, Washington (2003)

    Google Scholar 

  • Piil Damm, A.: Ethnic enclaves and immigrant labor market outcomes: quasi-experimental evidence. J. Labor Econ. 27(2), 281–314 (2009)

    Article  Google Scholar 

  • Piil Damm, A.: Neighborhood quality and labor market outcomes: evidence from a quasi-random neighborhood assignment of immigrants. J. Urban Econ. 79(1), 139–166 (2014)

    Article  Google Scholar 

  • Rosenbaum, J.: Changing the geography of opportunity by expanding residential choice: lessons from the Gautreaux program. Hous. Policy Debate 6, 231–269 (1995)

    Article  Google Scholar 

  • Ross, S.: Social interactions within cities: neighborhood environments and peer relationships. In: Brooks, N., Donaghy, K., Knaap, G. (eds.) The Oxford Handbook of Urban Economics and Planning, pp. 203–229. Oxford University Press, Oxford (2012)

    Google Scholar 

  • Sampson, R.: Moving to inequality: neighborhood effects and experiments meet social structure. Am. J. Sociol. 114(1), 189–231 (2008)

    Article  PubMed Central  PubMed  Google Scholar 

  • Sampson, R., Morenoff, J., Gannon-Rowley, T.: Assessing ‘neighborhood effects’: social processes and new directions in research. Annu. Rev. Sociol. 28, 443–478 (2002)

    Article  Google Scholar 

  • Sanbonmatsu, L., Kling, J., Duncan, G., Brooks-Gunn, J.: Neighborhoods and academic achievement: evidence from the Moving to Opportunity experiment. J. Hum. Res. 41(4), 649–691 (2006)

    Google Scholar 

  • Sanbonmatsu, L., et al.: Moving to Opportunity for Fair Housing Demonstration Program: Final Impacts. U.S. Department of Housing and Urban Development, Office of Policy Development and Research, Washington (2011)

    Google Scholar 

  • Sari, F.: Analysis of neighbourhood effects and work behaviour: evidence from Paris. Hous. Stud. 27(1), 45–76 (2012)

    Article  Google Scholar 

  • Schwartz, H.: Housing policy is school policy: economically integrative housing promotes academic achievement in Montgomery County, MD. Century Foundation, NY (2010)

    Google Scholar 

  • Sharkey, P.: An alternative approach to addressing selection into and out of social settings: neighborhood change and African American children’s economic outcomes. Sociol. Methods Res. 41, 251–293 (2012)

    Article  Google Scholar 

  • Tienda, M.: Poor people and poor places: deciphering neighborhood effects on poverty outcomes. In: Haber, J. (ed.) Macro-Micro Linkages in Sociology, pp. 244–262. Sage, Newbury Park (1991)

    Google Scholar 

  • Van Ham, M., Manley, D.: The effect of neighbourhood housing tenure mix on labor market outcomes: a longitudinal investigation of neighbourhood effects. J. Econ. Geogr. 10(2), 257–282 (2010)

    Article  Google Scholar 

  • Van Kempen, E.: Poverty pockets and life chances. Am. Behav. Sci. 41(3), 430–449 (1997)

    Article  Google Scholar 

  • Vartanian, T., Buck, P.: Childhood and adolescent neighborhood effects on adult income: using siblings to examine differences in ordinary least squares and fixed-effect models. Soc. Ser. Rev. 79, 60–94 (2005)

    Article  Google Scholar 

  • Weinberg, B., Reagan, P., Yankow, J.: Do neighborhoods affect work behavior? Evidence from the NLSY79. J. Labor Econ. 22(4), 891–924 (2004)

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by NICHD grant 5R01 HD47786-2, and Grants from the MacArthur Foundation, the Kellogg Foundation, the U.S. Department of Housing and Urban Development and the Annie E. Casey Foundation. The views represented in this paper do not necessarily reflect the views of these sponsors. Tama Leventhal and Xiaoming Li served as expert advisors to this project. Stefanie DeLuca, Greg Duncan, Lisa Gennetian, David Harding, Rucker Johnson, Jens Ludwig, Jeffrey Morenoff, Philip Oreopoulos, Gary Painter, and seminar participants at Glasgow University, the University of Michigan, and the University of Southern California contributed many helpful suggestions. The authors also gratefully acknowledge the programming assistance of Dr. Albert Anderson and the research assistance of Georgios Kypriotakis, Eun Lye Lee, Andrew Linn, Ana Santiago-San Roman, Lisa Stack and Rebecca Grace Stokan.

Author information

Authors and Affiliations

  1. Department of Urban Studies and Planning, Wayne State University, 656 W. Kirby St., Detroit, MI, 48203, USA

    George C. Galster

  2. Mandel School of Applied Social Sciences, Case Western Reserve University, 11235 Bellflower Road, Cleveland, OH, 44106, USA

    Anna M. Santiago

Authors
  1. George C. Galster
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna M. Santiago
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George C. Galster.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Galster, G.C., Santiago, A.M. Evaluating the potential of a natural experiment to provide unbiased evidence of neighborhood effects on health. Health Serv Outcomes Res Method 15, 99–135 (2015). https://doi.org/10.1007/s10742-014-0134-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10742-014-0134-9

Keywords

Search

Navigation