Skip to main content

Advertisement

SpringerLink
Log in

Difference-in-Difference in the Time of Cholera: a Gentle Introduction for Epidemiologists

  • Epidemiologic Methods (P Howards, Section Editor)
  • Published:
Current Epidemiology Reports Aims and scope Submit manuscript

Abstract

Purpose of Review

The goal of this article is to provide an introduction to the intuition behind the difference-in-difference method for epidemiologists. We focus on the theoretical aspects of this tool, including the types of questions for which difference-in-difference is appropriate, and what assumptions must hold for the results to be causally interpretable.

Recent Findings

While currently under-utilized in epidemiologic research, the difference-in-difference method is a useful tool to examine effects of population level exposures, but relies on strong assumptions.

Summary

We use the famous example of John Snow’s investigation of the cause of cholera mortality in London to illustrate the difference-in-difference approach and corresponding assumptions. We conclude by arguing that this method deserves a second look from epidemiologists interested in asking causal questions about the impact of a population level exposure change on a population level outcome for the group that experienced the change.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. John Snow provides population counts for 1851, and population estimates for 1849, but no population estimates for 1854. Therefore, we have used the 1851 data for both time points.

References

Papers of particular interest, published recently, have been highlighted as: • Of importance

  1. • Snow J. On the mode of communication of cholera. John Churchill; 1855. 216 p. The first recorded use of the difference-in-difference method.

  2. • Snow J. Snow on cholera, being a reprint of two papers. New York: New York, The Commonwealth fund; 1936. The first recorded use of the difference-in-difference method.

    Google Scholar 

  3. • Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health. 2018;39(1):453–69 An excellent review article for those interested in learning more about the applied use of difference-in-difference.

    Article  Google Scholar 

  4. • Difference-in-difference estimation | Columbia Public Health [Internet]. [cited 2020 Jul 17]. Available from: https://www.publichealth.columbia.edu/research/population-health-methods/difference-difference-estimation. A useful web-based tutorial on the difference-in-difference method.

  5. Subramanian SV, Jones K, Kaddour A, Krieger N. Revisiting Robinson: the perils of individualistic and ecologic fallacy. Int J Epidemiol. 2009 Apr;38(2):342–60.

    Article  CAS  Google Scholar 

  6. • Mostly Harmless Econometrics | [Internet]. [cited 2020 May 29]. Available from: https://www.mostlyharmlesseconometrics.com/. A popular econometrics textbook which provides detailed intuition on the statistics of the difference-in-difference method.

  7. • Castillo-Carniglia A, Ponicki WR, Gaidus A, Gruenewald PJ, Marshall BDL, Fink DS, et al. Prescription drug monitoring programs and opioid overdoses: exploring sources of heterogeneity. Epidemiology. 2019 Mar;30(2):212–20 Applied epidemiologic & public health papers using the difference-in-difference method.

  8. • Cerdá M, Wall M, Feng T, Keyes KM, Sarvet A, Schulenberg J, et al. Association of state recreational marijuana laws with adolescent marijuana use. JAMA Pediatr. 2017;171(2):142 Applied epidemiologic & public health papers using the difference-in-difference method.

    Article  Google Scholar 

  9. • Warto M, Parker M. Oops, I D-I-D it again! Advanced difference-in-differences models in SAS®. :17. Sample code with explanations for difference-in-difference regression modeling in SAS.

  10. • Spiegelman D, Hertzmark E. Easy SAS calculations for risk or prevalence ratios and differences. Am J Epidemiol. 2005;162(3):199–200 Sample code with explanations for difference-in-difference regression modeling in SAS.

    Article  Google Scholar 

  11. Hernan MA. Does water kill? A call for less casual causal inferences. Ann Epidemiol. 2016 Oct;26(10):674–80.

    Article  Google Scholar 

  12. • Santaella-Tenorio J, Cerdá M, Villaveces A, Galea S. What do we know about the association between firearm legislation and firearm-related injuries? Epidemiol Rev. 2016;38(1):140–57 Applied epidemiologic & public health papers using the difference-in-difference method.

    Article  Google Scholar 

  13. Westreich D, Cole SR. Invited commentary: positivity in practice. Am J Epidemiol. 2010 Mar 15;171(6):674–7.

    Article  Google Scholar 

  14. • Hamad R, Collin DF, Rehkopf DH. Estimating the short-term effects of the earned income tax credit on child health. Am J Epidemiol. 2018;187(12):2633–41 Applied epidemiologic & public health papers using the difference-in-difference method.

    PubMed  PubMed Central  Google Scholar 

  15. • Raifman J, Moscoe E, Austin SB, McConnell M. Difference-in-differences analysis of the association between state same-sex marriage policies and adolescent suicide attempts. JAMA Pediatr. 2017;171(4):350–6 Applied epidemiologic & public health papers using the difference-in-difference method.

    Article  Google Scholar 

  16. • Athey S, Imbens G. Design-based analysis in difference-in-differences settings with staggered adoption. ArXiv180805293 Cs Econ Math Stat [Internet]. 2018 Sep 1 [cited 2020 Aug 13]; Available from: http://arxiv.org/abs/1808.05293. A paper extending the difference-in-difference method to settings with staggered adoption of policies.

  17. • Goodman-Bacon A. Difference-in-differences with variation in treatment timing [Internet]. Cambridge: National Bureau of Economic Research; 2018 Sep [cited 2020 Mar 5] p. w25018. Report No.: w25018. Available from: http://www.nber.org/papers/w25018.pdf. A paper extending the difference-in-difference method to variations in treatment timing.

Download references

Acknowledgments

We thank Dr. Noah Haber & Dr. Samrachana Adhikari for thoughtful comments on an earlier draft of this manuscript. Any remaining errors are our own. EC was supported by NIH/NICHD K01 HD100222-01A1.

Author information

Authors and Affiliations

  1. Department of Population Health, New York University Langone Medical Center, New York City, NY, USA

    Ellen C. Caniglia

  2. Department of Epidemiology, Boston University School of Public Health, Boston, MA, USA

    Eleanor J. Murray

Authors
  1. Ellen C. Caniglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eleanor J. Murray
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eleanor J. Murray.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Epidemiologic Methods

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Caniglia, E.C., Murray, E.J. Difference-in-Difference in the Time of Cholera: a Gentle Introduction for Epidemiologists. Curr Epidemiol Rep 7, 203–211 (2020). https://doi.org/10.1007/s40471-020-00245-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40471-020-00245-2

Keywords

Search

Navigation