Time series robustness checks to test the effects of the 1996 Australian firearm law on cause-specific mortality
Many studies utilize time series methods to identify causal effects without accounting for an underlying time trend. We show that accounting for trends changes the conclusions in the study of Chapman et al. (JAMA, 316(3), 291–299, 2016), who evaluated the impact of the Australian firearm law in 1996. We also introduce a new empirical method that tests whether their empirical strategy can actually identify a causal effect that is also useful for panel analyses.
We use national data from the Australian Bureau of Statistics, assembled in annual counts of: total firearm deaths, firearm suicides, and firearm homicides. These data are used in an independent re-analysis of the impact of the 1996 Australian firearm law that accounts for underlying stochastic trends. We then estimate a series of artificially created interruptions using interrupted times series analysis in a time frame before 1996, to test for changes in the slope of mortality across several years prior to the actual regulatory changes. This tests whether the empirical model produces effects in years other than the year of the intervention, thereby testing if the results can simply be replicated at random using other interruption years.
Controlling for stochastic trends produces less statistical evidence of the impact of the firearm law on firearm mortality than previously reported by Chapman et al. (JAMA, 316(3), 291–299, 2016). Introducing artificial interruptions in 1990 through 1995 produces statistically significant decreases in all firearm-related mortality measures well above the expected type 1 error. Overall, 19 out of the 36 artificial interruption models we tested were found to be statistically significant, suggesting that the empirical model can be implemented in multiple non-intervention years with results similar to the true 1996 interruption year.
Current evidence showing decreases in firearm mortality after the 1996 Australian national firearm law relies on an empirical model that may have limited ability to identify the true effects of the law.
KeywordsAustralia Firearm regulation Interrupted time series Methods
- Andreyeva, E., & Ukert, B. (2017). Do firearm regulations work? Evidence from the Australian National Firearms Agreement.Google Scholar
- Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348–355.Google Scholar
- Branas, C. C., Kastanaki, A. E., Michalodimitrakis, M., Tzougas, J., Kranioti, E. F., Theodorakis, P. N., Carr, B. G., & Wiebe, D. J. (2015). The impact of economic austerity and prosperity events on suicide in Greece: A 30-year interrupted time-series analysis. British Medical Journal Open, 5(1), 1–11.Google Scholar
- Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Skokie, IL: Rand McNally.Google Scholar
- Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.Google Scholar
- Enders, W. (2014). Applied econometric time series (4th ed.). New York, NY: Wiley.Google Scholar
- Gagné, M., Robitaille, Y., Hamel, D., & St-Laurent, D. (2010). Firearms regulation and declining rates of male suicide in Quebec. Injury Prevention, ip-2009.Google Scholar
- Greene, W. H. (2003). Econometric analysis. Delhi, India: Pearson Education.Google Scholar
- Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York, NY: Houghton Mifflin.Google Scholar