Abstract
Objectives
Many studies utilize time series methods to identify causal effects without accounting for an underlying time trend. We show that accounting for trends changes the conclusions in the study of Chapman et al. (JAMA, 316(3), 291–299, 2016), who evaluated the impact of the Australian firearm law in 1996. We also introduce a new empirical method that tests whether their empirical strategy can actually identify a causal effect that is also useful for panel analyses.
Methods
We use national data from the Australian Bureau of Statistics, assembled in annual counts of: total firearm deaths, firearm suicides, and firearm homicides. These data are used in an independent re-analysis of the impact of the 1996 Australian firearm law that accounts for underlying stochastic trends. We then estimate a series of artificially created interruptions using interrupted times series analysis in a time frame before 1996, to test for changes in the slope of mortality across several years prior to the actual regulatory changes. This tests whether the empirical model produces effects in years other than the year of the intervention, thereby testing if the results can simply be replicated at random using other interruption years.
Results
Controlling for stochastic trends produces less statistical evidence of the impact of the firearm law on firearm mortality than previously reported by Chapman et al. (JAMA, 316(3), 291–299, 2016). Introducing artificial interruptions in 1990 through 1995 produces statistically significant decreases in all firearm-related mortality measures well above the expected type 1 error. Overall, 19 out of the 36 artificial interruption models we tested were found to be statistically significant, suggesting that the empirical model can be implemented in multiple non-intervention years with results similar to the true 1996 interruption year.
Conclusions
Current evidence showing decreases in firearm mortality after the 1996 Australian national firearm law relies on an empirical model that may have limited ability to identify the true effects of the law.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We also stratified the pre-treatment period into two unique periods, 1979–1986 and 1986–1996, and estimated artificial interruptions in 1982 and 1993, respectively. We similarly find statistically significant impacts of the law and the interaction in Chapman et al.’s (2016) model well above expected levels. Such subsample analysis speaks to the multiple comparison problem outlined in Hansen (2001), which may be a potential concern in the results of the five artificial interruptions in the same time series.
References
Abadie, A., Diamond, A., & Hainmueller, J. (2010). Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105(490), 493–505.
Andreyeva, E., & Ukert, B. (2017). Do firearm regulations work? Evidence from the Australian National Firearms Agreement.
Bernal, J. L., Cummins, S., & Gasparrini, A. (2017). Interrupted time series regression for the evaluation of public health interventions: A tutorial. International Journal of Epidemiology, 46(1), 348–355.
Branas, C. C., Kastanaki, A. E., Michalodimitrakis, M., Tzougas, J., Kranioti, E. F., Theodorakis, P. N., Carr, B. G., & Wiebe, D. J. (2015). The impact of economic austerity and prosperity events on suicide in Greece: A 30-year interrupted time-series analysis. British Medical Journal Open, 5(1), 1–11.
Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Skokie, IL: Rand McNally.
Carmichael, F., & Ward, R. (2001). Male unemployment and crime in England and Wales. Economics Letters, 73(1), 111–115.
Chapman, S., Alpers, P., Agho, K., & Jones, M. (2006). Australia’s 1996 gun law reforms: Faster falls in firearm deaths, firearm suicides, and a decade without mass shootings. Injury Prevention, 12(6), 365–372.
Chapman, S., Alpers, P., & Jones, M. (2016). Association between gun law reforms and intentional firearm deaths in Australia, 1979–2013. JAMA, 316(3), 291–299.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
D’Alessio, S. J., & Stolzenberg, L. (1995). The impact of sentencing guidelines on jail incarceration in Minnesota. Criminology, 33(2), 283–302.
Dugan, L. (2010). Estimating effects over time for single and multiple units. In A. Piquero & D. Weisburd (Eds.), Handbook of quantitative criminology (pp. 741–763). New York, NY: Springer.
Enders, W. (2014). Applied econometric time series (4th ed.). New York, NY: Wiley.
Gagné, M., Robitaille, Y., Hamel, D., & St-Laurent, D. (2010). Firearms regulation and declining rates of male suicide in Quebec. Injury Prevention, ip-2009.
Greene, W. H. (2003). Econometric analysis. Delhi, India: Pearson Education.
Hansen, B. E. (2001). The new econometrics of structural change: Dating breaks in U.S. labor productivity. The Journal of Economic Perspectives, 15, 117–128.
Humphreys, D. K., Gasparrini, A., & Wiebe, D. J. (2017). Evaluating the impact of Florida’s “stand your ground” self-defense law on homicide and suicide by firearm: An interrupted time series study. JAMA Internal Medicine, 177(1), 44–50.
Klieve, H., Barnes, M., & De Leo, D. (2009). Controlling firearms use in Australia: Has the 1996 gun law reform produced the decrease in rates of suicide with this method? Social Psychiatry and Psychiatric Epidemiology, 44(4), 285.
Laliotis, I., Ioannidis, J. P., & Stavropoulou, C. (2016). Total and cause-specific mortality before and after the onset of the Greek economic crisis: An interrupted time-series analysis. The Lancet Public Health, 1(2), e56–e65.
Lee, D. Y., & Holoviak, S. J. (2006). Unemployment and crime: An empirical investigation. Applied Economics Letters, 13(12), 805–810.
Lee, W.-S., & Suardi, S. (2010). The Australian firearms buyback and its effect on gun deaths. Contemporary Economic Policy, 28(1), 65–79.
McCleary, R., McDowall, D., & Bartos, B. J. (2017). Design and analysis of time series experiments. New York, NY: Oxford University Press.
Messner, S. F., Raffalovich, L. E., & McMillan, R. (2001). Economic deprivation and changes in homicide arrest rates for white and black youths, 1967–1998: A national time-series analysis. Criminology, 39(3), 591–614.
Pridemore, W. A., Chamlin, M. B., & Cochran, J. K. (2007). An interrupted time-series analysis of Durkheim’s social deregulation thesis: The case of the Russian Federation. Justice Quarterly, 24(2), 271–290.
Pridemore, W. A., Chamlin, M. B., & Trahan, A. (2008). A test of competing hypotheses about homicide following terrorist attacks: An interrupted time series analysis of September 11 and Oklahoma City. Journal of Quantitative Criminology, 24(4), 381–396.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. New York, NY: Houghton Mifflin.
Spittal, M. J., Pirkis, J., Miller, M., & Studdert, D. M. (2012). Declines in the lethality of suicide attempts explain the decline in suicide deaths in Australia. PLoS One, 7(9), e44565.
Tang, C. F., & Lean, H. H. (2007). Will inflation increase crime rate? New evidence from bounds and modified Wald tests. Global Crime, 8(4), 311–323.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ukert, B., Andreyeva, E. & Branas, C.C. Time series robustness checks to test the effects of the 1996 Australian firearm law on cause-specific mortality. J Exp Criminol 14, 141–154 (2018). https://doi.org/10.1007/s11292-017-9313-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11292-017-9313-3