Journal of Quantitative Criminology

, Volume 28, Issue 1, pp 191–218 | Cite as

American Terrorism and Extremist Crime Data Sources and Selectivity Bias: An Investigation Focusing on Homicide Events Committed by Far-Right Extremists

  • Steven M. Chermak
  • Joshua D. Freilich
  • William S. Parkin
  • James P. Lynch
Original Paper


This paper examines the reliability of the methods used to capture homicide events committed by far-right extremists in a number of open source terrorism data sources. Although the number of research studies that use open source data to examine terrorism has grown dramatically in the last 10 years, there has yet to be a study that examines issues related to selectivity bias. After reviewing limitations of existing terrorism studies and the major sources of data on terrorism and violent extremist criminal activity, we compare the estimates of these homicide events from 10 sources used to create the United States Extremist Crime Database (ECDB). We document incidents that sources either incorrectly exclude or include based upon their inclusion criteria. We use a “catchment-re-catchment” analysis and find that the inclusion of additional sources result in decreasing numbers of target events not identified in previous sources and a steadily increasing number of events that were identified in any of the previous data sources. This finding indicates that collectively the sources are approaching capturing the universe of eligible events. Next, we assess the effects of procedural differences on these estimates. We find considerable variation in the number of events captured by sources. Sources include some events that are contrary to their inclusion criteria and exclude others that meet their criteria. Importantly, though, the attributes of victim, suspect, and incident characteristics are generally similar across data source. This finding supports the notion that scholars using open-source data are using data that is representative of the larger universe they are interested in. The implications for terrorism and open source research are discussed.


Terrorism Far-right violence Selectivity bias 



The authors would like to thank the editors, and participants in the research seminar on methodological issues in terrorism that was convened at John Jay College of Criminal Justice in October, 2010 for helpful insights. The authors would also like to thank the reviewers and Erin Miller for their thoughtful suggestions for revision. In addition, we would like to thank Nick Corsaro and Jeremy Wilson for helping us think through the data analysis plan.


  1. Aho JA (1990) The politics of righteousness: Idaho Christian patriotism. University of Washington Press, SeattleGoogle Scholar
  2. Bakker E (2006) Jihadi terrorists in Europe, their characteristics and the circumstances in which they joined the Jihad: an exploratory study. Clingendael Institute, The HagueGoogle Scholar
  3. Biderman A, Lynch JP (1991) Understanding crime incidence statistics: why the UCR diverges from the NCS. Springer, New YorkGoogle Scholar
  4. Chen H, Reid E, Sinai J, Silke A, Ganor B (2008) Terrorism Informatics: knowledge management and data mining for homeland security. Springer, New YorkGoogle Scholar
  5. Chermak SM (2002) Searching for a demon. The media construction of the militia movement. Northeastern University Press, BostonGoogle Scholar
  6. Chermak SM, Gruenewald J (2006) Domestic terrorism and the media. Justice Q 23(4):428–461CrossRefGoogle Scholar
  7. Chermak SM, Freilich JD, Simone J Jr (2010) Surveying American state police agencies about lone wolves, far-right criminality, and far-right and Islamic jihadist criminal collaboration. Stud Confl Terrorism 33:1019–1041CrossRefGoogle Scholar
  8. Coates J (1995) Armed and dangerous: the rise of the survivalist right. Hill and Wang, New YorkGoogle Scholar
  9. Dobratz BA, Shanks-Meile SL (1997) White power, white pride! The white separatist movement in the United States. Twayne Publishers, New YorkGoogle Scholar
  10. Damphousse K, Shields C (2007) The morning after, assessing the effect of major terrorism events of prosecution strategies and outcomes. J Contemp Crim Just 23(2):154–174CrossRefGoogle Scholar
  11. Dugan L, LaFree G, Piquero AR (2005) Testing a rational choice model of airline hijacking. Criminology 43(4):1031–1065CrossRefGoogle Scholar
  12. Dyer J (1997) Harvest of rage: why Oklahoma City is only the beginning. Westview, BoulderGoogle Scholar
  13. Endler W, Sandler T (2006) The political economy of terrorism. Cambridge University Press, NYGoogle Scholar
  14. Freilich JD (2003) American militias: state-level variations in militia activities. LFB Scholarly Publishing, New YorkGoogle Scholar
  15. Freilich JD, Chermak SM (2009a) United States extremist crime database (ECDB), 1990–2008: preliminary results. Department of Homeland Security University Network Research and Education Summit. Washington, DCGoogle Scholar
  16. Freilich JD, Chermak SM (2009b) Preventing deadly encounters between law enforcement and American far-rightists. Crime Prev Stud 25:141–172Google Scholar
  17. Freilich JD, Chermak SM (2011) Introducing the United States extremist crime database (ECDB), 1990-2010. Paper under reviewGoogle Scholar
  18. Freilich JD, Pridemore WA (2006) Mismeasuring militias: limitations of advocacy group data and of state-level studies of paramilitary groups. Justice Q 23(1):147–162CrossRefGoogle Scholar
  19. Freilich JD, Almanzar NAP, Rivera CJ (1999) How social movement organizations explicitly and implicitly promote deviant behavior: the case of the militia movement. Justice Q 16(3):655–683CrossRefGoogle Scholar
  20. Freilich JD, Chermak SM, Caspi D (2009a) Critical events in the life trajectories of domestic extremist white supremacist groups: a case study analysis of four violent organizations. Criminol Public Policy 8(3):497–530CrossRefGoogle Scholar
  21. Freilich JD, Chermak SM, Simone J Jr (2009b) Surveying American state police agencies about terrorism threats, terrorism sources, and terrorism definitions. Terrorism Polit Violence 21(3):450–475CrossRefGoogle Scholar
  22. Green DP, Strolovitch DZ, Wong JS, Bailey R (2001) Measuring gay populations and antigay hate crime. Soc Sci Q 82(2):281–297CrossRefGoogle Scholar
  23. Gruenewald J, Freilich JD, Chermak SM (2009) An overview of the domestic far-right and its criminal activities. In: Perry B, Blazak R (eds) Hate crimes: hate crime offenders. Praeger, Westport, pp 1–21Google Scholar
  24. Hamm M (1993) American skinheads: the criminology and control of hate crime. Praeger, WestportGoogle Scholar
  25. Hamm M (1997) Apocalypse in Oklahoma City: Waco and Ruby Ridge revenged. Northeastern University Press, BostonGoogle Scholar
  26. Hamm M (2002) In bad company: America’s terrorist underground. Northeastern University Press, BostonGoogle Scholar
  27. Hamm MS (2005) Crimes committed by terrorist groups: theory, research and prevention. Final Report to the National Institute of Justice, September 2005Google Scholar
  28. Handler J (1990) Socioeconomic profile of an American terrorist. Terrorism 13:195–213CrossRefGoogle Scholar
  29. Hewitt C (2005) Political violence and terrorism in modern America: Christopher Hewitt: a chronology. Praeger Security International, New York CityGoogle Scholar
  30. Hewitt C (2003) Understanding terrorism in America: from the Klan to Al Qaeda. Routledge, New YorkGoogle Scholar
  31. Horgan J (2008) Interviewing terrorists: a case for primary research. In: Chen H et al (eds) Terrorism informatics: knowledge management and data mining for homeland security. Springer, New York, pp 27–50Google Scholar
  32. Johnson S, Braithewaite A (2009) Spatio-temporal modeling of insurgency in Iraq. Crime Prev Stud 25:9–31Google Scholar
  33. LaFree G, Dugan L (2004) How does studying terrorism compare to studying crime? Sociology of crime. Law Deviance 5:53–74CrossRefGoogle Scholar
  34. LaFree G, Dugan L (2007) Introducing the global terrorism database. Terrorism Polit Violence 19:181–204CrossRefGoogle Scholar
  35. LaFree G, Dugan L (2009) Research on terrorism and countering terrorism. Crime Justice 38:413–477Google Scholar
  36. LaFree G, Morris N, Dugan L, Fahey S (2006) Identifying global terrorist hot spots. In: Victoroff J (ed) Tangled roots: social and psychological factors in the genesis of terrorism. IOS Press, Amsterdam, pp 98–114Google Scholar
  37. Langer E (2003) A hundred little Hitlers: the death of a black man, the trial of a white racist, and the rise of the neo-Nazi movement in America. Picador, New YorkGoogle Scholar
  38. Leiken RS, Brooke S (2006) The quantitative analysis of terrorism and immigration: an initial exploration. Terrorism Polit Violence 18:503–521CrossRefGoogle Scholar
  39. Lum C, Kennedy LW, Sherley AJ (2006) The effectiveness of counter-terrorism strategies: a campbell systematic review. [Online]:
  40. Lynch JP, Addington LA (eds) (2007) Understanding crime statistics: revisting the divergence of the NCVS and the UCR. Cambridge University Press, CambridgeGoogle Scholar
  41. Maxwell C, Chermak S (2007) Creating a data archive to facilitate research on understanding and responding to terrorism. In Gunes OS, Al-Badayner DM (eds) Understanding terrorism: analysis of sociological and psychological aspects. NATO Science for Peace and Security. IOS Press, Amsterdam, pp 306–331Google Scholar
  42. Merari A (1991) Academic research and government policy on terrorism. Terrorism Polit Violence 3(1):88–102CrossRefGoogle Scholar
  43. Naday A, Freilich JD, Mellow J (2008) The elusive data on supermax confinement. Prison J 88(1):69–93CrossRefGoogle Scholar
  44. Neiwert DavidA (1999) In god’s country: the patriot movement in the Pacific Northwest. Washington State University Press, PullmanGoogle Scholar
  45. Ross JI (1992) Research note: contemporary radical right-wing violence in Canada: a quantitative analysis. Terrorism Polit Violence 4(3):72–101CrossRefGoogle Scholar
  46. Ross JI (1993) Research on contemporary oppositional political terrorism in the United States: merits, drawbacks and suggestions for improvement. In: Tunnel KD (ed) Political crime in contemporary America: a critical approach. Garland Publishing, New York, pp 101–116Google Scholar
  47. Sageman M (2004) Understanding terror networks. University of Pennsylvania Press, PhiladelphiaGoogle Scholar
  48. Schmid A (2004) Terrorism on trial: the definitional problem. Case West Reserve Univ Sch Law 36:375–419Google Scholar
  49. Silber MA, Bhatt A (2006) Radicalization in the West: the Homegrown threat. New York City Police Department Intelligence Division. Available from the WWW at
  50. Silke A (2001) The devil you know: continuing problems with research on Terrorism. Terrorism Polit Violence 13(4):1–14CrossRefGoogle Scholar
  51. Silke A (2008) Research on terrorism: a review of the impact of 9/11 and the global QWar on terrorism. In: Chen H et al (eds) Terrorism informatics: knowledge management and data mining for homeland security. Springer, New York, pp 27–50Google Scholar
  52. Smith BL (1994) Terrorism in America: pipe bombs pipe dreams. State University of New York Press, AlbanyGoogle Scholar
  53. Smith BL, Damphousse KR (1998) Terrorism, politics and punishment: a test of structural-contextual theory and the ‘liberation hypothesis. Criminology 36(1):67–92CrossRefGoogle Scholar
  54. Surette R (1999) Media echoes: systematic effects of news coverage. Justice Q 16(3):601–631CrossRefGoogle Scholar
  55. Townsley M, Johnson SD, Ratcliffe JH (2008) Space time dynamics of insurgent activity in Iraq. Secur J 21:139–146CrossRefGoogle Scholar
  56. Turchie T, Puckett K (2007) Hunting the American terrorist: the FBI’s war on homegrown terror. History publishing Company, PalisadesGoogle Scholar
  57. Victoroff J (2005) The mind of the terrorist: a review and critique of psychological approaches. J Conflict Resolut 49(1):3–42CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Steven M. Chermak
    • 1
  • Joshua D. Freilich
    • 2
  • William S. Parkin
    • 2
  • James P. Lynch
    • 3
  1. 1.School of Criminal JusticeMichigan State UniversityEast LansingUSA
  2. 2.John Jay College of Criminal JusticeNew YorkUSA
  3. 3.Bureau of Justice StatisticsWashingtonUSA

Personalised recommendations