Skip to main content

American Terrorism and Extremist Crime Data Sources and Selectivity Bias: An Investigation Focusing on Homicide Events Committed by Far-Right Extremists

Journal of Quantitative Criminology volume 28, pages 191–218 (2012)Cite this article

Abstract

This paper examines the reliability of the methods used to capture homicide events committed by far-right extremists in a number of open source terrorism data sources. Although the number of research studies that use open source data to examine terrorism has grown dramatically in the last 10 years, there has yet to be a study that examines issues related to selectivity bias. After reviewing limitations of existing terrorism studies and the major sources of data on terrorism and violent extremist criminal activity, we compare the estimates of these homicide events from 10 sources used to create the United States Extremist Crime Database (ECDB). We document incidents that sources either incorrectly exclude or include based upon their inclusion criteria. We use a “catchment-re-catchment” analysis and find that the inclusion of additional sources result in decreasing numbers of target events not identified in previous sources and a steadily increasing number of events that were identified in any of the previous data sources. This finding indicates that collectively the sources are approaching capturing the universe of eligible events. Next, we assess the effects of procedural differences on these estimates. We find considerable variation in the number of events captured by sources. Sources include some events that are contrary to their inclusion criteria and exclude others that meet their criteria. Importantly, though, the attributes of victim, suspect, and incident characteristics are generally similar across data source. This finding supports the notion that scholars using open-source data are using data that is representative of the larger universe they are interested in. The implications for terrorism and open source research are discussed.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Notes

  1. The ECDB has received funding to expand its focus to include far-left and Al Qaeda inspired violent criminal activities and financial crimes by far-rightists and Al Qaeda inspired offenders. Since the focus of the analysis is on the far-right homicides, these data will not be discussed.

  2. This study operationalizes the far-right as individuals or groups that subscribe to aspects of the following ideals: They are fiercely nationalistic (as opposed to universal and international in orientation), anti-global, suspicious of centralized federal authority, reverent of individual liberty (especially their right to own guns, be free of taxes), believe in conspiracy theories that involve a grave threat to national sovereignty and/or personal liberty and a belief that one’s personal and/or national “way of life” is under attack and is either already lost or that the threat is imminent (sometimes such beliefs are amorphous and vague, but for some the threat is from a specific ethnic, racial, or religious group), and a belief in the need to be prepared for an attack either by participating in or supporting the need for paramilitary preparations and training or survivalism. Importantly, the mainstream conservative movement and the mainstream Christian right are not included.

  3. We attempted to review an 11th source, the Bureau of Justice Assistance (BJA) and Department of Defense (DOD) funded Institute for the Study of Violent Groups (ISVG) that tracks crimes committed by political extremists since 2002. Our requests for a listing of far-right homicides committed in the U.S. went unanswered.

  4. We excluded the National Counterterrorism Center’s Worldwide Incidents Tracking System (WITS) database that includes terrorist acts committed since 2005 in our analysis because it only contained one far-right related homicide event. This homicide was included in the other sources we examined.

  5. Six of the sources we used only included acts committed to further a “political” or “ideological” objective. These sources assume that ideologically motivated crimes are committed for a higher purpose and are different than routine crimes committed to further personal interests. The ADL, SPLC, the media and our coders focused however, on crimes committed by extremist suspects as opposed to events. Thus, these sources are interested in all homicides committed by far-rightists (both ideological and non-ideological). Our study assumes that the ECDB categorization of a homicide event as either ideologically motivated or non-ideologically motivated is accurate. In this sense, we are privileging the ECDB. For example, if the ECDB categorizes homicide X as ideologically motivated, while the FBI or GTD categorize it as non-ideological we code that event as ideologically motivated. While we appreciate critiques that question why the ECDB is treated as the ground truth, we make this determination for a few reasons. Again, the ECDB’s ongoing data collection efforts focus almost exclusively on homicides committed by far-rightists and other extremists in the United States. Further, the ECDB systematically searched through a series of open sources to identify these events twice. Conversely, most other data collection efforts focus on a much larger geographic universe and have a much larger N.

  6. The FBI and databases like the GTD argue that while hate/bias-motivated crimes are related to terrorism, they are a separate phenomenon. Hate crimes are counted separately (e.g., the FBI’s UCR hate crimes report is distinct from the government’s annual terrorism reports). Non-hate crime ideologically motivated acts are thought to implicate broader political objectives that qualify as terrorist, while hate crimes do not qualify. Other sources disagree and sometimes conclude that ideologically motivated acts- anti- government or anti-minority- qualify and should be labeled terrorist. Here too we assume that the ECDB categorization of a homicide event as either a bias-motivated crime or not is accurate.

  7. The FBI and sources that rely upon its definition (e.g., the ATS and Hewitt) conclude that acts committed by lone wolves usually do not qualify as terrorists. Groups like the IRA or Al Qaeda, 1998- 2001, are organized entities that engage in ongoing criminal activities designed to harm American interests. Conversely, lone actors usually lack the logistical support and infrastructure to conduct repeated attacks and to remain a longstanding threat to government interests. Thus, lone wolves do not implicate the same threat level and do not qualify as terrorist. Other sources conclude that ideologically motivated acts, regardless of the organizational level of the suspects who commit it, should be labeled terrorist.

  8. The F.B.I. is charged with investigating domestic terrorism incidents and these incidents are subsequently prosecuted on the federal-level. Some sources again conclude that the jurisdiction of the prosecution is irrelevant and that what matters is whether the act is ideologically motivated.

  9. The FBI defines terrorism as “the unlawful use of force or violence against persons or property to intimidate or coerce a government, the civilian population, or any segment thereof, in furtherance of political or social objectives” (FBI, 1997).

  10. While Hewitt claimed to rely on the F.B.I.s definition, his precise methodology and validation scheme are unclear. He devotes only four pages, in an appendix, to his data sources and coding procedures. Further, as our analyses demonstrate Hewitt’s chronology includes lone actor, state-level and other attacks that should have been excluded under a strict application of the FBI’s guidelines.

  11. There was the possibility that a homicide that occurred outside the time frame would be included in a source. SLATT reports legal decisions as these homicides progress through the criminal justice system. For example, it reported on the numerous appeals and court decisions related to the Oklahoma City bombing. We treated these cases as being included in the source even though they fell outside the time frame. In addition, Hewitt’s data collection ended 6/30/2004. Thus, we compare this source to the only the 1990–2004 homicides. Finally, the GTD does not include any incidents from 1993 thus we only compared this source for 1990–1992 and 1994–2008.

  12. It is likely that a specific source relied on several of the other sources examined here to identify incidents. That is, these sources rely on other open sources, and thus the number of sources might be indicative of a single source that captured an incident, that was then used by the other sources.

  13. The catchment/recatchment methodology is similar to the capture and recapture methodology. Importantly though, we use the catchment/recatchment methodology here more as an analogy than a specific method largely because the harvesting of open source data is not sampling in the traditional sense and terrorism is not very prevalent. For these reasons it is not necessary to do formal estimation as is done in more traditional catchment/recatchment applications.

  14. We have excluded the FBI and ATS sources from these analyses because they focus only on federal cases.

  15. All of the suspects for these incidents were male and white.

References

  • Aho JA (1990) The politics of righteousness: Idaho Christian patriotism. University of Washington Press, Seattle

    Google Scholar 

  • Bakker E (2006) Jihadi terrorists in Europe, their characteristics and the circumstances in which they joined the Jihad: an exploratory study. Clingendael Institute, The Hague

    Google Scholar 

  • Biderman A, Lynch JP (1991) Understanding crime incidence statistics: why the UCR diverges from the NCS. Springer, New York

    Google Scholar 

  • Chen H, Reid E, Sinai J, Silke A, Ganor B (2008) Terrorism Informatics: knowledge management and data mining for homeland security. Springer, New York

    Google Scholar 

  • Chermak SM (2002) Searching for a demon. The media construction of the militia movement. Northeastern University Press, Boston

    Google Scholar 

  • Chermak SM, Gruenewald J (2006) Domestic terrorism and the media. Justice Q 23(4):428–461

    Article  Google Scholar 

  • Chermak SM, Freilich JD, Simone J Jr (2010) Surveying American state police agencies about lone wolves, far-right criminality, and far-right and Islamic jihadist criminal collaboration. Stud Confl Terrorism 33:1019–1041

    Article  Google Scholar 

  • Coates J (1995) Armed and dangerous: the rise of the survivalist right. Hill and Wang, New York

    Google Scholar 

  • Dobratz BA, Shanks-Meile SL (1997) White power, white pride! The white separatist movement in the United States. Twayne Publishers, New York

    Google Scholar 

  • Damphousse K, Shields C (2007) The morning after, assessing the effect of major terrorism events of prosecution strategies and outcomes. J Contemp Crim Just 23(2):154–174

    Article  Google Scholar 

  • Dugan L, LaFree G, Piquero AR (2005) Testing a rational choice model of airline hijacking. Criminology 43(4):1031–1065

    Article  Google Scholar 

  • Dyer J (1997) Harvest of rage: why Oklahoma City is only the beginning. Westview, Boulder

    Google Scholar 

  • Endler W, Sandler T (2006) The political economy of terrorism. Cambridge University Press, NY

    Google Scholar 

  • Freilich JD (2003) American militias: state-level variations in militia activities. LFB Scholarly Publishing, New York

    Google Scholar 

  • Freilich JD, Chermak SM (2009a) United States extremist crime database (ECDB), 1990–2008: preliminary results. Department of Homeland Security University Network Research and Education Summit. Washington, DC

  • Freilich JD, Chermak SM (2009b) Preventing deadly encounters between law enforcement and American far-rightists. Crime Prev Stud 25:141–172

    Google Scholar 

  • Freilich JD, Chermak SM (2011) Introducing the United States extremist crime database (ECDB), 1990-2010. Paper under review

  • Freilich JD, Pridemore WA (2006) Mismeasuring militias: limitations of advocacy group data and of state-level studies of paramilitary groups. Justice Q 23(1):147–162

    Article  Google Scholar 

  • Freilich JD, Almanzar NAP, Rivera CJ (1999) How social movement organizations explicitly and implicitly promote deviant behavior: the case of the militia movement. Justice Q 16(3):655–683

    Article  Google Scholar 

  • Freilich JD, Chermak SM, Caspi D (2009a) Critical events in the life trajectories of domestic extremist white supremacist groups: a case study analysis of four violent organizations. Criminol Public Policy 8(3):497–530

    Article  Google Scholar 

  • Freilich JD, Chermak SM, Simone J Jr (2009b) Surveying American state police agencies about terrorism threats, terrorism sources, and terrorism definitions. Terrorism Polit Violence 21(3):450–475

    Article  Google Scholar 

  • Green DP, Strolovitch DZ, Wong JS, Bailey R (2001) Measuring gay populations and antigay hate crime. Soc Sci Q 82(2):281–297

    Article  Google Scholar 

  • Gruenewald J, Freilich JD, Chermak SM (2009) An overview of the domestic far-right and its criminal activities. In: Perry B, Blazak R (eds) Hate crimes: hate crime offenders. Praeger, Westport, pp 1–21

    Google Scholar 

  • Hamm M (1993) American skinheads: the criminology and control of hate crime. Praeger, Westport

    Google Scholar 

  • Hamm M (1997) Apocalypse in Oklahoma City: Waco and Ruby Ridge revenged. Northeastern University Press, Boston

    Google Scholar 

  • Hamm M (2002) In bad company: America’s terrorist underground. Northeastern University Press, Boston

    Google Scholar 

  • Hamm MS (2005) Crimes committed by terrorist groups: theory, research and prevention. Final Report to the National Institute of Justice, September 2005

  • Handler J (1990) Socioeconomic profile of an American terrorist. Terrorism 13:195–213

    Article  Google Scholar 

  • Hewitt C (2005) Political violence and terrorism in modern America: Christopher Hewitt: a chronology. Praeger Security International, New York City

    Google Scholar 

  • Hewitt C (2003) Understanding terrorism in America: from the Klan to Al Qaeda. Routledge, New York

    Google Scholar 

  • Horgan J (2008) Interviewing terrorists: a case for primary research. In: Chen H et al (eds) Terrorism informatics: knowledge management and data mining for homeland security. Springer, New York, pp 27–50

    Google Scholar 

  • Johnson S, Braithewaite A (2009) Spatio-temporal modeling of insurgency in Iraq. Crime Prev Stud 25:9–31

    Google Scholar 

  • LaFree G, Dugan L (2004) How does studying terrorism compare to studying crime? Sociology of crime. Law Deviance 5:53–74

    Article  Google Scholar 

  • LaFree G, Dugan L (2007) Introducing the global terrorism database. Terrorism Polit Violence 19:181–204

    Article  Google Scholar 

  • LaFree G, Dugan L (2009) Research on terrorism and countering terrorism. Crime Justice 38:413–477

    Google Scholar 

  • LaFree G, Morris N, Dugan L, Fahey S (2006) Identifying global terrorist hot spots. In: Victoroff J (ed) Tangled roots: social and psychological factors in the genesis of terrorism. IOS Press, Amsterdam, pp 98–114

    Google Scholar 

  • Langer E (2003) A hundred little Hitlers: the death of a black man, the trial of a white racist, and the rise of the neo-Nazi movement in America. Picador, New York

    Google Scholar 

  • Leiken RS, Brooke S (2006) The quantitative analysis of terrorism and immigration: an initial exploration. Terrorism Polit Violence 18:503–521

    Article  Google Scholar 

  • Lum C, Kennedy LW, Sherley AJ (2006) The effectiveness of counter-terrorism strategies: a campbell systematic review. [Online]: http://db.c2admin.org/doc-pdf/Lum_Terrorism_Review.pdf

  • Lynch JP, Addington LA (eds) (2007) Understanding crime statistics: revisting the divergence of the NCVS and the UCR. Cambridge University Press, Cambridge

    Google Scholar 

  • Maxwell C, Chermak S (2007) Creating a data archive to facilitate research on understanding and responding to terrorism. In Gunes OS, Al-Badayner DM (eds) Understanding terrorism: analysis of sociological and psychological aspects. NATO Science for Peace and Security. IOS Press, Amsterdam, pp 306–331

  • Merari A (1991) Academic research and government policy on terrorism. Terrorism Polit Violence 3(1):88–102

    Article  Google Scholar 

  • Naday A, Freilich JD, Mellow J (2008) The elusive data on supermax confinement. Prison J 88(1):69–93

    Article  Google Scholar 

  • Neiwert DavidA (1999) In god’s country: the patriot movement in the Pacific Northwest. Washington State University Press, Pullman

    Google Scholar 

  • Ross JI (1992) Research note: contemporary radical right-wing violence in Canada: a quantitative analysis. Terrorism Polit Violence 4(3):72–101

    Article  Google Scholar 

  • Ross JI (1993) Research on contemporary oppositional political terrorism in the United States: merits, drawbacks and suggestions for improvement. In: Tunnel KD (ed) Political crime in contemporary America: a critical approach. Garland Publishing, New York, pp 101–116

    Google Scholar 

  • Sageman M (2004) Understanding terror networks. University of Pennsylvania Press, Philadelphia

    Google Scholar 

  • Schmid A (2004) Terrorism on trial: the definitional problem. Case West Reserve Univ Sch Law 36:375–419

    Google Scholar 

  • Silber MA, Bhatt A (2006) Radicalization in the West: the Homegrown threat. New York City Police Department Intelligence Division. Available from the WWW at http://www.nypdshield.org/public/SiteFiles/documents/NYPD_Report-Radicalization_in_the_West.pdf

  • Silke A (2001) The devil you know: continuing problems with research on Terrorism. Terrorism Polit Violence 13(4):1–14

    Article  Google Scholar 

  • Silke A (2008) Research on terrorism: a review of the impact of 9/11 and the global QWar on terrorism. In: Chen H et al (eds) Terrorism informatics: knowledge management and data mining for homeland security. Springer, New York, pp 27–50

    Google Scholar 

  • Smith BL (1994) Terrorism in America: pipe bombs pipe dreams. State University of New York Press, Albany

    Google Scholar 

  • Smith BL, Damphousse KR (1998) Terrorism, politics and punishment: a test of structural-contextual theory and the ‘liberation hypothesis. Criminology 36(1):67–92

    Article  Google Scholar 

  • Surette R (1999) Media echoes: systematic effects of news coverage. Justice Q 16(3):601–631

    Article  Google Scholar 

  • Townsley M, Johnson SD, Ratcliffe JH (2008) Space time dynamics of insurgent activity in Iraq. Secur J 21:139–146

    Article  Google Scholar 

  • Turchie T, Puckett K (2007) Hunting the American terrorist: the FBI’s war on homegrown terror. History publishing Company, Palisades

    Google Scholar 

  • Victoroff J (2005) The mind of the terrorist: a review and critique of psychological approaches. J Conflict Resolut 49(1):3–42

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the editors, and participants in the research seminar on methodological issues in terrorism that was convened at John Jay College of Criminal Justice in October, 2010 for helpful insights. The authors would also like to thank the reviewers and Erin Miller for their thoughtful suggestions for revision. In addition, we would like to thank Nick Corsaro and Jeremy Wilson for helping us think through the data analysis plan.

Author information

Authors and Affiliations

  1. School of Criminal Justice, Michigan State University, 512 Baker Hall, East Lansing, MI, 48824, USA

    Steven M. Chermak

  2. John Jay College of Criminal Justice, New York, NY, USA

    Joshua D. Freilich & William S. Parkin

  3. Bureau of Justice Statistics, Washington, DC, USA

    James P. Lynch

Authors
  1. Steven M. Chermak
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joshua D. Freilich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. William S. Parkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. James P. Lynch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Steven M. Chermak.

Additional information

This research was supported by the United States Department of Homeland Security Science and Technology Directorate’s University Program’s Division as well as its Human Factors and Behavioral Science Division both directly and through the National Consortium for the Study of Terrorism and Responses to Terrorism (START) through both research and education grants. However, any opinions, findings, and conclusions or recommendations in this document are those of the authors and do not necessarily reflect views of the U.S. Department of Homeland Security or START.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Chermak, S.M., Freilich, J.D., Parkin, W.S. et al. American Terrorism and Extremist Crime Data Sources and Selectivity Bias: An Investigation Focusing on Homicide Events Committed by Far-Right Extremists. J Quant Criminol 28, 191–218 (2012). https://doi.org/10.1007/s10940-011-9156-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10940-011-9156-4

Keywords

  • Terrorism
  • Far-right violence
  • Selectivity bias