Advertisement

Journal of Behavioral Medicine

, Volume 42, Issue 4, pp 811–829 | Cite as

State of the science: a scoping review of primary prevention of firearm injuries among children and adolescents

  • Quyen M. NgoEmail author
  • Eric Sigel
  • Allante Moon
  • Sara F. Stein
  • Lynn S. Massey
  • Frederick Rivara
  • Cheryl King
  • Mark Ilgen
  • Rebecca Cunningham
  • Maureen A. Walton
  • For the FACTS Consortium
Article

Abstract

Intentional and unintentional firearm injury is the second leading cause of death for youth, underscoring the need for effective primary prevention approaches that focus on increasing safe storage by caregivers and decreasing handling/carriage among youth. This article describes the state of the science for prevention of firearm injuries among children and adolescents. We applied PRISMA guidelines to present results from a scoping review using PubMed, Scopus, CINAHL, and CJ Abstracts for original research articles published between January 1, 1985 and March 1, 2018 in the U.S. focusing on primary screening or interventions for primary prevention of pediatric firearm injuries. In total, 46 articles met inclusion criteria: safe storage (23), screening (2), firearm handling/carriage/use (21). Across school, healthcare, and community settings, few evidenced-based programs exist, and data on firearm safety technologies are lacking. Programs have generally not employed rigorous designs, and/or assessed behavioral (e.g., carriage) or injury-related firearm outcomes. Evidenced-based prevention programs are needed to mitigate firearm morbidity and mortality among youth.

Keywords

Universal prevention Primary prevention Firearm injury Safe storage Firearm carriage Children Adolescents 

Background

Firearm-related injuries are the second leading cause of deaths among children and adolescents in the U.S. (Cunningham et al., 2018), with the majority of fatalities involving a handgun (Fowler et al., 2017). Among adolescents, homicide and suicide are the leading causes of firearm injuries and death (Fowler et al., 2017; Reich et al., 2002), whereas unintentional firearm injuries are more common among younger children. Younger children are also more likely than adolescents to be killed in incidents related to intimate partner violence (IPV; Fowler et al., 2017). Overall, firearm-related homicide rates among Black children are greater than 10 times the rates among White children or among Asian children (Fowler et al., 2017). In contrast, firearm-related suicide fatalities among White children and among First Nations children are 4–5 times higher than among Black, Asian, or Hispanic children (Fowler et al., 2017). This firearm-related morbidity among youth underscores the urgent need for evidenced-based prevention approaches to decrease firearm handling (e.g. playing with a found gun), carriage (e.g., bringing a gun to school, carrying a gun while out with friends), or use among children and adolescents; one primary way of doing this is by increasing safe storage of firearms by caregivers (e.g., use of gun safe or gun lock, bullets stored separately).

Recent reviews underscore high regional variability in estimates of the presence of guns in U.S. homes (Parikh et al., 2017), with less than 50% being stored safely (e.g., locked and unloaded; (Rowhani-Rahbar et al., 2016; Scott et al., 2018). The need to increase safe storage and prevent access by youth is further underscored for families of youth at risk for suicide, given that one-third of parents of children with a history of self-harm stored guns unlocked and unloaded (Scott et al., 2018). Moreover, one-third of adolescents report access to a handgun (Loh et al., 2010), highlighting the need to encourage safe storage and to decrease handling, carriage and use among adolescents as well as children.

The first step in establishing recommendations for primary prevention strategies (defined as interventions delivered to broad groups of individuals prior to injury) to reduce firearm access among youth (e.g., increase safe storage of all firearms, decrease handling/carriage among youth) requires understanding the current state of the science across foci of interventions (individual, family, and community level; firearm safety technology) and settings (e.g., homes, schools, healthcare, community). Prior reviews of the firearm literature, focusing on epidemiology and risk/protective factors (Crossen et al., 2015; Dowd, 1999; Fein & Mollen, 1999; Komro, 1999; Parikh et al., 2017; Williamson et al., 2014), recommend additional research to identify effective prevention approaches to reduce youth access to firearms (Parikh et al., 2017). Further, these reviews conclude that screening for firearms in the home, and counseling to increase safe storage, inconsistently occurs in healthcare settings (Roszko et al., 2016). For example, data from the American Academy of Pediatrics shows that only 21% of pediatricians “Always” try to identify families who have firearms in the homes (American Academy of Pediatrics, 2016). Nonetheless, counseling is effective in improving safe storage when combined with providing a firearm safe storage device (e.g., trigger locks which prevents the gun from being fired; (Rowhani-Rahbar et al., 2016). Of note, previous literature reviews have neither examined firearm handling, carriage, use, nor safe storage intervention efforts across multiple community settings (e.g., schools, healthcare, community centers and/or community-level).

This review describes the extant research literature pertaining to primary prevention strategies to reduce firearm access by children and adolescents under 18 years of age, exemplifying how these prevention efforts can be done without conflict with the Second Amendment and the legal rights of adults to own firearms. It is important to mention that there are no restrictions on physicians in asking and counseling about firearm storage (Rivara & Fan, 2017). Hereafter, the term children will refer to youth 0–9 years of age (e.g., preschool and elementary) and the term adolescents will refer to those between 10 and 17 years of age (e.g., middle and high school). Specifically, we conducted a scoping review to identify gaps in this literature for the purpose of informing future prevention and intervention work in child and adolescent firearm injuries. This review focuses on screening tools and interventions for: (1) safe storage and (2) firearm handling, carriage, and use.

Methods

We applied the 5-step scoping review method (Arksey & O’Malley, 2005) and utilized the PRISMA framework for reporting standards (Moher et al., 2009), which includes: (1) identifying the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; and (5) collating, summarizing and reporting results. We used the findings from the scoping review to establish recommendations for research and intervention to address current gaps in the science.

Literature search strategies

We conducted a scoping review using PubMed, Scopus, CINAHL, CJ Abstracts for English-language original research articles published between January, 1985 and March, 2018 on primary prevention of youth firearm related outcomes in the U.S. The initial comprehensive search was created in PubMed and subsequently duplicated in other databases. Searches combined keywords including firearm, firearm carriage, ownership, unintentional injury, intentional injury, self-inflicted injuries, youth, child, and adolescent. (See “Appendix” for example list of PubMed search terms). Citation tracking was conducted using the EndNote citation management software for articles that met inclusion criteria.

Eligibility criteria

Articles were eligible if the sample included youth (ages 0–17), or if the sample was of comprised parents/caregivers of youth. Articles were not included if they were a literature review of empirical articles, or non-empirical recommendations, such as opinion/perspective pieces or professional society statements. Also, we excluded studies for these reasons: international and non-English articles; were not focused on 0–17 year olds; articles without specific focus on firearms (e.g., general violence interventions without mention of firearm criteria in the title or abstract, intervention components or outcomes); could not obtain the abstract/article; and non-primary or non-universal prevention intervention articles (e.g., focused on epidemiology, risk/protective factors, policy, secondary prevention, case studies, or treatment/medical management of gunshot wounds).

Data abstraction and analysis

Title and abstract review was completed using the web-based program Rayyan (Ouzzani et al., 2016). To establish the protocol for determining eligibility, two content experts reviewed 100 title abstracts independently and blindly. Once the expert reviewers established eligibility criteria and achieved full consensus, trained research assistants applied the criteria for article identification and inclusion. During the title and abstract screening, the independent reviewers erred on inclusion. Data was extracted using a form that captured design, sample and setting, intervention description, outcomes, and limitations.

Results

Article identification and selection

The initial search strategy identified a total of 1780 articles (Fig. 1); removing duplicates (498) resulted in 1282 articles. The title and abstract of the 1282 articles were reviewed for inclusion, with 90 articles meeting eligibility criteria. Experts also identified 12 articles for potential inclusion. The full text of the 102 articles (search identified + expert identified) was reviewed, with 46 articles continuing to meet inclusion criteria included in the scoping review, which focused on: safe storage (23), screening (2), firearm carriage/handling/use (21).
Fig. 1

PRISMA diagram: Primary Intervention Article Identification (1985–2018)

Firearm safe storage

In total, 23 articles focused on primary prevention efforts for safe storage (e.g., safes and/or trigger locks), which include individual-level interventions with caregivers in health care settings as well as community-level interventions (see Table 1).
Table 1

Firearm safe storage and means restrictions

First author, date

Title

Design

Sample

Intervention

Outcome

Limitations

Training

Dingeldein et al. (2012)

Evaluation of a firearm injury prevention web-based curriculum

Two group RCT

n = 92 pediatric medical student residents in clinics

3 online training scenarios: 1 related youth firearm injury prevention

Intervention group showed significant improvements in attitudes and self-efficacy as compared to control for physician role in firearm safety at immediate posttest

Opt-in for training may indicate possibility for bias

No physician behaviors measured (patient counseling)

Health care settings

Kruesi et al. (1999)

Suicide and violence prevention: Parent education in the emergency department

Two group Pretest/Posttest, non-randomized

Parents of n = 103 youth (6–19 years mean age 13.7) in emergency department for mental health

Means restriction staff training: firearms, medications, alcohol

Among 15 parents with firearms at ED visit, those trained were significantly more likely to restrict means than those non-trained at 2 month follow-up

Small Sample size

Non-randomized design

Oatis et al. (1999)

Pediatric practice based evaluation of the Steps to Prevent Firearm Injury program

One group Pretest/Posttest

n = 1617 parents in pediatrician office; youth 0–17 years

Educational program

No statistically significant reductions in firearm ownership, storing guns outside of a locked container, or keeping a gun loaded over a 9–11 month follow up

Low posttest retention rate (23.6%)

Reliance on self-report

No control group

Grossman et al. (2000)

Firearm safety counseling in primary care pediatrics: A randomized, controlled trial

Two group RCT

n = 1295 parents at well-child visits (ages 2 months to 18 years)

Single session gun safety counseling

No statistically significant changes in the rate of acquisition of new guns, removal of guns, or proportion of purchasing trigger locks between the treatment and control groups at 3 month follow up

Impossible to blind treatment providers to group assignment

Intervention was not effective

Stevens et al. (2002)

A pediatric, practice-based, randomized trial of drinking and smoking prevention and bicycle helmet, gun, and seatbelt safety promotion

Two group RCT, 15 clinics; compara-tive efficacy

n = 2183 parent/5th and 6th grade youth dyads

Physician counseling, booster call and letter: alcohol/tobacco or safety (safe storage)

At 3-year follow-up, no differences by group in alcohol, tobacco, firearm storage, seatbelt; bike helmet use significantly increased

No control group, so potentially underpowered for comparative efficacy Lack of evidence of effectiveness of program

Albright and Burge (2003)

Improving firearm storage habits: Impact of brief office counseling by family physicians

Two group Pretest/Posttest

Non-randomized

n = 156 adult gun owners in family practice (41% children in home)

Physician counseling, with or without storage materials

Intervention participants in both groups were 3 times more likely to make safe changes in gun storage compared to participants in the control at 2–3 month follow up

Small sample size

Non-random assignment

Reliance on self-report

Carbone et al. (2005)

Effectiveness of gun-safety counseling and a gun lock giveaway in a Hispanic community

Two groups Pretest/Posttest, Non-randomized

n = 151 gun owning parents in a pediatric clinic (children 0–17 years, mean age 6–7)

Physician counseling, educational brochure, and gun lock

No significant differences between groups on number of guns removed from household Intervention group had a significantly greater frequency of locked storage of firearm and improved type of storage than controls at 1 month follow up

Baseline differences between intervention and control in gun storage

Non-randomized

Reliance on self-report

Barkin et al. (2008)

Is office-based counseling about media use, timeouts, and firearm storage effective?

Two group RCT

n = 137 pediatric practices, mostly moms with child (2–11 years)

Physician counseling about media, discipline, firearm storage, free cable locks

Approximately half of gun owners reported use of locks on all firearms Significant increases in use of cable locks by intervention as compared to control group over 6 month follow up

Reliance on self-report

Runyan et al. (2016)

Lethal Means Counseling for Parents of Youth Seeking Emergency Care for Suicidality

One group Pretest/Posttest

n = 114 parents with suicidal child (12–17 years) in psychiatric emergency services

Trained discharge counselors provided 5 min counseling on safe storage of medications and firearms, with free lock boxes

33 of 33 parents with guns at home indicated all were locked 2–3 weeks post-discharge

Reliance on self-report

No control group

Lock boxes only offered for medication storage, not gun storage

Community settings: trigger locks and safes, etc.

Coyne-Beasley et al. (2001)

Love our kids, lock your guns: a community-based firearm safety counseling and gun lock distribution program

One group Pretest/Posttest

n = 112 adults (63% had children)

Single session counseling, with up to 4 gun locks provided

Significant increase in storage of guns in locked compartment, use of gun locks, and asking friends about ownership and storage over 6 month follow up

Small sample size

No control group

Reliance on self-report

Roberto et al. (2002)

Promoting gun trigger-lock use: Insights and implications from a radio-based health communication intervention

Two group, Pretest/Posttest

Non-randomized

n = 1500 randomly selected from 2 counties (63% had children under 18 years)

Radio public service announcements with toll-free number to obtain free trigger-lock

17% of gun-owning households exposed to PSA 799 individuals called to get a trigger-lock At 6-week follow-up, 95% reported trying the lock and 87% reported that the lock was on one of their guns Significant increase in gun safety knowledge

No longitudinal comparison of treatment and control groups

Horn et al. (2003)

Community based program to improve firearm storage practices in rural Alaska

One group Pretest/Posttest

n = 40 randomly selected households with 2 or more guns in home (average 2.4 children under 16 years)

Distribution of one gun safe and 1 trigger lock and provision of gun storage education

AT 3-months, 86% of the provided safes were found to be locked with guns inside and 78% of participants reported that all guns were now secure in the household Only 30% of the trigger locks were reported to be in use at 3 month follow up.

Small sample size No comparison/control group

Reliance on self-report

Wafer and Carruth (2003)

Locks for Life: A gun lock distribution community health intervention program

One group Pretest/Posttest

n = 135 adults attending a local festival (majority had children in household)

Provision of a trigger locks (n = 216) and gun safety literature

216 gun locks distributed

Among, 19 participants, 14 reported using trigger locks and 5 with intent to use trigger locks at 3 month follow up

86% attrition

No comparison/control group

Sidman et al. (2005)

Evaluation of a community-based handgun safe-storage campaign

Two groups Pretest/Posttest, Non-randomized

n = 302 households with child (< 18 years) and a firearm

TV and radio safe storage campaign, billboards, and coupons for lock box

Increases in intervention and control counties in locking handguns between 1996 and 2001

Significant intervention effects found for households with at least 3 handguns relative to controls

Rate of lock box discount coupon was not confirmed

Non-random assignment

Reliance on self-report

Control counties had other gun safety initiatives

Grossman et al. (2012)

Improving firearm storage in Alaska Native villages: A randomized trial of household gun cabinets

Two group RCT

n = 255 households with at least 1 gun (~ 70% reported children in household)

Free metal gun cabinet installed, safety handout

Intervention group had significantly lower percentage of unlocked guns and ammunition in home compared to controls at 12-months follow-up

Gun cabinet used were designed for rifles and shotguns might not be suitable for urban or non-Native communities

Wargo et al. (2013)

Community gun safety in Central Pennsylvania

One group Pretest/Posttest

n = 58 adults attending a Head Start meeting (age 6 years or younger)

Gun safety education, gun lock instruction, and a free gun lock

Significant reduction in keeping guns in closet or unattended, and more using a gunlock at 12-month follow-up. No effects of loaded guns, storage of ammunition or reapply lock after removal

26% retention rate

Small sample size

No control group

Reliance on self-report

Simonetti et al. (2017)

Evaluation of a community-based safe firearm and ammunition storage intervention

One group Pretest/Posttest

N = 208 adults attending 2 community-based firearm safety events (53.4% reported children under 18 years)

Brief safety messages for means restriction to prevent suicide, free trigger lock or lock box with practice

Significant increase in firearms stored locked and unloaded. Non-significant increase in ammunition locked up. Significant increase in reports of using all 3 firearm safety procedures at 4–6 weeks

66% retention rate. No control group

Reliance on self-report

Lock box used only fit 1 handgun and some ammunition, limiting use for multiple guns or rifles/shot guns

Community settings: buyback programs

Callahan et al. (1994)

Money for guns: Evaluation of the Seattle Gun Buy-Back Program

One group Pretest/Posttest

Police records and trauma data; self-report surveys (n varied; 11–91 years old; 2.4% under age 21)

Gun buy-back program

1772 firearms relinquished, most commonly because they were no longer wanted. No statistically significant changes in firearm-related robberies, assaults, homicides, deaths, and admissions to the trauma center 6 months later

Program unable to detect changes in firearm-related outcomes due to variations in rates over time

No comparison/control community

Celeste Kallenborn et al. (2004)

Cease Fire Tampa Bay: A three-tiered approach to Firearm injury

One group, Post-test

n = 6 counties

Firearm curriculum 3rd and 6th graders/community education program parents of students

Gun buy-back program, firearm safety education for children and adults, give up to 3 gun locks

6981 firearms relinquished over ~ 3 years

No outcomes reported beyond the collection of guns

No comparison/control community

McGuire et al. (2011)

Goods for guns-the use of a gun buyback as an injury prevention/community education tool

One group Posttest only,

Non-randomized

n = 534 adults that learned of the buyback program (37% had children 14 years old or younger)

Buyback program, counseling and gun locks, gun destroying ceremony

1861 guns collected over 8 years; 75 gun locks distributed. Two-thirds said safety was the reason for relinquishing gun, citing children in the home. Downward trend in firearm injuries and mortality in buyback county compared to other counties in state

Outcomes of buyback program not assessed

Reliance on self-report

Study setting of police headquarters may have biased responses

Violano et al. (2014)

Gun buyback programs: a venue to eliminate unwanted guns in the community

One group Pretest

n = 301 adults

(27% reported having children in the home)

Gun buyback program

Relinquished firearms were less likely to have been kept locked; safety reported as the motivation for return. Those with additional firearms in the house were less likely to report having trouble storing the gun. Those knowing someone injured by a firearm were more likely to keep guns locked

Program outcomes not evaluated

No control/comparison group

Reliance on self-report

Green et al. (2017)

Are “goods for guns” good for the community? An update of a community gun buyback program

One group Pretest

n = 109 adults (21% had children who could access guns)

Gun buyback program at police departments

339 firearms relinquished, with safety as the most common reason;61% reported other guns at home; 21% reported that children could access them; 14% reported a household member with mental illness, suicide, or domestic violence

Survey response rate < 60%

No control/comparison group

Reliance on self-report

Kasper et al. (2017)

And the survey said… evaluating rationale for participation in gun buybacks as a tool to encourage higher yields

One group Pretest

n = 273 adults (48% had children living or frequenting their home)

Gun buyback program at police departments

943 firearms relinquished, with half citing safety reasons as motivation for return. Of households with guns remaining, 27% were potentially accessible to children. 42% of participants took gunlocks home

No comparison/control community

No behavioral outcome data reported, including firearm outcomes

Healthcare settings

In pediatric or health care settings, eight studies examined physician-provided educational messages to parents with children or adolescents in the home (See Table 1) around safe firearm storage (in a gun lock box or with trigger locks which prevent a gun from being fired), either alone or with provision of free locking devices. Generally, studies focused on safe storage as a way of: (1) reducing risk for accidental discharge by children or (2) restricting access to lethal means of self-harm for older children at risk for suicide.

Reducing risks for unintentional discharge of firearms When examining education of families without provision of safe storage devices, three studies (Grossman et al., 2000; Oatis et al., 1999; Stevens et al., 2002) found no change in firearm storage; however, one other study reported that participants were three times more likely to implement safe storage practices (Albright & Burge, 2003). When examining the combination of education and provision of a safe storage device, two randomized controlled trials (RCTs) found positive effects (Barkin et al., 2008; Carbone et al., 2005), particularly among younger children (< 12 years old). In the largest study that has been conducted, Barkin (Barkin et al., 2008) randomized 137 practices to provider-delivered motivational-interviewing based patient education on a range of safety behaviors, including information on safe firearm storage, car seats, and bike helmets, and provided free gun cable locks to families (with children 2–11 years old) that reported having firearms in the home (~ 24%); over a six-month follow-up, this study found a significant increase in safe firearm storage (Odds Ratio: 2.0; 22% difference between groups).

Restricting access to lethal means for self-harm Two studies examined safe storage counseling (talking with parents/caregivers about limiting youth access to potential methods of suicide, such as firearms and medications) in emergency departments (Kruesi et al., 1999; Runyan et al., 2016). One study showed that parents of children receiving safe storage counseling in the emergency department were 4 times more likely to limit access to firearms and medications compared to those parents who did not receive safe storage counseling (Kruesi et al., 1999). In a second study, psychiatric emergency clinicians (i.e., psychiatrists, psychologists, and social workers) participated in online training in safe storage counseling, after which 89% of youth presenting for suicidal ideation or attempt received safe storage counseling with provision of a free lock box, and at follow-up all caregivers reported locking firearms (Runyan et al., 2016). More research is needed in health care settings to optimize delivery of combined counseling, and provision of safety devices, and to measure impact on firearm injury-related outcomes. Additionally, more research is needed on measurement of safe storage as all but one of the studies (Grossman et al., 2000) above used self-report to measure safe storage.

Training of healthcare providers Finally, one study, separate from those in the two sections above, examined training resident physicians to deliver educational messages to patients during office visits, which is key to delivering safe storage messages to patients in healthcare settings (Dingeldein et al., 2012). The study found increases in reported self-efficacy in discussing firearm safety with patients after a web-based curriculum; however, effects on actual delivery of counseling, or on patient outcomes, were not evaluated (Dingeldein et al., 2012). Additional research is needed to determine whether and what aspects of these trainings may be effective in increasing delivery of counseling to patients and whether the counseling delivered has an impact on patient behaviors and outcomes such as safe storage of firearms or reductions in firearm injuries.

Community settings

Community-based interventions for safe firearm storage have generally included firearm lock box give-away programs, firearm buy-back programs, and/or community education programs. Eight studies evaluated locking device giveaways (Coyne-Beasley et al., 2001; Grossman et al., 2012; Horn et al., 2003; Roberto et al., 2002; Sidman et al., 2005; Simonetti et al., 2018; Wafer & Carruth, 2003; Wargo et al., 2013), with findings generally indicating that distribution of safe storage devices led to an increase in self-reports of safe firearm storage, regardless of where they were dispersed; however, none of the studies evaluated the effects on youth firearm-related injury.

Safe storage education programs Two community education initiatives report mixed results. Assessment of a radio public service announcement (PSA) about safe storage, with toll-free number to obtain trigger locks, found that ~ 20% of the population heard the PSA but only 3% of gun-owning households in the community called to obtain trigger locks. However, 87% of those who requested gun locks used them (Roberto et al., 2002). A multifaceted community safe firearm storage educational campaign (with lock box coupons) found a significant difference between individuals in intervention and control counties (i.e., individuals in intervention counties were more likely to lock guns) in a subset of firearm owners (with 3 or more firearms; intervention: 233% increase; control: 39% decrease) (Sidman et al., 2005).

Firearm buyback programs Finally, six studies examined firearm buyback programs (Callahan et al., 1994; Green et al., 2017; Celeste Kallenborn et al., 2004; Kasper et al., 2017; McGuire et al., 2011; Violano et al., 2014), consistently finding that reasons for participation were for safety, particularly in the context of having children at home, with one study finding younger participants more often cited needing money as the reason (Callahan et al., 1994). Although most studies do not assess firearm-related outcomes from these buyback programs, one study showed no effects on rates of assault injuries or death (Callahan et al., 1994) whereas another reported a non-significant decreasing trend on firearm injuries in the intervention versus control communities (McGuire et al., 2011).

Firearm handling, carriage, and use

In total, 2 studies focused on screening tools and 21 articles described primary prevention interventions for firearm handling, carriage, or use (see Table 2) conducted in healthcare, school, and community settings, with focus typically on children or adolescents.
Table 2

Prevention of firearm handling, carriage, and use

First author, date

Title

Design

Sample

Intervention

Outcomes

Limitations

Screening

Hayes and Sege (2003)

FiGHTS: a preliminary screening tool for adolescent firearms-carrying

One group, Pretest

n = 15000 high school students

None: Screening measure development

Analyses of YRBS data from 1999 to identify items (FiGHTS) related to firearm carriage, with five items identified: Fighting, Gender, Hurt while fighting, Threatened, and Smoker

Longitudinal predictive validity unknown

Goldstick et al. (2017)

Development of the SaFETy Score: A clinical screening tool for predicting future firearm violence

Two group Pretest/posttest, non-randomized

n = 599 drug using youth (14–24 years) in emergency department

None: Screening measure development

4 baseline questions (safety score) predicted firearm violence over 2 years: Serious fighting, community environment, friend weapon carriage (including gun), and firearm threats. 24-month follow up

Need for validation in future studies, including those without drug use entry criteria

Reliance on self-report

Child-focused programs (ages 0–9 years)

Hardy et al. (1996)

A firearm safety program for children: They just can’t say no

Two group RCT

n = 48 daycare children (4–6 years)

Individual education not to play with guns

Children who played with guns were more likely to interact with guns at home and show more aggression in their play. The intervention and control did not differ in gun play at 2-week follow up

Intervention was not effective. Small sample size

Hardy (2002)

Teaching firearm safety to children: Failure of a program

Two group RCT

n = 70 daycare children (4–7 years)

4 Lessons: Straight Talk about Risks

37 children (53%) played with the guns in the structured setting. No difference between groups in gun play at 1 week posttest

Lack of evidence of effectiveness of program. Possible contamination between groups

Liller et al. (2003)

Evaluation of the “Respect not Risk” firearm safety lesson for 3rd-graders

One group Pretest/Posttest

n = 433 3rd graders

1 lesson on firearm safety with letter for families about free gun locks

Significant increases in knowledge at posttest, and sharing with family at 1-week posttest

No control group

No behavioral outcome data reported, including firearm outcomes

Himle et al. (2004a)

An evaluation of two procedures for training skills to prevent gun play in children

Three group

RCT Posttest only

n = 31 preschool children (4–5 years)

Eddie Eagle Gun Safe Program and behavioral skills training (BST)

The Eddie Eagle group and the BST group scored significantly better on self-report than controls; only the BST group scored better on role-play assessments and no differences in groups based on in situ 1-week assessment

Small sample size

Less than half of consent forms given out were returned (selection bias)

Himle, Miltenberger, et al. (2004b)

Teaching safety skills to children to prevent gun play

One group Pretest/Posttest

n = 8 preschool children (4–5 years)

BST

Education alone was not effective, as in situ training was necessary for all children to require skills, which were maintained in all but 1 child up to 8 weeks

Small sample size

No control group

Miltenberger et al. (2005)

Teaching safety skills to children to prevent gun play: An evaluation of in situ training

One group post-test

N = 10 preschool children (4–5 years)

BST

All 10 children executed the safety skills following BST with in situ training in school and home assessments at 3-month follow-up

Study did not directly compare BST with and without in situ training

Jostad et al. (2008)

Peer tutoring to prevent firearm play: Acquisition, generalization

One group Pretest/Posttest

n = 6 preschool children (4–5 years)

BST delivered by 6–7 year old peers

Multiple BST sessions were needed for skill acquisition for half the children, the remaining required in situ training, evidence for maintenance over 1-year was mixed

Small sample size

No control group

Gatheridge et al. (2004)

Comparison of two programs to teach firearm injury prevention skills to 6 and 7-year-old children

Three group RCT

n = 45 children in after school programs (6–7 years)

4, 15-min sessions of: Eddie eagle, or BST

Although both interventions increased knowledge, only the BST intervention scored better than controls based on role play and in situ assessments (1 week later)

No long-term follow-ups or assessment outside of the research environment (e.g., home). Unclear if data collectors were blinded to condition assignment

Howard (2005)

Evaluation of age-appropriate firearm safety interventions

Four group RCT

n = 57 K and 1st graders, from 8 schools

4 groups: discussion alone, discussion and psychomotor, discussion, psychomotor and role plays, control

Knowledge about firearm safety increased in all intervention groups, but not in the controls. The discussion/psychomotor group had the greatest increase at 1-month follow-up

Small sample size

No behavioral outcome data reported, including firearm outcomes

Kelso et al. (2007)

Teaching skills to 2nd and 3rd grade children to prevent gun play

Three group

RCT,

Posttest only

n = 30 children 8–9 years

45 min Eddie eagle or BST

Both interventions were efficacious based on self-report; however, there were no differences between either intervention and control in role play and in situ assessments on gun play safety at immediate posttest

Small sample size

Lack of evidence of effectiveness of program on gun play behaviors

Gross et al. (2007)

Preliminary evaluation of a parent training program to prevent gun play

One group Pretest/Posttest

n = 4, children 4–7 years

Parent BST

BST was effective for 3 of 4 participants and was implemented with fidelity by 3 of the 4 parents during training

Small sample size

No control group

Kenny and Wurtele (2016)

Teaching preschoolers safety rules: A pilot study of injury prevention

Two group Pretest/Posttest non-randomized

n = 346 preschool children (3–5 years), primarily Latino

10 one-hour sessions BST

Intervention group showed significant improvement self-report than control at 2–3 month (average 75 days) follow-up

Differential sample size between groups (e.g., 45 in control). Non-randomized design. No behavioral outcomes assessed

Adolescent-focused programs (ages 1017 years)

Davidson et al. (1994)

The impact of the Safe Kids/Healthy Neighborhoods Injury Prevention Program in Harlem, 1988 through 1991

Two group Pretest/Posttest

Youth in Harlem community (under 17 years)

Multi-component: park clean-up, leisure activities, injury prevention including guns

After 3 years, the intervention community showed decreases in assaults, but no differences in gun injuries, relative to the comparison. Significant findings were for motor vehicle injuries; falls were unchanged

Intervention community had greater baseline injury rates than the comparison. Non-randomized design. The many interventions preclude determination of efficacy of particular components. Unclear intervention focus on firearms

Thomas (et al. 1998)

Breaking the cycle of violence among youth living in metropolitan Atlanta: A case history of kids alive and loved

One group Pretest/Posttest

n = 58 African-Americans (10–20 years; mean 15.8)

Support group for youth exposed to violence (lost a loved one from gun violence)

Increases in anecdotal reports of youth managing anger, conflict resolution, and increasing leadership; no systematic follow-up

No comparison/control group

No behavioral outcome data reported, including firearm outcomes

Tucker et al. (1999)

Violence prevention: Reaching adolescents with the message

One group Pretest/Posttest

n = 99 5th–8th graders

Videos, anger management, and paralyzed gunshot victim testimonial

Program acceptability ratings reported at 1-month follow up

No comparison/control group

No behavioral outcome data reported, including firearm outcomes

Durant et al. (2001)

Evaluation of a peaceful conflict resolution and violence prevention curriculum for sixth-grade students

Two group Pretest/Posttest non-randomized 4 schools

n = 704 middle school students (11–14 years)

13 session violence prevention curriculum

Relative to control, at posttest intervention groups showed significant decrease in carrying a concealed gun, intention to use violence, and use of violence at 2 week posttest

Baseline violence differences (more in intervention schools) could have confounded findings. Data not analyzed nested by school

Longer term outcomes unknown

Emde (2002)

Cops and Docs: A gun violence injury prevention program

One group Post-test

n = 5000 middle school students (13 years)

Cops and Docs program: 50-minute slides and discussion about gun violence

The program has been disseminated to school districts in WA and in 14 states; no follow-up

No control group

No outcome data reported despite post-test

Lack of evidence of effectiveness of program

Chang et al. (2005)

A multidisciplinary youth violence-prevention initiative: Impact on attitudes

One group Pretest/Posttest

n = 97 adolescents from community (mean 12.6 years)

Hospital tour showing video/slides of gun violence

Statistically significant reduction in beliefs supporting aggression and trend toward reduced likelihood of violence a month later (mean 25.8 days).

49% completed posttest

No control group

No behavioral outcome data reported, including firearm outcomes

Regan (2009)

Implementation and evaluation of a youth violence prevention program for adolescents

One group Pretest/Posttest

n = 20 9th–10th graders

4 sessions on gun and gang violence, field trip to trauma center

Significant improvement in perceived knowledge about the history of gang violence and its impact on American culture and methods to decrease gun violence immediately after program

Small sample size

No control group

No behavioral outcome data reported, including firearm outcomes

Kunkel et al. (2010)

A hospital-based violence prevention tour: A collaborative approach to empower youth

One group Pretest/Posttest

n = 185 adolescents from community (10–17 years)

Tour of hospital, discussion about gun violence, and PowerPoint slides

84% reported an increase in awareness of violent injuries, which was significantly higher among girls than boys, immediately after program

Small sample size

No control group

No behavioral outcome data reported, including firearm outcomes

Goldberg et al. (2010)

An analysis of inner-city students’ attitudes towards violence before and after participation in the ‘‘Cradle to Grave’’ (C2G) program

One group Pretest/Posttest

n = 88 8th and 9th graders

2-h program in trauma center with video about youth death from gunshot wound

Significant reduction in violent beliefs among those from a public school but not those from a charter school at 4-week follow up

No control group

No behavioral outcome data reported, including firearm outcomes

Screening tools

Two studies specifically examined firearm-related screening tools for identifying those at greatest risk for firearm-related injuries. Goldstick (Goldstick et al., 2017) recruited drug-using youth presenting to the emergency department and found that the 4-item SaFETy score predicted firearm violence over 2 years; however, this study oversampled those presenting for a fighting related injury (58%), all participants reported drug use (predominantly marijuana), and the sample included emerging adults (ages 14–24). Thus, replication is required to determine generalizability to more representative samples of youth. The five-item FIGHTS, was developed to screen youth for firearm carriage using cross-sectional YRBS data (Hayes & Sege, 2003), with 30% of adolescents screening positive (representing 82% of firearm carriers); however, longitudinal data is needed to verify the predictive validity of this tool.

Child-focused programs (ages 0–9 years)

Twelve studies have evaluated teacher-led programs to prevent children from handling firearms, with mixed overall findings. For example, two studies, using education only, found no effects (Hardy, 2002; Hardy et al., 1996), but one education program that included a letter to parents about how to obtain free gun locks, found significant improvements in knowledge among 3rd graders, who also reported sharing information with their families (Liller et al., 2003). Several studies suggest that behavioral skills rehearsal (e.g., Behavioral Skills Training) is necessary for skill acquisition, and is more effective than interactive education only (e.g., Eddie Eagle), on outcomes assessed in hypothetical situations (Gatheridge et al., 2004; Himle et al., 2004a; Howard, 2005; Jostad & Miltenberger, 2004; Miltenberger et al., 2005). Also, feasibility studies suggested that parents could be trained to teach their children firearm safety skills (Gross et al., 2007) and older peers (e.g., 6–7 year olds) can teach 4–5 year olds (Gatheridge et al., 2004; Himle et al., 2004b; Howard, 2005; Jostad & Miltenberger, 2004). However, this literature is limited by small samples sizes and either lack of assessment of behavioral outcomes (Kenny & Wurtele, 2016) or finding null effects for in situ “real life” situations (Himle et al., 2004a; Kelso et al., 2007). Thus, despite evidence that age appropriate firearm safety programs for children can change knowledge of firearm safety, effects on firearm handling (and injury) are unclear.

Adolescent-focused programs (ages 10–17 years)

School settings Among adolescents, three school-based programs have specifically focused on preventing weapon carriage/use, primarily among middle school students. For example, one study conducted a RCT to evaluate a 13-week school based violence prevention curriculum in middle schools (2 control, 2 intervention), finding that at 2-weeks post-intervention, students in the intervention arm carried concealed weapons less frequently than at pretest (DuRant et al., 2001). Other school-based programs show significant effects on knowledge (Regan, 2009) and/or do not report outcomes (Emde, 2002). Thus, additional research on school-based curricula is necessary to examine the impact on prevention of firearm carriage.

Healthcare settings Other adolescent-focused programs emphasizing medical consequences of gunshot wounds (e.g., videos, testimonials, with or without trauma center tour) show positive effects only on knowledge and beliefs regarding firearm consequences and safety (Chang et al., 2005; Goldberg et al., 2010; Kunkel et al., 2010); however, these studies have not had control groups nor measured behavioral outcomes (Tucker et al., 1999). Another program (Kids Alive and Loved; (Thomas et al., 1998) qualitatively described positive outcomes on anger management skills and assaultive behavior among youth who recently lost a loved one to violence but did not include a control group.

Community settings Finally, a community-level intervention that included multiple components, such as park cleaning/greening, leisure activities, injury prevention (including firearms), did not show any specific effects on firearm injuries (Davidson et al., 1994).

Discussion

It is imperative to develop primary prevention efforts to prevent firearm-related injuries and deaths among children and adolescents given the individual, social, and economic costs (Fowler et al., 2017; Howell & Abraham, 2013) by increasing safe storage among caregivers and decreasing firearm handling/carriage among youth. To inform such prevention efforts, this scoping review assessed the state of the science regarding primary prevention approaches. We reviewed the literature on interventions focused on the safe storage of firearms and firearm handling, carriage, and use, identifying few evidenced-based programs in home, school, healthcare, or community settings. Overall, the literature is plagued with methodological weaknesses, with few RCTs and infrequent evaluation of longer-term behavioral outcomes (e.g., safe storage, firearm carriage and play), as well as the lack of assessment of the impact on firearm injury and mortality. These weaknesses likely relate to the lack of funding for rigorous trials. Nonetheless, some tentative conclusions and future directions can be provided.

Firearm safe storage

Interventions in health care settings have been most rigorously evaluated, providing evidence that clinician screening and education around firearm safety, particularly when paired with the distribution of free gun locks, increases self-reports of safe firearm storage in homes with children (Rowhani-Rahbar et al., 2015). Given that routine screening and education across health care settings is lacking (Roszko et al., 2016), improving adoption of screening and firearm safety education is needed. As safe storage education in health care settings have often focused on children, extension to adolescents is urgently needed given rates of suicide and homicide. In addition, there is little data regarding the feasibility and acceptability of screening and counseling approaches in health care settings from the perspective of patients. Finally, pediatric health care visits are constrained by time allotted to the visit, limited opportunities to reinforce messaging over time, and are typically attended by mothers (Barkin et al., 2008), which means that counseling in this setting may fail to reach other adults or gun owners in the household.

Thus, from a social ecological perspective (Centers for Disease Control and Prevention, 2015; Urie, 1979), prevention programs in health care settings are necessary but are not sufficient to prevent all youth firearm-related injuries, with community-level approaches needed to complement such efforts. As evidence supports that providing free gun locking devices leads to an increase in self-reports of safer storage, there is rationale to expand such practices across settings. However, measurement of safe storage was almost solely through self-report which may be plagued by social desirability issues, indicating a need for further research on methods of assessing safe storage. The one study that included field observations of safe storage did not indicate whether this observation was a cursory visual inspection, or if it included query and assessment of firearms that may not have been stored safely but were out of the visual field of the research team (Grossman et al., 2000). Verification of self-report of firearm storage is needed using approaches that are acceptable to gun owners, as some findings suggest that home visits may not be acceptable (Sangvai et al., 2007) as well as very expensive. Finally, although firearm buyback programs increase the number of firearms relinquished, their efficacy on reducing youth firearm injuries is not yet determined and requires more rigorous study.

Firearm carriage, handling, and use

Focusing on children, school-based interventions that include behavioral skills training to prevent firearm handling increase knowledge and hypothetical skills, but lack data on real world situations and effects on firearm injuries. Despite this unknown efficacy, development of scalable approaches to reduce handling of firearms among preschool and elementary-aged youth seems warranted. Similarly, among adolescents, school-based programs have positive outcomes on knowledge and attitudes, but effects on firearm carriage, handling and use behaviors are largely unknown, with a notable exception showing positive effects on weapon carriage at post-test (DuRant et al., 2001). In parallel, interventions for adolescents that use scare tactics (e.g., videos, testimonials, and/or tours illustrating gunshot wounds and emergency care settings) lack evidence for changing firearm carriage and firearm injury. Note that general violence prevention studies were not included in this review, as they did not measure firearm specific outcomes. For example, a single session intervention (Project Sync) for adolescents presenting to the emergency department for any reason reduced aggression, including firearm aggression (Carter et al., 2016), and a collaborative care intervention for hospitalized adolescents reduced weapon carriage, including firearms (Zatzick et al., 2014). A potential next step for research in this area is to test the potential impact of these promising prevention programs, such as SaferTeens, Project Sync, and others in the CDC technical package on violence, specifically on firearm outcomes in fully powered studies (Blueprints, 2018; Caldwell et al., 2010; Carter et al., 2016; Center for Disease Control and Prevention, 2018; Cunningham et al., 2012; DuRant et al., 2001; Reischl et al., 2011; Walton et al., 2010; Zatzick et al., 2014; Zimmerman et al., 2011).

Further, although multi-level community programs provide an opportunity to increase impact on prevention of youth firearm injuries, only one study was identified in this review that focused specifically on children or adolescents (Davidson et al., 1994). Among broader community samples, data suggest that vacant lot clean up and greening projects have an impact on firearm assaults (Branas et al., 2011; Garvin et al., 2013). Specifically, in a recent cluster randomized trial that fell outside of our search period, removal of blighted vacant lots reduced shootings, with no evidence that the shootings were displaced to adjacent areas (Moyer et al., 2019). Moreover, a recent study demonstrated that implementation of primary violence prevention efforts across the social ecology in a community (e.g., individual intervention, mentoring, vacant lot clean up) reduced assault injuries; however, firearm-related outcomes (e.g., carriage, use, injuries) were not assessed (Heinze et al., 2016). Finally, several community-level interventions focused primarily on gang members, and/or older samples (e.g., adults over 18 years; Cease Fire: (Skogan et al., 2009); Safe Streets: (Duncan et al., 2014; Milam et al., 2016; Whitehill et al., 2013), show promise for reducing firearm injury. Altogether, these studies underscore the potential impact of multi-sector community wide interventions.

Limitations

Several limitations must be noted. First, this scoping review may not have been fully comprehensive (e.g., did not include targeted searching of specific journals or hand searches in journals without an online presence). Second, studies published between the initial search and publication of this article will not be included. Third, we did not include non-English articles or successful prevention intervention efforts outside of the United States. Given the unique context of U.S. firearm ownership, we felt it was appropriate to focus on the state of the science in the U.S. Next, our review required firearms to be in the title or abstract, which could have excluded articles that describe prevention programs that did not list firearms specifically; however, such studies are not likely to measure firearm specific behaviors if not stated. This review did not include policy-level changes in relation to youth firearm outcomes as those are described elsewhere, nor did it focus on interventions for high-risk samples (e.g., youth with a prior firearm and/or assault injury) as these likely require a dedicated focus of their own. Despite these limitations, this scoping review provides useful insights into future directions.

Future directions for research and practice

Overall, research on firearm safety for children and adolescents require a broader focus and more rigorous methods, including: (1) use of RCT designs to control for societal level events (e.g., legislative changes, mass shootings); (2) measurement of multiple firearm behavioral outcomes that go beyond self-report, including storage, carriage and use, as well as firearm injuries and deaths; (3) assessment of longitudinal outcomes beyond post-test; (4) inclusion of diverse samples by age (children and adolescents), gender (caregiver and youth), race/ethnicity, and socio-economic resources; (5) enhanced attention to fidelity of intervention delivery; (6) focus on multiple risks for adverse firearm outcomes across youth violence, suicide, IPV, and unintentional injuries; (7) integration of interventions across the social ecology; (8) attention to implementation of evidenced-based approaches (e.g., see Wolk et al., 2017), including measurement of cost effectiveness. This is particularly true for research and interventions focusing on safe storage, and firearm handling, carriage, and use among children and adolescents.

Of critical importance, we recommend using a participatory action framework (Baum et al., 2006) to partner with firearm owners to further research and practice of prevention of youth firearm-related injuries. Two recent studies which fell outside of our article search time frame indicated that parents, healthcare providers, staff and clinic leaders found firearm interventions acceptable and feasible (Beidas et al., 2018; Wolk et al., 2018). Wolk and colleagues (2018) also found that firearm owners and non-owners had overall similar responses with regard to implementing an evidence-based approach to firearm safety promotion (Wolk et al., 2018). However, many of the interventions reviewed failed to mention whether they received input from firearm-owning caregivers, potentially creating a disconnect between program focus and what is acceptable, including reasons for firearm ownership (e.g., safety/protection, hunting, recreation) and barriers to safe storage. In a parallel manner, involvement of firearm owners in refining interventions to prevent carriage and use of firearms by adolescents is considered essential.

There is mounting evidence that in healthcare settings, clinician-provided education in tandem with gun locks is a promising intervention approach for child and adolescent firearm safety. Moreover, Beidas et al. (2018) and colleagues (in press) found that primary care physicians who were provided training, provided education about firearm safety commonly, although not routinely. This parallels promising preliminary evidence that community-based gun lock distribution and education may also be an effective approach to decreasing firearm injuries among children and adolescents (Simonetti et al., 2018). Although gun buy-back programs have reported more mixed results, additional research into these strategies are also needed to fully evaluate efficacy (Callahan et al., 1994; McGuire et al., 2011). Finally, a two-pronged approach could be used, based on whether firearms are present in the home. For families without firearms in the home, safe storage counseling would focus on prevention of access in other homes where their children may play; for families with firearms in the home, safe storage counseling and provision of free locking devices would be tailored based on the adults in the home, the number/type of firearms present, and whether the youth was at risk for suicide.

Notably, the majority of behavioral interventions across community settings are delivered using an individual, in-person delivery modality. Alternatively, to promote fidelity and scalability, future studies should consider harnessing digital modalities to support screening, and intervention delivery (e.g., videos, text messaging, websites), which could be integrated with in-person delivery. Moreover, there were no studies utilizing mobile health approaches (e.g., social media, smartphone apps, wearables) for firearm safety for youth.

Finally, the lack of research testing the efficacy of firearm safety technology (e.g., trigger locks, personalized smart sensors) on improving safety and accessibility of firearms by adults, while preventing negative firearm outcomes among youth, is a serious gap in the field. Such studies could be informed by research examining ergonomics of firearm safety mechanisms (Cornell & Khasawneh, 2008) as well as the preferences of gun owners. It is unknown what segments of gun owners may prefer higher technology gun locks, or smart firearm technology. Academic researchers, gun owners and industry manufacturers could partner in fruitful collaborations to further advancements in firearm design and technology (e.g., biosensors) that could potentially enhance firearm safety for children and adolescents while respecting preferences of gun owners. Research here may learn from similar changes in acceptability of car and safety seat technology over the past two decades.

Conclusions

There is a clear need for evidenced-based primary prevention approaches to reduce firearm injuries among children and adolescents, which to date are limited due to methodological weaknesses and an overall paucity of research. Best practices for prevention are needed across settings, including homes, schools, health care, and broader communities with attention to scalability, potentially using digital health approaches. To accelerate the process of filling this knowledge gap, we encourage researchers to incorporate firearm-related measures and outcomes into longitudinal studies of youth and to consider adapting promising, evidenced-based interventions to address firearm outcomes. These adapted interventions could be tested using hybrid efficacy-implementation studies (Curran et al., 2012). Partnering with community stakeholders and firearm owning parents, using a participatory action approach (Baum et al., 2006), will be critical during adaptation to maximize acceptability, efficacy and impact on youth. Finally, firearm safety technology both in safe storage devise and in firearms themselves is an emerging area that requires study to determine acceptability, feasibility and benefits and risks for families of children and adolescents.

While we recognize and discuss the need for further research, we believe the overall currently available evidence indicates that safe-storage of firearms will reduce the risk of firearm injury to children and adolescents. Physicians in the U.S. are allowed to and should ask families about firearms in the homes where youth live and regularly visit. Counseling about safe storage and provision of locking devices can potentially reduce firearm injuries and death among youth.

Notes

Acknowledgements

We would like to thank Gurpreet Kaur Rana, Informationist at the Taubman Health Sciences Library, University of Michigan, for creating the search strategies. We would also like to thank Jhuree Hong and Jaemin Park, and Stephanie Kostolansky for assisting with title abstract screening, organizing and summarizing articles for review.

Funding

This review was funded by NIH/NICHD 1R24HD087149-01A1. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the funding agencies. Supported by the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health K23 AA022641.

Compliance with ethical standards

Conflict of interest

Quyen M. Ngo, Eric Sigel, Allante Moon, Sara F. Stein, Lynn S. Massey, Frederick Rivara, Cheryl King, Mark Ilgen, Rebecca Cunningham, and Maureen A. Walton declares that they have no conflict of interest.

Human and animal rights and Informed consent

All procedures followed were in accordance with ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000. Informed consent was obtained from all patients for being included in the study.

References

  1. Albright, T. L., & Burge, S. K. (2003). Improving firearm storage habits: Impact of brief office counseling by family physicians. The Journal of the American Board of Family Practice, 16, 40–46.CrossRefGoogle Scholar
  2. American Academy of Pediatrics. (2016). Survey: Gun violence prevention a big issue for most pediatricians. Retrieved from http://www.aappublications.org/news/2016/08/10/GunViolence081016. Accessed 9 Nov 2018.
  3. Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8, 19–32.CrossRefGoogle Scholar
  4. Barkin, S. L., Finch, S. A., Ip, E. H., Scheindlin, B., Craig, J. A., Steffes, J., et al. (2008). Is office-based counseling about media use, timeouts, and firearm storage effective? Results from a cluster-randomized, controlled trial. Pediatrics, 122, e15–e25.CrossRefGoogle Scholar
  5. Baum, F., MacDougall, C., & Smith, D. (2006). Participatory action research. Journal of Epidemiology and Community Health, 60, 854–857.CrossRefGoogle Scholar
  6. Beidas, R. S., Jager-Hyman, S., Becker-Haimes, E. M., Wolk, C. B., Ahmedani, B. K., Zeber, J. E., et al. (2018). Acceptability and use of evidence-based practices for firearm storage in pediatric primary care. Academic Pediatrics.  https://doi.org/10.1016/j.acap.2018.11.007.Google Scholar
  7. Blueprints for Healthy Youth Development. (2018). Blueprints for Healthy Youth Development. Retrieved from https://www.blueprintsprograms.org/. Accessed 9 Nov 2018.
  8. Branas, C. C., Cheney, R. A., MacDonald, J. M., Tam, V. W., Jackson, T. D., & Ten Have, T. R. (2011). A difference-in-differences analysis of health, safety, and greening vacant urban space. American Journal of Epidemiology, 174, 1296–1306.CrossRefGoogle Scholar
  9. Caldwell, C. H., Rafferty, J., Reischl, T. M., De Loney, E. H., & Brooks, C. L. (2010). Enhancing parenting skills among nonresident African American fathers as a strategy for preventing youth risky behaviors. American Journal of Community Psychology, 45, 17–35.  https://doi.org/10.1007/s10464-009-9290-4 CrossRefGoogle Scholar
  10. Callahan, C. M., Rivara, F. P., & Koepsell, T. D. (1994). Money for guns: Evaluation of the Seattle gun buy-back program. Public Health Reports, 109, 472.Google Scholar
  11. Carbone, P. S., Clemens, C. J., & Ball, T. M. (2005). Effectiveness of gun-safety counseling and a gun lock giveaway in a hispanic community. Archives of Pediatrics and Adolescent Medicine, 159, 1049–1054.  https://doi.org/10.1001/archpedi.159.11.1049 CrossRefGoogle Scholar
  12. Carter, P. M., Walton, M. A., Zimmerman, M. A., Chermack, S. T., Roche, J. S., & Cunningham, R. M. (2016). Efficacy of a universal brief intervention for violence among urban emergency department youth. Academic Emergency Medicine, 23, 1061–1070.  https://doi.org/10.1111/acem.13021 CrossRefGoogle Scholar
  13. Centers for Disease Control and Prevention. (2015). The social-ecological model: A framework for prevention. Atlanta, GA: CDC. Retrieved from https://www.cdc.gov/violenceprevention/overview/social-ecologicalmodel.html. Accessed 2 Nov 2018.
  14. Center for Disease Control and Prevention. (2018). Youth violence. Retrieved from https://www.cdc.gov/violenceprevention/youthviolence/index.html. Accessed 2 Nov 2018.
  15. Chang, D. C., Cornwell, E. E., III, Sutton, E. R., Yonas, M. A., & Allen, F. (2005). A multidisciplinary youth violence-prevention initiative: Impact on attitudes. Journal of the American College of Surgeons, 201, 721–723.CrossRefGoogle Scholar
  16. Cornell, M. A., & Khasawneh, M. T. (2008). An ergonomic comparison of firearm safety mechanisms. Ergonomics, 51, 1394–1406.CrossRefGoogle Scholar
  17. Coyne-Beasley, T., Schoenbach, V. J., & Johnson, R. M. (2001). Love our kids, lock your guns: A community-based firearm safety counseling and gun lock distribution program. Archives of Pediatrics and Adolescent Medicine, 155, 659–664.CrossRefGoogle Scholar
  18. Crossen, E. J., Lewis, B., & Hoffman, B. D. (2015). Preventing gun injuries in children. Pediatrics in Review, 36(2), 43–50.CrossRefGoogle Scholar
  19. Cunningham, R. M., Chermack, S. T., Zimmerman, M. A., Shope, J. T., Bingham, C. R., Blow, F. C., et al. (2012). Brief motivational interviewing intervention for peer violence and alcohol use in teens: One-year follow-up. Pediatrics, 129, 1083–1090.  https://doi.org/10.1542/peds.2011-3419 CrossRefGoogle Scholar
  20. Cunningham, R. M., Walton, M. A., & Carter, P. M. (2018). The major causes of death in children and adolescents in the United States. New England Journal of Medicine, 379, 2468–2475.CrossRefGoogle Scholar
  21. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50, 217.CrossRefGoogle Scholar
  22. Davidson, L. L., Durkin, M. S., Kuhn, L., O’Connor, P., Barlow, B., & Heagarty, M. C. (1994). The impact of the Safe Kids/Healthy Neighborhoods Injury Prevention Program in Harlem, 1988 through 1991. American Journal of Public Health, 84, 580–586.CrossRefGoogle Scholar
  23. Dingeldein, L., Sheehan, K., Krcmarik, M., & Dowd, M. D. (2012). Evaluation of a firearm injury prevention web-based curriculum. Teaching and Learning in Medicine, 24, 327–333.CrossRefGoogle Scholar
  24. Dowd, M. D. (1999). Childhood injury prevention at home and play. Current Opinion in Pediatrics, 11, 578–582.CrossRefGoogle Scholar
  25. Duncan, T. K., Waxman, K., Romero, J., & Diaz, G. (2014). Operation PeaceWorks: A community program with the participation of a Level II trauma center to decrease gang-related violence. Journal of Trauma and Acute Care Surgery, 76, 1208–1213.CrossRefGoogle Scholar
  26. DuRant, R. H., Barkin, S., & Krowchuk, D. P. (2001). Evaluation of a peaceful conflict resolution and violence prevention curriculum for sixth-grade students. Journal of Adolescent Health, 28, 386–393.CrossRefGoogle Scholar
  27. Emde, K. (2002). Cops & docs: A gun violence injury prevention program. International Journal of Trauma Nursing, 8, 54–56.CrossRefGoogle Scholar
  28. Fein, J. A., & Mollen, C. J. (1999). Interpersonal violence. Current Opinion in Pediatrics, 11, 588–593.CrossRefGoogle Scholar
  29. Fowler, K. A., Dahlberg, L. L., Haileyesus, T., Gutierrez, C., & Bacon, S. (2017). Childhood firearm injuries in the United States. Pediatrics, e20163486.Google Scholar
  30. Garvin, E. C., Cannuscio, C. C., & Branas, C. C. (2013). Greening vacant lots to reduce violent crime: A randomised controlled trial. Injury Prevention, 19, 198–203.CrossRefGoogle Scholar
  31. Gatheridge, B. J., Miltenberger, R. G., Huneke, D. F., Satterlund, M. J., Mattern, A. R., Johnson, B. M., et al. (2004). Comparison of two programs to teach firearm injury prevention skills to 6-and 7-year-old children. Pediatrics, 114, e294–e299.CrossRefGoogle Scholar
  32. Goldberg, A. J., Toto, J. M., Kulp, H. R., Lloyd, M. E., Gaughan, J. P., Seamon, M. J., et al. (2010). An analysis of inner-city students’ attitudes towards violence before and after participation in the “Cradle to Grave” programme. Injury, 41, 110–115.CrossRefGoogle Scholar
  33. Goldstick, J. E., Carter, P. M., Walton, M. A., Dahlberg, L. L., Sumner, S. A., Zimmerman, M. A., et al. (2017). Development of the SaFETy score: A clinical screening tool for predicting future firearm violence risk. Annals of Internal Medicine, 166, 707–714.  https://doi.org/10.7326/M16-1927 CrossRefGoogle Scholar
  34. Green, J., Damle, R. N., Kasper, R. E., Violano, P., Manno, M., Nazarey, P. P., et al. (2017). Are “goods for guns” good for the community? An update of a community gun buyback program. Journal of Trauma and Acute Care Surgery, 83, 284–288.CrossRefGoogle Scholar
  35. Gross, A., Miltenberger, R., Knudson, P., Bosch, A., & Breitwieser, C. B. (2007). Preliminary evaluation of a parent training program to prevent gun play. Journal of Applied Behavior Analysis, 40, 691–695.CrossRefGoogle Scholar
  36. Grossman, D. C., Cummings, P., Koepsell, T. D., Marshall, J., D’Ambrosio, L., Thompson, R. S., et al. (2000). Firearm safety counseling in primary care pediatrics: A randomized, controlled trial. Pediatrics, 106, 22–26.CrossRefGoogle Scholar
  37. Grossman, D. C., Stafford, H. A., Koepsell, T. D., Hill, R., Retzer, K. D., & Jones, W. (2012). Improving firearm storage in Alaska native villages: A randomized trial of household gun cabinets. American Journal of Public Health, 102, S291–S297.CrossRefGoogle Scholar
  38. Hardy, M. S. (2002). Teaching firearm safety to children: Failure of a program. Journal of Developmental and Behavioral Pediatrics, 23, 71–76.CrossRefGoogle Scholar
  39. Hardy, M. S., Armstrong, F. D., Martin, B. L., & Strawn, K. N. (1996). A firearm safety program for children: They just can’t say no. Journal of Developmental and Behavioral Pediatrics: JDBP, 17, 216–221.CrossRefGoogle Scholar
  40. Hayes, D. N., & Sege, R. (2003). FiGHTS: A preliminary screening tool for adolescent firearms-carrying. Annals of Emergency Medicine, 42, 798–807.CrossRefGoogle Scholar
  41. Heinze, J. E., Reischl, T. M., Bai, M., Roche, J. S., Morrel-Samuels, S., Cunningham, R. M., et al. (2016). A comprehensive prevention approach to reducing assault offenses and assault injuries among youth. Prevention Science: The Official Journal of the Society for Prevention Research, 17, 167–176.  https://doi.org/10.1007/s11121-015-0616-1 CrossRefGoogle Scholar
  42. Himle, M. B., Miltenberger, R. G., Flessner, C., & Gatheridge, B. (2004a). Teaching safety skills to children to prevent gun play. Journal of Applied Behavior Analysis, 37, 1–9.CrossRefGoogle Scholar
  43. Himle, M. B., Miltenberger, R. G., Gatheridge, B. J., & Flessner, C. A. (2004b). An evaluation of two procedures for training skills to prevent gun play in children. Pediatrics, 113, 70–77.CrossRefGoogle Scholar
  44. Horn, A., Grossman, D. C., Jones, W., & Berger, L. (2003). Community based program to improve firearm storage practices in rural Alaska. Injury Prevention, 9, 231–234.CrossRefGoogle Scholar
  45. Howard, P. K. (2005). Evaluation of age-appropriate firearm safety interventions. Pediatric Emergency Care, 21, 473–479.CrossRefGoogle Scholar
  46. Howell, E. M., & Abraham, P. (2013). The hospital costs of firearm assaults. Washington, DC: Urban Institute.Google Scholar
  47. J Celeste Kallenborn MBA, R., Kathie Gonzales BSN, R., Crane, N. B., Karen Pesce BSN, R., Sherry Swan BSN, R., & Flint, L. (2004). Cease Fire Tampa Bay: A three-tiered approach to firearm injury prevention. Journal of Trauma Nursing, 11, 6.Google Scholar
  48. Jostad, C. M., & Miltenberger, R. G. (2004). Firearm injury prevention skills: Increasing the efficiency of training with peer tutoring. Child & Family Behavior Therapy, 26, 21–35.CrossRefGoogle Scholar
  49. Kasper, R. E., Green, J., Damle, R. N., Aidlen, J., Nazarey, P., Manno, M., et al. (2017). And the survey said… evaluating rationale for participation in gun buybacks as a tool to encourage higher yields. Journal of Pediatric Surgery, 52(2), 354–359.CrossRefGoogle Scholar
  50. Kelso, P. D., Miltenberger, R. G., Waters, M. A., Egemo-Helm, K., & Bagne, A. G. (2007). Teaching skills to second and third grade children to prevent gun play: A comparison of procedures. Education and Treatment of Children, 30, 29–48.CrossRefGoogle Scholar
  51. Kenny, M. C., & Wurtele, S. K. (2016). Teaching preschoolers safety rules: A pilot study of injury prevention. Children’s Health Care, 45, 428–440.CrossRefGoogle Scholar
  52. Komro, K. A. (1999). Adolescents’ access to firearms: Epidemiology and prevention. Journal of Health Education, 30, 290–296.CrossRefGoogle Scholar
  53. Kruesi, M. J., Grossman, J., Pennington, J. M., Woodward, P. J., Duda, D., & Hirsch, J. G. (1999). Suicide and violence prevention: Parent education in the emergency department. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 250–255.CrossRefGoogle Scholar
  54. Kunkel, P., Thomas, C. J., Seguin, C., Dereczyk, D., Rajda, C., & Brandt, M.-M. (2010). A hospital-based violence prevention tour: A collaborative approach to empower youth. Journal of Trauma and Acute Care Surgery, 68, 289–293.CrossRefGoogle Scholar
  55. Liller, K. D., Perrin, K., Nearns, J., Pesce, K., Crane, N. B., & Gonzalez, R. R. (2003). Evaluation of the “Respect Not Risk” firearm safety lesson for 3rd-graders. The Journal of School Nursing, 19, 338–343.CrossRefGoogle Scholar
  56. Loh, K., Walton, M. A., Harrison, S. R., Zimmerman, M., Stanley, R., Chermack, S. T., et al. (2010). Prevalence and correlates of handgun access among adolescents seeking care in an urban emergency department. Accident Analysis and Prevention, 42, 347–353.  https://doi.org/10.1016/j.aap.2009.11.012 CrossRefGoogle Scholar
  57. McGuire, M., Manno, M., Rook, A., Maranda, L., Renaud, E., DeRoss, A., et al. (2011). Goods for guns—the use of a gun buyback as an injury prevention/community education tool. Journal of Trauma and Acute Care Surgery, 71, S537–S540.CrossRefGoogle Scholar
  58. Milam, A. J., Buggs, S. A., Furr-Holden, C. D. M., Leaf, P. J., Bradshaw, C. P., & Webster, D. (2016). Changes in attitudes toward guns and shootings following implementation of the Baltimore Safe Streets intervention. Journal of Urban Health, 93, 609–626.CrossRefGoogle Scholar
  59. Miltenberger, R. G., Gatheridge, B. J., Satterlund, M., Egemo-Helm, K. R., Johnson, B. M., Jostad, C., et al. (2005). Teaching safety skills to children to prevent gun play: An evaluation of in situ training. Journal of Applied Behavior Analysis, 38, 395–398.CrossRefGoogle Scholar
  60. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151, 264–269.CrossRefGoogle Scholar
  61. Moyer, R., MacDonald, J. M., Ridgeway, G., & Branas, C. C. (2019). Effect of remediating blighted vacant land on shootings: A citywide cluster randomized trial. American Journal of Public Health, 109, 140–144.CrossRefGoogle Scholar
  62. Oatis, P. J., Buderer, N. M. F., Cummings, P., & Fleitz, R. (1999). Pediatric practice based evaluation of the Steps to Prevent Firearm Injury program. Injury Prevention, 5, 48–52.CrossRefGoogle Scholar
  63. Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016). Rayyan: A web and mobile app for systematic reviews. Systematic Reviews, 5, 210.CrossRefGoogle Scholar
  64. Parikh, K., Silver, A., Patel, S. J., Iqbal, S. F., & Goyal, M. (2017). Pediatric firearm-related injuries in the United States. Hospital Pediatrics, hpeds. 2016-0146.Google Scholar
  65. Regan, M. E. (2009). Implementation and evaluation of a youth violence prevention program for adolescents. The Journal of School Nursing, 25, 27–33.CrossRefGoogle Scholar
  66. Reich, K., Culross, P. L., & Behrman, R. E. (2002). Children, youth, and gun violence: Analysis and recommendations. The Future of Children, 12, 5–23.CrossRefGoogle Scholar
  67. Reischl, T. M., Zimmerman, M. A., Morrel-Samuels, S., Franzen, S. P., Faulk, M., Eisman, A. B., et al. (2011). Youth empowerment solutions for violence prevention. Adolescent Medicine-State of the Art Reviews, 22, 581.Google Scholar
  68. Rivara, F. P., & Fan, M. D. (2017). Pediatricians, firearms, and the First Amendment. JAMA Pediatrics, 171, 723–724.CrossRefGoogle Scholar
  69. Roberto, A., Meyer, G., Johnson, A. J., Atkin, C., & Smith, P. (2002). Promoting gun trigger-lock use: Insights and implications from a radio-based health communication intervention. Journal of Applied Communication Research, 30, 210–230.CrossRefGoogle Scholar
  70. Roszko, P. J., Ameli, J., Carter, P. M., Cunningham, R. M., & Ranney, M. L. (2016). Clinician attitudes, screening practices, and interventions to reduce firearm-related injury. Epidemiologic Reviews, 38, 87–110.  https://doi.org/10.1093/epirev/mxv005 Google Scholar
  71. Rowhani-Rahbar, A., Simonetti, J. A., & Rivara, F. P. (2016). Effectiveness of interventions to promote safe firearm storage. Epidemiologic Reviews, 38, 111–124.Google Scholar
  72. Rowhani-Rahbar, A., Zatzick, D., Wang, J., Mills, B. M., Simonetti, J. A., Fan, M. D., et al. (2015). Firearm-related hospitalization and risk for subsequent violent injury, death, or crime perpetration: A cohort study. Annals of Internal Medicine, 162, 492–500.CrossRefGoogle Scholar
  73. Runyan, C. W., Becker, A., Brandspigel, S., Barber, C., Trudeau, A., & Novins, D. (2016). Lethal means counseling for parents of youth seeking emergency care for suicidality. Western Journal of Emergency Medicine, 17, 8.CrossRefGoogle Scholar
  74. Sangvai, S., Cipriani, L., Colborn, D. K., & Wald, E. R. (2007). Studying injury prevention: Practices, problems, and pitfalls in implementation. Clinical Pediatrics, 46, 228–235.CrossRefGoogle Scholar
  75. Scott, J., Azrael, D., & Miller, M. (2018). Firearm storage in homes with children with self-harm risk factors. Pediatrics, e20172600.Google Scholar
  76. Sidman, E. A., Grossman, D. C., Koepsell, T. D., D’Ambrosio, L., Britt, J., Simpson, E. S., et al. (2005). Evaluation of a community-based handgun safe-storage campaign. Pediatrics, 115, e654–e661.CrossRefGoogle Scholar
  77. Simonetti, J. A., Rowhani-Rahbar, A., King, C., Bennett, E., & Rivara, F. P. (2018). Evaluation of a community-based safe firearm and ammunition storage intervention. Injury Prevention, 24, 218–223.CrossRefGoogle Scholar
  78. Skogan, W., Hartnett, S., Bump, N., & Dubois, J. (2009). Evaluation of CeaseFire-Chicago. Northwestern University Illinois. Retrieved from https://www.ncjrs.gov/pdffiles1/nij/grants/227181.pdf. Accessed 2 Nov 2018.
  79. Stevens, M. M., Olson, A. L., Gaffney, C. A., Tosteson, T. D., Mott, L. A., & Starr, P. (2002). A pediatric, practice-based, randomized trial of drinking and smoking prevention and bicycle helmet, gun, and seatbelt safety promotion. Pediatrics, 109, 490–497.CrossRefGoogle Scholar
  80. Thomas, S. B., Leite, B., & Duncan, T. (1998). Breaking the cycle of violence among youth living in metropolitan Atlanta: A case history of kids alive and loved. Health Education & Behavior, 25, 160–174.CrossRefGoogle Scholar
  81. Tucker, J. B., Barone, J. E., Stewart, J., Hogan, R. J., Sarnelle, J. A., & Blackwood, M. M. (1999). Violence prevention: Reaching adolescents with the message. Pediatric Emergency Care, 15, 436–439.CrossRefGoogle Scholar
  82. Urie, B. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard.Google Scholar
  83. Violano, P., Driscoll, C., Chaudhary, N. K., Schuster, K. M., Davis, K. A., Borer, E., et al. (2014). Gun buyback programs: A venue to eliminate unwanted guns in the community. Journal of Trauma and Acute Care Surgery, 77, S46–S50.CrossRefGoogle Scholar
  84. Wafer, M. S., & Carruth, A. (2003). “Locks for life”: A gun lock distribution community health intervention program. Journal of Emergency Nursing, 29, 349–351.CrossRefGoogle Scholar
  85. Walton, M. A., Chermack, S. T., Shope, J. T., Bingham, C. R., Zimmerman, M. A., Blow, F. C., et al. (2010). Effects of a brief intervention for reducing violence and alcohol misuse among adolescents: A randomized controlled trial. JAMA, 304, 527–535.  https://doi.org/10.1001/jama.2010.1066 CrossRefGoogle Scholar
  86. Wargo, C., Erdman, D. A., Smith, J. G., Widom, K., & Reardon, J. (2013). Community gun safety in Central Pennsylvania. Journal of Trauma Nursing, 20, 67–73.CrossRefGoogle Scholar
  87. Whitehill, J. M., Webster, D. W., & Vernick, J. S. (2013). Street conflict mediation to prevent youth violence: Conflict characteristics and outcomes. Injury Prevention, 19, 204–209.CrossRefGoogle Scholar
  88. Williamson, A. A., Guerra, N. G., & Tynan, W. D. (2014). The role of health and mental health care providers in gun violence prevention. Clinical Practice in Pediatric Psychology, 2, 88.CrossRefGoogle Scholar
  89. Wolk, C. B., Jager-Hyman, S., Marcus, S. C., Ahmedani, B. K., Zeber, J. E., Fein, J. A., et al. (2017). Developing implementation strategies for firearm safety promotion in paediatric primary care for suicide prevention in two large US health systems: A study protocol for a mixed-methods implementation study. British Medical Journal Open, 7, e014407.Google Scholar
  90. Wolk, C. B., Van Pelt, A. E., Jager-Hyman, S., Ahmedani, B. K., Zeber, J. E., Fein, J. A., et al. (2018). Stakeholder perspectives on implementing a firearm safety intervention in pediatric primary care as a universal suicide prevention strategy: A qualitative study. JAMA Network Open, 1, e185309–e185309.CrossRefGoogle Scholar
  91. Zatzick, D., Russo, J., Lord, S. P., Varley, C., Wang, J., Berliner, L., et al. (2014). Collaborative care intervention targeting violence risk behaviors, substance use, and posttraumatic stress and depressive symptoms in injured adolescents: A randomized clinical trial. JAMA Pediatrics, 168, 532–539.CrossRefGoogle Scholar
  92. Zimmerman, M. A., Stewart, S. E., Morrel-Samuels, S., Franzen, S., & Reischl, T. M. (2011). Youth empowerment solutions for peaceful communities: Combining theory and practice in a community-level violence prevention curriculum. Health Promotion Practice, 12, 425–439.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Quyen M. Ngo
    • 1
    • 2
    Email author
  • Eric Sigel
    • 3
  • Allante Moon
    • 4
  • Sara F. Stein
    • 5
    • 6
  • Lynn S. Massey
    • 1
  • Frederick Rivara
    • 7
  • Cheryl King
    • 8
  • Mark Ilgen
    • 9
  • Rebecca Cunningham
    • 4
  • Maureen A. Walton
    • 8
  • For the FACTS Consortium
  1. 1.Department of Emergency MedicineUniversity of Michigan School of MedicineAnn ArborUSA
  2. 2.Institute for Research on Women and GenderUniversity of MichiganAnn ArborUSA
  3. 3.Section of Adolescent Medicine Department of PediatricsUniversity of Colorado School of MedicineAuroraUSA
  4. 4.School of Public HealthUniversity of MichiganAnn ArborUSA
  5. 5.Department of PsychologyUniversity of Michigan School of MedicineAnn ArborUSA
  6. 6.School of Social WorkUniversity of MichiganAnn ArborUSA
  7. 7.Department of PediatricsUniversity of WashingtonSeattleUSA
  8. 8.Department of PsychiatryUniversity of Michigan School of MedicineAnn ArborUSA
  9. 9.VA Center Clinical Management Research (CCMR)VA Ann Arbor Healthcare SystemAnn ArborUSA

Personalised recommendations